ML 122: Comparing LLMs on Colab (20 pts extra)

What You Need

Purpose

To use and compare various Large Language Models, running them in a Google Colab under your own control, the way many companies will want to.

Opening the Github

In a browser, go to
A colab gradio web UI for running Large Language Models
Scroll down to the "Colab" section, as shown below.

Stanford's Alpaca

On the "gpt4-x-alpaca-13b-native-4bit-128g" line, right-click the "Open in Colab" button, outlined in red in the image above, and open it in a new tab.

A Google Colab page opens, with a block of code. Run the code.

A pop-up warns that the colab was not authored by Google. Click "Run anyway".

After about five minutes, you'll see a public URL for your GPT server ending in "gradio.live", outlined in red in the image below.

Opening the Web Interface

Click the public URL. The Web interface opens, as shown below.

Paste in this query, and then press the Enter key, or click the Generate button. 

Why is the sky blue?
The answer is correct, as shown below.

Letter Search

Clear the history, by scrolling to the bottom of the page and clicking the "Clear history" button, and clicking Confirm.

Now enter this query: 

Does the letter "e" appear in the word "ketchup"?
The answer is not accurate, as shown below.

Flag ML 122.5: Link Limitations (10 pts)

View your Colab.

The flag is covered by a green rectangle in the image below.

Clearing your Colab

Close the Web interface for the LLM.

In Google Colab, from the menu bar, click Runtime, "Disconnect and delete runtime".

Click Yes.

Close the Colab tab.

Meta's LLaMA

On this page:
A colab gradio web UI for running Large Language Models
Launch the stable-vicuna-13B-GPTQ-4bit-128g model.

Run the queries above. Compare the answers to the answers from the other model. The "ketchup" answer is especially interesting.

Flag ML 122.6: Params (10 pts)

View your Colab. Scroll back a few lines.

The flag is covered by a green rectangle in the image below.

Posted 5-19-23
Flags 1-4 removed, flags 5 and 6 added 12-11-23