ML 122: Comparing LLMs on Colab (20 pts extra)

What You Need

To use and compare various Large Language Models, running them in a Google Colab under your own control, the way many companies will want to.

In a browser, go to

A colab gradio web UI for running Large Language Models

Scroll down to the "Colab" section, as shown below.

On the "vicuna-13b-GPTQ-4bit-128g" line, right-click the "Open in Colab" button, and open it in a new tab.

A Google Colab page opens, with a block of code. Run the code.

A pop-up warns that the colab was not authored by Google. Click "Run anyway".

After about five minutes, you'll see a public URL for your GPT server ending in "gradio.live", outlined in red in the image below.

Click the public URL. The Web interface opens, as shown below.

Paste in this query, and then press the Enter key, or click the Generate button.

Why is the sky blue?

The answer is correct, as shown below.

Clear the history, by scrolling to the bottom of the page and clicking the "Clear history" button, and clicking Confirm.

Now enter this query:

Does the letter "e" appear in the word "ketchup"?

The answer is not accurate, as shown below.

Flag ML 122.5: Link Limitations (10 pts)
View your Colab.
The flag is covered by a green rectangle in the image below.

Close the Web interface for the LLM.

In Google Colab, from the menu bar, click Runtime, "Disconnect and delete runtime".

Click Yes.

Close the Colab tab.

On this page:

A colab gradio web UI for running Large Language Models

Launch the stable-vicuna-13B-GPTQ-4bit-128g model.

Run the queries above. Compare the answers to the answers from the other model. The "ketchup" answer is especially interesting.

Flag ML 122.6: Params (10 pts)
View your Colab. Scroll back a few lines.
The flag is covered by a green rectangle in the image below.

Posted 5-19-23
Flags 1-4 removed, flags 5 and 6 added 12-11-23
Model changed 6-29-24