How Do LLMs Work? A First Look

Next-Word Prediction

When you chat with an LLM like ChatGPT, it might look like the model writes a whole response at once. But that's not what's happening. LLMs actually generate text one word at a time. For each word, the model looks at everything that's been written so far and asks: "What word is most likely to come next?"

The tool below lets you see this process in action. Type a sentence and the model will show you its predictions for the next word — along with how confident it is in each option. Try different starting phrases and see what patterns you notice.

Temperature

So the model assigns a probability to every possible next word. But when it's time to actually pick one, how does it decide? That's where temperature comes in.

At low temperature, the model almost always picks its top prediction — the "safest" choice. At high temperature, it's more willing to pick less likely options, which makes the output more varied and surprising (but also less predictable).

The tool below sends the same prompt to the model at two different temperatures so you can compare the results side by side.

Things to notice:

• Does the model always have one clear "best" prediction, or are there cases where several options seem equally likely?
• How does changing the temperature affect which words get chosen?
• Can you find a prompt where the model is very confident about what comes next? One where it seems uncertain?

A Simpler Model

You just saw that an LLM predicts the next word by looking at all the words that came before it. But what if a model only looked at the last word to make its prediction? That's what a Markov chain does.

The Markov Babbler below builds a simple model from a piece of text. It reads through the text and counts how often each word is followed by each other word. Then it uses those counts to generate new text — picking the next word based only on the current word, with no memory of anything earlier in the sentence.

The walkthrough will show you exactly how this works step by step. Pay attention to what the model can and can't do — it will help you think about what makes LLMs different.

Things to notice:

• How does the output compare to what the LLM generated? What's similar? What's different?
• The Markov model only looks at one word of context. What kinds of mistakes does it make because of that?
• What would you need to add to this simple model to make the output more coherent?

What Are You Curious About?

Based on what you explored today, what questions do you have about how LLMs work? Think about what surprised you, what you couldn't explain, or what you want to understand better.

Enter at least 3 questions below. They can be about any aspect of how language models work.

Investigate

Discuss this question with your partner. Use what you observed in the demos to reason about possible answers. You can also use other resources if helpful, but focus on building your own understanding.