Why does ChatGPT do that?
Eight mysteries, solved: you can explain to a friend why it answers at all — and why it sometimes makes things up.
- 1Why does it miscount the r’s in "strawberry"?
Tokens · Beginner · 3 min
- 2How does it know "king" is close to "queen"?
Embeddings · Beginner · 5 min
- 3How does it know what "it" refers to?
Self-attention · Beginner · 5 min
- 4What's inside the box?
The Transformer · Builder · 5 min
- 5Does it plan what it's going to say?
Next-token prediction · Beginner · 3 min
- 6Why does the same prompt give different answers?
Temperature and sampling · Beginner · 3 min
- 7How did it learn all this?
Training: loss and gradient descent · Builder · 8 min
- 8Why is it so polite (and so cautious)?
RLHF · Builder · 5 min
- 9Why does it forget what I said 20 messages ago?
coming · opens from the tokens station
- 10Teach it my docs, or let it read them?
coming · opens from the embeddings station
- 11Why does it invent sources that don't exist?
coming · opens from the sampling station
- 12What’s an "agent", and why is everyone saying it?
coming · opens from the play station
- 13Why does it ‘think’ before answering?
coming · opens from the sampling station
- 14Why does the free model feel dumber?
coming · opens from the layers station
- 15Why does the free tier say 429?
coming · opens from the tokens station
- 16How does a 30 GB model fit on my laptop?
coming · opens from the layers station
- 17How does it read a picture?
coming · opens from the tokens station
- 18Why is the second call cheaper?
coming · opens from the tokens station
- 19Why does it break my JSON?
coming · opens from the sampling station
- 20Why can a web page hijack my AI agent?
coming · opens from the play station
- 21How does AI draw a picture from noise?
coming · opens from the sampling station
- 22Why does the model ignore my instructions?
coming · opens from the tokens station