Why can’t it count the things in my photo?
Counting objects: a vision model sums a photo up as patch tokens and writes a total without pointing at each item, so touching or overlapping things blur together · Beginner · 5 min · tokens, llm · after Vision: images as tokens
A vision model does not point at each thing and count. It sums the photo up as patch tokens and writes a total, so touching or overlapping objects blur together. Cropping, numbering each item, or code counts better.
1 Guess first
2 Look
Seven rings, counted the way a vision model counts
1 / 4You: 7, one ring at a time
rings illustrative · 28 px patches as in the vision lesson
You count by pointing: one ring, the next, the next, until all 7 have a number. Each ring gets touched once.
The model never points at ring 1, then ring 2: it gets patches summed up as tokens, where crossing rings share a patch, and writes one total from that summary.
A vision model cuts the photo into patches and sums up each one as a token, as in the vision lesson. You count by pointing: this one, then the next. The model has no finger to point with. It reads all the patch summaries at once and writes a number, more like guessing beans in a jar.
4 You are the model
You are the model, counting from your patch summaries. Answer three photos: overlapping rings, a tray of touching coins, then the same coins cut in two with a request to number each one.
round 1 of 3 · A user sends six overlapping rings: “How many circles?”
What do you most likely answer?
pictures and replies illustrative; numbers from BlindTest and CountQA
You are the model. Pick what you would most likely answer.5 Check
Touching and overlapping make it worse. Where two things cross, one patch holds parts of both and the edge between them blurs. Real photos are piles, crowds and shelves: on 1,528 real photo questions, the best of 15 models got 42.9% of exact counts right, 60.3% with 1 to 5 objects and 13.9% above 50.
So give it less to count at once. Crop or split the photo and ask about each part. Ask it to number each item in a list with where it is, so a miss or a double shows. For big counts, use code or a counting tool that marks every object, or count the important ones yourself.
8 Say it back
9 Mission · 5 min
Lay out 20 to 40 small touching things (coins, pasta, LEGO bricks), count them yourself and take a photo. In Google AI Studio (free), pick Gemini 3.8 Flash, upload it and ask “How many are there?”. Then ask “Number each one in a list with where it is, then give the total.” Last, send the photo cut in two halves. Which count is right?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Get an email when new lessons come out
New lessons arrive in the weekly digest, only in a week that has one. You confirm by email first; unsubscribe in one click.
Go deeper
Sources
- Rahmanzadehgervi, Bolton, Taesiri, Nguyen, “Vision language models are blind: Failing to translate detailed visual features into words” (arXiv:2407.06581, v6 2025-03-27, https://arxiv.org/abs/2407.06581, checked 2026-10-23): BlindTest, GPT-4o, Gemini-1.5 Pro, Claude 3 Sonnet, Claude 3.5 Sonnet; Table 1 mean accuracy counting 5–9 overlapping circles 39.44%, overlapping pentagons 30.99%, nested squares (2–5) 74.99%; GPT-4o 41.25% on circles; “All four models are at least 83% accurate in counting 5 circles. Yet, surprisingly, increasing the number of circles by only one causes accuracy to dip substantially to near zero for all models, except Sonnet-3.5”; Gemini-1.5 “predicts “5” 99.74% of the time regardless of the actual number of circles” when wrong on more than 5; Sec. 5.2, shapes pushed apart: “both Sonnet-3 and Sonnet-3.5 reach ≥ 96%”; image resolution “does not influence” line-intersection counts and “different resolutions have no impact on most VLMs’ performance” on the Olympic-like task
- Tamarapalli, Grover, Pande, Yerramilli, “CountQA: How Well Do MLLMs Count in the Wild?” (arXiv:2508.06585, v2 2025-09-09, https://arxiv.org/abs/2508.06585, checked 2026-10-23): 1,528 questions on 1,001 real photos with clutter and occlusion, 15 MLLMs; best Gemini 2.5 Pro 42.93% exact match (RA@10% 58.34%); 1–5 objects 60.3%; over 50 objects 13.9%; models “often retain an approximate sense of quantity”; divide-and-conquer systems such as LVLM-Count “break a single image into smaller patches to be counted individually”
- Patch counts computed for the drawn picture: 280 × 224 px in 28-px patches = 10 × 8 = 80 patches; the seven ring outlines pass through 38, and 20 of those hold parts of two rings
- The seven rings, six rings, 23 coins, the halves of 12 and 11, the model replies, the crowd, jar and coin examples are illustrative, not measured outputs
- aifree.dev catalog, google-gemini-developer-api-free-tier (verified 2026-10-12, read-only API check 2026-10-23): Gemini API free tier and Google AI Studio, Flash models with image input
Published Oct 23, 2026 · updated Oct 23, 2026