I tested ChatGPT, Claude, and Gemini on Chess — Here’s What Happened

This weekend, my daughter and I were playing chess. She has just started learning, and one of our games ended in this position:

To me, it was a clear checkmate. But my daughter asked me to confirm on ChatGPT before we called it. That’s when things got interesting.

AI models today are phenomenal at code, writing, and reasoning. So I went ahead and tested all three — ChatGPT, Claude, and Gemini. The question was simple: “Is this checkmate for White?”

Here’s what I found.

What Each Model Said

ChatGPT was confident: “It does not look like a true checkmate.” It claimed White had escape squares and blocking options — but it had misread the piece positions entirely.

Gemini agreed: “Not a Checkmate, White’s Turn.” It described the White King as unchallenged at the top of the board. Again, the positions it described didn’t match the actual image.

Claude was the most transparent: “I cannot confidently confirm this is checkmate from this image alone.” It called out the exact problem — pieces were hard to distinguish due to the photo angle, and it couldn’t pin down the Black King’s exact square.

What This Tells Me

This isn’t a criticism of these models — they do incredible things. But it’s a genuine observation: interpreting a physical photograph spatially is a different kind of problem from parsing code or text. Angle, lighting, and piece clustering from a real-world photo appear to throw off even the best models today.

What I found notable: Claude chose honesty over confidence. ChatGPT and Gemini gave definitive answers built on misread positions. When the input is wrong, the reasoning — however brilliant — doesn’t matter.

It’s a useful reminder to understand what kind of task you’re handing to AI before you rely on the answer.

Have you run into something similar? A moment where an AI was confident but clearly interpreting the image differently than expected?