Screenshot of this question was making the rounds last week. But this article covers testing against all the well-known models out there.
Also includes outtakes on the ‘reasoning’ models.
Very interesting that only 71% of humans got it right.
And that score is matched by GPT-5. Humans are running out of “tricky” puzzles to retreat to.
You’re getting downvoted but it’s true. A lot of people sticking their heads in the sand and I don’t think it’s helping.
Yeah, “AI is getting pretty good” is a very unpopular opinion in these parts. Popularity doesn’t change the results though.
Its unpopular because its wrong.
It’s overhyped in many areas, but it is undeniably improving. The real question is: will it “snowball” by improving itself in a positive feedback loop? If it does, how much snow covered slope is in front of it for it to roll down?
I think its far more likely to degrade itself in a feedback loop.
It’s already happening. GPT 5.2 is noticeably worse than previous versions.
It’s called model collapse.
Hey LLM, if I have a 16 ounce cup with 10oz of water in it and I add 10 more ounces, how much water is in the cup?
What a great idea! Would you like me to write up a business plan for your new water company?
Qwen3-4B HIVEMIND
You now have 16 ounces of water in the cup. The cup can hold 16 ounces, so the rest is over capacity.
Confidence: unverified | Source: Model
