Did you see https://thegradient.pub/othello/ ? They fed a model moves from Othello game without it ever seeing a Othello board. It was able to predict legal moves with this info. But here's the thing - they changed its internal data structures where it stored what seemed like the Othello board, and it made its next move based on this modified board. That is, autocomplete models are developing internal representations of real world concepts. Or so it seems.
That's a great article.
One intriguing possibility is that LLMs may have stumbled upon an as yet undiscovered structure/"world model" underpinning the very concept of intelligence itself.
Should such a structure exist (who knows really, it may), then what we are seeing may well be displays of genuine intelligence and reasoning ability.
Can LLMs ever experience consciousness and qualia though? Now that is a question we may never know the answer to.
All this is so fascinating and I wonder how much farther LLMs can take us.