There's some semantics here.
Large neural networks have been shown to build parallel models of the world that they learn through their training data. A chess model trained only on chess move notation does have an interior board and state that it tracks, despite never being told about a board, the number of pieces, how they move, etc.
LLMs do something similar with language, mapping tokens to ideas in ways that can produce meaningful and grammatically correct output without needing to be formally taught any of this.
Our brains also do things like this - we observe X and then try to intuit Y, the truth of the world and reason for X.
Where this breaks down is humans can then take our guess for Y and propose Z, and try to find it out in the world. If we can't, then Y is probably wrong, and our new understanding of the world is Y* that is consistent both with X and the absence of Z.
I cannot stress enough how far we are from machines that can do this. They can learn to approximate Y through training, but they have no means to extend that to propose an unobserved Z, no means to update their internal state given the results of that observation (without fully retraining the model on new data - which makes it a new model), and no way to question X given Z - they can talk at great length about data biases but can literally never excise bad data from their training weights. There have been experiments with training models on time-boxed datasets (e.g. stuff up to AD 1880) and asking it about modern concepts - predictably hilarious results.
All the current 'AI' stuff is smoke and mirrors around this very hard problem. The machines can talk, but they can't think (Z) and can't learn (Y*).