There is (it's the kv cache), but it's recomputable from the session transcript.

Part of why comparing human consciousness and possible AI equivalents is not easily apples to apples.

I can't take a human and create a backup state to restore to, or fork into multiple versions of the same mind that then goes down different paths, or pause and then resume things.

Humans do have consciousness continuity breaks and it's quite possible we only have an illusion of continuity vs actual continuity at a neurological level. So just as a transformer has seams at generating the individual token as well as at a context window, humans have consciousness seams (sleep, anesthesia, highway hypnosis, short term memory capacity, etc). But there are very wide differences between human consciousness as an effectively constrained singleton and AI as able to be quite multiple and resumable from an initial single state.