This is a fundamental issue of the Matrix protocol, yes. For regular small scale use it doesn't matter so much, and the state history gets reset every time you do a room upgrade, which is another annoying "feature" of Matrix, but it eases the fundamental problem a bit.
But IMHO the Matrix protocol is a child of the Bitcoin hype era and is built on a similar data-structure that is inherently impossible to scale and the developers of Matrix should have realized that early on. Their bosses back then actually did, but they spun it off as a separate company and got some crypto-currency investments so the can was kicked down the road and here we are...