for me, it's that it breaks randomly. all. the. time.
With element, it was constant across two different homeservers I tried. Either logging me out, or just randomly unverifying my client and failing to decrypt messages.
with fluffychat, it's been more stable, but recently it's taken to logging me and a friend of mine out at random just like Element. After logging back in, it constantly tells me there's an error fetching the notification (even though I'm not launching from a notification) due to the account not existing.
The protocol feels incredibly fragile, since it seems every implementation trips over the same footguns, and it makes the user experience horrible.