I'm sorry, how are the glasses supposed to react to "Hey Meta" if they are not listening the entire time? Did people get so stupid in the meantime?

Yes any smart device with a microphone does this and they keep a protocol of everything stored on the device even in offline mode so they can upload it as soon as they are connected. Even if you never connect to the internet authorities can easily extract the protocol from the device if they get their hands on it. Those are months of conversations that you don‘t even remember.

My wife asked me why I carry a gun around the house. I said, ‘Decepticons.’ We laughed, the toaster laughed. I shot the toaster. Good times.

And someone somewhere has to replay your gibberish mumbles over and over until they understand it to train the AI