The best explanation I've heard, as an agnostic, is this is our misperception of this life vs an eternal (infinite) afterlife.
Sure, bad people might live really fun lives for maybe 100 years while good people might suffer for the same. But then, for millions and millions and gazillions of millennia after that, both are punished or rewarded for how they chose to live those 100 years.
My friend's baby had a bunch of vaccinations the other day. Horrible day for the kid, unfun and stabby. But in a dozen years, will she remember it? Nope. But that day, that's all she could comprehend. Similarly, for this brief go around, yeah, what we can see and experience is all we can really comprehend. But the promise of at least Christianity, but I think many other religions, is that the next life is infinite and all that matters.