You could install jellyfin on a computer and point it at your media folders. Use your browser to navigate content or in one of the other jellyfin clients on the devices you want to consume your content on.
post
I used mp3tag back in the day to do mp3 and aac files. It’s possible it could work with videos.
And: https://alternativeto.net/software/mp3tag/
If you filter by your OS (and I suggest: open source), you might find one that does that.
And: I’d be surprised if ffmpeg isn’t able to do it. Here’s a random search result that may work: https://www.bannerbear.com/blog/how-to-set-a-custom-thumbnail-for-a-video-file-using-ffmpeg/
all 2 comments