I used mp3tag back in the day to do mp3 and aac files. It’s possible it could work with videos.
And: https://alternativeto.net/software/mp3tag/
If you filter by your OS (and I suggest: open source), you might find one that does that.
And: I’d be surprised if ffmpeg isn’t able to do it. Here’s a random search result that may work: https://www.bannerbear.com/blog/how-to-set-a-custom-thumbnail-for-a-video-file-using-ffmpeg/