this post was submitted on 09 Jul 2025
165 points (100.0% liked)

Selfhosted

49336 readers
471 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

I think a lot of people have heard of OpenAI’s local-friendly Whisper model, but I don’t see enough self-hosters talking about WhisperX, so I’ll hop on the soapbox:

Whisper is extremely good when you have lots of audio with one person talking, but fails hard in a conversational setting with people talking over each other. It’s also hard to sync up transcripts with the original audio.

Enter WhisperX: WhisperX is an improved whisper implementation that automatically tags who is talking, and tags each line of speech with a timestamp.

I’ve found it great for DMing TTRPGs — simply record your session with a conference mic, run a transcript with WhisperX, and pass the output to a long-context LLM for easy session summaries. It’s a great way to avoid slowing down the game by taking notes on minor events and NPCs.

I’ve also used it in a hacky script pipeline to bulk download podcast episodes with yt-dlp, create searchable transcripts, and scrub ads by having an LLM sniff out timestamps to cut with ffmpeg.

Privacy-friendly, modest hardware requirements, and good at what it does. WhisperX, apply directly to the forehead.

you are viewing a single comment's thread
view the rest of the comments
[–] hoshikarakitaridia@lemmy.world 3 points 1 day ago (1 children)

Long videos or voice notes where you're usually just looking for a small snippet.

[–] irmadlad@lemmy.world 2 points 1 day ago (2 children)

Now that's an interesting angle. I am a mediocre musician on my best day, but sometimes I incorporate phrases and lyric snippits in a piece. I wonder if I could use WhisperX to find those words or phrases from a stack of songs. For instance, I did a piece that used a line from Jimi Hendrix's 'If 6 were 9' where he says 'I'm the one who's gotta die when it's time for me to die. So let me live my life the way I want to.' I wonder if WhisperX could pick that out of a stack of Jimi Hendrix songs.

[–] dgdft@lemmy.world 2 points 1 day ago* (last edited 1 day ago) (1 children)

You should be able to get decent results if you pipe your tracks through demucs first to isolate the vocals.

https://github.com/adefossez/demucs

Vanilla whisper will probably be better than whisperX for that use case though.

Depending on how esoteric your music library is, you can also build a lyrics DB with beets: https://beets.readthedocs.io/en/stable/plugins/lyrics.html

[–] irmadlad@lemmy.world 2 points 1 day ago

I use UVR for vocal isolation. It just works, but that shouldn't be a problem. I'll check it out. At the worst, I'll learn something.

[–] hoshikarakitaridia@lemmy.world 2 points 1 day ago (1 children)

It might take a while, but when your PC is working on it you are not and searching for words might be easier ^^

I'm excited to hear how well it works ^^

[–] irmadlad@lemmy.world 2 points 1 day ago

I'm always excited to try new stuff. You never know. A use case might develop that you didn't think of.