What local LLMs do you run ? And how do you run them on your device ?
What specific use-case do you use it for ?
1 Like
I like to run Whisper locally.
The API version is definitely better, but having a local STT model saves money and allows you to basically make your own local Alexa for free, amongst other fun perks.
2 Likes
Any application you built with Whisper locally ?
Lol, “applications” are a loose term in my little home lab, because I tend to just play around with things and build more interconnected systems and automation scripts than dedicated apps.
Whisper is great on-hand when I don’t feel like typing something, or want to “say” a command which I could then use to run an ahk script or some other script.
Now I am working on a larger project, but my lips are still zipped on that one.