Do you run Local LLMs on your system for privacy?

What local LLMs do you run ? And how do you run them on your device ?

What specific use-case do you use it for ?

1 Like

I like to run Whisper locally.

The API version is definitely better, but having a local STT model saves money and allows you to basically make your own local Alexa for free, amongst other fun perks.


Any application you built with Whisper locally ?

Lol, “applications” are a loose term in my little home lab, because I tend to just play around with things and build more interconnected systems and automation scripts than dedicated apps.

Whisper is great on-hand when I don’t feel like typing something, or want to “say” a command which I could then use to run an ahk script or some other script.

Now I am working on a larger project, but my lips are still zipped on that one.