Personally I am in agreement with Simon Willison. Just give up.
My advice is mainly for the Uber-paranoid crowd. There are solutions. The solutions basically neuter the whole LLM experience, but they could better than anything out there. Filters can be evaded (on both input and output). But if you think about making a “walled garden”, through embeddings, similar to what AOL did in the 90’s, you get a unicorn candy canes and popcorn experience.
… But look what happened to AOL.
So, for generic business interfaces, that aren’t meant to wow people. Sanitized proxy prompts might we worth looking into.
But yeah, for most of us, just do your best and keep building better stuff!