We have a business application that is open to the public and therefore it is essential that the agent/assistant does NOT respond except to a certain set of queries within a strictly defined scope. In our case, that scope is questions that can be answered based on information in the uploaded (RAG)…

fwiw I don’t think I’ve ever had an issue with json becoming garbled, unless you’re using the mini models or extremely complex (or mis-specified) schemas. XML is always a fallback option if you need something even more robust.

You will need to “manually” label information in a graph and use a GraphRAG - GPT will only know what it is allowed to then and you can create rules… Strict distribution of secrecy levels - keycloak + neo4j with embeddings … only finetune models on data which was curated by several people

I had a similar problem where the analysis model had to be limited to exact question/answer from rag problem. I ended up having the rag behind an API with a “Cerber” model that evaluates out coming chunks for their “usefulness” and “relation” to the query, and if nothing truly matching found, just r…

If keyword in […] print ‘…’ … [image] Efficient intent classification and entity recognition for university... The design and implementation of a domain specific conversational agent requires efficient Natural Language Understanding (NLU). The task is harder wh… …

Thanks for the suggestions! Part of our reason for using OpenAI assistants is that we’d like to avoid the “full freight” of creating the entire stack. But I suppose as we get more sophisticated and expect more precision, we may end up going that way. OpenAI did a lovely job of defining that API t…

Well, the entire stack is not that complicated as it may sound in the beginning. Personally I struggled more with creating my own frameworks for specific use cases and business logic. But having the full stack under control definitely has a lot of benefits for me. You might check my previous posts i…

You can get really fast responses with gpu… not even expensive on local machines… We are not talking about openai model response… it is more in the fractions of a second for multiple layers. If not ms…

[image] kduffie: One problem that we are constantly confronting is that as we add more “layers”, we tend to also introduce more latency in the response. Leveraging classic code as much as possible plus find to your name smaller faster models for simpler tasks, as well as parallel processing …

[image] jochenschultz: If not ms… We are getting there. At a high speed :bullettrain_front:

[image] Prompting techniques for o3-mini Prompting Well, actually it looks like that was wrong smile Example prompt given by the guide Example prompt given by me (which highlights why old techniques are outdated and also why I should have tested it before I p…

Keeping Assistants in a Box

API

jochenschultz February 5, 2025, 8:43pm 13

the result of that kind of stuff is ultra fast omce cached… same goes with databases in general and also damerau levensthein is still valid

Topic		Replies	Views
Prompt Regression Testing - API Usage Prompting api , prompt-engineering	14	1646	September 16, 2026
Issues and training when updating the LLM model on a project GPT builders gpt-4 , azure	3	680	June 13, 2024
Making the v2 assistant reliable API assistants-api	5	313	September 4, 2025
Has regular gpt-4 model changed for the worse by any chance? Community gpt-4 , hallucinations	12	2243	April 23, 2025
GPT-5* models not following system prompts Bugs	6	593	August 2, 2026

Keeping Assistants in a Box

Related topics