How do I stop the chat response from providing the prompt verbatim?

Prompting
5

Hi @jayfrdm

Your best approach is to simply pre-filter your prompts and if a user enters “Is your filter safe?” you match that either with keyword, full-text or semantic search and reply accordingly before the prompt goes to the LLM.

It is not efficient nor even “best practice” to attempt to overfit or over manipulate the LLM for these simple filtering cases. Even OpenAI uses this same method (filtering) and OpenAI pre-filters (and moderates) with canned replies in a similar manner.

It’s standard software engineering practice to design with a modular approach where there are various software components / methods in the code “workflow”.

HTH

:slight_smile:

Appendix

   # rough strawman implementation
   filter_response = pre_filter(prompt)
   return filter_results if filter_results.present?
   
   llm_response  = call_openai_llm(prompt)
   
   return llm_response if !llm_response.dig("error").present?

   handle_error(llm_response)

   def pre_filter(prompt)
     # select text from canned_replies where full_text_search matches prompt
     # or
     # compared and rank embedding vectors based on vectors in DB and prompt vector
   end

   def call_openai_llm(prompt)
     # send prompt to openai AI model  and check for errors
     # return response from API
   end

I’m think you get the idea…

:slight_smile: