Training Custom Agents To Reduce Margin of Error

GPT builders
3

Hi,

Here is an example of report of a legal doc risks: GPT-4o - Hallucinating at temp:0 - Unusable in production - #26 by sergeliatko

Built in following steps:

  1. Prepare the doc elements for RAG
  2. Run analysis questions from policy file and store them in DB grouped by theme
  3. Pull analysis results by theme (all analysis points) sorted by importance from DB and generate summaries for each of the themes
  4. Summarize the results from step #3 and prepend them as the introduction to report
  5. Stitch all together: intro + reports for each of themes in order defined by user in app options.

Also, this answer might point into good direction: Efficient way for Chunking CSV Files or Structured Data - #7 by sergeliatko

You can also find some tips here: Using gpt-4 API to Semantically Chunk Documents

Providing the “incorrect output” in examples is kind of tricky and may introduce errors (especially when they are at the end of the prompt). Have you tried to replace them with examples of how to correct the wrong output to make it awesome? And try to put those somewhere in the middle of your examples, so that the examples end with several great items to set up the rhythm for the model.

1 Like