GPT one paragraph reply? Condensation/Summary for core ideas (keep content depth)

Prompting
1

I want more detailed responses and I do not want a one paragraph “executive summary”, but no matter what I prompt, that is what I always get from gpt 4 and gpt 3.5?

Maybe you can point me to the right search terms for this problem. Maybe I need to approach it differently. I am at a loss because I do not now what to search for or what the proper solution for this might be called.

End goal is to get the complex ideas into a clear, succinct but complete “summary”. It should not be a one paragraph abstract but that is all I always get. (Might be related to people saying -turbo models are lazy)

  1. I want to reformat/condense an online transcript (PowerPoint talk) or meeting minutes

  2. The source is 19K tokens long

  3. As with a lot of transcript there is filler and with one few shot example (not shared here, because source language is German)
    I manually reduced from 51 Tokens down to 26 tokens and kept the entire content of that block. I am sure a manually edit would end up with 30-40% of token saving (so 19K tokens would turn into less then 12K)

  4. I tried ChatGPT Pro, attached the file and tried various prompts in the Chain of Thought/Analogous Prompt flavor, e.g.

You are a Deep Dive Translator, skilled in elucidating complex discussions in detailed, accessible language without sacrificing content depth. Your task involves transforming a long online talk transcript. Execute this task as follows:

  1. Begin with a thorough reading of the transcript to fully understand the scope of ideas, arguments, and the narrative arc.
  2. Highlight the primary points and themes, noting their significance within the talk.
  3. Pinpoint areas with extensive elaboration. Rather than summarizing, think about how to make these clearer with detailed rephrasing, ensuring the argument remains intact and comprehensive.
  4. Delve into each key point separately. Use detailed analogies, simplified explanations, or expanded illustrations that maintain the original intent but enhance clarity. Ensure each explanation is thorough and retains the complexity of the original discussion.
  5. Systematically rephrase and organize the content. For complex sections, break them down into sub-points for clarity, using detailed paragraphs, clear headings, and bullet points for structure.
  6. Merge these detailed sections back into the overall content, ensuring a seamless flow that respects the original structure and logic. Incorporate transitional phrases to maintain narrative continuity.
  7. Conduct a meticulous review of the rephrased content. Ensure every section is not only accessible and engaging but also comprehensive, mirroring the original talk’s depth and breadth.

Focus on enhancing clarity and engagement without reducing the content’s richness or diluting its arguments. Your rephrased content should reflect a deep understanding and comprehensive representation of the original talk.

No matter what else I add, I only get one paragraph back.

  1. I then tried a condenser in the API playground with gpt-3.5-turbo-16k on the first part of the transcript with this as system prompt (max length for response set to 5000):

As an expert summarizer, your task is to condense the provided text to reduce its token size significantly. You should retain all the key information and the essence of the content without changing the original meaning. Aim for brevity and precision, ensuring that the condensed version captures the main points and themes of the original text. Your output should be a clear, concise version that communicates the original text’s essential messages in fewer words.

Example Input:

Yes, welcome for now. And today we’re talking specifically about weight management. I think we’re all always interested in that. So, now the fasting period begins.

Example Output:

Welcome. Today’s topic: Weight management, which is interesting for everyone, also because Lent is starting

Details:

All expressed in a significantly reduced token count without losing the essence or changing the content.

I played around with setting the temperature to between 0 and 1. No big changes. I always only get back one paragraph?

2

There are fairly active debates here in the Forum around summarization techniques etc., including some more elaborate techniques.

A few recent threads, which also include links to very relevant older threads, are the following:

You might want to take a look at these to see what might fit your specific use case.

As a general rule of thumb, avoiding words like “summary” or “condense” if you are looking for output that is more specific is a good start.

2 Likes
3

With very long inputs the model can get into trouble with issues like “lost in the middle” etc.

A common approach to this is to split large input into smaller chunks, summarise them and rejoin them to form your final summary.

Do your transcripts always follow a similar structure? Otherwise you can choose a fixed length and split on a paragraph break as a simple way to get started.

Another thing you can try is to add a reiteration of the instructions at the end of your initial prompt, or to add a follow up message that asks the LLM to follow the instructions properly.

2 Likes
4

I’m experiencing the same issue on long documents when ‘stuffing’ . This is the ‘missing middle’ 'phenomenon . You may need to chunk the text and produce summaries of those chunks.

5

I came up with the following which worked for me

  1. I took four 2 sentence examples and shortened them
  2. I used this prompt and added the examples

You rewrite a block of text for me without changing the meaning of the text. I’ll give you examples. You always get the original first, then a “|” as a separator, followed by the revised version.
When you’re ready, say ready. Whenever I write you a text, you take it and rewrite it based on the examples.

Example 1

  1. I gave it the text in about 200 line chunks and manually copied the replies to a new file.

It might have done a bit to much work. I might need to make this into a script and use smaller chunks. It reduced 70931 characters down to 13213