Instruction following in API Usage

API
1

Hello everyone!

I am hoping someone can help improve my code (below) so that it closely follows the instruction.

The key part of my instruction is that I want the summary to be under 50 words. Is there anything I can edit in my code that help with that?

Code:
import os
import pandas as pd
import numpy as np

import openai
from openai import OpenAI
api_key = os.getenv(‘OPENAI_API_KEY’)
openai.api_key = api_key

client = OpenAI()

simple function to count the number of words

def count_words(string):
words = string.split()
return len(words)

Using gpt-4-0613

def get_summary(text):
try:
# Making a request to the chat API endpoint
response = openai.chat.completions.create(
model=“gpt-4-0613”,
messages=[
{“role”: “system”, “content”: “You are a helpful assistant.”},
{“role”: “user”, “content”: f"Summarize the following text in under 50 words: {text}"}
]
)

    summary = response.choices[0].message.content
    return summary.strip()
except Exception as e:
    return str(e)

When I run this code, the output is 104 words, even though the instructions were to create a summary in under 50 words.

count_words(
get_summary(large_string)
)
104

Any thoughts?

My input text (large_string in the code above) is the first few paragraphs from the book Wizard of Oz. About 5000 words.

2

Hi!

Here is one idea:

user:

Could you try to write a summary of the wizard of oz in 50 words or less?

assistant:

Kansas girl Dorothy is whisked away by a tornado to the magical land of Oz. With her dog Toto and friends Scarecrow, Tin Man, and Cowardly Lion, she follows the Yellow Brick Road to seek the Wizard’s help to return home, confronts the Wicked Witch, and learns the power of self-reliance.

if < 50, great!

else:

user:

Unfortunately, that’s 51 words. Can we optimize it?

assistant:

Kansas girl Dorothy, with Toto, meets Scarecrow, Tin Man, Lion in magical Oz. They seek the Wizard, confront a witch, and discover inner strength on the Yellow Brick Road.

In summary: sometimes things work out in a single go. Sometimes, you need to allow the AI to refine the result.

1 Like
3

Thank you!
Your suggestion looks like it is using the web interface of ChatGPT, correct? I am hoping to integrate the word limitations directly inside my Python code for the API.

4

NP :slight_smile:

I used the api playground with gpt-4-1106-preview i think

in the if not escalation your message structure would look something like this:

messages = [
  {“role”: “system”, “content”: “You are a helpful assistant.”},
  {“role”: “user”, “content”: f"Summarize the following text in under 50 words: {text}"}
  {"role": "assistant",  previous_output}
  {"role": "user", "Unfortunately, that’s " + word_count + "  words. Can we optimize it?"}
]
5

I think what is going on here is that you are triggering a particular part of finely-tuned behavior, where a “summary”, when mentioned by name, results in a particular type of output by the AI model.

The AI, dealing internally with tokenized parts of language, doesn’t have a full perception of words as entities. Planning to end at a particular point, without going back to edit your work, requires preplanning of the presentation of contents, which is another high bar for the AI.

I will provide you with a method for success.

system

You are a helpful AI writing assistant.

user

Document:

{document_text}

-–

AI Task:

From the text above, create a five sentence overview of the topic and contents. Each sentence of the digest shall be about 10 words long, and shall progress the presentation of information extracted from the text logically, so that the final response product can arrive at 50 words total.
No other chat is desired, just the abstract of the document.

AI output (source)

NASA’s James Webb Space Telescope captured detailed images of Uranus. The telescope revealed Uranus’ rings, moons, and dynamic atmospheric features. Infrared views show a seasonal polar cap and various bright storms. Webb’s data will help understand Uranus’ extreme seasons and atmospheric phenomena. These insights aid the study of exoplanets and our solar system.

Word Count: 53

AI output 2 (source)

Chimpanzees in Ivory Coast exhibit unique tool-use filmed by researchers. They create special water-dipping sticks with chewed, brush-like tips. These tools are specifically designed for drinking, a behavior not seen before. An absorption experiment confirmed the efficiency of the longer brush tips. This “drinking culture” aids survival in dry conditions for this endangered sub-species.

Word Count: 54

Modify sentence instruction from 10 words to 9 words if the target is below instead of around 50 words.

6

Thank you @_j and @Diet ! This is very helpful.

1 Like