How to reliably identify the content extracted rather than generated by openAI

API
,
1

Hey,

I know this is a double-up to a post in the feature request, but this might be best suited in the area of the forum. Hence the repost.

So I’m attempting to do the following:

Input some text and request chat GPT do to 2 things:

  1. Extract specific information from the input
  2. Suggest content for the information that was not extractable

Is there a way to do this reliably in one call?
As I imagine that this would quickly be done in one call. Requesting information extract, flagging on our side what has been completed, the requesting for generation of empty fields.

Once again, this, unfortunately, requires an additional request, which is more time and more $$$.

Example:

	resp, err := c.openaiClient.CreateChatCompletion(
		c.ctx,
		openai.ChatCompletionRequest{
			Model:    "gpt-3.5-turbo-16k",
			Messages: [{ Role: 'user', Content: 'My name is Alex, I am 42 years old'}],
			Functions: []openai.FunctionDefinition{{
			Name:        "extract_info",
			Description: "Extract info from text",
			Parameters:  json.RawMessage(`{
				"properties": {
					"name": {
						"type": "string",
						"description": "Character name"
					},
					"personality": {
						"type": "string",
						"description": "Character personality"
					},
					"generated_content": {
						"type": "string",
						"description": "AI Generated content",
						"enum": ["name", "personality"]
					},
				},
				"type": "object",
				},
			)
		}`
	)

I would expect the generated_content to list the keys that have been AI-generated.

While we do it manually and this is touch and go it feels like this could be a great addition to the Functions API.

Unless there is another reliable way of achieving this?

1 Like
2

It is usually preferable to have the AI look at data in passes, rather than trying to do everything at once, in this case you could extract the data and check it in conventional code for missing fields and then generate a reply asking for the missing information, that would require no API calls and be programmatically simple. You could then use the AI to parse the answer to fill in the missing fields and again check for completeness. Repeat this until all fields have been filled in or the user selects another service or exits.

3

Thanks, that is the direction we would take if there was not API way to achieve this.

Thanks for the response!

We do have a rather complex request and breaking it down clearly seems to be the way to go

4

Not sure what you mean by no API? If you are trying to SDK text into the chatgpt site that will end up causing issues, could you clarify?

5

Something similar to Functions or any way to try and fulfill the feature either via prompt or the API. We messed with prompts a lot and getting an aiRecommended field that has an array of strings to list all suggested vs extracted info was very shaky.

6

Yea, I think sometimes it’s easy to forget that “functions” are really another bit of text in the prompt to the AI, yes it has been fine tuned to more reliably respond with a json formatted string and there is some extra signalling logic to trigger a new finished_reason flag, but it’s still the LLM completing a prompt.

The real power is in the consistency of the response and the fact you can force that response in any given situation which removes the doubt that a return string will contain a usefully formatted reply.

I usually recommend people to go watch a series of 1hour free courses by Andy Ng and some of the OpenAI staffers over on Short Courses | Learn Generative AI from DeepLearning.AI they are centred around building commercial bot solutions for business, they are firmly aimed at beginners who have a basic grasp of python and prompting in general, but they show some excellent best practice method for constructing and splitting multi part prompting.

If you have any further questions after watching those, please reach out.

2 Likes
7

Even i want to know more about it .
I tried to make chat gpt remember a conversation but it forgets very fast.I think the model can remember chat for a shortlived period.
I gave a huge text from a research paper to chatgpt and wanted to extract information like Numericals , data, terminology, and etc. from that input text. I asked chatgpt to remember it. chatgpt gave information from the text upto 3 prompts but then it forgot that it had to relay information from the input text and instead gave me information from its training model database.
If their is a way to make it remember certain command then it would be very helpful