Console Log Security or Receiving Other People's Responses?

API
1

I’m a little confused, and sorry if this has been asked a ton, I couldn’t find it while searching. Is this like a security feature? Are we not able to console log the actual responses for security reasons? Or am I receiving other people’s API responses? I’m getting random arrays of messages about like covid and stuff, it looks like someone is training/pre-training a model to summarize.

Here’s what logged for the object before sending it to gpt-3.5-turbo:

{ role: 'user', content: 'Hello!' }

Here’s what logged from the OpenAI response.

[
  {
    role: 'system',
    content: 'Original Text: "The COVID-19 pandemic has had a devastating impact on the global economy, leading to widespread business closures and job losses. Governments around the world have implemented various measures to mitigate the economic damage, including stimulus packages and financial assistance programs. However, the road to recovery remains uncertain as the virus continues to spread and new variants emerge. Businesses are adapting to new ways of operating, with a focus on remote work and digital transformation. The pandemic has also highlighted the importance of healthcare and the need for investment in medical research and infrastructure. As vaccinations roll out worldwide, there is hope for a gradual recovery, but the long-term effects of the pandemic on the economy are yet to be fully understood."\n' +
      '\n' +
      'Improved Summary: The COVID-19 pandemic has severely impacted the global economy, resulting in business closures and job losses. Governments worldwide have implemented measures to support economic recovery, but uncertainties remain as the virus spreads. Businesses are adapting through remote work and digital transformation. The pandemic highlights the importance of healthcare and investment in medical research. Vaccinations offer hope for gradual recovery, but the long-term economic effects are still unclear.'
  },
  { role: 'user', content: 'Hello!' }
]

Here’s what got sent to the frontend (Our service is called willow):

willowToServer: {"title":"New Chat","chatId":null,"messages":[{"role":"user","content":"Hello!"}],"chatType":"willow","userId":"6518c9fd1d7014190fc8b0ad"}
{
  title: 'New Chat',
  chatId: null,
  messages: [ { role: 'user', content: 'Hello!' } ],
  chatType: 'willow',
  userId: '6518c9fd1d7014190fc8b0ad'
}
Message: Hello!
{
  title: 'New Chat',
  userId: new ObjectId("6518c9fd1d7014190fc8b0ad"),
  messages: [
    {
      role: 'user',
      content: 'Hello!',
      name: new ObjectId("6518c9fd1d7014190fc8b0ad"),
      embedding: []
    }
  ],
  chatType: 'willow',
  currentContext: ''
}
{
  title: 'New Chat',
  userId: new ObjectId("6518c9fd1d7014190fc8b0ad"),
  messages: [
    {
      role: 'user',
      content: 'Hello!',
      function_call: null,
      name: new ObjectId("6518c9fd1d7014190fc8b0ad"),
      embedding: [],
      _id: new ObjectId("6518ec72720122b8e293fc73")
    }
  ],
  chatType: 'willow',
  currentContext: '',
  _id: new ObjectId("6518ec72720122b8e293fc72")
}

Then I got some about the Industrial Revolution lol:

encodedText   0 undefined
0
encodedText Hello 9906 1 9906
1
encodedText Original Text: "The industrial revolution, which occurred in the late 18th and early 19th centuries, was a period of significant advancements in manufacturing, agriculture, and technology. It led to the transformation of society and the shift from an agrarian economy to an industrial one. The revolution began in Britain and then spread to Europe and North America. It was characterized by the mechanization of production processes, the development of new machines and technologies, and the growth of factories. This resulted in increased productivity, urbanization, and a rise in population. The industrial revolution also had wide-ranging social and economic impacts. It created a new class system with industrial capitalists, workers, and a growing middle class. It led to the improvement of transportation and communication systems, as well as the emergence of new industries and the expansion of trade. Overall, the industrial revolution had a profound and lasting effect on society and laid the foundation for modern industrialization."

Improved Summary: The industrial revolution occurred in the late 18th and early 19th centuries, bringing significant advancements in manufacturing, agriculture, and technology. It led to a transformation from an agrarian to an industrial economy, beginning in Britain and spreading to Europe and North America. This revolution involved mechanization, new technologies, and the growth of factories, resulting in increased productivity, urbanization, and population growth. It also created a new class system, improved transportation and communication, and promoted the emergence of new industries and expanded trade. The industrial revolution had a lasting impact on society and paved the way for modern industrialization. 18902,2991,25,330,791,13076,14110,11,902,10222,304,279,3389,220,972,339,323,4216,220,777,339,24552,11,574,264,4261,315,5199,83787,304,15266,11,30029,11,323,5557,13,1102,6197,311,279,18475,315,8396,323,279,6541,505,459,40574,8997,8752,311,459,13076,832,13,578,14110,6137,304,13527,323,1243,9041,311,4606,323,4892,5270,13,1102,574,32971,555,279,7852,2065,315,5788,11618,11,279,4500,315,502,12933,323,14645,11,323,279,6650,315,35159,13,1115,19543,304,7319,26206,11,16036,2065,11,323,264,10205,304,7187,13,578,13076,14110,1101,1047,7029,82105,3674,323,7100,25949,13,1102,3549,264,502,538,1887,449,13076,87408,11,7487,11,323,264,7982,6278,538,13,1102,6197,311,279,16048,315,18386,323,10758,6067,11,439,1664,439,279,49179,315,502,19647,323,279,14800,315,6696,13,28993,11,279,13076,14110,1047,264,28254,323,29869,2515,389,8396,323,17551,279,16665,369,6617,13076,2065,2266,82210,22241,25,578,13076,14110,10222,304,279,3389,220,972,339,323,4216,220,777,339,24552,11,12967,5199,83787,304,15266,11,30029,11,323,5557,13,1102,6197,311,264,18475,505,459,40574,8997,311,459,13076,8752,11,7314,304,13527,323,31135,311,4606,323,4892,5270,13,1115,14110,6532,7852,2065,11,502,14645,11,323,279,6650,315,35159,11,13239,304,7319,26206,11,16036,2065,11,323,7187,6650,13,1102,1101,3549,264,502,538,1887,11,13241,18386,323,10758,11,323,30026,279,49179,315,502,19647,323,17626,6696,13,578,13076,14110,1047,264,29869,5536,389,8396,323,63675,279,1648,369,6617,13076,2065,13 313 18902
313
[
  {
    role: 'system',
    content: 'Original Text: "The industrial revolution, which occurred in the late 18th and early 19th centuries, was a period of significant advancements in manufacturing, agriculture, and technology. It led to the transformation of society and the shift from an agrarian economy to an industrial one. The revolution began in Britain and then spread to Europe and North America. It was characterized by the mechanization of production processes, the development of new machines and technologies, and the growth of factories. This resulted in increased productivity, urbanization, and a rise in population. The industrial revolution also had wide-ranging social and economic impacts. It created a new class system with industrial capitalists, workers, and a growing middle class. It led to the improvement of transportation and communication systems, as well as the emergence of new industries and the expansion of trade. Overall, the industrial revolution had a profound and lasting effect on society and laid the foundation for modern industrialization."\n' +
      '\n' +
      'Improved Summary: The industrial revolution occurred in the late 18th and early 19th centuries, bringing significant advancements in manufacturing, agriculture, and technology. It led to a transformation from an agrarian to an industrial economy, beginning in Britain and spreading to Europe and North America. This revolution involved mechanization, new technologies, and the growth of factories, resulting in increased productivity, urbanization, and population growth. It also created a new class system, improved transportation and communication, and promoted the emergence of new industries and expanded trade. The industrial revolution had a lasting impact on society and paved the way for modern industrialization.'
  },
  { role: 'user', content: 'Hello' }
]

A little strange. It’s happening on like every new chat.

2

You are discussing another service that is not OpenAI.

Users of the OpenAI API send exactly what they want, and get back only the response generated by the AI.

I’ve seen some evidence in the past that ChatGPT (the web chatbot) has been injected with some pre-training questions in conversation history, like “what is the capital of Paris France” (or it picks those out of fine-tune). In general, multi-shot examples before a conversation starts are a way of showing the AI how to respond, and also a way to prevent the AI from being immediately re-trained on user input.

Non-user messaging should not be transmitted to the user by the service, unless they’ve moved the responsibility of chat management onto client software.

3

I’m not though, I’m using gpt-3.5-turbo… I’m using openai “4.10.0”.

import { secretsManager } from '../security/secrets_manager'
import OpenAI from 'openai'
import { AzureKeyCredential, OpenAIClient } from '@azure/openai'
import { getModelMax } from './utils'
import { lengthCheckpoint } from './summarize'
import type { ChatCompletionMessageParam } from 'openai/resources/chat'
import type mongoose from 'mongoose'

const openai = new OpenAI({ apiKey: secretsManager.openai })

export const convertMessagesToString = async (messages) => {
	let text = ''
	for (const message of messages) {
		text += message.role + ' : \n' + message.content + '\n'
	}
	return text
}

export const chatOpenAI = async (
	model: string,
	messages: ChatCompletionMessageParam[],
	userId: mongoose.Types.ObjectId | null = null
) => {
	const modelMax = await getModelMax(model)

	// TODO: Implement Function Calling and Handle Accordingly
	const runningMessages = await lengthCheckpoint(messages, modelMax)
	console.log(runningMessages)
	openai.apiKey = secretsManager.openai
	const userIdString = userId?.toString() ?? 'null'
	let completion
	if (userIdString === 'null') {
		completion = await openai.chat.completions.create({
			messages: runningMessages as ChatCompletionMessageParam[],
			model: 'gpt-3.5-turbo',
			stream: false
		})
	} else {
		completion = await openai.chat.completions.create({
			messages: runningMessages as ChatCompletionMessageParam[],
			model: 'gpt-3.5-turbo',
			stream: false,
			user: userIdString
		})
	}
	const contextString = await convertMessagesToString(runningMessages)
	return {
		contextString: contextString,
		completion: completion.choices[0].message.content
	}
}
4

I see…sort of.

I might suggest logging the actual API json within the library right before it is sent, along with dumping the full “choices” reply you receive back from the AI.

You say “logged from the OpenAI response”. However, then you show the messages format that would be sent to the API, with a system message that is not programming the chatbot to act as your application, but a system message with the covid text. Then show a “sent to our frontend” that has your internal format with just the hellos. Very confusing presentation of what is being sent to and what is being received from the actual OpenAI API.

I might look close at where you’re getting the data in that ChatCompletionMessageParam from, log it right before you send it, and see if you’re sending an incorrect database retrieval, or you’re putting message containers in strings in message containers.

If it’s not your own chat shown, could it be a particular user stuffing the AI with their own bizarre data attempts?

5

Nah I figured it out. It looks like OpenAI changes our messages to be worded better. Because I ended up asking it about what’s going on via the API and it did this:

[
  {
    role: 'system',
    content: "No, I'm referring to the unrelated message that appears along with the console logs every time I start a new chat. It seems to be a message that is not related to my query or the generated response. Is there a way to prevent this from happening?"
  },
  {
    role: 'user',
    content: "No, like I'm calling OpenAI's API, and I'm receiving random text. It looks like it's other user conversations that you're having with other users."
  }
]

The reason for the other ones were that I pass the messages through a summary prompt each time and it’s recognizing that I’m trying to get a summary via my summary prompt and replaces it with its own better version of it. So yeah like you were saying it’s sending the zero-shot prompt for it.

6

What? OpenAI doesn’t rewrite your wording for you, nor can the AI answer questions about OpenAI services.

In your last reply, you show a system message and a user message – but I don’t know where you would have obtained or how you created that. They are not typical roles.

This is what should be SENT to OpenAI as the “messages” property in your API request:

'{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "system",
        "content": "You are WillowGPT, an AI writing assistant"
      },
      {
        "role": "user",
        "content": "Hello, help me write a paper about chatbots"
      }
    ]
  }'

(and then more user/assistant chat history messages can be inserted before the most recent question)

“system” acts as the programming for a chatbot, telling the AI its identity and purpose, and should always be sent.

Then you get back a reply, from which to parse the response:

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "gpt-3.5-turbo-0613",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Sure, here's a paper that discusses the use and programming...",
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 22,
    "completion_tokens": 155,
    "total_tokens": 177
  }
}
1 Like
7

Hi @josh8

Your code is using both:

Where’s your endpoint deployed? Is it Azure OpenAI service or directly from OpenAI?

1 Like