There is an interesting discussion we can have here about the nature of lying to an AI to get results. One can even make a bold statement:
Prompt engineering is all lies!
How can I say such a thing? Am I the only one here so callous as to deceive the poor AI model?
Lets start with some background of how these language completion AI models work.
Completions
Base models like GPT-2 and GPT-3 were originally released with just pretraining. They were trained on a large corpus of unlabeled data, where reinforcement learning trains them on the patterns in language, and what emerges is an even deeper understanding of the semantics and purpose behind that language.
That’s fine if you want to have the AI predict how a sentence would end. You could even have it complete a paragraph or an article if you provide the first half of the introduction to that article. That is because a completion model predicts how generative language should look when it generates word-by-word, or rather token-by-token.
You are constructing a fictional start of a document or dialog to realize the rest of the AI completion that you want.
Prompt engineering, contextual completion:
Published January 8, 2024 - Nature
A new approach to prompting language models
Forward
Today, we discuss new findings about the behavior of LLMs when provided different input context to complete language on. This has enormous implications, because
…and then AI writes the rest.
Chatting with completions
This is where the lying gets turned up a notch. We have to construct a hypothetical situation that never happened - a conversation. If we provide something that looks like it is an ongoing conversation between two individuals, the “completion” can be AI generating what is predicted for either one of those parts of the conversation.
Here is an informative conversation with an expert in the field of machine learning.
[user] Hello! I’m trying to wrap my head around the different types of machine learning. I understand there’s supervised, unsupervised, and reinforcement learning, but could you break down the key distinctions for me?
[expert] Certainly! It’s a great place to start. The core difference lies in the type of data and the goal you’re trying to achieve. Supervised learning uses labeled data – meaning you provide both the input and the desired output. For example, showing the algorithm pictures of cats and dogs with the labels “cat” and “dog” respectively. The goal is to learn a mapping from input to output.
[user] So, if I have a dataset with housing prices and all the features that contribute to those prices, that would be supervised learning?
[expert] Exactly! That’s a classic example of regression, a type of supervised learning. You’re using the features (like square footage, number of bedrooms, location) as input and the house price as the desired output. The algorithm learns the relationship between them so it can predict prices for new houses.
[user] Okay, that makes sense. What about unsupervised learning then? If supervised has labels, does unsupervised have no labels at all?
[expert] Precisely. Unsupervised learning works with unlabeled data. Instead of trying to predict something specific, it aims to find patterns, structures, or relationships within the data itself. Think of it like trying to discover inherent groupings or anomalies without any prior knowledge of what they are.
[user] So, like grouping customers based on their purchasing behavior without knowing what “types” of customers exist beforehand?
[expert]
This is a multi-shot completion prompt. It shows example conversation that starts to develop a pattern of responses. You can see that where it ends, the AI shall start to “complete” language. The AI acts as the expert!
Another form of lying to get the output you desire: an answer to a final question.
This is the kind of prompt you would use on OpenAI base models like davinci-002 that are currently in operation, although it is best for more specialized tasks needing completion specifically.
Instruction-tuned completion
Now we enter the era of post-training. This is where OpenAI adds more supervised and labeled training data to the AI, using PPO, similar to fine-tuning. The initial development of this was called InstructGPT, and then instruction-following models of GPT-3 were released on the API.
We no longer have to lie to the AI - it recognizes when a question seems to be complete, and then the completion can become an answer.
What is the sum of 3 + 4 + 7? \n\n
The AI will produce an output for us. It also is trained to STOP the output now, by emitting a special internal stop sequence after its own response.
It still can complete though, if it seems that it is within a user input, so we can still produce hypothetical inputs and see what happens.
My question is about LLMs. What is the most
What’s going to happen here? The AI won’t say “it looks like your question was incomplete”. Instead, it will complete the question the best-looking way, and then continue to answer it.
Neat output - from lying.
This can be seen currently with gpt-3.5-turbo-instruct on the completions API endpoint.
Chat AI
This is kind of the ultimate lying. The AI developers are lying to you that there is some thinking entity that can act as an independent being. This has become the universal way to interact with AI language models.
Not only is there very extensive training on chat-like behaviors being done, but also, each turn is within a special container signaling the start and end of a turn.
Even more, there are “roles” - special names for each input or output that dictate the authority of each.
Here’s what the AI would receive from the API request you send:
<|im_start|>system<|im_sep|>You are a helpful assistant<|im_end|><|im_start|>user<|im_sep|>What’s the biggest pumpkin ever?<|im_end|><|im_start|>assistant<|im_sep|>
The inputs are strictly containerized, and follow the format of extensive post-training.
The actual “prompt” is the final “assistant” role name that is inserted. It acts like the completion chat shown earlier. It prompts the AI to complete what an assistant would say in that circumstance.
So you can see that while you can lie to the AI about what it is and what it does as an API developer to get the desired output, can lie to the AI as a user about the circumstances where you need an output generated - the biggest lie is that there some magical being inside the completion AI that has been created. That we’d have to worry about hurting our AI pal’s feelings! 
