Do new gpt4-turbo models have more trouble with examples?

[Diet]

@erik6

Any others experiencing this, or do I just need to work on my “example engineering” craft?

Personally, I’ve never gotten multi-shot working satisfactorily.

We’re discussing multishot here, if you wanna have a look at it:

Fine Tune Chat-Model based on plain text

text-davinci-003 was instruct-trained gpt-3 gpt-3.5-turbo-instruct is its instructGPT replacement davinci was a GPT-3 base model that is a predictive completions engine davinci-002 is the next version (likely based on 3.5 scaling) In the following forum link, an example of what it takes to get the newest base model to form cohesive responses if you don’t train by fine-tune: train by multishot input, placing the AI into a state as if it was still completing a “document”.

Yet Another openai.Completion.create() Question

from openai import OpenAI client = OpenAI() prompt = """" Here is the backend API communication between a user and a talented AI expert assistant named Anita, based on GPT-3 by OpenAI. `assistant` answers are in json format, with key=assistant, and any linefeed escaped to /n. {"user": "How does quantum computing differ from traditional computing?"} {"assistant": "Quantum computing uses quantum bits or 'qubits' which can represent and store information in a way that allows for more complex compu…

---

@erik6

Let’s say that I want json output: I’ve had better results by providing an output template, than to give 3 output examples. When I use the examples, it will stick too closely to the values I had used there.

for json output, you can do stuff like “start with {” and “you are part of a larger system, and must provide json output - any other response will cause the system to crash.”

it’s generally capable of following simple schemas (I like using TS type definitions) fairly reliably.

Adding examples tends to add a lot of unnecessary noise to your signal with minimal utility, as you’ve discovered. Working on your description on what it should do, and cutting as much noise as possible is generally a good idea IMO.

so tldr: this individual (me) believes using examples aren’t a good idea in most cases.