Providing a whole example vs providing a small part of an example

API
1

Hello,

As stated in the documentation, providing examples for the LLM is highly recommended. However, my desired output is huge, so if I provided a whole example output, I wouldn’t have enough tokens for the actual output. Take a look at this example.

Provide a very detailed 10 step explaniation on what to do with my life to achieve these goals based on the context I provided about myself? Each step has to be at least 4 sentences. Strictly use a JSON array with each element containing a JSON object. An example array looks like this:

"[{
                 "step": 1
                 "content": ابدأ بممارسة تمارين القلب وتمارين القوة في صالة الألعاب الرياضية. ركز على القيام بمزيج من الاثنين للحصول على جسم رشيق ومتناسق. تساعد تمارين القلب مثل الجري وركوب الدراجات والسباحة على حرق السعرات الحرارية وفقدان الوزن بشكل أسرع. ستساعد تمارين القوة مثل القرفصاء والرفعة المميتة والسحب على بناء العضلات الخالية من الدهون.
                 "exercises": ["running", "cycling", "swimming", "squats", "deadlifts", "pull-ups"] // State all the exercises mentioned in English
               },
               {
                "step": 2
                "content": الحصول على قسط كاف من النوم والراحة. اهدف إلى النوم لمدة 8 ساعات ليلاً للتعافي من التمارين. الراحة مهمة لاستعادة العضلات ونموها. وأخيرا، البقاء دوافع ومتسقة. من المهم تقييم تقدمك وإجراء التغييرات عند الضرورة للبقاء على المسار الصحيح.
                "exercises": [] // State all the exercises mentioned in English
               },
               // Do 7 more steps
              ]

Ignore the Arabic text; it’s irrelevant lol. Now, I would assume it makes a huge difference whether I manually write the 10 steps or whether I say, Write 7 more steps. I care more about speed, and I am wondering what’s better overall. I do care about the number of tokens I use, but I care more about speed.

  1. Manually write all 10 steps.
  2. Manually write 2 steps, and tell the LLM to do the rest

Thanks in advance!

2

Hi,

It is almost always better to split your large tasks up into sections, I understand your requirement is speed, and you could try the GPT-3.5-16K model to give you extra size, but performance always degrades after 3 or 4 tasks per prompt, and is best when a single task is performed per prompt, find some way to keep your users informed about the status of the command and give yourself more time to produce a high quality result.

3

Well, when I say

Provide a very detailed 10 step explanation

Each step has to be at least 4 sentences

Strictly, use JSON

I would assume these are considered three tasks which I inferred as the recommended maximum? Also, I will give the gpt-3.5-16K model a shot. Thanks, btw for helping me! I’ll update my code appropriately.

4

Ok, well lets use reductio ad absurdum to explore your proposed prompt, lets say that instead of the AI, I asked you to do the task and instead of 10 steps, I asked for 100,000 steps. Would you still class that as 3 tasks? asking for 10 steps is going to take essentially all of the AI’s attention, anything it does after that will be less capable, and because the request for 10 things in also surrounded by other requests, that itself will be degraded.

Think of the AI’s attention like a time limit, if it only has time to do 10 things well, then asking for more will reduce the quality of all.