Hi everyone, i am using the RefineDocumentsChain from lnagchain, now i want to use an FewSotPrompt to provide an additional example.
I dont know how to implement it - any help?
Hey there!
What exactly do you mean? Have you done “few shot” prompting before? Or are you asking for more about the langchain side of this?
1 Like
Hey there,
Well I am testing Prompt Priciples - one of which being "Few Shot "
My goal is to apply the different priciples in order to optimize the result.
I have a Document which i want to summarize. The documents token number eceeds max token of 4096 for gpt-3.5. So i need to chunk it, and in order to have the whole information from each chunk , I pass it through the RefineDocumentChain- this chain is apparently the right one for summaries.
→ this works fine
Now I have seen the FeShotPromptTemplate of Langchain and am trying to integrate it, to also pass an example to it. My understanding is, it feeds a prompt, with an example of the query and result. In my case the query is the document and the result the summary.
Now i want to look my prompt like this:
summarize this “document_to_be_summarized (>4096 token)”
format the output accoring to this “example”
Can i use a FewShotTemplatePrompt and pass it through my chain?
1 Like
So I’ll be the first one to admit I have very mixed feelings about langchain, but I understand many people like using it.
I’m guessing you looked at this already?
See, it’s a bit difficult for me to express Few Shot Prompting easily, because tbqh it’s one of the first “techniques” I learned on my own and just kind of did subconsciously. A name wasn’t given to it until later.
Few shot prompting is essentially just this:
Q: What time is it?
A: It is 5pm
Q: I live in France
A: It is 22:00
From what the refine docs say, it seems it’s an initial Q/A snippet that’s fed into a chat completion instance with extra context to change the initial “A” of the answer if it is needed.
“Few Shot Prompting” in general is just clarification or correcting the model when it’s initial answer is considered insufficient for whatever reason. It’s a more…“natural” method of conversation, at least to me.
Say you ask a model about a doc. You ask:
What is the plot summary of Mary Sue in this doc?
And it responds:
Sure! Mary Sue saved the world from dark evil forces and everyone lived happily ever after!
Now, if the answer is sufficient, you have a single shot prompt.
If you wanted it tweaked a little, you could just ask again (in the same conversation thread or chain):
Okay, that's cute, but I kinda wanted the summary in l33t speak
And it responds:
Sure! M4ry Su3 s4v3d th3 w0rld fr0m d4rk 3v1l f0rc3s 4nd 3v3ry0n3 l1v3d h4pp1ly 3v3r 4ft3r!
And now it has become a few-shot prompt.
This is also why I have my gripes with LangChain, because I notice it tends to overcomplicate things a little bit, and it ends up confusing people over concepts that are not that confusing; using LangChain is what ends up being confusing.
All of this to say, if you want an example of a Few Shot prompt for querying your doc, all you need is an instance where it provides an answer, you as the querant say “uhm, actually” and then it retrieves the correct answer after you provide the clarification or refinement. This kind of prompting is also far more common with code.
Does this help?
I understand what few shot promot is in this case - yet i dont have a question to be answered , but the task of summarizing a content. the problem being the content is larger than what gpt can process in one go. which means the prompt will be too large to process in one go. is there a way to work around that ?
My apologies.
When I say Query/Question, that is specifically referring to you as the user of a language model providing input to the LLM. Every single input is considered a request or query regardless. You are asking the model either for something, or to do something, in this case a summary.
There we go! That’s a huge help! Thank you for providing that information 
.
This doesn’t sound like something a few shot prompting mechanism might work well for. If it were me, I would parse it into meaningful chunks (pages, chapters, etc), and summarize the chunks. Once each chunk is summarized, I would loop through and then summarize and combine all those initial summaries to get a finalized result.
If you are trying to stuff a prompt with few shot examples on top of the actual prompt and are running out of room you should consider fine-tuning a model instead:
Fine-tuning improves on few-shot learning by training on many more examples than can fit in the prompt, letting you achieve better results on a wide number of tasks. Once a model has been fine-tuned, you won’t need to provide as many examples in the prompt. This saves costs and enables lower-latency requests.
With the fine-tuned model you’ll have more room and have better success trying to chunk it up.
If you still don’t have enough you can try asking a model to distill the information. GPT-4 has a context length of ~128k though. If you are reaching that level of information a sizeable amount may be ignored/skipped over.
1 Like