Welcome to the discussion thread for the “Foundational must read GPT/LLM papers” topic! This is your space to dissect, debate, and delve deeper into the papers mentioned in the main thread. Perhaps you’re grappling with some complex concepts in a paper, or you’ve stumbled upon an intriguing idea that you’d like to explore further. Maybe you have real-world experiences that confirm or contradict the theories proposed. Whatever your thoughts or questions, this is the place to share and learn together. As we navigate this rapidly-evolving field, remember, the goal isn’t just to keep up with the literature, but to enhance our collective understanding and further the discourse. All insights are welcome. Let’s engage, challenge, and inspire each other to push the boundaries of our knowledge on GPT and Large Language Models.
It’s an interesting paper, but it looks like an attempt to lay claim to a fairly simple idea. Interleaving requests to authoritative material and reasoning about them improves the accuracy of responses.
Did it really require a paper? I’m also deeply skeptical about results beyond - this seems to work better in some situations.
We have a discord where I’ve provided some extensive examples of this, but one technique I’d like to see is a ‘paper’ or at least a blog about is just having GPT4 provide alternative responses and then pick the best one. The advantage of this approach is that it can be applied to any q/a and evaluation would be straightforward.
It’s a fairly simple idea and hopefully a paper has been written on it. GPT4 wasn’t much help, so it’s probably been written since cutoff.
I feel like a lot of LLM papers suffer from this issue, but what may be obvious for us as developer’s who spent a long time playing with LLMs might not be well described in the litterature.
I could argue as a physicist and say that all of Einstein’s special theory of relativity is just common sense applied to the Michelson–Morley experiment. But this knowledge wouldn’t have been widely available to someone without a background in Physics.
There’s many, many requirements to writing a good scientific paper and only one of them is having an idea, the rest is a lot of hard work describing, testing and providing results. My biggest requirement is always “can the reader replicate the results?”
I liked the ReAct paper because they provided enough information and explanation for me to replicate their results, they have a GitHub if you want to play with it yourself:
If you want I can also post my own implementation using GPT-3.5 (the original paper used DaVinci)
I think the ReAct paper is a good place to start for people who want to know more about “agent’s” without throwing their API key at something like AutoGPT
Fair enough, and don’t get me wrong, it’s a great idea. Many great ideas are often quite simple in retrospect.
I suppose my biggest issue is I just don’t believe the paper.
Well, directionally I do, but it really feels like one of those papers ‘great, it seems to work, now let’s make up some math so people will think we understand why it works and will cite it and we can say it was our idea.’
I guess I’ve just read too many of those papers and have become overly cynical.
It’s fine to be sceptical even cynical when trying to “review” a paper, that’s what this thread is for
From my experience the methods shown in the react paper does improve the responses I get, it does really well at chained tasks like “find x information and do y calculation considering c” but it has a high failure rate on question that are loosely defined.
Here’s an example of what I’m talking about: I gave the agent the task “write about your own abilities”
In the first attempt it responded “I am an AI who can read Wikipedia and do calculations”
(This is true, I gave it access)
In the second attempt it found the wiki on psychic abilities, filled it’s context window with nonsense and responded “I have psychic abilities…”
