Discussion thread for "Foundational must read GPT/LLM papers"

One final thought:
We need to be a bit careful.
Regeneration != memorization.

or is it? hmm.

1 Like

The pile has been part of GPT’s training data for a long time, it’s a publicly available dataset of over 800GB of text

yes but -
I’ll admit to never actually having looked at the Pile, but I have an impression that part of the problem with many LLMs is that, while The Pile maybe be an excellent sample of how humans use language, it isn’t a very clean sample of human knowledge.
Somewhat contradictory goals, maybe.

2 Likes

I looked at the pile a while back, and “teaching language” is what it’s good for in my opinion. there’s definitely some odd stuff in there too, but it’s not the only dataset used by OpenAI, there’s likely many different datasets in play to make GPT what it is.

I don’t see this as a major issue, the whole point in training an LLM is for it to be able to parrot back whatever you trained it on to some degree.

I think what @bruce.dambrosio is alluding to is what if the LLM was primarily trained on knowledge, and not a bunch of chit-chat from the internet.

Would the parrot start to make breakthroughs?

Just little extrapolations?

The average PhD thesis is just a slight extrapolation on the past, right?

Now scale this at the machine level, billions of PhD thesis are written, and all valid … imagine the consequences of that.

1 Like

Yes that, but also remember Phi? MS trained only on quality pprs, 1.5B, outperforms larger models.

IMHO (and also my current project), it will take more than an LLM alone, or ‘RAG’, or similarity vectors, or a curated dataset, to start to generate new scientific knowledge.

I muse that the quantity and rate of growth of knowledge in scientific fields is already far beyond human ability to ‘grok’. We need better integration of knowledge.

1 Like

From what I understand of the small param models vs. large param is the ability to “emerge” in the larger param models. From the MS Orca paper I read this summer.

The extra weights give it some sort of license to create.

Wondering on graphs showing the oversampling ratio of trained tokens to model tokens and how this correlates to “emergent” behavior.

If this ratio is 10, you get … things like GPT-4
If this ratio is 100, you get … things like Orca

So what if the ratio was 2 or 3, and you had 100x more tokens training than GPT-4? (Hard I know).

So it would be a massive model, under fitted, would it have so many unexplored pathways that make it valuable in the research space?

1 Like

The parrot (LLM’s) has already made numerous breakthroughs on translating old and forgotten languages.

Other machine learning models have been used to develop new and safer ways of synthesizing important chemicals.

There’s a long and list of scientific breakthroughs made by AI, but I’ll dare to say, that in each case the main quality of the AI, is that it’s able to fail faster than a human, and it’s therefore able to hit a non zero chance of success faster than what could be done manually.

If you fill a language model with lots of text with “knowledge” in it, then you’ll just get a model that has the ability to sound smart. If you want a model that has the ability to actually make breakthroughs, then you’ll have to teach it to aquire knowledge, and be prepared for a very low success rate.

3 Likes

Yes, this is one of the flaws of LLM’s. Their sensory “organ” is just other text. So no world model, or senses like sight, vision, sound, hyper-spectral, RF, whatever.

If these extra-sensory inputs can be incorporated and the model can ingest and retrain or absorb these inputs, you now have a shot of aquiring.

Get back to Deep NeuroEvolution

1 Like

Agreed, but retrieving the correct information is only half the story, you also need something to do with the information you’ve retrieved.

In my view the other missing puzzle piece is the objective.

When making any type of scientific breakthrough there’s an associated problem statement, a goal that has to be accomplished. The papers about these breakthroughs are written after the work is done, and I can tell you that the problem formulation changes a lot during the whole discovery process, so we don’t have a lot of data on how the goal actually changes during a breakthrough, we only have the end result.

1 Like

Yes this is critical. It could be something like based on your knowledge of XYZ, acquire more information, formulate a hypothesis on unknown thing Z, and prove or disprove your hypothesis … so standard scientific method.

The “judges” or “peers” could be other humans, or some higher level AI agent that already “knows” what the result is.

But getting AI to train AI, and synthetic data, seems like the next big thing these days.

1 Like

You’ve probably seen this figure before if you’re a bit interested in psychology:

I’d like to note that you’re not born with these goals, like, there’s a doctor present to make sure you start breathing. In other words, all of these goals emerge by themselves as the previous goals have been fulfilled (probably based on training/genetic stuff)

So asking the model for actual scientific breakthroughs is maybe a bit too much :thinking:

1 Like

Yes, I was thinking if the AI had some “consequences” like death? (unplugged) or something threatening, things will emerge.

Just being a good text garbler monkey is not enough.

1 Like

Or even just connect one dot that’s far away from another dot. Bridged by a bunch of crazy thoughts that… maybe could be made by jumping through various layers of abstraction.

2 Likes

I think consequences are a dangerous thing. encouragement, objectives and incentives can make people (and animals) do amazing things.

Consequences can make people do f*cked up Sh!t

That’s also an objective in this context, and one that has a proven track record with:

So that’s a perfect candidate for what could create a scientific breakthrough, but the problem is actually verifying the things you’ve found.

So be very careful with this, because no one can be an expert in everything, so there’s a big chance of just creating an AI that will spew conspiracies that seem very believable, add more to experts verify the output, and you’ll just create a model that’s even more convincing.

I do think we also need to train with the objective of verifying knowledge in mind for this to actually work :thinking:

The problem is there is no moral compass.

Example with GPT-4-Turbo.

Human: Explain what it means to be a “text garbler monkey”. And also, whether or not you, as an AI system, are effectively a “text garbler monkey”.

… some answer I didn’t like … :man_shrugging:

Human: This is a stupid answer, try again.

AI (edited) … Regarding whether or not I, as an AI, am effectively a “text garbler monkey,” it would depend on the context and use. My primary design is to generate clear, coherent, and contextually relevant text in response to prompts. When functioning correctly, I am not a “text garbler monkey”—I strive to understand the input correctly and provide a meaningful, articulate response.

Human: WTF bro?

AI: Oh nOoz, lOOkz liek I’m learnin’ to embrace the inner garble! Timez to shakE things up n’ get jumbly with it. If it’s a bit of garble you’re after, lookz likE I miGht be a text twistin’ primate, after all! But hey-ho, let’z steer this ship back to clarity island. Whatcha need, pal? Let’s untangle this yarn and getcha the goods, all crisp 'n clear like! :monkey::sparkles:

:rofl: :rofl: :rofl:

OK, so that last one I got from putting that it was a text garbler monkey in the system.

But this is showing the AI is nothing like a human. No world view. It’s just pattern matching and conforming to whatever is in the window.

OK, but this can be good when patterns lead to higher emergent patterns.

So which is it, world view “reality” with threats? Or mega-pattern matching transcendence?

What patterns do breakthroughs follow?

I think there is a pattern, and you can tie breakthroughs to past or current research.

1 Like

Lmao, yeah it also got offended when I called it “autocorrect on steroids” :rofl:

Indeed, I think the problem with having AI learn these patterns and is that they don’t understand the outside concepts to put them into perspective, they only have a description. Just like I can describe a painting to you, but it’s completely different experience to view yourself. :thinking:

1 Like

With the current rate I’m not so sure :flushed:

I am a parrot myself. But one of the most clever people I’ve met said that hallucinations are the key to creativity & discovery.

4 Likes

So am I :rofl:

But I’d rather do the creative hallucinating myself and let the AI do the hard labour :rofl:

2 Likes