Been there… lead to countless chats about how to flipp energy around, building fusion reactors on the moon,… antimatter,… even dyson spheres… pretty interesting.
But keep in mind:
For any system, including AGI (Artificial General Intelligence), the rate of growth and advancement is constrained by the available energy. This is because energy is required for computation, data processing, and physical operations.
So there is a limit - while compound interest has none.
Except when we agree on a new value. Would you maybe just give me all you own just because I exist and would everyone else do the same?
This way my sole existence would create unlimited value, decrease inflation (I promise I won’t spend all), and would also make me pretty happy.
I believe AGI can only exist if it’s driven by an inovative computing architecture, which would be more energy-efficient. I can’t quite imagine an AGI driven by Von Neumann architecture
I don’t think it’s the right approach to simplify the problem and say that it’s all just about software and hardware. Because, we could continue in that direction and say “it’s all about understanding electrons,” but that won’t help us reach AGI (or whatever the goal may be).
Instead, we should take a different approach and first try to develop a conceptual idea of such a thing, and then further elaborate on it to the point where we can talk about implementing specific software and hardware.
And that’s generally the problem with understanding LLMs: people think that achieving cognitive functions (like planning) in a machine is just a matter of scaling up LLMs.
Both viewpoints here are interesting and highlight different aspects of how we might approach the development of AGI. While improving hardware can undoubtedly speed up computational efficiency, I agree with the idea that simply scaling software and hardware won’t necessarily lead to AGI. The conceptual framework and understanding of cognition play a crucial role.
We should first focus on developing a more robust theoretical foundation for AGI, beyond just hardware improvements. This could mean studying human cognition or other biological systems to inform the design of more complex AI models, which could, in turn, lead to more targeted software and hardware development.
Scaling LLMs (large language models) alone might improve their capabilities, but without a clear understanding of how to structure AI to perform higher-level cognitive tasks like planning and reasoning, we could hit limits.
What are your thoughts on balancing hardware innovation with conceptual advancements in AI theory? I’m curious how others see this interplay between theory and technical infrastructure.
I agree with Yann LeCun’s assertion that LLMs are only an off-ramp on the highway to ultimate intelligence. I believe we need to return to understanding the realization of intelligence in biological entities (artificial neurons and artificial neural networks, apart from their initial motivation, actually have little in common with biological ones and are their rough approximation). Along with intelligence, we should better understand what a cognitive model is and how a mental model arises from it, and how intelligence and cognition are intertwined.
Once we grasp the theoretical side, we will know which direction to take in implementing ultimate intelligence. Everything else is a method of trial and error in a space with an enormous number of options (yes, perhaps someone might succeed with this approach).
Yann has been instrumental in bringing AI to where it is today, but… he’s been wrong about a number of things recently, and I think this is another one.
The LLM may not be the final form, but it’s for sure going to be a huge central part of it.
Interesting read: ARC Prize Testing And Notes On Openai’S New o1 Model
“Does all this mean AGI is here if we just scale test-time compute? Not quite… To beat ARC-AGI this way, you’d need to generate over 100 million solution programs per task. Practicality alone rules out O(x^n) search for scaled up AI systems.”
