While I am deeply skeptical such a thing is possible without a breakthrough, it’s reasonable to think a breakthrough isn’t totally out of the cards given how many people are now working and investing in this.
I’m curious what the world will look like post-ASI. Will it be like the culture series as Elon says hopefully?
Given the demo in the openai about a computer teaching a kid math, I went looking for some research on how society might really be impacted if these things come true. The lack of thoughtful research on this topic is quite surprising.
This is a good paper by deepmind, but only orthogonally addresses it Foundational must read GPT/LLM papers - #100 by qrdl
I thought this comment was very insightful from the quora post on this paper:
EDIT: A lot of people asked why do I believe it would take much shorter time to move from Level 2 to Level 6 than from Level 1 to 2. This is because of the exponential nature of learning. Think of the famous problem where a person asks the king to give him a gift of rice on a chessboard where each square would have a double of previous square. The early squares had very less and by the time you got beyond the 20th square it is hard to fit even in a large palace.
The human at the 90th percentile is not substantially different from the human at the 50th percentile. We still have 2 eyes & 2 ears to learn new info and similar brain size. On the other hand, once the machine reaches the 50th percentile, it can dramatically grow its learning & processing — by not just adding more processors & memory, but also spawning 1000s of virtual agents that would teach each other and get smart in a recursive way.
There’s an interesting recent paper on singularity and llm agents learning in a recursive way:
https://chatgpt.com/share/99a45ac0-c5d4-498d-b5f8-615a3776871e
My sense is that the comment is correct, but it may be we hit something that isn’t really “competent agi” and more just a reasonable facsimile (eg, mixture of experts MoE). I think there is another level of AGI which isn’t merely competent, in fact it doesn’t even have to be competent, but it does need to be capable to learn its way to the singularity. Maybe we will get there before level 2, maybe not.
It seems to me more thought needs to start going into this, something with milestones and tracking that we are properly evolving and adapting as a society as it occurs.
One idea is rather than separating things into levels of AGI, it should be more about capabilities. For example, voice cloning and generative video as a capability will be quite impactful and won’t necessarily require that much intelligence - just raw computing power. Given the demand for entertainment, I could see us hitting this before ASI itself.
Each area of society will have to adapt to new capabilities as they are achieved. Having a plan for each capability and measuring metrics to show that they’ve been surpassed will probably make us more prepared.
As mentioned above, the most critical milestone will be the system that can learn in an unbounded manner. It’s baseline doesn’t have to be that bright - it just needs to be shown to be scaling with time and compute.