Creating Next-Gen LLMs: Sakana.ai's Evolutionary Approach

A Japanese startup called “sakana.ai” has developed a technology that enables the automatic merging of multiple LLMs to generate a new one.

It appears that major companies such as NTT Group, KDDI, and Sony Group are backing the startup.

The concept revolves around combining “grandchild” models, which are derived from exceptional “child” models.

Although model merging has been likened to alchemy, it could potentially be a powerful method for creating new LLMs from existing open-source ones.

What are your thoughts on this?

1 Like