This paper blew me away: Can LLMs Be Computers? | Percepta
The entire agentic ecosystem today, from Claude Code to Codex to OpenClaw, is based on the principle of surrounding LLMs - which are simply token distribution generators - with external tools (Python, git, curl, APIs, MCPs), and a state machine. An LLM cannot reliably “execute” a mathematical expression - it can talk/reason about it; it can (if seen in the training data) “guess” the result; it can produce Python code to run (externally) in a Python sandbox, and use the result in its answer.
What the folks in this paper did, was turn the LLM into a virtual machine (VM), and effectively use the transformer weights as a fast WebAssembly interpreter. The result: you give an LLM a program, it ACTUALLY executes the program by emitting a stack trace (internally), and it outputs the result!
They took a vanilla PyTorch Transformer architecture (couple of dozen lines of code), and embedded WebAssembly in model weights by training the transformer to predict state transitions of a WASM virtual machine, so the attention and MLP layers collectively implement the interpreter’s logic. Programs are fed as tokens, and the network simulates execution step-by-step.
If it holds true, this could be an absolute game changer. Imagine you somehow integrate this into a GPT model, i.e. you effectively embed a code interpreter “inside” the LLM weights. Then instead of using costly/inefficient thinking/reasoning tokens, you are instead using an internal logarithmic-complexity state emission to literally compute or “run” something, before finally emitting tokens.
Anyway, it got me very excited, and I’m wondering if OpenAI peeps are looking into this?
