What Comes After Language Models
I keep seeing scientific discovery framed as a sufficiently ambitious prediction or data-compression problem. A recent position paper by Tom Zahavy challenges that view by separating reasoning into three capabilities. Induction finds general patterns in examples. Deduction works out what follows…
I keep seeing scientific discovery framed as a sufficiently ambitious prediction or data-compression problem. A recent position paper by Tom Zahavy challenges that view by separating reasoning into three capabilities. Induction finds general patterns in examples. Deduction works out what follows from a set of assumptions. Abduction proposes a new explanation when neither the existing rules nor the available data point to one. Today’s AI systems are increasingly capable at the first two. The paper argues that the third, the conceptual “jump,” remains largely missing. General relativity is the central example. Newtonian gravity was working extraordinarily well, so an optimization system minimizing prediction error would have had little reason to replace it. Einstein instead used physical thought experiments, including the imagined experience of a freely falling observer and an accelerating elevator, to arrive at the equivalence of gravity and acceleration. A capable AI might derive the field equations once handed that principle. The harder question is whether it could invent the principle in the first place. The practical takeaway is not that LLMs will never invent. It is that language and formal reasoning may not be sufficient on their own. Better prompting, longer reasoning traces, and more powerful search can help a model explore an existing conceptual space. They do not necessarily help it create a new one. Zahavy’s proposed direction is an interactive world model that acts as a synthetic laboratory, where an agent can manipulate a simulated environment, run counterfactual experiments, and use the results to form new hypotheses. This is still a research hypothesis, not a proven recipe. But for anyone betting that scaling LLMs is the main road to AGI, it raises a useful question: What if the bottleneck is not more reasoning within language, but a better way to ground new concepts in experience? Subscribe to our weekly newsletter The post What Comes After Language Models appeared first on Gradient Flow.Source: Gradient Flow — Published — Category: Models