AGI Will Not Be a Model
Frontier models may become essential components of artificial general intelligence. That does not make the models themselves generally intelligent.
Strap yourself in. This is going to be a long one.
Every few months, a new model is released which appears to move the artificial general intelligence debate forward. The most recent example is OpenAI’s GPT-6 Astra. When this happens, the inevitable plays out: the model passes another professional exam, solves a difficult mathematical problem, generates a working application from a prompt, discovers something novel in science or convincingly simulates a person.
The immediate question is always the same: Is this AGI?
I think the question is being asked at the wrong level.
These achievements tell us something important about model capability, but they do not necessarily tell us whether the system possesses general intelligence. A model can produce an extraordinary result without being able to learn from the experience, incorporate what happened into a persistent understanding of the world or become better at navigating the next unfamiliar situation because of it.
Frontier models are not AGI. They may become extremely powerful components of future AGI systems, but capability alone is nowhere near an adequate threshold. The real gate, I think, is architectural.
The definitions already contain the problem
There is no settled definition of AGI. OpenAI’s long-standing definition describes it as “highly autonomous systems that outperform humans at most economically valuable work”, while Google DeepMind researchers have proposed measuring progress through the breadth and depth of a system’s capabilities, with autonomy considered separately. These definitions are useful, but they are intentionally focused on what a system can do rather than how it works.
There is a sensible reason for that. Intelligence need not reproduce human biology or follow a particular engineering blueprint, and if a machine can reliably perform a broad range of difficult tasks it would be strange to dismiss it simply because its internal machinery looks unfamiliar.
The problem is that a purely capability-based definition risks treating general intelligence as a sufficiently large collection of outputs. If a system performs well enough across enough domains, eventually it qualifies, while questions about how it acquired those abilities, whether it can form genuinely new ones and whether its knowledge continues to develop can become secondary.
I think that is a much bigger omission than it initially appears. Learning and adaptation are not simply another two capabilities to add to a benchmark alongside maths and coding. They are part of what makes intelligence general in the first place.
A capable model is not the same thing as an intelligent system
Today’s frontier models are trained across enormous bodies of human-produced information, which gives them remarkable breadth. They can write, analyse, translate, code, classify, summarise and reason across domains that once required completely separate specialised systems.
That breadth can look general because the boundary around the training process is mostly invisible to the person using the model. You ask a question and receive a useful answer. You correct it and the model responds appropriately. It may remember your correction for the remainder of the conversation and behave differently from that point onwards. From the outside, this looks quite a lot like learning.
But in most cases the underlying model has not changed. The correction remains inside the temporary context being supplied to it and, once that context disappears, the apparent learning largely disappears with it.
The model may contain an extraordinary amount of knowledge, but it does not independently maintain a coherent body of knowledge that it continuously updates through experience. It may demonstrate a skill without preserving the process through which it acquired that skill, or reason its way through a contradiction in one conversation without changing how it handles the same type of contradiction later.
I think that distinction matters.
A reference library is not automatically memory simply because a model can search it, any more than a task loop becomes agency because a model can repeatedly call tools. A large context window is not continual learning, and a convincing statement of uncertainty is not necessarily evidence that the system has a reliable internal estimate of its own uncertainty.
All of these things can make a model substantially more useful, and they are probably going to be parts of whatever comes next, but they can also create the appearance of an integrated intelligence without actually producing one.
The ability to perform is different from the ability to become
Most AGI arguments focus on performance: what can the system do now?
The harder question, however, is developmental: what can the system become through its own continuing experience?
A generally intelligent system should be able to encounter a new pattern, recognise that its existing concepts are inadequate and form a better one. It should learn new knowledge and skills incrementally rather than requiring another industrial training run, preserve what it learns, connect it to what it already knows and revise both when the evidence changes.
It should be able to learn through instruction, demonstration, observation, exploration, success and failure, and sometimes from a single unusually informative event. It also has to operate with incomplete and contradictory information rather than requiring a clean benchmark with a correct answer sitting somewhere behind it.
Over time, the system should become better at navigating the world because it has experienced more of it. A system that repeatedly encounters the world as though it has just been switched on may be astonishingly broadly capable, but it is not really developing.
That, for me, is a much more useful distinction between producing an answer and maintaining something resembling a mind, and I am using “mind” here in the functional rather than mystical sense.
The gate is architectural
If AGI arrives, I doubt it is going to be a single model sitting behind a chat interface. It is much more likely to emerge from an architecture in which a frontier model, or possibly several different models, forms part of a larger system involving memory, learning, perception, planning, attention, action, feedback and some ability to monitor and alter its own behaviour.
Listing those ingredients is the easy part. The hard problem is getting them to operate as one coherent system.
Memory has to influence reasoning when it is relevant without flooding every decision with everything the system has ever encountered. New learning needs to update old knowledge without destroying useful capabilities. Goals need to shape attention. Actions need to create experience, experience needs to alter the world model, and that changed model then needs to influence what happens next. Somewhere in all of this the system also has to distinguish an important exception from random noise and a genuine contradiction from something that merely looks contradictory because the context has changed.
None of that is solved by attaching an ever-growing folder of text to a prompt.
Which raises the obvious question: if model performance is not enough, what would actually count?
Our working checklist for AGI
There probably will not be one magical benchmark where a machine crosses a clean line and everyone agrees that AGI has arrived. I do think, though, that there are a set of capabilities that would need to exist together before we at Cerebral Circuit would be comfortable using the term.
This is our working checklist. It will almost certainly change as the technology develops, but it describes the threshold we currently think matters.
1. It must be able to recognise and understand new things
A generally intelligent system needs to do more than match an input against patterns it already knows. It should be able to recognise familiar entities from partial, noisy or ambiguous information, work out which categories something belongs to and predict what is likely to happen next, while also recognising when something does not fit what it already knows.
That final part is important because intelligence cannot be general if every unfamiliar thing simply gets forced into the nearest existing category. At some point the system needs the functional equivalent of saying: I do not think the concepts I currently have are good enough to explain this.
2. It must be able to form new concepts
Recognising novelty is not particularly useful if the system cannot do anything with it. An AGI should be capable of forming new categories, abstractions and relationships from experience rather than relying entirely on concepts embedded during training.
That might mean learning a principle from a small number of examples, identifying a higher-order pattern, revising the boundaries of an existing category or carrying an idea learned in one domain into a materially different situation where the superficial details no longer match.
This is one of the places where benchmark performance can be deceptive because frontier models have already been trained on an enormous amount of abstraction performed by humans. They can manipulate concepts with extraordinary sophistication, but the more interesting question for AGI is whether they can develop useful new abstractions after deployment when their existing conceptual machinery turns out not to be enough.
3. It must actually learn
This is probably the biggest one for us.
A generally intelligent system must be able to acquire durable knowledge and skills during ordinary operation. Not simply hold something inside a context window, retrieve a note somebody stored in a database or wait until another mountain of data has been collected for the next training run. Experience itself has to be capable of changing the system’s future behaviour.
That learning should happen through instruction, demonstration, observation, experimentation, success, failure and sometimes a single sufficiently informative event, depending on what is being learned.
It also has to work in both directions. The system needs to acquire new knowledge, but it must also be able to discover that something it previously believed was incomplete or wrong and change accordingly. A mind that can only accumulate information but cannot properly revise itself eventually becomes a very large collection of mistakes.
4. It must have integrated memory
Memory is not simply storage. A generally intelligent system probably needs several kinds of it: some equivalent of working memory for what it is dealing with now, longer-term factual knowledge, memories of particular experiences, learned procedures and some way of remembering intentions or commitments that need to be acted on later.
The important word here is integrated.
A memory only becomes cognitively useful if it can influence what the system notices, reasons about, predicts and does when that memory becomes relevant. Dumping every previous conversation into a vector database and retrieving vaguely similar passages later may be useful engineering, but calling the database “long-term memory” does not magically make the surrounding system generally intelligent.
There is also the opposite problem. A system that remembers everything indiscriminately is likely to become unusable fairly quickly. It needs some way of deciding what deserves to be retained, what can be compressed, which details have become obsolete and what can safely be forgotten.
5. It must maintain and revise a world model
General intelligence requires some usable representation of how the world works: entities, relationships, causes, constraints, other agents and change through time.
That system should be able to draw conclusions from incomplete evidence, separate an observation from an inference, recognise contradictions, reason about causes rather than association alone and ask what might happen if some part of the situation were different.
Most importantly, that world model has to be revisable through experience. If I touch a hot stove and burn myself, what just happened changes my expectations about hot stoves. An intelligent machine needs some functional equivalent of that process, including when what it is learning is considerably more abstract than “hot things hurt”.
6. It must be able to pursue goals through time
General intelligence cannot exist entirely in isolated question-and-answer turns. A system should be able to understand a broad objective, break it into useful subgoals, identify dependencies and constraints, plan a course of action, observe what actually happens and change the plan when reality inevitably refuses to cooperate.
This is part of why I think agent methodology is pointing in the right direction, even though the word “agent” is currently doing an absurd amount of work.
Much of what the industry calls an AI agent today is still a frontier model placed inside a loop. The system receives a goal, generates a step, calls a tool, reads the result and generates another step. Add retrieval, a scratchpad and a database of previous interactions and the whole thing begins to look considerably more persistent.
That is useful engineering and it can automate meaningful work, but much of the coherence is still supplied from outside. Humans define the goal, choose which tools are available, decide what gets remembered, determine when the work is finished and repair the process when it starts wandering off into the weeds.
For AGI, more of that continuity has to belong to the system itself.
7. It must decide what matters
I suspect salience is one of the least appreciated problems in the whole AGI discussion.
No intelligent system can attend to everything, which means something has to decide which inputs deserve attention, which features of those inputs matter, which memories are relevant, which contradiction is worth investigating, which goal takes priority and which experience contains something important enough to learn from.
The same problem exists at several levels at once. A sudden movement in a visual field may need immediate attention, while a small change in someone’s wording might completely alter the meaning of a request. A failed action might be irrelevant bad luck or evidence that the system’s entire model of the problem is wrong. A tempting short-term action may also need to be ignored because it conflicts with a much more important goal further down the line.
Humans currently provide a remarkable amount of this cognitive scaffolding for AI systems. We decide what goes into the prompt, which documents are available, what tools exist, what the objective is and what information gets stored. Engineers define the retention rules, the context limits and often the conditions under which the system should stop.
A genuinely general system would need much more control over its own attention, learning and memory. Not unlimited control, obviously, because that creates an entirely different class of problems, but enough that it can determine what is cognitively important rather than merely processing whatever a human has chosen to place in front of it.
8. It must notice when its own thinking is failing
An AGI does not need human consciousness, an inner monologue or some philosophical sense of self in order to qualify as generally intelligent.
It does, however, need some functional metacognition. It should be able to recognise uncertainty, identify gaps in its knowledge, notice when its current reasoning strategy is failing and try another one. It should have some ability to distinguish between things it knows with strong evidence, things it remembers imperfectly and things it is effectively guessing.
Current models can tell you that they are uncertain, sometimes very convincingly. That is not necessarily the same thing as having a reliable mechanism for knowing when they should be uncertain, which is the part that actually matters.
9. It must interact with an environment and learn from the consequences
We are not convinced that AGI necessarily needs a robot body, but we are increasingly convinced that it needs some kind of ongoing environment in which its actions have consequences.
That environment could be physical, digital or some mixture of both. What matters is that the system can take an action, observe the result and learn from the difference between what it expected to happen and what actually happened.
Without that feedback loop, intelligence remains strangely detached from consequence. A model can contain millions of descriptions of people making mistakes and still never have its own understanding of the world corrected because it made one.
10. All of this has to work together
This is the bit we think matters most, because you could probably build a benchmark for every item above and eventually produce a machine that passed them independently without producing anything we would want to call AGI.
Intelligence is not a shopping list of modules.
Learning has to alter memory, and memory has to alter reasoning. Reasoning has to shape planning, while goals influence attention and attention determines what gets learned in the first place. Actions generate experience, that experience changes the system’s understanding of the world and the changed understanding then affects what the system notices and does next.
All of that has to remain reasonably coherent as the system develops over time.
That integration is the architectural problem, and it is also why I think so much of the current argument about whether GPT-whatever, Gemini-whatever or the next frontier model has “reached AGI” is premature. We are still mostly measuring the capability of one extraordinarily powerful component and treating improvements in that component as though they automatically imply equivalent progress in the larger cognitive system.
They do not.
So what would fail the test?
There are some fairly straightforward disqualifiers that fall out of this view.
A system does not qualify simply because it can outperform people across a long list of economically valuable tasks if it cannot acquire durable new knowledge and skills through ordinary interaction. If its apparent learning vanishes when the context window closes, if it cannot revise what it knows when better evidence appears, or if it cannot adapt to a genuinely unfamiliar environment without a human rebuilding the process around it, then something fairly fundamental is still missing.
I also do not think the threshold should depend on whether the machine convincingly simulates a human, claims to want something or manages to persuade observers that there is somebody “inside” it. Those are interesting questions and may become extremely important when we start talking about consciousness, moral status or safety, but they are not reliable measures of general intelligence.
The test I care about is whether the capabilities form part of a coherent and developing whole.
If somebody showed us a system tomorrow and claimed it was AGI, we would want to know whether it could encounter something genuinely unfamiliar and develop a useful new concept for it, learn something during ordinary operation and still know it later, allow that learning to change what it notices and how it behaves, revise an existing belief when reality contradicts it and transfer what it has learned into a materially different situation.
We would want to know whether it can maintain goals and plans across time without somebody constantly reconstructing the context around it, decide what information actually matters, notice when its own reasoning has become unreliable and change strategy, then act in an environment and learn from the difference between what it expected to happen and what actually happened.
Most importantly, we would want to know whether all of those things are happening inside one coherent system that continues developing through its own experience.
If the answer to most of that is no, I do not particularly care how many exams it can pass. The model may be astonishing, but it is not yet generally intelligent.
Why this distinction matters before AGI exists
This is not only an argument about some hypothetical future machine. Organisations using AI today are already discovering, on a much smaller scale, that model capability on its own does not determine whether an AI system actually works.
The best-performing model does not automatically produce the best working system because reliability depends on how that model is connected to context, memory, tools, data, permissions, feedback and human judgement. The questions are basically miniature versions of the architectural problems above: what does this system know, where did that knowledge come from, what should it remember, what can it act on, how does it know whether the action worked, what happens when the evidence changes and who notices when it is confidently pursuing the wrong goal?
These are often treated as secondary implementation details that get added once the clever model has done the interesting part. In practice, they are frequently the things that determine whether the clever model survives contact with an actual organisation.
Frontier models are going to keep improving, probably dramatically, and they will continue producing results that look increasingly like evidence that AGI has arrived. Some of those achievements will be genuinely extraordinary and some may force us to revise parts of the checklist above.
But I think we need to stop confusing the expansion of model capability with the emergence of general intelligence.
We are measuring intelligence at the component level when the thing we claim to be measuring only really makes sense at the level of the complete system.
AGI will not simply be a model that can do enough things. It will be a system capable of continually developing what it knows, what it can do and how it decides what matters, then carrying that development forward into whatever it encounters next.
Until then, the architecture remains the gate.