Self-improving AI is often framed as a model capability: a model improves its reasoning, learns from its own outputs, or acquires new skills. I think the model is only one layer.
The infrastructure around the model—and the way the system is deployed—can also be designed for self-improvement. An agentic system can improve how it allocates computation, evaluates and revises actions, and composes or reconfigures workflows for different tasks.
A self-improving system should learn not only better answers, but better ways to think, act, and organize its own computation.
Thinking control
The first level governs what happens inside inference. Rather than treating decoding as passive sampling, we can allocate search, evaluate partial trajectories, and steer generation toward desired behavior.
GenARM uses an autoregressive reward model to guide generation token by token. Transfer Q★ treats decoding as value-guided control under distribution shift. Collab and FusionRoute extend control across multiple models, deciding how their complementary capabilities should contribute during generation.
The central question is not simply whether more test-time compute improves accuracy. It is how to spend computation where it changes the answer, while keeping the resulting policy coherent and auditable.
Action control
An agent must do more than produce a plausible chain of thought. It must choose actions, observe their consequences, identify failure, and recover. Self-improvement therefore needs an explicit feedback loop between reasoning and behavior.
Agentic Critical Training turns critique and revision into a learning signal for multi-stage agents. Instead of treating self-critique as decorative prose, it makes proposal, inspection, and correction part of a controlled training process.
This view also changes evaluation. We should ask whether an agent can recognize that a plan is failing, localize the failure, and adapt its actions—not merely whether its final response resembles a reference answer.
Workflow control
The third level operates above any single reasoning trace. Complex agents rely on tools, critics, retrievers, simulators, and specialized models. A fixed pipeline cannot be ideal for every task.
FlowBank explores a reusable portfolio of workflow components that can be precomputed, selected, and recomposed. The system improves not only by changing model parameters, but by learning which organization of computation works best for each query.
This suggests a broader research program: agentic infrastructure should behave less like a static software stack and more like an adaptive policy. It should evaluate its own organization, preserve useful workflow fragments, and change how capabilities are invoked in deployment.
Beyond the model layer
More capable base models will certainly matter. But self-improving AI will also require systems that can examine how those capabilities are being used. The important boundary is no longer simply training versus inference. It is between systems that execute a fixed procedure and systems that can evaluate, adapt, and improve the procedure itself.