Sense, Model, Think, Predict, Imagine, Act: The Cognitive Cycle Behind Every Agent

EveryEvery system this module studies — a thermostat, a chess program, a warehouse robot, a large language model calling tools on its own behalf — shares one property: it sits inside an environment, takes in information about that environment, and acts back on it. That loop is the single idea underneath the word "agent." This page sets out a model of what can happen inside that loop, from the simplest possible version to the richest one currently understood, and gives every later page in this module a shared vocabulary and a shared diagram to build from.

The loop has a well-known ancestor

TheThe best-known version of this loop outside computer science is Colonel John Boyd's OODA loop — Observe, Orient, Decide, Act — developed for fighter-pilot decision-making and later applied far beyond it¹. Boyd's own diagrams make a point worth carrying into this module directly: OODA is not a clean circle. Every stage feeds back into Orient, which Boyd treated as the loop's centre of gravity rather than one stage among four — and, critically, his diagrams include a direct line from Orient straight to Act, bypassing Decide entirely. That shortcut represents trained expertise: someone who has done something enough times recognises a pattern and responds without consciously deliberating over it. Keep that shortcut in mind — this page's model has a direct equivalent, and it matters for the same reason.

Six stages, not four

ThisThis module works with a more granular breakdown of the same loop: Sense, Model, Think, Predict, Imagine, Act. Each of the first five names a distinct kind of work; none of them is optional in principle, though — as later pages in this module will show — plenty of real, well-engineered systems deliberately implement only some of them.

• Sense is Boyd's Observe: raw information entering the system from the environment.
• Model and Think are what Boyd compresses into Orient and Decide — this module treats them as separable because they answer different questions. Model asks what is the situation, right now. Think asks what should be done about it.
• Predict is usually left implicit in accounts of this loop, and it deserves to be explicit: before acting, a system capable of it runs its model forward to ask what a candidate action would actually lead to.
• Imagine is not a synonym for Predict, and the distinction is worth being precise about. Predict runs the model forward once, for the likely case. Imagine runs it forward for cases that haven't happened at all — deliberately counterfactual, adversarial, or simply never-yet-encountered scenarios, built by recombining fragments of what the system already knows rather than extrapolating the present forward.
• Act is Act.

This also lines up with the taxonomy of agent types most AI courses already teach²: a reflex agent only needs Sense and Act. A model-based agent adds Model. A goal-based agent adds Think. A utility-based agent adds Predict, since evaluating which future is best requires generating more than one candidate to compare. This module's cycle isn't a rival to that taxonomy — it's the same stages laid out as one pipeline instead of four separate agent categories, which makes it easier to see exactly which piece a given real system has and hasn't implemented.

The full model

NothingNothing in this diagram is a straight pipeline, deliberately. Real cognitive architectures — and the real agentic systems this module asks you to design — look like this: a small number of components doing distinct jobs, densely cross-connected, not a single line from sensor to actuator.

Perception is where Sense happens. Two things can shape what gets perceived before Think ever runs: Short Term memory holds the current, specific situation as it's being taken in, and Abstraction feeds a schema-driven expectation back into Perception — prior general knowledge shaping what's noticed and how it's initially read, not just what's done with it afterward.

Short Term and Abstraction are not two separate stages joined by a gate. The connection between them is itself the mechanism worth naming: complementary learning systems theory holds that a fast system encodes specific individual experiences while a slow system gradually extracts statistical regularities across many such experiences, and generalisation is the output of that slow, repeated consolidation process, not a step that happens before it³. Classification, in other words, is not a separate node in this model. It's what the Short Term → Abstraction edge is.

Episodic Memory is a genuinely separate store from Abstraction, holding specific past experiences rather than generalised ones — the classic distinction between remembering a particular dinner and knowing generally what dinners involve⁴. It feeds two different downstream jobs. Into Imagination, because imagining a novel scenario draws on recombined fragments of real episodic memory rather than being generated from nothing^{5 6}. And directly into Reason/Plan, because episodic memory supplies causal precedent — a sense of what tends to follow from what — that a purely classificatory abstraction doesn't provide on its own, and which a genuine "what if" evaluation depends on.

Procedural Memory is the mechanism behind Boyd's Orient-to-Act shortcut, made explicit rather than left as an unexplained bypass. It is not a bare wire from Perception to Action — something has to determine which action a given perception calls for, and that something is a fast, pattern-matched lookup, built up over time by exactly the same kind of repetition that builds Abstraction, but resulting in a compiled skill rather than a declarative category^{9 10}. This is also why the edge feeding it from Short Term is drawn differently from the others: it represents a slow background formation process, not a live, moment-to-moment data flow.

Representation is where Model actually happens: the system's working picture of the current situation, built from Short Term and Abstraction, and tightly coupled to Imagination rather than sitting downstream of it — a live situational model and an imagined variant of it inform each other continuously, not in one fixed order.

Reason/Plan is where Think and Predict happen, drawing on Representation, Episodic Memory, and Imagination together, and connecting bidirectionally to Action — planning is revised by what happens when a plan meets the world, not just handed downstream once.

Why Imagine is worth its own name

TheThe distinction drawn above between Predict and Imagine isn't a stylistic preference — it's the finding behind a specific line of research, and it's worth knowing the line has a genuinely unusual origin. Hassabis and Maguire's neuroscience work proposed that imagining a fictitious scene and recalling a real episodic memory share the same underlying construction process⁵, a claim developed further as the constructive episodic simulation hypothesis: imagining the future works by flexibly recombining fragments of past experience into a scenario that never actually happened⁶. A decade later, Hassabis co-authored a paper updating memory-systems theory specifically for the needs of artificial agents⁷, and the following year co-authored Imagination-Augmented Agents — a reinforcement-learning architecture that operationalises exactly this mechanism, training an agent to generate and interpret imagined rollouts rather than relying on a single forward prediction⁸. That's not an analogy borrowed for teaching purposes. It's the same person's own research, followed from the neuroscience of human imagination directly into the architecture of an artificial agent capable of it.

A note on where this connects beyond this module

Hassabis'sHassabis's "scene construction" — the claim that a scene, whether perceived, remembered, or imagined, is actively assembled rather than passively retrieved — is making the same underlying claim as Goffman's frame analysis, just at a different level of description: what counts as "what's going on here" is constructed, not given, whether the construction happens in a hippocampus or in a social interpretation. Readers who go on to this site's material on frame analysis for software will meet the same idea again, argued from sociology rather than neuroscience.

Not every agent needs the whole model

ThisThis is the full picture, and it's deliberately more than most real systems implement. A reflex agent uses only Perception, Procedural Memory, and Action — nothing else in this diagram is live for it. That's frequently the correct engineering choice, not a lesser one: a system restricted to a fast, compiled, procedural response is more deterministic, more testable, and lower-latency than one that reasons or imagines its way to an answer, and this site's material on trustworthy software makes the case for exactly those properties in their own right. Later pages in this module walk through specific agent types against this same diagram, showing precisely which parts of it each one uses, which parts it deliberately leaves out, and why leaving them out is often a defensible design decision rather than a missing feature.

References