An AI agent is an LLM that can take actions in the world and observe the results - repeating that loop until it completes a goal. Agents turn a single prompt-response pattern into an autonomous process that can browse the web, write and run code, query databases, and coordinate with other agents. Understanding what happens inside that loop - and where it fails - is the foundation of AI engineering in 2026.
What an AI agent is
The word "agent" is overloaded. The precise definition used by most AI engineers in 2026 is: an LLM placed in a loop with tools and memory, tasked with completing a goal autonomously. Three components are required:
A chatbot is not an agent - it responds once per turn and has no persistent goal. A coding assistant that can write, execute, observe the error, and self-correct is an agent. The key distinction is the autonomous loop that continues until the goal is satisfied.
The agent loop
Every agent implementation - regardless of framework (LangChain, LlamaIndex, AutoGen, custom) - executes a variant of the same loop. The ReAct pattern (Yao et al., 2022) formalised it as alternating Reasoning and Acting steps, which maps cleanly to how modern tool-calling APIs work:
- 1
Receive task
entryThe system prompt defines the agent's persona, available tools (as JSON Schema), and any constraints. The user message or triggering event sets the goal. Everything is placed in the LLM's context window. - 2
Reason and plan
thinkThe LLM generates a chain-of-thought (explicit or implicit). It evaluates the current state against the goal, considers which tool to invoke, and produces a structured tool-call object or a final text response. Strong models do this in a single forward pass. - 3
Execute tool call
actThe agent runtime intercepts the tool call (before it reaches the user), validates arguments against the tool's schema, and invokes the implementation - a Python function, an MCP server, a REST API, or a subprocess. - 4
Observe result
observeThe tool result is appended to the conversation context as a "tool result" message. The LLM can now see what happened: the file contents, the code output, the search results, or the error message. - 5
Loop or terminate
decideThe LLM re-enters the reasoning step with the updated context. If the goal is achieved (or the model decides it cannot proceed), it emits a final text response and the loop exits. Otherwise it goes back to step 2.
Planning strategies
Planning is how an agent decides the sequence of actions to reach a goal. The right strategy depends on task complexity and how predictable the environment is.
| Strategy | How it works | Good for | Failure mode |
|---|---|---|---|
| ReAct (default) | Interleave Thought and Action in each turn; no upfront plan | Short tasks, dynamic environments | Gets stuck in local loops on complex multi-step tasks |
| Plan-and-execute | First generate a full plan; then execute each step | Longer tasks with stable subtasks | Plan becomes stale if early steps produce unexpected results |
| Tree of Thoughts | Branch multiple reasoning paths; evaluate and prune | Tasks requiring search over a decision tree (puzzles, code) | High token cost; slow |
| Self-reflection / Reflexion | After each attempt, critique the result and retry with updated strategy | Code generation, tasks with a verifiable success criterion | Can overcorrect; requires a reliable success signal |
Tool calling
Tool calling is the mechanism by which an LLM requests execution of a specific function with specific arguments. Modern model APIs (Anthropic, OpenAI, Google) expose this as a first-class feature: you pass tool definitions as JSON Schema in the API call, and the model returns a structured tool-call object rather than raw text when it wants to invoke one.
- Definition - each tool has a name, a description (written for the model, not a human), and a parameters schema. The description is a prompt - write it carefully or the model will misuse the tool.
- Parallel tool calls - modern models can emit multiple tool calls in a single response. Run them in parallel where possible; collapsing to sequential execution is the most common avoidable latency sink in agent implementations.
- Error handling - always return a structured error (not an exception) to the model. It can then decide whether to retry, try a different tool, or inform the user. Exceptions that bubble up to the agent runtime typically terminate the loop unexpectedly.
- Side effects and idempotency - clearly separate read-only tools from write tools. Gate destructive operations (send email, delete record) behind an explicit user-confirmation step; never let the model invoke them unsupervised.
The Model Context Protocol (MCP) standardises how tool definitions are discovered and invoked, allowing agent frameworks to connect to any MCP-compliant server without bespoke integration code.
Memory
Memory determines what the agent knows now and what it can recall across sessions. There are four distinct types, each with different storage, retrieval, and update mechanics:
Multi-agent systems
A single-agent architecture hits a ceiling: the context window limits how much state it can hold, and a single LLM making all decisions is a bottleneck. Multi-agent systems distribute work across specialised sub-agents coordinated by an orchestrator.
| Pattern | Structure | Good for |
|---|---|---|
| Orchestrator-worker | A supervisor agent breaks the task into subtasks and delegates to specialised worker agents | Long, decomposable tasks (research + write + review) |
| Parallel fan-out | Multiple agents tackle independent subtasks simultaneously; results are aggregated | Embarrassingly parallel work (scraping N websites, analysing N files) |
| Pipeline | Each agent hands its output to the next stage (extract -> classify -> summarise) | Sequential transformation pipelines with clear stage boundaries |
| Debate / critic | One agent generates; another critiques and flags errors; a third resolves conflicts | High-stakes outputs (code review, legal summary, medical triage) |
Communication between agents uses the same tool-calling mechanism as human-to-agent communication: one agent's tool call dispatches to another agent rather than a function. This keeps the architecture uniform and observable.
Guardrails and evals
An agent that works 95% of the time is not production-ready. Guardrails limit the blast radius when the model makes a mistake; evals measure how often it does.
- Input guardrails - classify user messages before the agent processes them. Reject prompt-injection attempts, off-topic requests, and inputs that violate your terms of service.
- Tool guardrails - validate tool arguments before execution. Sanitise paths, cap API call volumes, and require human approval for irreversible actions.
- Output guardrails - run the agent's final answer through a classifier before returning it to the user. Check for hallucinated citations, PII leakage, and off-policy responses.
- Evals - test the agent against a fixed set of tasks with known correct outputs. Run evals on every model upgrade and every system-prompt change. Tools like LangSmith, Braintrust, and Arize Phoenix make this tractable.
When NOT to use an agent
Agents add latency, cost, and failure modes. Use a simpler pattern when it suffices:
- Single-turn tasks - if the task can be completed in one LLM call (classification, summarisation, extraction), a simple prompt with no loop is faster, cheaper, and more reliable.
- Deterministic workflows - if the sequence of steps is always the same, hard-code it. A pipeline script is more reliable than an agent reinventing the same path on every run.
- High reliability requirements - agents compound errors across steps. If each step is 95% reliable, five steps in sequence have a 77% success rate. For mission-critical operations, constrain the agent tightly or replace it with deterministic code.
- Latency-sensitive paths - a single LLM call takes 1-5 seconds; an agent loop over 10 steps takes 10-50 seconds. For real-time user-facing features, keep the loop shallow or run steps in parallel.
For how tools are standardised across agent hosts, see MCP Explained. For giving agents access to a knowledge base at scale, see RAG Explained. To understand what the agent engineer role looks like day to day, see the AI Agent Engineer guide.
Sources & further reading
- 1ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al., 2022 - arXiv 2210.03629
- 2Tool Use with Claude — Anthropic
- 3OpenAI Function Calling Guide — OpenAI
- 4Model Context Protocol - Introduction — MCP Community