LLM Agents and Tool Use: From Chatbot to Autonomous System

From Prediction to Action

A base LLM predicts the next token given a context window. That is all it does. It has no persistent state, no ability to observe external information beyond what was put in the context, and no mechanism to affect anything outside the inference process itself. An agent built on an LLM must add all of these capabilities through architectural scaffolding around the model.

The minimal agent loop has four components: a context window that holds the agent's current state and history, a set of tools the model can call, an execution layer that routes tool calls to the appropriate functions and returns results, and a termination condition that determines when the agent has completed its task. The model runs in a loop: it reads the current context, decides either to call a tool or to produce a final response, and the loop continues until it terminates.

Function Calling and Structured Output

Early tool use relied on prompt engineering: the model was instructed to produce tool calls in a specific text format (e.g., JSON enclosed in special tags), and a parser extracted the function name and arguments from the raw text output. This is fragile because the model might not follow the format precisely or might produce malformed JSON.

Modern APIs expose native function calling: the developer defines a set of tools with their names, parameter schemas, and descriptions, and the model is trained to produce structured function call objects rather than free text when it decides to use a tool. The model's output is either a text response or a function call, with validated parameter types. This structured output is more reliable than prompt-based tool use and allows the model to learn appropriate tool selection patterns during fine-tuning.

Toolformer (Schick et al., arXiv:2302.04761) demonstrated that models could be taught to decide when and how to use tools through a self-supervised approach: the model generates candidate tool calls, executes them, and learns from which calls improved the loss on downstream tokens. This training approach allows tool use to be learned from existing text corpora without manually labeled examples of when tools should be used.

ReAct and the Interleaved Reasoning Pattern

ReAct (Yao et al., arXiv:2210.03629) established the standard prompting pattern for LLM agents. The model is instructed to produce reasoning traces that explain its thinking and actions in alternating steps. A thought step describes what the model is reasoning about and why it is choosing a particular action. An action step specifies the tool to call and its arguments. An observation step records the tool's output. This cycle repeats until the model produces a final answer.

The reasoning trace serves two functions. First, it improves task performance by encouraging the model to decompose complex tasks before acting, reducing the rate of incorrect tool calls caused by shallow pattern matching. Second, it produces an interpretable record of the agent's decision process, which is important for debugging failures and for the human oversight of agentic systems in high-stakes applications.

The reasoning trace is not proof that the model understood the task. It is a record of what the model said it was thinking. These are not the same thing.

Code Execution as a Tool

Code execution is qualitatively different from other tools because it allows the model to compute precise answers to quantitative questions rather than relying on pattern-matched approximations. A model asked to calculate compound interest or to sort a list by multiple criteria can write and execute code rather than attempting the computation through token prediction, which is unreliable for complex arithmetic.

Program-Aided Language Models (PAL, Gao et al., arXiv:2211.10435) demonstrated that asking models to solve mathematical and symbolic problems by generating Python code, then executing that code to get the answer, substantially outperforms chain-of-thought prompting on tasks with verifiable numerical answers. The key insight is that code execution offloads precise computation to an interpreter, allowing the model to focus on the parts of the task where language understanding and planning add value.

Memory in Agentic Systems

Within a single task, the context window serves as working memory. But most enterprise agentic applications require memory that persists across sessions: knowledge about a user's preferences, results from previous tool calls, learned patterns from past interactions. This external memory can take several forms.

Episodic memory stores records of past interactions and their outcomes, retrieved by semantic similarity at query time. A customer service agent can retrieve records of previous conversations with the same customer and use them to provide continuity across sessions.

Semantic memory stores structured knowledge, typically in a vector database or knowledge graph, that the agent can query when it needs factual information not in its context window.

Procedural memory stores patterns about how to accomplish specific types of tasks, which can be retrieved and incorporated into the agent's current reasoning to avoid repeating problem-solving from scratch.

The interaction between external memory retrieval and context window management creates engineering challenges. A retrieved memory may be relevant enough to be worth including but long enough to crowd out other important context. Deciding what to include, what to summarize, and what to omit requires either careful heuristics or a secondary model that handles context management.

Error Compounding and Recovery

The most significant practical failure mode of agentic systems is error compounding across multi-step tasks. In a single-turn question-answering task, the model's errors are contained: a wrong answer is a wrong answer. In an agentic task with ten sequential tool calls, an error in step three can make all subsequent steps not just wrong but confidently wrong, because the model is reasoning about a state of the world that does not exist.

Recovery from mid-task errors without human intervention is difficult because the model cannot generally distinguish between a correct context and a context built on an earlier error. Once the context window reflects an incorrect state of the world, the model acts consistently with that incorrect state, producing further errors that reinforce the original mistake.

Mitigation strategies include checkpoint-based approaches (requiring human verification at defined decision points), self-consistency checking (running the same task multiple times and flagging discrepancies), and explicit uncertainty tracking (maintaining a representation of what the agent is and is not confident about). None of these fully eliminates the problem, and for high-stakes enterprise workflows, they are complementary rather than alternatives.

Action Blast Radius as a Design Constraint

Every tool an agent can call has an associated blast radius: the scope of irreversible consequences that a single call can cause. A tool that reads a file has a blast radius of zero. A tool that sends an email on behalf of a user, deletes a database record, or executes a financial transaction has a blast radius that may affect people and systems outside the agent's immediate task.

Designing agentic systems for enterprise use requires an explicit inventory of tool blast radii and a policy for human approval at the blast radius boundary. Low-blast-radius operations (reads, queries, drafts) can be automated without human review. High-blast-radius operations (sends, deletes, payments) should require explicit human approval before execution, regardless of how confident the model appears in its reasoning trace.