The Enterprise AI Agent Stack: What to Build, What to Buy, and What to Wire Together

Most enterprise AI agent discussions start and end at the model layer. Which LLM provider, which model version, which context window. The actual engineering challenge is five layers deeper, and the decisions you make at those layers determine whether your agent pilot turns into a repeatable enterprise capability or a well-funded science project.

This post is a framework for CIOs, CTOs, and Chief AI Officers who are past the proof-of-concept stage and need to make real architecture decisions. It covers the five layers of an enterprise AI agent stack, a build vs buy vs wire analysis for each, a comparison of the leading open-source frameworks, and a practical 12-week pilot architecture you can hand to your engineering team.

If you have not yet connected your enterprise knowledge to a structured data layer, read the knowledge graph post first. The agent stack described here assumes you have something structured for your agents to reason against. Without it, your agents will hallucinate with great confidence and at great cost.

---

Part One: The Five Layers of an Enterprise AI Agent Stack

An AI agent is a system that perceives its environment, reasons about it, takes actions, and observes the results of those actions. In an enterprise context, that simple loop requires five distinct infrastructure layers. Each layer has its own engineering challenges, its own market of vendors, and its own build vs buy calculus.

Layer 1: The Model Layer

The model layer is the LLM itself: the system that performs reasoning, generates text, and decides what tools to call. For most enterprises, this is an API call to a foundation model provider. The model layer is the most visible part of the agent stack and the one most discussed in board rooms, but it is also the layer where the build vs buy decision is clearest. Very few organizations have the resources or the training data to build a competitive foundation model. The differentiation at the model layer is minimal compared to the layers above it.

What does matter at the model layer: context window size (how much your agent can hold in memory per turn), tool-calling reliability (how consistently the model formats tool call requests), and cost per token (which becomes significant at enterprise query volumes). These are evaluation criteria, not build criteria.

Layer 2: The Tool Layer

Tools are what give agents the ability to act in the world beyond generating text. A tool is any function the agent can invoke: querying a database, calling an API, reading a file, submitting a form, executing code, searching a knowledge graph. The research that formalized this pattern, Schick et al. at Meta AI (Toolformer, arXiv:2302.04761, NeurIPS 2023), demonstrated that language models can learn to use tools from minimal demonstrations. What the paper did not address is the enterprise challenge: your tools are not generic web APIs. They are your CRM, your ERP, your contract management system, your CMDB, your data warehouse, and dozens of other internal systems with authentication requirements, rate limits, schema idiosyncrasies, and data governance obligations.

The tool layer is where enterprises create the most differentiated value from AI agents. The tools you expose determine what your agents can do. No vendor can build these integrations for you. This is the layer you build.

Layer 3: The Orchestration Layer

The orchestration layer manages the agent loop: how the model, the tools, and the context interact over multiple turns. The foundational pattern here is ReAct (Yao et al., arXiv:2210.03629, ICLR 2023): a loop in which the agent reasons about the current state, acts by calling a tool, observes the result, and repeats until the task is complete or a stopping condition is met. This Reason-Act loop, simple as it sounds, is the core of every production agent system.

What makes orchestration complex at enterprise scale: managing context across long multi-step tasks, handling tool failures and retries, coordinating multiple agents working in parallel on subtasks, enforcing approval gates before high-stakes actions, and logging every decision for auditability. The orchestration layer is where most engineering effort concentrates in a mature agent deployment.

Layer 4: The Memory Layer

Memory is the mechanism by which agents retain and retrieve information across turns, sessions, and tasks. There are four distinct memory types in enterprise agent systems:

• In-context memory: The current conversation and tool call history in the LLM's context window. Fast, but bounded by context window size and expensive at scale.
• External retrieval (RAG): Documents and text chunks retrieved from a vector database using semantic similarity. Effective for unstructured knowledge, but imprecise for structured relationships.
• Structured knowledge (knowledge graph): Entity relationships queryable via SPARQL or Cypher. High precision for multi-hop reasoning about connected enterprise data.
• Persistent state: Task state, user preferences, and intermediate results stored in a conventional database and surfaced to the agent as needed.

A mature enterprise agent stack uses all four. The memory layer is where the knowledge graph, described in detail in the previous post, connects to the agent framework. An agent that can query a structured knowledge graph directly, as a tool, produces materially more precise answers on entity-specific questions than one that retrieves only document chunks.

Layer 5: The Observability Layer

The observability layer answers the questions your CTO and CFO will ask: what is each agent doing, how much does it cost per task, where does it fail, and how do you know when it is wrong? The NIST AI Risk Management Framework (AI RMF 1.0, January 2023) identifies traceability and explainability as core requirements for trustworthy AI systems in organizational contexts. For an agent system, this means logging every LLM call, every tool invocation, every token consumed, and every decision point. Without this, you cannot diagnose failures, audit compliance with internal policies, or build a business case for scaling.

The observability layer is frequently the last thing built and the first thing that causes a pilot to fail at scale.

---

Part Two: The Build vs Buy vs Wire Decision at Each Layer

Model Layer: Buy

Buy the model layer. Use the best available foundation model for your task via API. The selection criteria are context window size, tool-calling reliability, latency, and cost. Evaluate on your own data: a model that performs well on generic benchmarks may underperform on your domain-specific queries. Run structured evaluations before committing to a provider for your most critical agent workflows. Then reevaluate every six months, because the model market moves faster than any enterprise procurement cycle.

The one exception: regulated industries with strict data residency requirements may need to run open-weight models on internal infrastructure rather than calling external APIs. This is a viable choice but adds significant ML engineering overhead and generally results in lower model capability than the frontier API providers.

Tool Layer: Build

Build the tool layer. This is not optional. Your enterprise tools are your competitive advantage. No off-the-shelf agent framework knows how to query your specific contract management system, interpret your vendor risk scoring model, or navigate your ERP's data model. Every tool your agents use is a function your engineering team writes, documents, tests, and maintains.

Good tool design for agents follows a principle of minimal viable interfaces: each tool should do one thing, return structured data, and fail explicitly. Agents that call tools with ambiguous return shapes or silent failure modes accumulate errors across the agent loop and produce incorrect final outputs. Design each tool as though an external developer will use it, because in a sense they will: the LLM is the developer calling your API.

As your tool library grows, consider maintaining a tool registry: a structured catalogue of available tools with descriptions, input schemas, output schemas, and example uses. The agent's system prompt or tool configuration references this registry, allowing you to compose specialized agents that only have access to the tools relevant to their task.

Orchestration Layer: Wire

Wire the orchestration layer using open-source frameworks rather than building the loop from scratch. The core agent loop (reason, act, observe, repeat) is solved engineering. What varies is how you configure it: which agents get which tools, how handoffs between agents work, what the stopping conditions are, and where human approval gates interrupt the loop.

The frameworks worth evaluating for enterprise use are described in Part Three. The decision between them is largely about whether your agent tasks are single-agent (one agent, one task) or multi-agent (several agents collaborating on a complex task), and whether your orchestration logic is best expressed as a graph of states (LangGraph) or as a conversation between agent roles (AutoGen, CrewAI).

Avoid the temptation to build a custom orchestration layer from scratch to avoid framework dependencies. The frameworks have invested significantly in the edge cases: context truncation, tool call retry logic, streaming, interruption handling. Building that yourself delays your value delivery by months and does not produce a meaningfully better result for most enterprise agent tasks.

Memory Layer: Hybrid

The memory layer requires a hybrid strategy. Buy the vector database and the embedding infrastructure (Pinecone, Weaviate, pgvector, or your cloud provider's managed offering). Build the knowledge graph or wire it from an open-source stack (Neo4j Community, Apache Jena). Build the persistent state store on top of your existing database infrastructure. Wire the retrieval logic, combining vector search and graph traversal into compound context assembly that the agent receives as structured tool call results.

The critical engineering decision in the memory layer is how you handle context budget: at any given agent turn, your total available context (the sum of system prompt, conversation history, retrieved knowledge, and tool call history) must fit within the model's context window at a cost you can afford. Agents that hit context limits mid-task either fail silently or produce degraded outputs. Build context budget management into your orchestration layer from day one.

Observability Layer: Wire

Wire the observability layer. There are mature open-source and commercial tools for LLM tracing (LangSmith, Helicone, Arize, Langfuse) and it is not worth building tracing infrastructure yourself. What you do build on top of these tools: task-level success metrics specific to your business outcomes, cost attribution dashboards that map agent spend to the business workflows they support, and alerting on anomalous behavior (unusually long reasoning chains, repeated tool failures, unexpected cost spikes).

The observability layer is also where your AI governance obligations live. Every agent call that affects a customer, a financial record, or a regulated process should produce a complete audit trail: which agent, which model, which tools were called, what inputs were provided, what outputs were produced, and what action was taken downstream. This is not a nice-to-have. The EU AI Act's requirements for high-risk AI systems include logging obligations that apply to agent systems used in credit, hiring, and other regulated domains, with penalties for providers and deployers at the 3% and 7% tiers of global annual turnover respectively.

---

Part Three: Framework Comparison for Enterprise Use

Four approaches dominate enterprise agent implementation. They differ along two axes: how much of the loop they automate, and how they model multi-agent coordination.

LangGraph

LangGraph (github.com/langchain-ai/langgraph) models the agent orchestration problem as a directed graph where nodes represent agent actions and edges represent state transitions. This graph-based approach is well suited for complex, multi-step agent tasks where the flow is not linear: tasks that branch based on intermediate results, require parallel subtask execution, or need to backtrack when a tool call fails. LangGraph provides built-in support for human-in-the-loop interruption at any graph node, making it compatible with enterprise approval workflows where certain agent actions (submitting a contract, approving a purchase) require human sign-off before proceeding.

The learning curve is meaningful: developers need to think in terms of graph states and transitions, not sequential function calls. For teams already using LangChain, LangGraph is a natural extension. For teams without that background, the mental model takes time to internalize.

AutoGen

AutoGen (Wu et al., arXiv:2308.08155, Microsoft Research, 2023) models multi-agent coordination as a conversation between agents with defined roles. A UserProxyAgent represents the human or the triggering system. An AssistantAgent represents the LLM-powered reasoner. A GroupChatManager orchestrates multi-agent conversations. This conversational model maps well to collaborative tasks: code review, document drafting, multi-step analysis where different agents contribute different expertise.

AutoGen's strength is that the multi-agent coordination pattern is explicit and readable: you can see who said what to whom and why the agent did what it did. Its limitation for strict enterprise workflows is that the conversational model can be less predictable than a graph: agents can go "off script" in ways that are difficult to constrain without careful system prompt engineering.

CrewAI

CrewAI (github.com/crewAIInc/crewAI) offers the lowest barrier to entry for multi-agent systems. It uses a role-based model: you define a crew of agents, each with a role, a goal, and a set of tools. Tasks are assigned to agents based on their role. The framework handles the coordination. CrewAI is the right choice for teams building their first multi-agent system who want to demonstrate value quickly. It is less suited to highly complex workflows that require fine-grained control over agent state, tool access, or coordination logic.

Custom ReAct Loop

For teams who need maximum control, the custom ReAct loop (after Yao et al., arXiv:2210.03629) remains viable. You write the while loop, the tool dispatcher, the context management, and the stopping logic yourself. This approach has no framework dependency and can be tightly optimized for your specific agent tasks. The cost: you own all the edge cases the frameworks have already solved. This is the right choice when you have a single, well-defined agent task type (not a general-purpose agent) and when your engineering team has the capacity to maintain a custom loop indefinitely.

HITL = Human in the Loop, the ability to interrupt the agent and require human approval before proceeding.

---

Part Four: Wiring the Knowledge Graph into the Agent Stack

If you have built or are building an enterprise knowledge graph, connecting it to your agent stack is the most direct path to agent precision. The connection is straightforward in architecture and non-trivial in implementation.

The Graph as a Tool

Define your knowledge graph as a tool the agent can call. The function signature should be explicit: query_knowledge_graph(entity_id: str, depth: int, relationship_types: list[str]) -> dict. The agent calls this tool when it needs to understand relationships: who owns this contract, what risks are linked to this vendor, which regulations apply to this data type. The tool executes a graph query (SPARQL, Cypher, or Gremlin depending on your stack) and returns a structured subgraph as a JSON object that the agent includes in its reasoning context.

The depth parameter is critical. A depth of 1 returns only the immediate neighbors of the entity. A depth of 2 returns neighbors of neighbors. Deep traversals (depth 3 or more) can return extremely large subgraphs. Define sensible defaults and hard limits to prevent context overflow.

Combining Graph and Vector Retrieval

The most powerful enterprise memory pattern combines graph traversal with vector search in a compound retrieval tool. The agent first queries the knowledge graph to identify relevant entities and their direct relationships (structured, precise). It then uses entity identifiers from the graph query to constrain a vector search, retrieving only documents associated with those specific entities (contextually relevant). This compound retrieval pattern addresses a fundamental limitation of pure vector RAG: a vector search for "Vendor X risk" returns whatever documents are most similar to that phrase, regardless of whether they are actually about the current relationship with that specific vendor. The graph query first confirms the identity and attributes of Vendor X, and the constrained vector search then retrieves only documents tagged to that entity.

Context Budget Management in Practice

At each agent turn, your context budget might allocate roughly as follows: system prompt with tool definitions (a fixed cost), retrieved knowledge graph subgraph (variable, bounded), retrieved document chunks (variable, bounded), conversation and tool call history (grows with task length). When the sum approaches the model's context limit, you must either summarize the history, truncate retrieved context, or route to a longer-context model at higher cost.

Build context budget management as a module in your orchestration layer, not as an afterthought. It should calculate the estimated token cost of each context component before assembly and enforce a priority order if truncation is necessary: system prompt and current tool results are highest priority, old history is lowest.

---

Part Five: A Practical 12-Week Pilot Architecture

The 12-week timeframe is long enough to build something real and short enough to maintain organizational momentum. The goal of the pilot is not a comprehensive agent platform: it is one working agent, doing one task well, with observable behavior and measurable outcomes.

Choose Your First Agent Task Carefully

The ideal first agent task has three properties: high current cost (it takes skilled people significant time to do it today), clear evaluation criteria (you know a correct answer when you see it), and bounded scope (it requires connecting two or three systems, not twenty). Examples that meet these criteria: vendor risk summarization that combines contract data and news; contract clause comparison across a defined clause library; IT incident triage that queries the CMDB and cross-references the runbook knowledge base.

Avoid first tasks that require the agent to take irreversible high-stakes actions. Use the first pilot to demonstrate reasoning and retrieval quality. Reserve autonomous action (submitting approvals, updating records, sending communications) for later iterations where you have established trust in the agent's output quality.

The 12-Week Build Sequence

The overlap in the timeline is intentional. You do not finish the data model before starting stack setup. You start both in parallel, with the understanding that the stack setup team will have early, incomplete data to work with. This is preferable to the alternative: a waterfall where stack setup waits for a perfectly finished data model and you miss your 12-week window.

Week 1 to 3: Data Model and Entity Taxonomy

Define the entities and relationships your agent task requires. For a vendor risk agent: Vendor, Contract, Risk Assessment, Regulatory Requirement, Contact. For a contract comparison agent: Contract, Clause, Clause Library Entry, Obligation. The entity taxonomy should cover the minimum necessary to answer your target questions, not the maximum possible representation of your business. You will expand it later.

Week 2 to 4: Stack Setup

Stand up your orchestration framework (LangGraph or AutoGen for complex tasks, CrewAI for a first pilot). Configure your development, staging, and production environments. Set up your observability stack: a trace logger that captures every LLM call, tool call, and token count from day one. It is much harder to add observability retroactively than to build it in from the start. Stand up your vector database and your graph database if you are including structured knowledge.

Week 3 to 6: Tool API Development

Write, test, and document your tool functions. Each tool should have: a docstring or description the LLM will read to decide when to call it, explicit input and output schemas, error handling that returns structured error messages rather than exceptions, and a test suite with representative inputs. This is the phase where enterprise agents succeed or fail. If your tools are poorly defined, poorly documented, or unreliable, the LLM will use them incorrectly regardless of how well you configure everything else.

Week 5 to 8: Orchestration Layer Configuration

Wire the orchestration framework to your tools and your model provider. Build the agent graph or role configuration. Implement context budget management. Add your approval gates for any high-stakes tool calls. Run end-to-end tests with representative task inputs and evaluate outputs manually: does the agent reach the correct conclusion? Does it use tools in the correct sequence? Where does it fail, and why?

Week 7 to 10: Observability and Evaluation

Complete your observability layer. Define your task-level success metrics: for a vendor risk agent, success might be "correctly identifies the three highest-risk vendors from a set of ten" on a held-out test set. Build an evaluation harness that runs your agent against this test set automatically. This evaluation harness is how you measure the impact of every change you make to prompts, tools, context management, and framework configuration. Without it, you are optimizing blind.

Week 9 to 12: Enterprise Pilot Deployment

Deploy to a limited set of internal users. Not everyone, not every use case: a specific team with a specific task that the agent is built to support. Instrument every interaction. Collect qualitative feedback from users. Monitor cost per task and compare it to the current human cost of the equivalent work. This data is your business case for the next phase: extending the agent to additional tools, additional tasks, or additional users.

---

Part Six: What to Defer

Three things enterprises commonly build in the first agent pilot that should wait:

Fully autonomous action. The first pilot should recommend, not execute. Build a "human in the loop" gate at every point where the agent would take an action with consequences: submitting a form, updating a record, sending a message. The gate does not need to be permanent, but it needs to exist during the pilot so you accumulate evidence of output quality before removing it.

General-purpose agents. An agent that can do anything does nothing well. The first agent does one task. The tool library expands after you have validated output quality on the first task. Agents that try to cover too much scope during a pilot produce inconsistent results that undermine organizational confidence in the technology.

Custom model fine-tuning. Most enterprise agent tasks do not require fine-tuning. They require better tools, better context, and better prompts. Defer fine-tuning until you have exhausted the gains available from retrieval, tool design, and prompt engineering. Fine-tuning is expensive, it degrades over time as the base model provider updates their weights, and it is unnecessary for the majority of enterprise agent workflows.

---

The Real Constraint Is Not the Technology

The enterprise AI agent stack is, at this point, a solved engineering problem. The frameworks are mature. The model APIs are reliable. The observability tooling exists. The real constraint on enterprise agent adoption is organizational: the discipline to define a narrow, well-scoped first task; the patience to invest properly in the tool layer before expecting results; and the governance infrastructure to monitor agent behavior in the same way you would monitor any other system that touches customer data or business-critical processes.

Organizations that build the tool layer well, invest in observability from the start, and resist the temptation to make the first agent general-purpose will have a working, measurable agent capability at the end of twelve weeks. Those that try to shortcut those three things will have an interesting demonstration that does not survive contact with real production workloads.

The knowledge graph and the agent stack are two halves of the same infrastructure investment. The knowledge graph organizes what your organization knows. The agent stack deploys that knowledge as an active reasoning capability. Neither is complete without the other.