Most enterprise teams choose their AI approach based on what the team already knows, not what the use case actually requires. The result is fine-tuned models that go stale, RAG pipelines that retrieve noise, and context windows that grow without bound. A practitioner decision framework for choosing the right approach before you build.
Enterprise AI teams spend significant time debating whether to fine-tune a model, build a RAG pipeline, or rely on context engineering. The debate is often framed as a capability comparison: which approach produces better outputs? That is the wrong question. Fine-tuning, retrieval-augmented generation, and context engineering are not competing approaches to the same problem. They solve different problems. Choosing between them requires understanding what problem you actually have, not which technique your team finds most interesting.
The confusion is understandable. All three approaches result in an LLM producing outputs that are more useful for a specific use case than the base model would produce without intervention. But they achieve that improvement in fundamentally different ways, with fundamentally different cost structures, latency profiles, maintenance requirements, and failure modes. A team that builds a fine-tuned model for a use case that needed RAG will spend months training a model that goes stale the moment the underlying data changes. A team that builds a RAG pipeline for a use case that needed fine-tuning will spend months tuning retrieval for a problem that was actually about output format, not about information access.
This post provides a decision framework based on the four variables that actually determine the right choice: data freshness requirements, customization depth, data residency constraints, and cost and latency targets. The framework produces a recommendation, not a tutorial on how to implement the chosen approach.
Before applying the decision framework, it is worth being precise about what each approach changes in the system, because that precision is what makes the framework work.
Shapes model behavior by controlling what enters the context window on each call: system prompt instructions, few-shot examples, retrieved content, and structured data. Nothing about the model's weights changes. The model uses its general capabilities to follow the instructions provided at runtime. Often called "prompt engineering," but the enterprise version involves significant architectural decisions about what to include, how to structure it, and how to keep it within cost and latency bounds.
Extends context engineering by adding a retrieval layer. Before each model call, a search system retrieves relevant chunks from a document store, knowledge base, or database and injects them into the context window. The model reasons over the retrieved content alongside the user's query. The original RAG architecture was introduced in Lewis et al. (arXiv:2005.11401, 2020). In enterprise deployments, the retrieval system and the document store are the primary engineering surfaces, not the model itself.
Updates the model's weights on a task-specific dataset, changing what the model "knows" and how it behaves by default without any runtime instructions. A fine-tuned model produces domain-appropriate outputs without needing extensive system prompt guidance. Modern enterprise fine-tuning typically uses parameter-efficient methods like LoRA (Hu et al., arXiv:2106.09685, 2021), which update a small fraction of the model's parameters rather than the full weight set, reducing compute cost significantly while preserving most of the benefit.
Context engineering and RAG affect what the model sees at inference time. Fine-tuning affects what the model is. If your use case requires the model to know something, that information can be injected at inference (context/RAG) or baked in at training (fine-tuning). If your use case requires the model to behave in a specific way (specific tone, format, reasoning pattern, or domain dialect), that behavior can be instructed at inference (with diminishing returns as complexity increases) or instilled at training. Understanding which of these two axes your use case sits on is most of the decision.
Five use case characteristics determine approach fit more than any other factors. The matrix below shows how each approach performs on each characteristic.
Two patterns stand out in the matrix. Fine-tuning dominates on depth-of-customization requirements: when the use case demands a specific tone, format, reasoning style, or domain dialect, fine-tuning is the only approach that instills that behavior reliably without extensive per-call instruction overhead. Context engineering and RAG dominate on data currency: when the underlying information changes frequently, both approaches can reflect updated information at inference time without retraining, while a fine-tuned model's knowledge is frozen at the point of its training data.
The data residency row is the most commonly overlooked in enterprise procurement. Both context engineering and RAG with cloud model APIs transmit the context content to the model provider's infrastructure on every call. Only fine-tuning enables full on-premises deployment of a model that produces domain-appropriate outputs without sending data to a cloud API. For regulated industries handling protected information, this consideration alone may resolve the decision before any other variable is evaluated.
Run through these four questions in order. Each question either resolves the decision or narrows it. Most use cases are resolved by question one or two.
If the information the model needs to produce correct outputs changes weekly or faster, fine-tuning is not viable as the primary approach. Retraining and deploying a fine-tuned model on that cadence requires a data collection, training, and evaluation pipeline that most enterprise teams do not have and is not worth building for this purpose. RAG or context engineering is the right primary approach, with fine-tuning potentially layered on top for format and style consistency.
If the information changes on a monthly or slower cadence, and the volume of it is large enough that it cannot be included in a context window, RAG becomes the primary approach. If the information is stable and the volume is manageable, context engineering may be sufficient.
If the use case requires the model to know specific things (facts, records, documents, current prices, internal policies), the requirement is about information access. RAG or context engineering provides that. If the use case requires the model to behave in a specific way (write in a particular format, use domain-specific terminology correctly, reason through a specific type of problem in a consistent sequence), the requirement is about behavior. Fine-tuning is significantly more reliable than prompt engineering for deep behavioral requirements, particularly as the complexity and consistency requirements increase.
If the use case involves data that cannot leave a specific regulatory boundary or network perimeter, and the model API is cloud-hosted, both context engineering and RAG will cause that data to cross the boundary on every call. This is not a configuration setting. It is an architectural consequence of how cloud LLM APIs work: the context payload is transmitted to the provider's infrastructure for processing. If residency is a hard requirement, the only approaches that satisfy it are those that use a self-hosted model, which means fine-tuning a model that can run on-premises, or using a smaller open-weights model with context engineering in a private deployment. Research on cost-efficient inference strategies like FrugalGPT (Chen, Zaharia, Zou, arXiv:2310.11409) is relevant here: fine-tuning a smaller, specialized model for a constrained task can achieve quality comparable to a large cloud model at a fraction of the inference cost, while also satisfying residency requirements.
Context engineering has the lowest setup cost and the highest per-call cost at scale, because every call carries the full context payload. RAG has moderate setup cost (the retrieval infrastructure, the document processing pipeline, the embedding model) and moderate per-call cost (retrieval adds latency and compute). Fine-tuning has the highest upfront cost (data collection, annotation, training, evaluation, deployment) and the lowest per-call cost, because the fine-tuned model produces outputs with minimal context overhead. For high-volume deployments, fine-tuning's cost curve eventually crosses below RAG and context engineering. The volume threshold depends on the training data cost, the model size, and the query volume. At lower volumes, the upfront investment does not amortize.
Once the framework resolves the primary approach, there are architecture decisions within that approach that determine whether the deployment works in practice.
The system prompt in a context-engineered deployment is not a one-time decision made at launch. It is a configuration file that should be version-controlled, reviewed before changes, and tested against a regression set after each modification. Prompt drift, where incremental additions to the system prompt cause earlier instructions to lose weight or become contradicted, is the primary failure mode in long-running context-engineered deployments. Treating the system prompt as code, with the same change management discipline, is the primary architectural safeguard.
Context windows are large but not free. Token cost scales with context size, and context size grows as few-shot examples, structured data, and conversation history accumulate. Define a token budget per component (system prompt, retrieved content, history, user input) and enforce it architecturally. Unbounded context growth is the most common cause of cost overruns in context-engineered deployments, and it is entirely preventable if budgeted at design time rather than monitored reactively.
A RAG system's output quality cannot exceed its retrieval quality. If the retrieval layer surfaces irrelevant, outdated, or incomplete chunks, the model will produce responses based on those chunks regardless of its general capability. Retrieval evaluation deserves as much engineering attention as model evaluation. Common failure modes include semantic mismatch between query and document embeddings (addressed by hybrid search combining dense and sparse retrieval), chunk boundary problems (sentences that span chunk boundaries lose context), and staleness (documents that have been updated but whose embeddings have not been refreshed). Each requires an explicit architectural mitigation, not a prompt adjustment.
Fine-tuning quality is primarily a function of training data quality, not quantity. A small dataset of high-quality examples, where each example represents the exact behavior the model should learn, consistently outperforms a large dataset of noisy examples. The annotation process for fine-tuning data, specifically writing input-output pairs that capture the desired behavior, is the most important and most consistently underinvested phase of a fine-tuning project. Plan for significant annotation effort before writing any training code. For LoRA-based fine-tuning, a few hundred high-quality examples is often sufficient for format and style adaptation; deeper behavioral changes require more.
The three approaches sit at different points in the inference pipeline. Understanding where each one operates is the prerequisite for building hybrid architectures correctly.
Context engineering operates at runtime, RAG operates at retrieval time, and fine-tuning operates at training time. A hybrid architecture combines all three, but each layer has a distinct failure mode and a distinct owner. Assigning ownership of each layer to a specific engineering role before the build starts is the difference between a maintainable system and one that degrades silently over time.
The team structure depends on which approach the framework selects. These are the minimum configurations that can deliver a working pilot, not an enterprise-scale deployment. Each role below is a functional responsibility, not necessarily a separate headcount: in smaller organizations, one person may cover multiple roles.
Pilot duration: 2 to 4 weeks to working prototype. Main risk: prompt drift as requirements accumulate.
Pilot duration: 4 to 8 weeks to working prototype. Main risk: retrieval quality issues that are not caught without a labeled evaluation set.
Pilot duration: 8 to 14 weeks to working prototype. Main risk: annotation quality and insufficient training data volume for behavioral consistency.
Pilot duration: 14 to 20 weeks. Main risk: dependency between fine-tuning and RAG phases causes delays if the fine-tuning evaluation gate takes longer than planned.
The roadmap below applies to RAG and fine-tuning deployments. Context engineering pilots compress to a single phase. Hybrid deployments run the fine-tuning track first, then run the RAG track starting at Phase 2.
Scope
Deliverable
Working prototype with a documented quality baseline and a failure mode inventory.
Go / No-Go Gate
The prototype meets a minimum quality threshold on the labeled evaluation set, and the failure modes are either addressed or have a documented mitigation plan. If neither is true, do not advance to Phase 2.
Scope
Deliverable
Hardened system passing security review, with runbook and monitoring in place.
Go / No-Go Gate
Security review passed, load test passed, runbook reviewed and approved by the team that will own the system in operation. If security review is not passed, rollout does not proceed.
Scope
Deliverable
Full deployment with defined ownership, update cadence, and deprecation plan.
Success Criteria
Quality metrics hold at full deployment volume. Update cadence is operational. The team that built it is not the only team that can operate it.
For each key component in the stack, the decision between building, buying, and configuring has a different answer. Getting this wrong is one of the most common sources of scope creep in enterprise AI programs.
The tooling that lets domain experts produce training examples and the harness that evaluates output quality against the labeled test set should be built internally. These are the components most specific to the organization's domain and quality standards. No vendor product will match the exact annotation criteria or evaluation dimensions that the domain experts define. Budget for this as an engineering task, not a procurement task.
The vector store for RAG deployments is a commodity infrastructure component. The decision criteria are: does it integrate with the organization's existing data platform, does it support hybrid search (dense and sparse), and does it meet the data residency requirements? Evaluate vendors in the managed vector database category. Do not build a custom vector store: the engineering cost is high and the maintenance burden compounds. The retrieval strategy on top of the vector store is what differentiates the system, not the store itself.
For organizations without existing ML training infrastructure, managed fine-tuning platforms in the AI infrastructure category significantly reduce the time to first training run. Evaluate options based on: supported model families, LoRA and QLoRA support, data residency and security certifications, and integration with the organization's existing model serving infrastructure. For organizations with existing GPU clusters, configuring open-source fine-tuning frameworks is often preferable to a managed platform, particularly for sensitive data where the training data cannot leave the organization's network.
The model API itself is a configuration surface, not a build target. System prompts, token budgets, output format constraints, and tool definitions are all configuration. The anti-pattern is treating these as engineering outputs that require a full development cycle to change. They should be in version control, testable in isolation, and deployable without a full release cycle. The system prompt is a configuration file; it should be treated like one from the start of the project.
LLM observability (latency, token usage, quality metrics, error rates) should be added by configuring the organization's existing observability stack, not by building a custom monitoring system. The metrics that matter for LLM deployments, specifically output quality drift and retrieval relevance drift over time, require custom instrumentation at the application layer, but the underlying time-series storage, alerting, and dashboarding should use existing infrastructure. The custom work is defining what to measure, not how to store or visualize it.
Building a custom document ingestion pipeline, chunking library, or embedding service from scratch is almost never justified. These are well-solved problems with established open-source implementations and managed services. The engineering effort should go into the retrieval strategy (what to retrieve, how to rank it, how to evaluate it) and the data model (how documents are chunked, what metadata is preserved), not the underlying infrastructure. The exception is organizations with unusual data formats or classification requirements that no existing tool supports.
These are the failure modes that recur across enterprise AI deployments, regardless of approach. Each one has an early signal and a mitigation. The mitigation is most effective when implemented at design time, not after the failure is observed in operation.
What happens: Source documents are updated, but the vector store is not refreshed on the same cadence. The retrieval layer continues to surface outdated content with high confidence scores. Output quality drops, but without retrieval monitoring the degradation is not detected until users report errors.
Early signal: Increase in user-reported factual errors, or a drop in output quality scores on the evaluation set run on a weekly cadence.
Mitigation: Define the document update cadence and build the re-embedding pipeline before deployment. Run the evaluation set on a fixed schedule and alert on quality drops above a defined threshold. Document freshness timestamps in document metadata and surface them in responses where relevance.
What happens: Requirements accumulate in the system prompt incrementally. Each addition looks small in isolation. Over time, the prompt contains contradictions, redundancies, and instructions that override earlier instructions in ways that are not intended. The model's behavior becomes unpredictable and the system prompt becomes too large to reason about as a single document.
Early signal: A regression in behavior on test cases that were passing before the last prompt change.
Mitigation: Treat the system prompt as a versioned configuration artifact from day one. Run a regression test suite on every change. Define a size limit for the system prompt and require architectural review when the limit is approached.
What happens: The annotation phase is underestimated. Annotators produce examples quickly but inconsistently. The training data contains examples of conflicting behavior. The fine-tuned model learns a blend of correct and incorrect behavior, and the blend shifts unpredictably with different inputs.
Early signal: High variance in output quality across similar inputs after fine-tuning, or a fine-tuning loss curve that does not converge cleanly.
Mitigation: Produce annotation guidelines before annotators begin, require inter-annotator agreement checks on a sample of examples, and evaluate a small training set (fewer than 100 examples) before committing to full data collection. If the small set does not produce the target behavior, the annotation criteria need revision before scaling.
What happens: The team builds a RAG or context engineering deployment over a cloud model API. During security review, it is discovered that the data being passed in the context window is classified at a level that prohibits transmission to the cloud provider's infrastructure. The architecture must be redesigned, the timeline extends, and the team rebuilds against a self-hosted model.
Early signal: This risk has no early signal because it is a requirements gap, not an implementation failure.
Mitigation: Complete a data classification exercise for all data types that will enter the context window before selecting the model deployment architecture. Include the security architect in the architecture decision, not the architecture review. "Include in review" and "include in decision" are different and the former does not prevent this risk.
What happens: A team with fine-tuning expertise builds a fine-tuned model for a use case that needed RAG. A team with RAG experience builds a retrieval system for a use case where the bottleneck was model behavior. The resulting system underperforms, the team spends months optimizing the wrong component, and the approach is eventually abandoned or supplemented with the correct one.
Early signal: The team's initial proposal names an approach before completing the four-question framework.
Mitigation: Require the four-question framework output to be documented and reviewed before any architectural proposal is written. The framework output should drive the proposal, not validate it after the fact.
The framework produces different recommendations for the same broad use case description depending on the specific requirements. Here are three scenarios that illustrate why the requirements drive the decision.
Deployment: A global law firm wants to deploy an AI assistant that helps associates review commercial contracts, flag non-standard clauses, identify missing provisions, and summarize key terms. The firm's contract templates and clause library are updated quarterly. Associates use the assistant for dozens of contracts per day.
Decision analysis: The information the model needs (clause definitions, standard vs. non-standard language, firm-specific templates) is updated quarterly: slow enough for fine-tuning to be viable, but the firm also has a large and growing body of contracts that would benefit from retrieval. The requirement is partly behavioral (the model must reason about contract language in the firm's specific analytical framework) and partly informational, surfacing relevant precedent and template provisions. Data residency is a concern: client contract data cannot leave the firm's network under many client agreements.
Recommendation: Fine-tune a mid-size open-weights model on the firm's historical contract review annotations to instill the analytical framework, then layer RAG over the fine-tuned model to surface relevant precedent and template provisions at query time. Deploy the fine-tuned model on-premises to satisfy data residency. This hybrid approach is more expensive to build than either approach alone, but it is the only configuration that satisfies all three requirements simultaneously. The on-premises fine-tuned model also has lower per-query cost than a cloud API at the query volumes legal associates generate.
Deployment: A large retailer wants to deploy an AI copilot for its customer support agents. The copilot should surface relevant policy information, order details, and return procedures in response to agent queries. Product catalog and pricing data changes daily. Support policies change several times per year.
Decision analysis: The information requirement (product data, order details, policies) changes on a daily to monthly cadence. Fine-tuning cannot track daily catalog changes. Data residency is not a constraint; the retailer's customer support infrastructure already runs in a cloud provider. Query volume is high but the per-query context size is moderate. The behavioral requirement is minimal: the copilot needs to summarize and surface information, not produce domain-expert analysis.
Recommendation: RAG over a cloud model API. The retrieval layer connects to live catalog and order data via tool calls for real-time data, and to a vector store of policy documents for less-frequently-updated content. The system prompt handles the behavioral requirements, which are simple enough to not require fine-tuning. This is the lowest-setup-cost option that satisfies all requirements. If the support team identifies a consistent tone or format issue that the system prompt cannot reliably fix, a light fine-tuning pass for style can be added later without redesigning the architecture.
Deployment: An asset manager wants to automate the summarization of quarterly earnings call transcripts into a standard internal format. The format includes sections for guidance, key risks, management commentary, and analyst questions. The format has been developed internally over several years and is not well-represented in general training data.
Decision analysis: The information is the transcript itself, which is provided in full on each call, so there is no retrieval requirement. The requirement is purely behavioral: the model must produce output in a specific format that the firm has developed and that does not conform to standard summarization conventions. Data residency constraints apply for certain transcripts under confidentiality agreements. The volume of transcripts is moderate but the per-transcript context is large.
Recommendation: Fine-tune a smaller open-weights model on historical earnings call transcripts and their human-written summaries in the firm's internal format. The fine-tuned model can be self-hosted to satisfy residency requirements. Context engineering with the full transcript as input handles the information access. No retrieval layer is needed: the transcript is the document. This is the highest-fidelity approach for a format-consistency requirement, and fine-tuning a smaller model for this specific task is likely to produce more consistent format adherence than prompting a large general model, at lower per-call cost.
The right approach is not determined by the use case category ("contract review" or "customer support") but by the specific combination of data currency, behavioral depth, residency, and volume that applies to that deployment. The same general use case description can lead to completely different recommendations depending on those four variables. Running the four-question framework before committing to an architecture is the primary way to avoid building the wrong system.
The cost of choosing the wrong approach is paid twice: once in the initial build, and again in the remediation when the approach fails to meet requirements at scale.
Teams that fine-tune a model for a use case that needed RAG face a specific failure mode: the model's knowledge is frozen at training time, and the use case requires knowledge that changes. The fine-tuned model produces confidently incorrect outputs as the underlying data drifts from the training distribution. Remediation requires retraining with updated data, which means rebuilding the data collection and annotation pipeline that was already built once. The cost is the original fine-tuning cost plus the cost of the remediation cycle, plus the cost of operating a model that was producing incorrect outputs during the interval.
Teams that build RAG for a use case that needed fine-tuning hit a different ceiling: retrieval quality cannot compensate for missing behavioral capability. If the model does not know how to reason about the domain, surfacing more relevant documents does not improve the output quality. The team spends significant effort optimizing retrieval (chunking strategies, embedding models, rerankers) when the actual bottleneck is model behavior. Remediation requires recognizing that retrieval is not the bottleneck and introducing fine-tuning, which means a significant architecture expansion after the initial deployment.
Context engineering has the lowest time-to-first-result: a working pilot can be built in days. The ongoing cost is in prompt maintenance and token cost at scale. RAG requires building a retrieval pipeline, a document store, an embedding process, and a retrieval evaluation framework. The engineering effort is weeks to months, not days. Fine-tuning requires data collection, annotation, training infrastructure, model evaluation, and deployment infrastructure. It is the highest upfront investment and the lowest per-query cost at scale. The decision between them is partly an investment timing decision: how much upfront cost is acceptable to reduce ongoing cost?
For high-volume use cases, fine-tuning's lower per-query cost eventually crosses below RAG and context engineering. The crossover depends on query volume, model size, and annotation cost. For low-volume use cases, the upfront investment rarely amortizes within a reasonable horizon, and RAG or context engineering is the correct choice even if fine-tuning would produce marginally better outputs. For use cases with strict data residency requirements, fine-tuning a self-hosted model eliminates cloud API cost entirely, which often makes the build investment the clear economic choice regardless of volume.
Map your use case to the four-question framework before any architecture discussion. Write down the answers: how often the data changes, whether the requirement is informational or behavioral, what the data residency constraints are, and what the volume and cost ceiling is. Then look at the fit matrix for the approach that matches those answers. The architecture discussion should start from the framework output, not from the team's familiarity with a particular approach. If the framework output contradicts what the team was planning to build, that is a signal worth examining before the build starts, not after.
These are the questions a CTO, Chief AI Officer, or VP of Engineering should be able to answer before committing budget and team capacity to an implementation. For each question, a good answer and a red flag are provided.
Question: How often does the information this use case requires change, and can the chosen approach reflect those changes within the acceptable lag window?
Good answer: "Our data changes weekly. We are using RAG with a re-embedding pipeline that runs nightly. The acceptable lag is 24 hours and the pipeline meets that requirement."
Red flag: "We are fine-tuning the model. We will retrain when needed." No defined cadence and no owner is a program risk, not an implementation detail.
Question: Is the primary gap between the base model and the required output about what the model knows, or about how the model behaves?
Good answer: "We ran 50 test cases against the base model with a detailed system prompt. Format consistency failed on 30 percent of cases despite explicit instructions. The requirement is behavioral and the system prompt alone is not sufficient. Fine-tuning is justified."
Red flag: "We are fine-tuning because our use case is specialized." Specialization is not by itself evidence that fine-tuning is needed. The test is whether the system prompt fails at the required consistency level.
Question: What is the classification of data that will enter the context window on each call, and is the chosen deployment model consistent with the handling requirements for that classification?
Good answer: "We completed a data classification exercise. The data entering the context window is classified as internal confidential. Our cloud provider's data processing agreement covers that classification and the legal team has confirmed it."
Red flag: "We will handle data residency in the security review." Security review after architecture selection is too late. This decision gates the architecture, not the other way around.
Question: Do you have a labeled retrieval evaluation set, and what is the minimum retrieval precision you will accept before deploying?
Good answer: "We have 200 labeled query-document pairs. We are targeting a minimum precision at 5 of 0.75. We will not deploy until that threshold is met on the full evaluation set."
Red flag: "We will evaluate retrieval quality qualitatively." A RAG deployment without a labeled evaluation set has no basis for a deployment decision and no way to detect quality degradation after deployment.
Question: Who are the annotators, what are the annotation guidelines, and have you evaluated a sample of their output before committing to full data collection?
Good answer: "The annotators are three senior associates in the relevant domain. We have written annotation guidelines that define what constitutes a correct output. We evaluated 50 examples and measured inter-annotator agreement. The agreement rate was acceptable and we are proceeding to full data collection."
Red flag: "We are using a general annotation service." Domain annotation requires domain expertise. General annotation services produce volume, not quality, for specialized tasks.
Question: What is the per-query cost at expected volume, at two times expected volume, and at five times expected volume? Is the model sustainable at each level?
Good answer: "At expected volume, context engineering costs approximately X per thousand queries. At five times volume, the cost is still within our AI budget allocation. We have modeled the crossover point where fine-tuning a self-hosted model would be cheaper and it is at Y queries per day, which we do not expect to reach within 18 months."
Red flag: "We will monitor cost after launch." Cost surprises at scale are a program risk, not an operational adjustment. Model them before the build starts.
Question: Who owns the system after the pilot team moves on? What is the update cadence, and who approves updates before they reach the deployment environment?
Good answer: "The platform engineering team owns the system post-pilot. The system prompt is updated through a defined change management process. RAG index updates run automatically and are monitored. Fine-tuning updates require ML Engineer review and a passing evaluation run before deployment."
Red flag: "The team that built it will maintain it." Pilot teams dissolve. A system without a permanent owner and a defined update process degrades to a liability.
Question: Does this use case require a combination of approaches? If so, has each layer's responsibility been explicitly defined and assigned?
Good answer: "Yes. Fine-tuning handles behavioral consistency; RAG handles current information access. The fine-tuned model is the behavioral layer, the RAG layer is the information layer. Each has a separate owner and a separate evaluation methodology."
Red flag: "We are adding RAG to the fine-tuned model to improve results." Adding layers without defined responsibilities and separate evaluation creates a system where no individual failure mode can be diagnosed cleanly.
The most capable enterprise AI deployments typically layer all three approaches: a fine-tuned model provides behavioral consistency and domain adaptation, a RAG layer provides access to current and specific information, and context engineering provides runtime control and session-specific customization. The legal services scenario above is an example of this pattern: fine-tuning for analytical behavior, RAG for precedent retrieval, context engineering for per-session instructions.
The layered approach is more powerful than any single approach and more complex to build and maintain. The decision framework above is designed to identify which layer is the most important one for a given use case, so that resources are concentrated on the right investment rather than distributed equally across all three. Most use cases are under-served by one primary approach rather than by the absence of the other two. Identifying that primary gap is what the framework resolves.
As model capabilities improve and costs decrease, the threshold at which context engineering alone is sufficient will continue to rise. Longer context windows reduce the need for RAG in many use cases. More capable base models reduce the need for fine-tuning for shallow behavioral adaptation. The framework's questions will remain stable even as the answers shift: a use case that required fine-tuning for consistent format adherence today may not require it once the base model's instruction-following is sufficiently reliable. Revisiting the framework at each major model generation, rather than treating the initial architecture decision as permanent, is part of running a well-maintained AI deployment.