Building the Pilot

What "Good Enough" Means On-Site

In a product engineering organization, "good enough" means meeting quality, reliability, and scalability standards set by the engineering team. Code review, test coverage, performance benchmarks, and security audits all gate what ships. These standards exist for good reasons: the product runs at scale, for many customers, and failures have wide impact.

On a customer site, in a pilot phase, the standard is different. A pilot does not need to handle every edge case. It does not need to scale to the full production volume. It does not need to be maintainable by anyone who was not in the room. What it needs to do is demonstrate, with real customer data, that the proposed approach works for the specific problem identified in discovery.

An FDE who applies product-quality standards to a pilot will deliver something excellent in twelve weeks that the customer needed in two. The customer will have lost confidence in the engagement and moved on. Speed is not a compromise in an FDE pilot. It is the definition of success at this stage.

The 80/20 Rule for Pilot Scope

In most enterprise AI deployments, 80 percent of the value comes from 20 percent of the possible features. An FDE's job is to identify that 20 percent and build only that. The remaining 80 percent can be scoped for the product team, deferred to a future phase, or deprioritized entirely if the 20 percent delivers the success criteria.

The hardest part of applying this rule is resisting customer requests for additional features. Every stakeholder interaction generates new ideas. The operations team wants an export button. The compliance team wants an audit log. The data team wants a custom dashboard. All of these are reasonable requests and none of them should be in the pilot.

The discipline required here is organizational. When a stakeholder makes a feature request during the build phase, the right response is to add it to a backlog document explicitly labeled "Phase 2 candidates" and share it with the stakeholder. This acknowledges the request, demonstrates that you heard it, and protects your build timeline simultaneously.

Three Over-Engineering Traps

Trap 1: Building for scale you do not have. A pilot processes hundreds of documents. The customer's production volume will be tens of thousands. An FDE who builds the scale infrastructure before the pilot succeeds is engineering for a commitment the customer has not yet made. Build for the pilot volume. If the pilot succeeds, the product team will rebuild for scale anyway.

Trap 2: Designing the perfect data model. Enterprise AI pilots almost always reveal that the data is messier than the discovery sprint suggested. An FDE who designs a comprehensive data model before seeing the actual data will spend days mapping it when the data arrives in a format they did not expect. Use a flexible structure for the pilot: a simple schema that captures what is needed, with field names that make sense to the customer, and no more.

Trap 3: Abstracting too early. Product engineers are trained to abstract: if you build a thing twice, make it a library. On a pilot, abstraction before the approach is validated wastes time. If the pilot fails, the abstracted library ships with it. Build the direct solution first. If it succeeds and the approach is validated, abstract during productization, not during piloting.

Rapid AI Prototyping On-Site

An FDE builds in three layers. The data layer handles ingestion and normalization of whatever the customer provides. The AI processing layer contains the core logic: LLM calls, retrieval, prompt chains, and output validation. The interface layer is whatever the customer stakeholder can evaluate: a simple web page, a command-line script, a spreadsheet output, or an API endpoint that feeds into a system the customer already uses.

Most FDE pilots can be built with a small number of components. For AI-document processing pilots: a document loader, a text splitter, a vector store, a retrieval step, and an LLM call with a carefully designed prompt is often the complete technical architecture. The sophistication is in the prompt engineering and the output validation, not in the infrastructure.

FDEs who try to use the latest framework for every component end up debugging framework issues instead of customer problems. Use the simplest component that works for the pilot. For vector storage, a local file-based store is fine for a pilot of hundreds of documents. For LLM calls, a direct API call with a simple retry wrapper is fine. The complexity that scales comes later.

# Minimal RAG pilot for document Q&A
# Three-layer pattern: data, AI, interface

import json
from pathlib import Path

# --- LAYER 1: DATA ---
def load_documents(folder: str) -> list[dict]:
    """Load text from customer documents. Keep it simple."""
    docs = []
    for p in Path(folder).glob("*.txt"):
        docs.append({"id": p.stem, "text": p.read_text()})
    return docs

def chunk_text(text: str, size: int = 400) -> list[str]:
    """Split text into overlapping chunks."""
    words = text.split()
    chunks = []
    for i in range(0, len(words), size - 50):
        chunks.append(" ".join(words[i:i+size]))
    return chunks

# --- LAYER 2: AI ---
def retrieve(query: str, chunks: list[str], top_k: int = 3) -> list[str]:
    """
    Placeholder retrieval. In a real pilot, replace with
    a vector store (e.g. Chroma, FAISS) and embeddings.
    """
    # For demonstration: return first top_k chunks
    return chunks[:top_k]

def answer_question(query: str, context: list[str]) -> str:
    """Call LLM with retrieved context."""
    # Replace with your LLM API call
    combined = "\n\n".join(context)
    prompt = (
        f"Answer the following question using only the context provided.\n"
        f"If the answer is not in the context, say 'Not found in documents'.\n\n"
        f"Context:\n{combined}\n\n"
        f"Question: {query}"
    )
    # return llm_call(prompt)
    return f"[LLM response to: {query}]"

# --- LAYER 3: INTERFACE ---
def run_pilot(docs_folder: str, question: str) -> dict:
    docs = load_documents(docs_folder)
    all_chunks = []
    for doc in docs:
        all_chunks.extend(chunk_text(doc["text"]))
    context = retrieve(question, all_chunks)
    answer = answer_question(question, context)
    return {"question": question, "answer": answer,
            "sources": [c[:100]+"..." for c in context]}