Module 1: What You Actually Need to Know About AI

A vendor just spent 45 minutes demoing an AI platform. Your team is excited. Your CFO is skeptical. You need to ask the right question, and you are not sure what that is. This module gives you the mental model that makes every AI conversation cleaner: what these systems are, how they fail, and what the words people use actually mean.

Why the Vocabulary Matters Before the Strategy

Most C-suite AI discussions go wrong before they start. A board member asks about "AI strategy" and the CTO responds with a roadmap of tools and APIs. A CFO asks about ROI and gets a slide about model accuracy. A CISO asks about security and gets a conversation about prompt injection that nobody else in the room can follow.

The gap is vocabulary. Not technical depth, vocabulary. You do not need to understand how a transformer is trained. You do need to understand what the word "hallucination" actually means, why it is not a bug that vendors can patch, and what it implies for any business process that routes AI outputs to customers or decision-makers.

This module gives you exactly that: the four concepts that unlock every other conversation about AI.

The executive test If you can explain these four things to a skeptical board member without using jargon, you will outperform most people who call themselves AI strategists: what tokens are, why models hallucinate, what fine-tuning actually does, and what an agent actually is.

Concept 1: Tokens and Context Windows

A large language model does not read text the way a person reads text. It processes sequences of tokens, where a token is roughly a word fragment: "contract" might be one token, "un-" might be another, and "believable" might be two. When you send a message to an AI system, it is converted into this sequence of tokens before the model sees it.

The context window is the number of tokens the model can process at once. Think of it as working memory. Everything the model knows about your specific conversation, your company's documents, your previous messages, has to fit inside this window. If your contract is 80,000 words and the model's window holds 100,000 tokens, the whole contract fits. If it does not fit, something gets cut.

Why this matters for the executive: when a vendor says their model "understands your documents," they mean it processes those documents within the context window at inference time. They do not mean the model has learned your documents permanently. If you send a document to the model today and do not send it tomorrow, the model has no memory of it. This distinction matters enormously for how you architect AI systems that use internal company knowledge.

Concept 2: Why Models Hallucinate

Hallucination is the single most important failure mode for executive decision-makers to understand, because it is structural, not a bug. No vendor can promise you a model that never hallucinates. Here is why.

A language model is trained to predict the next most plausible token in a sequence. It is extraordinarily good at producing text that sounds correct, cites things with confidence, and follows the expected structure of an answer. But it does not know what it does not know. When it reaches the edge of what its training data covered, it does not stop and say "I am uncertain here." It continues generating plausible-sounding text.

The result is confident incorrectness. A model asked about a specific legal precedent might cite a real case name, a plausible-sounding ruling, and a credible date, and all three might be wrong. The text will read as authoritative.

Cause 1

Training Cutoff

The model was trained on data up to a certain date. Events, regulations, and company data after that date are unknown to it unless provided in context.

Cause 2

Missing Context

The model does not have access to your internal data unless you provide it. When asked questions about your organization, it fills gaps with plausible-sounding guesses.

Cause 3

Plausibility Bias

The model is optimized to produce text that sounds correct, not text that is correct. Under uncertainty, it chooses the most statistically plausible continuation.

Implication

Design for Oversight

Any AI system where outputs reach customers, regulators, or high-stakes decisions without human review is a system with unquantified hallucination risk baked in.

Concept 3: Prompting, Fine-Tuning, and RAG

Three techniques determine how AI systems use company-specific knowledge. Confusing them is one of the most common and costly mistakes in enterprise AI procurement.

Prompting means giving the model instructions and context at the time of each request. You include the relevant document, the task description, and the constraints in the message. The model uses that context to generate a response. No training happens. The model is unchanged. This is the fastest and most flexible approach, and it is appropriate for a wide range of tasks.

Fine-tuning means taking a pre-trained model and continuing to train it on your data so that it learns the patterns, vocabulary, and behavior that are specific to your domain. The weights of the model change. After fine-tuning, the model has internalized your style, your terminology, and your domain knowledge. This is expensive, requires substantial labeled data, and is appropriate when you need consistent behavior at scale on a narrow task.

Retrieval-augmented generation (RAG) connects the model to a search system over your documents. At inference time, the system retrieves relevant documents and places them in the model's context window before generating a response. The model's weights do not change. This is the standard approach for giving an AI system access to company knowledge without fine-tuning.

The question to ask every vendor When a vendor says their system has been "trained on your data" or "knows your company," ask specifically: is this prompting, fine-tuning, or RAG? Each has different cost, latency, update frequency, and privacy implications. A vendor who cannot answer clearly is a vendor to interrogate further.

Concept 4: What an Agent Actually Is

The word "agent" is used to mean everything from a simple chatbot to a fully autonomous system that can make purchases, send emails, and execute code. This range is enormous and the distinction matters for your risk and governance decisions.

An AI agent is a system that uses an AI model to take actions, not just produce text. The agent observes a situation, decides on an action, executes that action using a tool (search, API call, database write, code execution), and then observes the result before deciding on the next action. This loop continues until the task is complete or the agent is stopped.

The executive implications are significant. An agent that can send emails on your behalf, place orders in your ERP system, or modify documents is an agent that can make consequential mistakes at scale, faster than a human can notice. The governance question for any agent deployment is not "does it work?" but "what is the blast radius if it gets it wrong, and who approved that?"

The Four Questions to Ask in Every AI Briefing

With these four concepts in hand, you can cut through vendor presentations and internal proposals with four questions that surface the issues that matter.

Question 1: Where does the knowledge come from? Is it in the model's training data, retrieved at runtime from a document store, or passed in the prompt? This determines how stale the knowledge can get, how private it is, and how the system handles updates to company data.

Question 2: What happens when the model is wrong? Is there a human review step before the output reaches a customer, a document, or a downstream system? If not, what is the expected error rate and who is accountable for errors that occur?

Question 3: Is this a model or an agent? If it is an agent: what tools does it have access to? What is the maximum damage it can cause in a single run? Who approved the scope of those tools?

Question 4: What does success look like in 90 days? Vendors will answer this question with accuracy numbers. You want process metrics: what specific business outcome changes, how it is measured, and what the baseline was before the AI was involved.

Python · Counting tokens before sending to an AI API

# Estimate token count before sending to an AI model
# Useful for checking whether your document fits the context window
# tiktoken is OpenAI's tokenizer; many other models use similar approaches

import tiktoken

def estimate_tokens(text: str, model: str = "gpt-4") -> int:
    """
    Returns approximate token count for a given text.
    One token is roughly 0.75 words for English text.
    """
    enc = tiktoken.encoding_for_model(model)
    return len(enc.encode(text))

# Example: check if a contract fits in context
with open("contract.txt", "r") as f:
    contract_text = f.read()

token_count = estimate_tokens(contract_text)
context_limit = 128_000  # typical large-context model limit

if token_count > context_limit:
    print(f"Document too large: {token_count} tokens exceeds {context_limit} limit.")
    print("Options: chunk the document, summarize sections, or use a larger-context model.")
else:
    remaining = context_limit - token_count
    print(f"Document fits. {token_count} tokens used, {remaining} tokens remaining.")

Try This

Before your next AI vendor briefing, write down in one sentence each: what problem you are solving, where the knowledge will come from, and what human review step exists before the AI output reaches a consequential endpoint. Share that one-paragraph brief with your team before the meeting. Watch how it changes the questions that get asked.