What Is Prompt Engineering?

Most people think prompting is just typing questions. The reality is that a language model is a complex probability machine, and how you phrase your input changes the probability distribution over every possible output. Prompt engineering is the practice of designing those inputs carefully, so that the outputs you want become far more likely than the outputs you do not.

This is not a minor distinction. Two people asking what looks like the same question can get radically different results because one phrased it in a way that matched the model's internal patterns and the other did not. The model does not read your intention. It reads your words, the order they appear in, the examples you included or omitted, and the framing you chose. Every one of those factors shapes the output.

This post explains what prompt engineering actually is, why the same question worded differently produces different answers, and what the discipline covers at a foundational level. If you want to move from knowing this conceptually to applying it systematically, the free course at Prompt Engineering Fundamentals covers everything here and goes much deeper.

What a Prompt Actually Is

A prompt is the text you send to a language model. That text is converted into tokens, a sequence of numeric identifiers, and passed through the model's layers. The model then produces a probability distribution over every possible next token, samples from that distribution, and repeats the process until it generates a complete response or reaches a stopping condition.

This means the model is not "thinking about your question" in any human sense. It is running a computation over the numeric representation of what you wrote. The computation is highly sensitive to the input: different phrasing activates different internal patterns, which shifts the probability distribution, which changes what comes out. That sensitivity is the foundation of prompt engineering.

A complete prompt usually contains several distinct components, though not all are always visible to you as the user. The most important ones are the system prompt, the user message, and any context you provide alongside them.

The system prompt

The system prompt is a set of instructions that sits at the start of the conversation and shapes how the model behaves throughout. It is usually set by the developer or application, not the end user. A customer support chatbot might have a system prompt that says "You are a helpful support agent for [Company]. Respond only to questions about our product. Keep your tone professional and concise." That instruction colors every response the model generates, even though the user never sees it.

Understanding system prompts matters because they reveal why the same underlying model can behave very differently depending on the application wrapping it. The model you access through one product might feel formal and cautious. The same model through a different product might feel creative and expansive. The system prompt is often the explanation.

The user message

The user message is what you type. It carries the most weight in determining what the model says next, because it is the most recent context the model processes. But "most recent" does not mean "most important in isolation." The model attends to the entire prompt context, including the system prompt, prior conversation turns, and any documents you have included. The user message is the final instruction layered on top of everything else.

Context and documents

Modern language models can handle long contexts, sometimes hundreds of thousands of tokens. This means you can include reference documents, code, tables, or prior outputs inside the prompt itself. What you include in that context is a design decision. Including irrelevant material can confuse the model. Including the right material can dramatically improve response quality. Selecting and structuring context is one of the most practically valuable prompt engineering skills.

Why the Same Question Gets Different Answers

If you ask a language model "summarize this article" and then ask "provide a three-sentence summary of this article aimed at a non-technical reader," you will get different outputs. Both requests are about summarizing. The second one is almost certainly more useful. The difference is specificity, but the mechanism behind that difference is worth understanding.

The model generates each token by sampling from a probability distribution shaped by everything it has seen in its training data and everything you wrote in your prompt. Certain patterns in your prompt activate learned associations from training. "Three-sentence" connects to patterns where the model has seen structured, constrained summaries. "Non-technical reader" connects to patterns where the model has generated explanations for general audiences. Both signals shift the distribution toward more useful outputs.

This is why vague prompts tend to produce vague answers. A prompt like "tell me about AI" leaves most of the distribution open. The model samples from a vast range of possible responses about AI, and the one it lands on may not match what you actually wanted. A more constrained prompt narrows the distribution toward the outputs you are looking for.

The model does not read your intention. It reads your words, the order they appear in, and the patterns they activate from training.

Zero-Shot vs. Few-Shot Prompting

Two foundational techniques in prompt engineering are zero-shot and few-shot prompting. Understanding the difference between them is essential for anyone working with language models seriously.

Zero-shot prompting

Zero-shot prompting means giving the model a task with no examples. You describe what you want and let the model rely entirely on what it learned during training. For many straightforward tasks, this works well. Ask a capable model to translate a sentence, classify a sentiment, or answer a factual question, and it can do so without seeing any demonstration of the task format.

Zero-shot prompting is fast and flexible. Its limitation is that for novel or nuanced tasks, the model may not interpret the task the way you intended. Without an example, the model has to infer what "correct" looks like from the description alone. That inference can go wrong in subtle ways.

Few-shot prompting

Few-shot prompting adds examples to the prompt. You show the model one or more input-output pairs that demonstrate the task, and then present your actual input. The model uses those examples as a template for its response. Brown et al. (arXiv:2005.14165) demonstrated that adding even a small number of examples to a prompt can dramatically improve performance across a wide range of tasks, without changing the model's weights at all.

The examples do not have to be drawn from the exact domain you are working in. They just need to illustrate the pattern you want. If you are asking the model to extract specific fields from unstructured text, showing it two examples of input text and the correctly extracted fields tells it far more than any description of the task could.

Few-shot prompting is one of the most reliable techniques you have access to. When a zero-shot prompt gives you inconsistent results, adding examples is usually the first thing to try.

Temperature and Why Models Are Non-Deterministic

You may have noticed that asking a language model the same question twice sometimes produces different answers. This is not a bug. It is a feature controlled by a parameter called temperature.

Temperature controls how the model samples from the probability distribution over possible next tokens. At a low temperature (close to 0), the model almost always picks the highest-probability token, making its outputs highly consistent and deterministic. At a higher temperature, the model samples more broadly, introducing variety into its outputs. This is useful for creative tasks, where a single "best" answer does not exist, and less useful for factual tasks, where you want the model to reliably produce the correct answer.

Most APIs and interfaces expose temperature as a setting you can adjust. For structured tasks like data extraction, classification, or code generation, a lower temperature produces more reliable results. For creative writing, brainstorming, or generating diverse options, a higher temperature encourages variation.

Understanding temperature helps explain some of the puzzling behavior people observe when using language models. The model that gave you a perfect answer yesterday might give you a slightly different one today if temperature is set above zero. That variability is not a malfunction. It is the probability distribution doing what it is designed to do.

What Prompt Engineering Is as a Discipline

Prompt engineering is the systematic practice of designing, testing, and refining prompts to get reliable, high-quality outputs from language models. It sits at the intersection of linguistics, software engineering, and applied cognitive science.

As a discipline, it has developed rapidly since large language models became widely accessible. Researchers have formalized techniques that practitioners discovered through trial and error: chain-of-thought prompting, which asks the model to reason step by step before answering (Wei et al., arXiv:2201.11903); self-consistency, which generates multiple reasoning paths and aggregates the results (Wang et al., arXiv:2203.11171); least-to-most prompting, which breaks complex problems into subproblems and solves them in order (Zhou et al., arXiv:2205.10625); and the ReAct pattern, which interleaves reasoning and tool use (Yao et al., arXiv:2210.03629).

Each of these techniques emerged from the observation that how you structure a prompt, not just what you ask, significantly affects what the model produces. That observation has been validated across models, tasks, and domains.

What prompt engineering is not

Prompt engineering is not a set of magic words or secret tricks that unlock hidden model capabilities. It is not manipulation in the way that jailbreaking attempts are. It is not a substitute for model selection, fine-tuning, or proper system design when those are the right tools for the job.

What it is, more precisely: the application of structured thinking about how language models work to the design of inputs that reliably produce the outputs you need. At scale, the difference between a prompt that works 60% of the time and one that works 95% of the time can mean thousands of hours of human review or the viability of an entire workflow.

Prompt Sensitivity: The Core Challenge

One of the most counterintuitive properties of language models is their sensitivity to small changes in wording. Adding a period at the end of an instruction, changing "list" to "enumerate," or moving the context before the question instead of after it can all change the output in ways that are hard to predict in advance.

This sensitivity is not fully understood even by researchers who build these models. It arises from the way the models are trained: on human-generated text where subtle differences in phrasing carry real semantic differences. The model has learned to treat those differences as meaningful, because in the training data they were. Transferring that sensitivity to your specific use case requires testing, iteration, and often a degree of patience.

The practical implication is that prompt engineering is empirical. You form a hypothesis about what will work, test it against real inputs, evaluate the outputs, and refine. The best practitioners treat it as a structured experiment, not a one-time creative exercise.

Prompt Injection: A Risk Worth Knowing

As you work with language models in any context where external inputs are involved, be aware of prompt injection. This is a class of attack where malicious text included in the input attempts to override the instructions in the system prompt. Greshake et al. (arXiv:2302.12173) documented this pattern systematically: an attacker who can control what content the model sees can potentially redirect its behavior in ways the application developer did not intend.

This matters practically for anyone building applications that process user-submitted content, web pages, or documents. Understanding what prompt injection is, and that it exists, is the first step toward designing prompts and systems that are more resistant to it.

Knowledge Check

Most people think prompting is just typing questions.

It's not. Prompt engineering is the practice of designing inputs so that the outputs you want become far more likely.

Key things I learned from this post:
- The system prompt you never see shapes every response
- Few-shot prompting (showing examples) dramatically improves consistency
- Temperature controls how deterministic vs. varied the model's outputs are
- Prompt engineering is empirical: form a hypothesis, test it, iterate

If you're using AI tools without thinking about this, you're leaving a lot on the table.

Full explanation: https://arjunjaggi.com/blog/what-is-prompt-engineering