LLMs from Scratch
Advanced 30 min Module 2 of 6
Module 2 of 6

Tokenization and Embeddings

You type "unhappiness" and the model never sees that word. It sees three pieces: "un", "happi", "ness". Why? Because tokenization turns language into a vocabulary a neural network can manage, and the choice of how to split words has measurable consequences for cost, quality, and multilingual capability. By the end of this module you will understand exactly how BPE works, why vocabulary size is a design tradeoff, and how the model turns those tokens into vectors it can reason over.

By the end of this module you will be able to

Why Not Just Use Words?

The naive approach to representing text is to assign every word its own entry in a vocabulary. The English language has over 170,000 words in active use, and that is before you add technical terms, names, code, URLs, or any other language. A word-level vocabulary large enough to cover real-world text would be enormous. Worse, it would still fail on unknown words. "Tokenization" might be in the vocabulary; "tokenisation" (British spelling) and "tokenize" might not be.

Character-level tokenization fixes the unknown-word problem: every character is always in the vocabulary. But it creates a new problem. The sentence "The cat sat" becomes 11 tokens instead of 3. Sequences become much longer, which is expensive: attention cost scales quadratically with sequence length in the standard implementation. A model trained on character tokens needs to learn much longer-range dependencies to connect meaning.

Subword tokenization is the compromise. Split words into pieces such that common words stay whole, while rare words are broken into recognizable parts. "unhappiness" might become ["un", "happi", "ness"] or ["un", "happiness"] depending on the tokenizer. The model learns representations for these subwords, and can compose representations for words it has never seen by combining familiar pieces.

Byte Pair Encoding (BPE)

BPE was originally a data compression algorithm, adapted for NLP tokenization in Sennrich et al. (arXiv:1508.07909). The algorithm is iterative. Start with a vocabulary of individual characters. Find the most frequent pair of adjacent symbols in the training corpus. Merge that pair into a single new symbol. Add the merged symbol to the vocabulary. Repeat until the vocabulary reaches the desired size.

For example: starting with the text "low low low low lower lower newest newest newest widest widest", the most frequent adjacent pair might be "e" and "s". Merge them into "es". Now "es" is a single token and any word containing "es" will use one token for that pair instead of two. Repeat: find the next most frequent pair in the updated corpus. Continue until you reach, say, 50,000 merges.

The resulting vocabulary contains both individual characters and common subwords. Very common words appear as single tokens. Rare words decompose into familiar pieces. GPT models use a variant of BPE that operates on bytes rather than characters, which handles any Unicode text without special cases. This is why the GPT tokenizer can handle Japanese, Arabic, and emoji, even though those require multiple bytes per character.

Fig 1 · BPE Merge Process: First Three Iterations
STEP 0: CHARACTERS l o w l o w e r n e w e s t w i d e s t Vocab: a,b,c...z Most freq pair: e,s STEP 1: MERGE "e"+"s" l o w l o w e r n e w es t w i d es t Vocab: + "es" Most freq pair: es,t STEP 2: MERGE "es"+"t" l o w l o w e r n e w est w i d est Vocab: + "est" Most freq pair: l,o

WordPiece and SentencePiece

WordPiece, used in BERT and its derivatives, is similar to BPE but uses a different merge criterion. Rather than merging the most frequent pair, it merges the pair that maximizes the likelihood of the training data under a language model. In practice, this often produces similar tokenizations to BPE, but WordPiece's greediness in the merge step gives it different behavior on rare words.

A distinctive feature of WordPiece is its prefix notation. Word-initial subwords have no special marker. Subwords that appear inside a word are prefixed with "##". So "tokenization" might become ["token", "##ization"]. This allows a model to know, from the token itself, whether it starts a word or continues one.

SentencePiece (Kudo and Richardson, arXiv:1808.06226) takes a different approach. Rather than starting from words, it treats the entire input text as a raw stream of Unicode characters, including whitespace. Whitespace is represented by a special character (usually the "▁" symbol). This makes SentencePiece language-agnostic: it works equally well on languages that do not use spaces between words, like Japanese, Chinese, and Thai. It is the tokenizer used in many multilingual models including models in the T5 and LLaMA families.

Why Vocabulary Size Is a Design Tradeoff

Vocabulary size is not a free variable. It touches compute, quality, and cost simultaneously.

A larger vocabulary means fewer tokens per piece of text. "tokenization" is one token, not three. Sequences are shorter, which is good: shorter sequences are cheaper to process (attention cost is quadratic in sequence length), and the model needs to model fewer long-range dependencies. The embedding matrix is larger (vocabulary size times d_model), but this cost is usually smaller than the savings from shorter sequences.

A smaller vocabulary produces longer sequences for the same text. It also means more token collisions: similar strings might map to the same token, losing distinctiveness. On the other hand, every token in the vocabulary has been seen more frequently during training, so its representation is better learned. With a very large vocabulary, rare tokens may appear only a handful of times in training data, resulting in poor embeddings.

GPT-4's tokenizer uses a vocabulary of roughly 100,277 tokens. This is a deliberate choice: large enough that common English words and code constructs are single tokens, small enough that each token appears often enough in training to learn a good embedding. Multilingual models often need larger vocabularies to cover many scripts without excessive fragmentation of non-English text.

Practical implication Non-English text and code consume more tokens than equivalent English prose in many models. A 1,000-word essay in Japanese will use more tokens than the same essay in English, because many Japanese characters require multiple tokens under a vocabulary primarily built for English. This is why vocabulary size and composition are active research areas for multilingual models.

Token Embeddings: From Integers to Vectors

After tokenization, each token is an integer index. "the" might be token 262. "transformer" might be token 47221. These integers are fed into an embedding lookup table: a matrix of shape (vocabulary_size, d_model). Row 262 is the learned vector for "the". Row 47221 is the learned vector for "transformer".

The embedding matrix is initialized randomly and updated by gradient descent during training. After training, tokens with similar meanings or grammatical roles cluster together in the d_model-dimensional space. This is not programmed in; it emerges from the model minimizing prediction error. The word "happy" will end up close to "joyful" because they appear in similar contexts in the training data, and similar contexts produce similar gradient updates.

These embeddings capture relationships that surprised researchers when they were first studied. The famous example from Mikolov et al. (arXiv:1301.3781) on word2vec: the vector for "king" minus "man" plus "woman" is close to the vector for "queen". Linear algebra on token embeddings reflects analogical relationships in language. Modern LLM embeddings are richer and higher-dimensional, but the same principle applies.

Positional Embeddings: Adding Location

Token embeddings alone do not tell the model where a token appears in the sequence. The sentence "The dog bit the man" and "The man bit the dog" contain the same tokens in different order. Without positional information, the transformer would produce identical representations for both.

The original transformer added positional encodings using sine and cosine functions of different frequencies: position p and dimension i get PE(p, 2i) = sin(p / 10000^(2i/d_model)) and PE(p, 2i+1) = cos(p / 10000^(2i/d_model)). These fixed encodings allow the model to compute relative distances between positions from their encoded values.

Most modern models use learned positional embeddings: a separate embedding table with one vector per position, learned during training. GPT-2 uses learned positional embeddings up to a maximum sequence length (1,024 for GPT-2 small). More recent models use rotary positional embeddings (RoPE) or ALiBi, which encode relative rather than absolute position and generalize better to sequences longer than those seen in training.

BP
BPE
Merges the most frequent adjacent character pairs iteratively. Used by GPT models. Byte-level BPE handles all Unicode without special cases.
WP
WordPiece
Merges pairs that maximize training data likelihood. Used by BERT. Marks non-initial subwords with "##" prefix to preserve word boundary information.
SP
SentencePiece
Treats raw bytes including whitespace as input. Language-agnostic: handles Japanese, Chinese, Arabic, and other scripts equally well.
Interactive 2: Vocabulary Size Tradeoff Visualizer Try it

Adjust vocabulary size and sequence length to see how they interact with memory and compute requirements. A larger vocabulary reduces sequence length but increases the embedding matrix size.

50K
1024
Embedding matrix: -- params    At 4 bytes each: --
Python · Simple BPE Implementation
from collections import Counter

def get_pairs(vocab):
    """Count all adjacent symbol pairs in the vocabulary."""
    pairs = Counter()
    for word, freq in vocab.items():
        symbols = word.split()
        for i in range(len(symbols) - 1):
            pairs[(symbols[i], symbols[i+1])] += freq
    return pairs

def merge_pair(pair, vocab):
    """Merge the most frequent pair in all words."""
    merged = ' '.join(pair)
    new_vocab = {}
    for word, freq in vocab.items():
        new_word = word.replace(merged, ''.join(pair))
        new_vocab[new_word] = freq
    return new_vocab

# Starting vocabulary: characters + end-of-word marker
vocab = {
    'l o w </w>': 5,
    'l o w e r </w>': 2,
    'n e w e s t </w>': 6,
    'w i d e s t </w>': 3,
}

print("Initial vocabulary:", list(vocab.keys()))

# Run 5 BPE merges
for i in range(5):
    pairs = get_pairs(vocab)
    if not pairs:
        break
    best = pairs.most_common(1)[0][0]
    vocab = merge_pair(best, vocab)
    print(f"Merge {i+1}: {best} -> {''.join(best)}")
    print(f"  Updated: {list(vocab.keys())}")
Try this

Go to platform.openai.com/tokenizer and paste in three versions of the same content: a paragraph in English, the same paragraph translated to another language, and a block of Python code. Compare the token counts. You will find that code and non-English text typically use significantly more tokens per character than English prose. This is a direct consequence of vocabulary composition: most tokens in GPT tokenizers correspond to common English subwords.

Why does the same word sometimes cost different numbers of tokens?
Two reasons. First, whitespace: many tokenizers treat "hello" and " hello" (with a leading space) as different tokens, because the space is often part of the token. The word at the start of a sentence may tokenize differently than the same word in the middle. Second, capitalization: "Hello" and "hello" may be separate tokens, or "Hello" may decompose into "He" and "llo" while "hello" is a single token. These are genuine design choices in each tokenizer, not bugs. They reflect what patterns appear most frequently in the training corpus used to build the vocabulary.
Knowledge check
What is the key difference between BPE and SentencePiece tokenization?
Correct. SentencePiece works directly on the raw byte stream, treating whitespace as just another character. This makes it language-agnostic: it handles Japanese, Chinese, Arabic, and other scripts that do not use spaces without special preprocessing.
Not quite. The key distinction is that SentencePiece does not assume whitespace as a word boundary. It treats the raw text stream including spaces as input, which makes it work equally well across all languages and scripts.
If you double the vocabulary size while keeping the embedding dimension d_model constant, what happens to the embedding matrix?
Correct. The embedding matrix has shape (vocabulary_size, d_model). Doubling vocabulary_size while holding d_model constant doubles the number of parameters in the embedding table. This is one real cost of a larger vocabulary.
Not quite. The embedding matrix is vocabulary_size x d_model. If vocabulary_size doubles and d_model is unchanged, the matrix has twice as many rows and thus twice as many parameters. Sequence length is a separate effect of vocabulary size.
Before you go
Reflection: Why might a model trained on English text perform worse on code, even though code is often in English characters?
You might also like
Was this module helpful?

Need help with tokenization strategy for a multilingual or domain-specific LLM deployment?

Schedule a call with Arjun

References

  1. Sennrich, R. et al. "Neural Machine Translation of Rare Words with Subword Units." arXiv:1508.07909. arxiv.org/abs/1508.07909
  2. Kudo, T. and Richardson, J. "SentencePiece: A simple and language independent subword tokenizer." arXiv:1808.06226. arxiv.org/abs/1808.06226
  3. Devlin, J. et al. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." arXiv:1810.04805. arxiv.org/abs/1810.04805 (WordPiece)
  4. Mikolov, T. et al. "Efficient Estimation of Word Representations in Vector Space." arXiv:1301.3781. arxiv.org/abs/1301.3781
  5. Su, J. et al. "RoFormer: Enhanced Transformer with Rotary Position Embedding." arXiv:2104.09864. arxiv.org/abs/2104.09864
  6. Press, O. et al. "Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation." arXiv:2108.12409. arxiv.org/abs/2108.12409 (ALiBi)
← Module 1: How Transformers Work Module 3: Pretraining →