Every technical term used in this book, in plain words.
Activation function — a small maths function applied between layers that lets a network learn non-straight-line patterns. SwiGLU is the common one now.
Adapter — a small set of extra weights added to a frozen model. LoRA produces adapters.
Attention — the mechanism by which each token looks at earlier tokens and decides which ones matter for predicting the next one.
Base model — a model trained only to predict the next token. It has not yet been taught to follow instructions.
bf16 — a 16-bit number format that keeps a wide range of magnitudes at the cost of precision. The default for language model training.
BPE (Byte Pair Encoding) — a tokenizer method that repeatedly joins the most common pair of neighbouring pieces into one new token.
Break-even volume — the number of requests per month above which running your own model becomes cheaper than paying a hosted API.
Calibration data — a small sample of text used during quantization to decide which weights matter most.
Catastrophic forgetting — when teaching a model something new makes it forget things it used to know.
Causal mask — a block that stops the model from seeing future tokens, so it cannot cheat by copying the answer.
Chat template — the special markup that marks where the user’s turn ends and the assistant’s begins.
Checkpoint — a saved copy of the model’s weights partway through training.
chrF — a score from 0 to 100 for how close a model’s answer is to the approved answer, counted in small pieces of characters. Works well for Indian languages, where word-based scores break on sandhi and spelling variants.
Code-mixing — switching between languages inside one message, as in Hinglish (“claim status kaise check karu?”). Normal in real customer messages.
Contamination — when test data has accidentally ended up in the training data, making evaluation scores meaningless.
Context window — how many tokens the model can look at in one go.
Continued pretraining (CPT) — carrying on the pretraining of an already-trained model, usually on new data such as a new language.
Cross-lingual retrieval — finding a passage in one language or script from a question in another, e.g. a Tamil question finding an English policy paragraph.
Decode — the slow part of generation: writing the answer one token at a time.
Distillation — training a small model to copy a larger one.
DPO (Direct Preference Optimization) — training on pairs of better and worse answers, without needing a separate reward model.
Embedding — the vector of numbers that represents one token.
Embedding model — a model that turns a whole sentence or passage into one vector, so that texts with similar meaning get similar vectors. Used for search and RAG.
Escalation tag — a fixed marker such as [ESCALATE] that a support model adds when a human must take over. Making it a tag makes it measurable.
Fertility — how many tokens one average word becomes. Lower is better.
Fine-tuning — further training of an existing model on new data.
FlashAttention — a way of computing attention that never writes the full attention matrix to memory. Same result, much less memory.
Frontier API — a large, general model from a big provider, used over the internet and paid per token.
FSDP (Fully Sharded Data Parallel) — splitting the model itself across several GPUs so a large model fits.
Gateway — the program between customers and the model server that cleans input, adds the client’s rules, checks the output, routes urgent cases to people and keeps masked logs.
GGUF — a file format for quantized models, used by llama.cpp for local running.
GQA (Grouped-Query Attention) — several query heads share one set of keys and values, which saves a lot of memory when serving.
Gradient accumulation — running several small batches and adding up their gradients before updating, to imitate a large batch.
Gradient checkpointing — throwing away intermediate values during the forward pass and recomputing them later, to save memory.
GRPO (Group Relative Policy Optimization) — generate several answers, score them, and push the model toward the ones that scored above average.
Head — one independent attention operation. Models run several in parallel.
Hybrid search — combining meaning-based search (embeddings) with letter-based search, so both paraphrases and exact codes are found.
Importance matrix — llama.cpp’s calibration step: sample text shows which weights matter most, and quantization protects those. Use text in your own languages.
KV cache — stored keys and values from earlier tokens, kept during generation so they are not recomputed.
LoRA (Low-Rank Adaptation) — training small add-on matrices instead of updating the whole model.
Memory bandwidth — how many gigabytes per second a device can read from memory. On small devices it sets the writing speed: roughly bandwidth divided by model size.
Merging an adapter — adding a LoRA adapter’s small matrices into the base model’s weights for good, producing one ordinary model file.
MoE (Mixture of Experts) — many small expert layers with a router that uses only a few of them per token.
MRR (Mean Reciprocal Rank) — a search score: 1 if the right answer is ranked first, 1/2 if second, 1/3 if third, averaged over all questions.
Multi-LoRA serving — one server holding one base model and many LoRA adapters, choosing the adapter per request: one GPU serving many clients.
muP — a way of setting hyperparameters on a small model so they transfer correctly to a large one.
Neuro-symbolic — combining a neural model with explicit rule-based code.
Normalization — rescaling values so they stay in a sensible range. RMSNorm is the common one now.
Overfitting — when a model learns your specific training file instead of general patterns. Also called memorisation.
p95 latency — the response time that 95% of requests beat; 1 request in 20 is slower. Closer to what customers complain about than the average.
Panini — the grammarian who codified Sanskrit in the Ashtadhyayi, around four thousand rules that form a near-formal system.
Perplexity — a measure of how surprised a model is by some text. Lower is better.
PII (Personally Identifiable Information) — data that identifies a person: phone, card, national ID numbers, email. Mask it in logs.
Pre-tokenization — the rough splitting of text, usually by a regular expression, that happens before the tokenizer’s main algorithm runs.
Prefill — the fast part of generation: reading the whole prompt at once before writing anything.
Pretraining — the first, large training run on raw text.
Projector — the small network that maps image or audio vectors into a language model’s space.
QLoRA — LoRA applied on top of a quantized model. The cheapest way to fine-tune.
Quantization — storing weights with fewer bits to save memory.
RAG (Retrieval-Augmented Generation) — searching your documents and putting the relevant passages into the prompt, instead of expecting the model to know everything.
Recall@k — the share of questions whose right passage appears in the top k search results.
Reciprocal rank fusion (RRF) — a simple way to merge several ranked lists: each item scores the sum of 1 / (60 + rank) over the lists.
Replay data — general data mixed into specialised training to stop the model forgetting what it already knew.
Residual connection — a side road that lets information skip past a block. Essential for training deep networks.
Reward hacking — when a model finds a way to score well without actually being better.
RLHF (Reinforcement Learning from Human Feedback) — training a reward model on human preferences, then optimising against it.
RLVR (Reinforcement Learning with Verifiable Rewards) — using an automatic checker as the reward, for tasks where correctness can be verified.
RMSNorm — a simpler, faster form of normalization that skips subtracting the mean.
Romanized text — a language written in Latin letters instead of its own script, such as Roman Urdu or “Tanglish”. Usually has no standard spelling.
RoPE (Rotary Position Embeddings) — encoding position by rotating query and key vectors, so the model learns about distance between tokens.
Router — the small network in an MoE model that decides which experts handle which token.
Samasa — a Sanskrit compound word, formed by joining several words.
Sandhi — the Sanskrit rules by which sounds change where words join.
Seed data — the small, human-written and human-reviewed set of examples a project starts from.
SFT (Supervised Fine-Tuning) — training on example question and answer pairs.
SwiGLU — a gated activation function, standard in modern models.
Sycophancy — when a model learns to agree with the user rather than be correct.
Synthetic data — examples written by a model rather than a person. Useful for growing training data; never use it as test data.
System prompt — instructions placed before the user’s message that set the rules the model must follow, such as a company’s support policy.
Teacher and student — in distillation, the large model that produces examples (teacher) and the small model trained on them (student).
Token — one piece of text, as cut by the tokenizer.
Tokenizer — the tool that cuts text into tokens.
Transliteration — rewriting text in another script without translating it, such as Roman Urdu “card” to Urdu script “کارڈ”.
TTFT (Time To First Token) — how long until the first word of a reply appears. What makes a chat feel responsive.
Unicode normalization — converting text so that visually identical strings are stored identically. NFC is the usual form.
Vector index — stored embeddings of every passage, searched by similarity. For small collections, just a matrix.
Virama — the Devanagari mark that removes a consonant’s inherent vowel, used to form conjunct letters.
ZWNJ (zero-width non-joiner) — an invisible character that stops two letters from joining. Common and inconsistent in Urdu text.