Step 12 — Teach it to follow instructions - Build Your Own Language Model

Goal: turn a model that continues text into a model that answers questions, and then into one that follows your rules.

Why this step matters¶

Your model from Step 11 predicts what comes next. Ask it a question and it may well reply with more questions, because that is what it saw in the data.

Supervised fine-tuning (SFT) fixes this. You show it thousands of examples of the pattern “someone asks, someone answers well”, and it learns the shape of that pattern.

What you do¶

1. Get instruction data in Sanskrit¶

This is the hard part. Options, roughly best to worst:

Written by native speakers and scholars. Expensive, slow, and by far the best.

Translated from English instruction datasets. Cheap. But translated instructions produce a model that sounds translated — the phrasing, the assumptions, and the register are all wrong.

Generated by a large model. Cheapest. Check the licence, and check the quality by hand on a sample. Generating question-and-answer pairs from your own corpus works better than generating them from nothing, because the content is grounded in real text.

Mix them. But note: a few thousand genuinely excellent examples beat a hundred thousand mediocre ones. This is not a slogan, it has been measured repeatedly.

Task types worth including for Sanskrit:

Translate this passage into English
Analyse the grammar of this line
Split this compound
Answer this question about this text
Write in the style of this text

2. Choose a chat template and never change it¶

The chat template is the special markup that says where the user’s turn starts and the assistant’s turn starts.

If you train with one template and serve with a different one, the model breaks in confusing ways that look like a quality problem rather than a formatting problem.

This is one of the most common bugs in the whole field.

3. Mask the loss on the prompt¶

Train the model only on the answers, not on the questions. You do not want it learning to generate user questions.

4. Choose full fine-tuning, LoRA, or QLoRA¶

Here the advice differs from Step 11, and the reason matters:

LoRA works very well for teaching new behaviour — a style, a format, a way of responding.
LoRA works less well for teaching new knowledge.

Step 11 already gave the model the Sanskrit knowledge. This step is teaching behaviour. So LoRA is a good choice here, and full fine-tuning would be a good choice there.

5. Do the same for Urdu¶

Use a separate adapter so you can compare, and so you can serve either one from the same base model.

Where people usually get stuck¶

The chat template mismatch.

Print your fully formatted training example, character by character, including every special token. Then print what your serving code produces. Compare them.

They must be identical. Not similar. Identical.

You are ready to move on when¶

Your model answers a Sanskrit question with a Sanskrit answer, in the right format, without wandering off into unrelated text.

🧑‍💻 Runnable code for this step¶

Instruction tuning (SFT) changes the shape of your data from raw text to pairs. Keeping prompt and completion separate is what lets the trainer score the answer only, not the parroted question:

# one line of sanskrit_sft.jsonl
{"prompt": "Translate this Sanskrit line into English:\nअहिंसा परमो धर्मः।",
 "completion": "Non-violence is the highest virtue."}

from trl import SFTTrainer, SFTConfig
from peft import LoraConfig

cfg = SFTConfig(output_dir="./adapter", num_train_epochs=3,
                per_device_train_batch_size=2, gradient_accumulation_steps=8,
                learning_rate=2e-4, max_length=1024,
                completion_only_loss=True,     # ← score the answer, not the question
                bf16=True, report_to="none")

trainer = SFTTrainer(model=model, args=cfg, train_dataset=ds,
                     peft_config=LoraConfig(r=16, lora_alpha=32,
                                            task_type="CAUSAL_LM",
                                            target_modules="all-linear"),
                     processing_class=tokenizer)
trainer.train()

💼 Instruction tuning for a business (Season 2)¶

Everything above still applies. A business adds one more demand: the model must follow rules that someone else wrote, every time, in the customer’s own language and script. This section is the design guide behind Season 2’s three language episodes.

1. Write the rules down as a system prompt first¶

Before any data, write the client’s rules in plain language, in prompts.json. From Episode 2:

Answer ONLY from the POLICY text. Never invent amounts, dates or coverage. Reply in the customer’s own style: Hindi in Devanagari if they wrote in Devanagari, Hinglish in Latin letters if they wrote in Latin letters. … If the customer is complaining, threatening legal action, asking to cancel, or describing a medical emergency, end with the tag [ESCALATE].

Three reasons to start here:

The client can read and approve it. Nobody can approve a weight file.
Every training example then demonstrates the rules. The adapter learns to obey the prompt, not to memorise answers.
Changing a rule later (a new phone number, a new priority) often needs only a prompt edit and a short re-train, not a new dataset.

2. Make every important behaviour measurable¶

A rule you cannot check is a rule you cannot sell. Season 2 turns each rule into something a script can grade:

Business rule	How it becomes measurable
hand angry, legal or emergency cases to a human	reply ends with `[ESCALATE]`
never invent a source	reply starts with `[NOT_FOUND]` for unknown verses
the bank’s code routes on the intent	reply is JSON: `{"intent": ..., "reply": ...}`
Hinglish in, Hinglish out	most letters of the reply are in the expected script
cite the exact verse	the chapter.verse reference matches

You will see each of these again in Step 10, where the grading code lives.

3. One human-readable seed format¶

Clients send spreadsheets, not trainer files. Season 2 keeps one simple format that a native speaker can review line by line:

{"id": "train-support-hi-latn-03", "task": "support", "lang": "hi-latn",
 "input": "POLICY: ...\nCUSTOMER: claim status kaise check karu?",
 "output": "Aap app ke 'My Claims' section mein status dekh sakte hain, ...",
 "meta": {"checks": ["chrf", "script:auto", "escalate"], "script": "latin"}}

common/csv_to_seed.py turns a client’s CSV into this format; common/make_dataset.py validates it and converts it to the chat prompt/completion format the trainer needs. Two separate files, two separate jobs: the seed file is for people, the trainer file is for the machine.

4. Code-mixed and romanized text is the normal case¶

Most real customers in South Asia do not type in textbook script. They write Hinglish, Roman Urdu, “Tanglish”, “Tenglish” and “Kanglish”: their language in Latin letters, with English words mixed in, and no standard spelling.

Put romanized examples in training and in the test set, as their own language code (hi-latn, ur-latn, ta-latn). Otherwise you will never notice the model failing on half your real traffic.
Normalize spelling variants before training and at serving time. Episode 3’s normalize.py maps bht, bohot and bahut to one form, and fixes Arabic look-alike letters inside Urdu script. Run the same normalizer on training data, test data and live messages.
Reply in the customer’s script. Make it a rule and a check, not a hope.

5. Teach the model when not to answer¶

Every Season 2 dataset includes examples where the right answer is to refuse or hand off: questions the policy does not cover, angry customers, verses outside the catalogue (including one made up on purpose). Without these, a fine-tuned model becomes more confidently wrong, because every example it saw had an answer.

A good starting mix: roughly one example in four or five should be a hand-off or refusal.

6. Start with dozens, grow to hundreds, keep the test set human¶

The seed sets in Season 2 are small on purpose (21 to 46 training examples per episode). They prove the recipe. A client project grows them:

Collect real questions (FAQ, past tickets) → csv_to_seed.py.
Expand the phrasing with a bigger “teacher” model → expand_with_teacher.py (Step 17). The answers stay human-approved.
Have a native speaker review the synthetic rows.
Never put synthetic rows in the test set. You would be grading the student on the teacher’s homework.

7. Watch the token budget in Indian scripts¶

The same sentence can cost 2–4 times more tokens in Tamil or Kannada script than in English (Step 3). A long system prompt plus a long policy text plus a long answer can exceed max_length, and the trainer then cuts off the end of the answer, which teaches the model to stop mid-sentence. train_lora.py prints the median and maximum tokens per example and warns you before this happens.