Goal: understand speech models, and see why this may be the highest-impact area of all.
Why this step matters¶
Many people who speak South Asian languages read them less easily than they speak them. Voice is the natural interface. And speech recognition quality for most of these languages is poor.
This is where a small team can do the most good.
For Sanskrit there is something more specific. Sanskrit is a recited language. Chanting follows strict rules of metre and pronunciation that have been preserved orally for thousands of years. That structure is both a challenge and an unusual opportunity: you have rule-based ground truth for what the audio should sound like.
What to cover¶
Speech recognition. Encoder-decoder models that turn audio into text. The Whisper family is the usual starting point.
Self-supervised speech representations. Models trained on unlabelled audio, which matters a lot when labelled audio is scarce.
Discrete audio tokens. The key idea for you: audio can be turned into a sequence of discrete tokens using a neural codec. Once it is tokens, everything you learned about text applies directly — same architecture, same training, same tricks.
Text to speech. Generating Sanskrit recitation with correct metre.
End-to-end speech models. Audio in, audio out, with no text step.
Sanskrit-specific opportunities:
Recitation checking — does this chant follow the correct metre and pronunciation?
Aligning existing recordings with existing texts to build training data
Pronunciation teaching tools