The question.
Nepali model quality cannot be established with English-only benchmarks, translated prompts alone, or a single perplexity number.
The research compares adapting an open-weight multilingual model with a smaller from-scratch baseline. It treats data provenance, compute accounting, and native-speaker assessment as first-class outputs.
The proposed approach.
- Curate licensed Nepali and bilingual text with deduplication and benchmark decontamination
- Compare tokenizer fertility and comparable byte-level loss before changing the vocabulary
- Run compute-matched adaptation baselines with English replay to study forgetting
- Compare supervised tuning and preference optimization using held-out Nepali tasks
Target architecture.
Provenance-aware corpusTokenizer studyTraining runsInstruction tuningNepali evaluationModel card
The difficult parts.
- Limited high-quality Nepali instruction data
- Benchmark contamination and translated-test artifacts
- Balancing Nepali gains against multilingual retention
How to evaluate it.
- Reading comprehension, grounded QA, instruction following, and code-switching
- Native-speaker judgments with a documented rubric
- Ablations for tokenizer changes, corpus mixtures, and compute cost
Planned deliverables.
- A corpus and data-provenance report
- Reproducible training configurations and checkpoints if the study is executed
- An evaluation protocol and model-card template
Design principles.
Measure Nepali capability directly.
A training recipe is more useful when its data and evaluation are reproducible.