Training Data Bias Compounds at Every Layer for Underrepresented Languages

A new paper from Avijit Roy and Proma Roy, published on arXiv on 12 August 2026, maps where AI infrastructure fails speakers of underrepresented languages. The answer: everywhere upstream. Tokenization, training corpora, and evaluation benchmarks each impose penalties that compound. Bengali serves as the case study—a language spoken by roughly 4% of the global population, with numbers specific enough to be operational.

Bengali accounts for less than 0.5% of global web content, an 8:1 under-representation against its population share. In major multilingual training corpora, the English-to-Bengali training-token deficit sits at 67:1. A model trained on standard multilingual data sees 67 English tokens for every Bengali token before a single weight updates. This is not downstream bias from RLHF or fine-tuning; it is baked into the data pipeline.

Tokenization adds a second penalty. Bengali uses an alphasyllabary script that BPE tokenizers trained on English-heavy corpora fragment into significantly more tokens per word—higher fertility—than equivalent English text. The Token Tax paper (analyzing 16 African languages across 10 LLMs) confirms the mechanistic consequence: fertility reliably predicts accuracy, with higher fertility correlating with lower accuracy. The reason is structural. Attention compute scales quadratically with sequence length. Doubling token count quadruples training cost and inference time. A 67:1 data deficit combined with elevated fertility means Bengali-language tasks face both under-training and computational penalty on every standard transformer.

A companion audit comparing GPT-4o, Qwen2.5-7B, and Mistral-7B found tokenization penalties large enough to matter for underserved-language educational AI—including Bengali offline tutoring systems and commercial API tools. The paper also flags Yoruba, a Latin-script language with tonal diacritics, produces higher GPT-4o token fragmentation than Bengali in the same corpus. Non-Latin script is not the only vulnerability; any script poorly covered in tokenizer vocabulary training data is exposed.

The connectivity layer compounds the problem. Internet penetration in rural Bangladesh sits at 36.5% versus 71.4% urban, meaning the population most dependent on offline-first AI tools is least represented in training corpora. Roy and Roy argue offline-first deployment should be treated as equity infrastructure, not a low-bandwidth workaround. Quantized models and edge inference are not niche optimizations; they are the only viable delivery mechanism for users without reliable connection.

For teams building multilingual inference stacks, three audit points emerge. First, measure tokenizer fertility ratios per target language before selecting a tokenizer—do not infer from English benchmarks. GPT-4o, Qwen2.5, and Mistral diverge meaningfully on non-Latin scripts, and the choice propagates through context window size, memory pressure, and API costs. Second, audit corpus composition against population share, not just data availability. A 67:1 ratio is not a budget constraint; it is a design choice that persists through the model's lifetime. Third, evaluation benchmarks built primarily on English or high-resource languages will not surface these gaps. A model that scores 90% on MMLU can still fail systematically on morphologically complex input from underrepresented languages.

The regulatory angle is live. EU GPAI obligations active since August 2025 require providers to publish training data summaries. EU AI Act Article 10, which governs high-risk systems including AI in education (full enforcement December 2027), mandates that training datasets be sufficiently representative and examined for bias that could discriminate—with penalties up to €15M or 3% of global annual turnover. Structural language under-representation at corpus level is precisely the gap a representativeness audit will surface.

Structural bias in multilingual AI does not begin at inference and cannot be patched at fine-tuning. Audit the tokenizer, audit the corpus, and expect regulators to ask for documentation of both.