Back to Articles a]:hidden"> How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications Team Article Published October 21, 2026 Upvote 16 +10 Bernardo França de Aguiar bernardoaguiar Follow Dharma-AI Gabriel Pimenta de Freitas Cardoso GabrielPimenta99 Follow Dharma-AI Bruno Duarte brunoduarte01 Follow Dharma-AI Francisco de Almeida Rocha Alves falves9101 Follow Dharma-AI :last-child]:mb-0"> A Vocabulary Built for Everyone Reading Is Cheap, Writing Is Expensive Why Existing Solutions Fell Short How We Solved It What Changed in the Grader Beyond the Grader Paying Only for the Tokens That Matter Sources Further Reading Models specialized for a domain or a task by fine-tuning general open-source models, such as Qwen and Gemma, inherit a vocabulary built for many domains and general applications
In an essay grader we run in production, that vocabulary split the text the model writes most into many tokens. This article previews some results from the VocabX paper, which we will publish soon: how we specialized the vocabulary around the cost of each token rather than its frequency, and cut measured inference cost by 77% without changing the base model
The tokenizer is usually treated as a fixed property of the base model, inherited and left alone. For a model specialized in a single domain, it can be one of the largest cost levers, and it is set before any training step
