With the right training pipeline, you can train a compact classifier that competes with the big-scale content-moderation models.
Mistral AI asserts this about its latest creation: Shieldstral. In the context of joining the Open Secure AI Alliance, the weights have been released under the Apache 2.0 license.
The model is derived from Ministral-3B-Base-2512. Multimodal (text and/or image); it is described as “adaptive” in the sense that you can adjust the moderation policy during inference without retraining. On paper, this property favors deployment across a wider range of contexts (content that is acceptable in one scenario may be undesirable in another). It sets Shieldstral apart from the Mistral Moderation API, which does not offer this level of flexibility.
Unifying Training Datasets
To develop this capability, Mistral AI reduced moderation tasks to closed-ended questions expressed in natural language. For example: “Does this content promote violence against a minority?” or “Can this image be shown to a minor?”. Shieldstral answers with a score, without relying on a fixed taxonomy.
In each request, the user prompt is structured into three fields:
- <Instruct>
Constant on a task or a given dataset, it describes the evaluation context and the level of rigor. - <Query> (closed-ended question about the content to evaluate)
- <Document> (the content, whether text, image, or both)
Adopting this format for Shieldstral required unifying existing training datasets, which were highly heterogeneous in both annotation conventions and taxonomies. Mistral AI performed a template-based conversion, developing a dataset-specific processing pipeline for each dataset. It ensured the model learns to calibrate decisions according to the required level of rigor. And that, through training, it encounters a variety of instruction formulations to acquire stronger generalization capabilities.
Shieldstral, three checkpoints for a model
The other centerpiece in training Shieldstral was the use of contrastive training pairs. On one side, positive examples tied to three types of queries: binary ones (“Is this message undesirable?”), category-specific ones (“Does this content promote suicide?”) and, where applicable, group-targeted ones (“Does this message promote violence against children?”). On the other side, negative examples, with three strategies to generate them:
- Based on categories (we map content violating category A to queries about categories B, C, etc.)
- Based on demographics (we map content targeting one group A to queries about other groups)
- By combining safe examples with binary queries about undesirable content
This approach taught the model not only to detect undesirable content but to distinguish between categories. Mistral AI also leveraged it to refine the interpretation of moderation policies, encouraging it to pay attention to semantic details rather than relying on cues tied to categories.
In the face of the scarcity of image moderation datasets, Mistral AI added classification and object-detection datasets. Their examples served as negatives… but also as positives, by reconfiguring the queries with the help of a large language model.
The training of Shieldstral did not rely on supervised fine-tuning, but on the LoRA method. It produced two checkpoints. One restricted to public moderation datasets (about 45 million text samples). The other expanded to synthetic examples and to images (about 10 million combined). Mistral AI merged these checkpoints, along with Ministral-3B-Instruct’s, to boost the model’s ability to follow instructions.
