Instructions to use CIRCL/vulnerability-severity-classification-roberta-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use CIRCL/vulnerability-severity-classification-roberta-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="CIRCL/vulnerability-severity-classification-roberta-base")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("CIRCL/vulnerability-severity-classification-roberta-base") model = AutoModelForSequenceClassification.from_pretrained("CIRCL/vulnerability-severity-classification-roberta-base", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification
Severity classification
This model is a fine-tuned version of roberta-base on the dataset CIRCL/vulnerability-scores.
The model was presented in the paper VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification [arXiv].
Abstract: VLAI is a transformer-based model that predicts software vulnerability severity levels directly from text descriptions. Built on RoBERTa, VLAI is fine-tuned on over 600,000 real-world vulnerabilities and achieves over 82% accuracy in predicting severity categories, enabling faster and more consistent triage ahead of manual CVSS scoring. The model and dataset are open-source and integrated into the Vulnerability-Lookup service.
Note: since VulnTrain 3.4.0, the dataset is split so that records of the same vulnerability (for example a CVE and its GitHub advisory) never appear in both train and test. Accuracy measured this way (74%) reflects performance on vulnerabilities the model has never seen. Earlier figures, including the one in the abstract above, were measured on a split where many test descriptions also appeared in training. Details in the technical report Improving vulnerability-scores: Leakage-Free Splits and Consistent CVSS 4.0 Scoring.
You can read this page for more information.
Model description
It is a classification model and is aimed to assist in classifying vulnerabilities by severity based on their descriptions.
Labels:
| Label | CVSS score |
|---|---|
| Low | < 4.0 |
| Medium | 4.0 β 6.9 |
| High | 7.0 β 8.9 |
| Critical | β₯ 9.0 |
How to get started with the model
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch
model_name = "CIRCL/vulnerability-severity-classification-roberta-base"
revision = "b868a7d"
tokenizer = AutoTokenizer.from_pretrained(model_name, revision=revision)
model = AutoModelForSequenceClassification.from_pretrained(model_name, revision=revision)
model.eval()
test_description = "SAP NetWeaver Visual Composer Metadata Uploader is not protected with a proper authorization, allowing unauthenticated agent to upload potentially malicious executable binaries \
that could severely harm the host system. This could significantly affect the confidentiality, integrity, and availability of the targeted system."
inputs = tokenizer(test_description, return_tensors="pt", truncation=True, padding=True)
# Run inference
with torch.no_grad():
outputs = model(**inputs)
predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
# Print results
print("Predictions:", predictions)
predicted_class = torch.argmax(predictions, dim=-1).item()
print("Predicted severity:", model.config.id2label[predicted_class])
Training data
Revision 58b4833
of CIRCL/vulnerability-scores,
built with VulnTrain from CVE records, GitHub, PySec and CSAF (Red Hat, Cisco, CISA) advisories:
| Split | Rows | Labelled rows |
|---|---|---|
| train | 686,695 | 584,245 |
| validation | 85,405 | 73,419 |
| test | 85,167 | 72,240 |
- Group-aware split (80/10/10). Rows that share a description, an identifier or an alias are kept in the same split, so no vulnerability appears in both train and test.
- Labels come from the first available CVSS score, in the order v4.0, v3.1, v3.0, v2.0. CVSS 4.0 scores are read with base metrics only (CVSS-B), whatever the source.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: adamw_torch_fused
- lr_scheduler_type: linear
- warmup_steps: 0
- weight_decay: 0.01
- num_epochs: 5
The checkpoint with the best validation accuracy (epoch 3) is the one published.
Training took 4 h 32 min on 4 x NVIDIA L40S and used 4.74 kWh, for 0.50 kg COβeq
(measured with CodeCarbon, see emissions.csv).
Results
Evaluated once on the leakage-free test split (72,240 labelled rows):
- Accuracy: 0.7419 (95% CI 0.7387 β 0.7451)
- F1 Macro: 0.6496
| Class | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Low | 0.5947 | 0.3563 | 0.4456 | 2,599 |
| Medium | 0.7790 | 0.8162 | 0.7971 | 32,602 |
| High | 0.7262 | 0.7409 | 0.7335 | 28,209 |
| Critical | 0.6659 | 0.5839 | 0.6222 | 8,830 |
Training results
Validation metrics at each epoch:
| Epoch | Step | Training Loss | Validation Loss | Accuracy | F1 Macro |
|---|---|---|---|---|---|
| 1.0 | 18258 | 2.3938 | 2.6531 | 0.7289 | 0.6162 |
| 2.0 | 36516 | 2.1821 | 2.5989 | 0.7411 | 0.6482 |
| 3.0 | 54774 | 1.8636 | 2.6522 | 0.7488 | 0.6568 |
| 4.0 | 73032 | 1.6322 | 2.9344 | 0.7481 | 0.6566 |
| 5.0 | 91290 | 1.3337 | 3.1119 | 0.7473 | 0.6622 |
Previous versions
The previous model, trained on the earlier random split, is revision
10d9861
(dataset revision 34e6a14, test accuracy 81.17%, 73.02% on the test rows without a twin in train).
Framework versions
- Transformers 5.22.2
- Pytorch 2.15.0
- Datasets 4.8.5
- Tokenizers 0.23.3
Provenance
This model card is generated by VulnTrain 3.5.0 at the end of the training run, from the values the run measured.
bom.cdx.json: machine-readable record of the run (CycloneDX ML-BOM): base model, dataset revision, metrics, training options and energy use.emissions.csv: energy use and COβ emissions of the training, measured with CodeCarbon.
Citation
@article{bonhomme2025vlai,
title = {{VLAI}: A {RoBERTa}-Based Model for Automated Vulnerability
Severity Classification},
author = {Bonhomme, C{\'e}dric and Dulaunoy, Alexandre},
journal = {arXiv preprint arXiv:2507.03607},
year = {2025}
}
@techreport{bonhomme2026vulnscores,
title = {Improving vulnerability-scores: Leakage-Free Splits and
Consistent {CVSS} 4.0 Scoring},
author = {Bonhomme, C{\'e}dric},
institution = {CIRCL - Computer Incident Response Center Luxembourg},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23760802},
url = {https://doi.org/10.5281/zenodo.23760802}
}
- Downloads last month
- 14,028
Model tree for CIRCL/vulnerability-severity-classification-roberta-base
Base model
FacebookAI/roberta-baseDataset used to train CIRCL/vulnerability-severity-classification-roberta-base
Spaces using CIRCL/vulnerability-severity-classification-roberta-base 2
Collection including CIRCL/vulnerability-severity-classification-roberta-base
Paper for CIRCL/vulnerability-severity-classification-roberta-base
Evaluation results
- Accuracy on CIRCL/vulnerability-scorestest set self-reported0.742
- F1 Macro on CIRCL/vulnerability-scorestest set self-reported0.650