whisper-small-basque

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2714
• Wer: 18.3719

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.4242	0.11	500	0.5947	47.5547
0.2999	0.21	1000	0.4454	34.5728
0.2505	0.32	1500	0.3934	28.3430
0.22	0.42	2000	0.3644	23.5473
0.2131	0.53	2500	0.3424	22.3772
0.2006	0.63	3000	0.3280	22.1647
0.1779	0.74	3500	0.3161	21.0050
0.182	0.84	4000	0.3064	20.5056
0.1724	0.95	4500	0.2966	20.1589
0.1319	1.05	5000	0.2923	19.8163
0.1415	1.16	5500	0.2891	19.7441
0.1362	1.26	6000	0.2865	19.6843
0.1327	1.37	6500	0.2828	18.9393
0.1336	1.47	7000	0.2795	19.5790
0.1326	1.58	7500	0.2760	18.4957
0.1361	1.68	8000	0.2753	18.8217
0.1275	1.79	8500	0.2735	18.5988
0.1288	1.89	9000	0.2714	18.3244
0.1185	2.0	9500	0.2705	18.7763
0.1023	2.1	10000	0.2714	18.3719

Framework versions

• Transformers 4.36.0
• Pytorch 2.1.1+cu121
• Datasets 2.8.0
• Tokenizers 0.15.2