mbert-loraxs-qa-LTr64-qkvd-iscale-0.3-lastL

This model is a fine-tuned version of bert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0840
• Exact Match: 65.0921
• F1: 78.9812

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
2.0686	0.2818	2500	1.7376	48.8963	63.5693
1.5655	0.5636	5000	1.3825	57.4174	71.5560
1.3910	0.8455	7500	1.3046	59.4533	74.3331
1.2793	1.1273	10000	1.2840	60.5970	75.3199
1.2960	1.4091	12500	1.1857	62.3470	76.7101
1.2280	1.6909	15000	1.1902	62.9189	77.0271
1.1623	1.9727	17500	1.1520	63.1591	77.4446
1.1020	2.2545	20000	1.1328	63.9140	77.9905
1.1474	2.5364	22500	1.1294	63.8911	77.9421
1.1249	2.8182	25000	1.1161	64.3372	78.4890
1.0010	3.1000	27500	1.1339	64.5888	78.6491
1.0704	3.3818	30000	1.0824	65.1493	78.9885
1.0596	3.6636	32500	1.0855	64.9777	78.8150
1.0874	3.9454	35000	1.0838	65.1264	78.9291
1.0874	4.0	35484	1.0840	65.0921	78.9812

Framework versions

• PEFT 0.19.1
• Transformers 5.9.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.5
• Tokenizers 0.22.2