Error while finetuning using Aut Train
Hi Team,
I am getting this below error while using AutoTrain to finetuning the model..
Please help to resolve this issue..
Thank You
INFO | 2025-01-09 15:16:54 | autotrain.backends.local:create:20 - Starting local training...
INFO | 2025-01-09 15:16:54 | autotrain.commands:launch_command:514 - ['accelerate', 'launch', '--num_machines', '1', '--num_processes', '1', '--mixed_precision', 'fp16', '-m', 'autotrain.trainers.text_classification', '--training_config', 'autotrain-ryo6d-huva8/training_params.json']
INFO | 2025-01-09 15:16:54 | autotrain.commands:launch_command:515 - {'data_path': 'autotrain-ryo6d-huva8/autotrain-data', 'model': 'answerdotai/ModernBERT-base', 'lr': 5e-05, 'epochs': 3, 'max_seq_length': 128, 'batch_size': 8, 'warmup_ratio': 0.1, 'gradient_accumulation': 1, 'optimizer': 'adamw_torch', 'scheduler': 'linear', 'weight_decay': 0.0, 'max_grad_norm': 1.0, 'seed': 42, 'train_split': 'train', 'valid_split': 'validation', 'text_column': 'autotrain_text', 'target_column': 'autotrain_label', 'logging_steps': -1, 'project_name': 'autotrain-ryo6d-huva8', 'auto_find_batch_size': False, 'mixed_precision': 'fp16', 'save_total_limit': 1, 'token': '*****', 'push_to_hub': True, 'eval_strategy': 'epoch', 'username': 'sk4444', 'log': 'tensorboard', 'early_stopping_patience': 5, 'early_stopping_threshold': 0.01}
INFO | 2025-01-09 15:16:54 | autotrain.backends.local:create:25 - Training PID: 185855
INFO: 127.0.0.1:33276 - "POST /ui/create_project HTTP/1.1" 200 OK
INFO: 127.0.0.1:39830 - "GET /ui/is_model_training HTTP/1.1" 200 OK
The following values were not passed to accelerate launch
and had defaults used instead:
--dynamo_backend
was set to a value of 'no'
To avoid this warning pass in values for each of the problematic parameters or run accelerate config
.
INFO: 127.0.0.1:39830 - "GET /ui/is_model_training HTTP/1.1" 200 OK
INFO: 127.0.0.1:33276 - "GET /ui/accelerators HTTP/1.1" 200 OK
INFO | 2025-01-09 15:16:58 | main:train:50 - loading dataset from disk
INFO | 2025-01-09 15:16:58 | main:train:72 - loading dataset from disk
ERROR | 2025-01-09 15:16:59 | autotrain.trainers.common:wrapper:215 - train has failed due to an exception: Traceback (most recent call last):
File "/root/anaconda3/envs/autotrain/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1038, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/autotrain/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 740, in getitem
raise KeyError(key)
KeyError: 'modernbert'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/anaconda3/envs/autotrain/lib/python3.11/site-packages/autotrain/trainers/common.py", line 212, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/autotrain/lib/python3.11/site-packages/autotrain/trainers/text_classification/main.py", line 106, in train
model_config = AutoConfig.from_pretrained(config.model, num_labels=num_classes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/autotrain/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1040, in from_pretrained
raise ValueError(
ValueError: The checkpoint you are trying to load has model type modernbert
but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
ERROR | 2025-01-09 15:16:59 | autotrain.trainers.common:wrapper:216 - The checkpoint you are trying to load has model type modernbert
but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
INFO | 2025-01-09 15:17:03 | autotrain.app.utils:get_running_jobs:40 - Killing PID: 185855
INFO | 2025-01-09 15:17:03 | autotrain.app.utils:kill_process_by_pid:90 - Sent SIGTERM to process with PID 185855
You need to install the dev version of transformers until 4.48 is released: pip install git+https://github.com/huggingface/transformers