BartForQuestionAnswering cannot train adapter

[LRY1994]

Environment info

• adapter-transformers version:
• Platform:
• Python version:
• PyTorch version (GPU?):
• Tensorflow version (GPU?):
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Information

Model I am using (BartForQuestionAnswering):
model = BartForQuestionAnswering.from_pretrained(args.base_model)

Language I am using the model on English

Adapter setup I am using (if any):
adapter_config= PrefixTuningConfig(flat=True, prefix_length=args.prefix_token_num)
adapter_config= PfeifferConfig(
reduction_factor=args.adapter_reduction_factor,
)

The problem arises when using:

• the official example scripts: (give details below)
• my own modified scripts: (give details below)

model = BartModel(config)
adapter_config= PrefixTuningConfig(flat=True, prefix_length=args.prefix_token_num)
model.add_adapter( args.adapter_names, config=adapter_config )
model.train_adapter( args.adapter_names)

model = BartForQuestionAnswering.from_pretrained(args.base_model)
model.load_adapter(adapter_dir, with_head=False,set_active=True)
model(**batch)

It comes:
Traceback (most recent call last):
File "./src/eval_extractive_qa/run.py", line 251, in
trainer.train()
File "/home/linry/prompt-closed-qa/src/eval_extractive_qa/utils/trainer.py", line 189, in train
tr_loss = self.train_epoch(train_dataloader,epoch)
File "/home/linry/prompt-closed-qa/src/eval_extractive_qa/utils/trainer.py", line 133, in train_epoch
outputs = train_model(
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/models/bart/modeling_bart.py", line 1665, in forward
outputs = self.model(
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/adapters/context.py", line 104, in wrapper_func
with cls(self, *args, **kwargs) as ctx:
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/adapters/context.py", line 86, in init
model.forward_context(self, *args, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/adapters/model_mixin.py", line 787, in forward_context
context.prefix_states = self.base_model.prefix_tuning(*args, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/adapters/prefix_tuning.py", line 218, in forward
prefix_states[name] = self.prefix_tuningsname
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(input, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/adapters/prefix_tuning.py", line 116, in forward
return {k: v(batch_size) for k, v in self.items()}
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/adapters/prefix_tuning.py", line 116, in
return {k: v(batch_size) for k, v in self.items()}
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(input, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/adapters/prefix_tuning.py", line 52, in forward
key_values = self.control_trans(embs) # batch_size x prefix_length x n_layers2input_size
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/container.py", line 139, in forward
input = module(input)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/data/linry/anaconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

Expected behavior

[adapter-hub-bert]

This issue has been automatically marked as stale because it has been without activity for 90 days. This issue will be closed in 14 days unless you comment or remove the stale label.