AttributeError: 'NoneType' object has no attribute 'output_adapter_gating_scores'

[CrazyLenmon]

Hi, I tried to train adapter with CLIPVisionModel.
There was nothing inside the context when I used the parallel bottleneck adapters. I have searched, but not found similar issues that I encountered so far. It's possibly something wrong with ForwardContext.get_context() which returns None.

When I tried to train the model and it showed the errors.

File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1537, in train
return inner_training_loop(
File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1854, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2744, in training_step
self.accelerator.backward(loss)
File "/opt/conda/lib/python3.10/site-packages/accelerate/accelerator.py", line 1958, in backward
self.deepspeed_engine_wrapped.backward(loss, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 167, in backward
self.engine.backward(loss, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
ret_val = func(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1964, in backward
self.optimizer.backward(loss, retain_graph=retain_graph)
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/zero/stage_1_and_2.py", line 2040, in backward
self.loss_scaler.backward(loss.float(), retain_graph=retain_graph)
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/fp16/loss_scaler.py", line 63, in backward
scaled_loss.backward(retain_graph=retain_graph)
File "/opt/conda/lib/python3.10/site-packages/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/opt/conda/lib/python3.10/site-packages/torch/autograd/init.py", line 200, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
File "/opt/conda/lib/python3.10/site-packages/torch/autograd/function.py", line 274, in apply
return user_fn(self, *args)
File "/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py", line 141, in backward
outputs = ctx.run_function(*detached_inputs)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1538, in _call_impl
result = forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/adapters/models/clip/modeling_clip.py", line 150, in forward
hidden_states = self.output_adapters(hidden_states, residual, layer_norm=None)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/adapters/methods/bottleneck.py", line 372, in forward
return self.bottleneck_layer_forward(hidden_states, residual_input, layer_norm)
File "/opt/conda/lib/python3.10/site-packages/adapters/methods/bottleneck.py", line 348, in bottleneck_layer_forward
state = self.compose(adapter_setup, state)
File "/opt/conda/lib/python3.10/site-packages/adapters/methods/adapter_layer_base.py", line 472, in compose
state = composition_func(adapter_setup, state, lvl=0)
File "/opt/conda/lib/python3.10/site-packages/adapters/methods/adapter_layer_base.py", line 308, in compose_stack
state = self.compose_single(adapter_stack_layer, state, lvl=lvl + 1)
File "/opt/conda/lib/python3.10/site-packages/adapters/methods/bottleneck.py", line 233, in compose_single
output_gating=context.output_adapter_gating_scores,
AttributeError: 'NoneType' object has no attribute 'output_adapter_gating_scores'

[CrazyLenmon]

This problem is quite wired. When I add some codes like "if context is None" before it, ForwardContext.get_context() would not return a Nonetype. I don't know the reason. Something like Schrödinger' codes.

[calpt]

Hey, sorry for missing this. Would you mind sharing a minimal example we can run to reproduce this? Thank you!

[mkgs210]

i also have this problem when i try to use

gradient_checkpointing_enable()
enable_input_require_grads()

i use preiffer adapter for text classification task

[mehulrk18]

@calpt I am also facing the same issue with the Bottleneck adapters from the AdapterHub. I am training LLaMA for summarization. It is working fine with LoRA config but not with BottleNeck Adapters (all configurations). Can you please help me with this?

[calpt]

Thanks all for reporting this issue!

The issue seems to be happening only if gradient checkpointing is enabled (please provide counter examples if this is not the case).

For Bottleneck adapters, the issue should be resolved by #730. However, it still persists for a few other adapter types in combination with gradient checkpointing, see #732.