ResNet pretained does not load

[rracariu]

With Julia 1.11.3 and Metalhead v0.9.5 ResNet would fail to load,

using Metalhead
model = ResNet(50; pretrain=true)
ERROR: ArgumentError: Tried to load Base.OneTo(0) into (:pad, :k, :stride) but the structures do not match.
Stacktrace:
  [1] loadmodel!(dst::Flux.MaxPool{2, 4}, src::Tuple{}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:95
  [2] loadmodel!(dst::Tuple{…}, src::Tuple{…}; filter::Function, cache::IdSet{…})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:105
  [3] loadmodel!(dst::Flux.Chain{Tuple{…}}, src::@NamedTuple{layers::Tuple{…}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:105
  [4] loadmodel!(dst::Tuple{…}, src::Tuple{…}; filter::Function, cache::IdSet{…})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:105
  [5] loadmodel!(dst::Flux.Chain{Tuple{…}}, src::@NamedTuple{layers::Tuple{…}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:105
  [6] loadmodel!(dst::Tuple{Flux.Chain{…}, Flux.Chain{…}}, src::Tuple{@NamedTuple{…}, @NamedTuple{…}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:105
  [7] loadmodel!(dst::Flux.Chain{Tuple{…}}, src::@NamedTuple{layers::Tuple{…}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:105
  [8] loadmodel!(dst::ResNet, src::@NamedTuple{layers::@NamedTuple{layers::Tuple{…}}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:105
  [9] loadmodel!(dst::ResNet, src::@NamedTuple{layers::@NamedTuple{layers::Tuple{@NamedTuple{layers::Tuple{…}}, @NamedTuple{layers::Tuple{…}}}}})
    @ Flux ~/.julia/packages/Flux/BkG8S/src/loading.jl:90
 [10] loadpretrain!(model::ResNet, artifact_name::String)
    @ Metalhead ~/.julia/packages/Metalhead/sKQks/src/pretrain.jl:57
 [11] ResNet(depth::Int64; pretrain::Bool, inchannels::Int64, nclasses::Int64)
    @ Metalhead ~/.julia/packages/Metalhead/sKQks/src/convnets/resnets/resnet.jl:33
 [12] top-level scope
    @ REPL[9]:1

versioninfo()
Julia Version 1.11.3
Commit d63adeda50d (2025-01-21 19:42 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 16 × Apple M3 Max
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
Threads: 1 default, 0 interactive, 1 GC (on 12 virtual cores)

[remi-garcia]

It seems to be an issue with Flux.jl loadmodel! changes v0.15 FluxML/Flux.jl#2584

[adrhill]

I think this issue has been fixed in FluxML/Flux.jl#2598.

[remi-garcia]

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.5 (2025-04-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.11) pkg> st
Status `~/.julia/environments/v1.11/Project.toml`
  [587475ba] Flux v0.16.3 `https://github.com/FluxML/Flux.jl.git#master`
  [dbeba491] Metalhead v0.9.5 `https://github.com/FluxML/Metalhead.jl.git#master`
Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> using Flux, Metalhead

julia> model = ResNet(18; pretrain=true, nclasses=10)
ERROR: ArgumentError: Tried to load Base.OneTo(0) into (:args, :expr_string, :func, :kwargs) but the structures do not match.
Stacktrace:
  [1] loadmodel!(dst::Function, src::Tuple{}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:95
  [2] loadmodel!(dst::Parallel{…}, src::@NamedTuple{…}; filter::Function, cache::IdSet{…})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
  [3] loadmodel!
    @ ~/.julia/packages/Flux/6pNhw/src/loading.jl:90 [inlined]
  [4] loadmodel!(dst::Tuple{…}, src::Tuple{…}; filter::Function, cache::IdSet{…})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
  [5] loadmodel!(dst::Chain{Tuple{…}}, src::@NamedTuple{layers::Tuple{…}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
  [6] loadmodel!(dst::Tuple{…}, src::Tuple{…}; filter::Function, cache::IdSet{…})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
  [7] loadmodel!(dst::Chain{Tuple{…}}, src::@NamedTuple{layers::Tuple{…}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
  [8] loadmodel!(dst::Tuple{…}, src::Tuple{…}; filter::Function, cache::IdSet{…})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
  [9] loadmodel!(dst::Chain{Tuple{…}}, src::@NamedTuple{layers::Tuple{…}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
 [10] loadmodel!(dst::ResNet, src::@NamedTuple{layers::@NamedTuple{layers::Tuple{…}}}; filter::Function, cache::IdSet{Any})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:105
 [11] loadmodel!(dst::ResNet, src::@NamedTuple{layers::@NamedTuple{layers::Tuple{@NamedTuple{…}, @NamedTuple{…}}}})
    @ Flux ~/.julia/packages/Flux/6pNhw/src/loading.jl:90
 [12] loadpretrain!(model::ResNet, artifact_name::String)
    @ Metalhead ~/.julia/packages/Metalhead/lxP8T/src/pretrain.jl:57
 [13] ResNet(depth::Int64; pretrain::Bool, inchannels::Int64, nclasses::Int64)
    @ Metalhead ~/.julia/packages/Metalhead/lxP8T/src/convnets/resnets/resnet.jl:33
 [14] top-level scope
    @ REPL[3]:1
Some type information was truncated. Use `show(err)` to see complete types.

Unfortunately, I still have the error using the main Flux.jl, and Metalhead.jl, branch

[remi-garcia]

Flux 0.14

julia> model = ResNet(18; pretrain=false, nclasses=10)
ResNet(
  Chain(
    Chain(
      Chain(
        Conv((7, 7), 3 => 64, pad=3, stride=2, bias=false),  # 9_408 parameters
        BatchNorm(64, relu),            # 128 parameters, plus 128
        MaxPool((3, 3), pad=1, stride=2),
      ),
      Chain(
        Parallel(
          addact(NNlib.relu, ...),
          identity,
          Chain(

Flux 0.16

julia> model = ResNet(18; pretrain=false, nclasses=10)
ResNet(
  Chain(
    Chain(
      Chain(
        Conv((7, 7), 3 => 64, pad=3, stride=2, bias=false),  # 9_408 parameters
        BatchNorm(64, relu),            # 128 parameters, plus 128
        MaxPool((3, 3), pad=1, stride=2),
      ),
      Chain(
        Parallel(
          PartialFunction(
            "",
            Metalhead.addact,
            (relu,),
            NamedTuple(),
          ),
          identity,
          Chain(

In both cases: Metalhead v0.9.5

[cirobr]

Same issue here, for Julia 1.11, Metalhead v0.9.5 . Had to downgrade to Metalhead v0.9.4 to load pretrained parameters, AND MLUtils to v0.9.5 to make DataLoader work. In summary, what works for me with pretrained Metalhead is:

• Julia 1.11
• Metalhead 0.9.4
• MLUtils 0.9.5
• Flux 0.14.25

[adrhill]

For reference, (because it took me a while to find), this is introduced by these four LOC:

Metalhead.jl/src/Metalhead.jl

Line 11 in c533d51

using PartialFunctions

Metalhead.jl/src/layers/utilities.jl

Line 19 in c533d51

addact(activation = relu, xs...) = activation(sum(xs))

Metalhead.jl/src/convnets/resnets/core.jl

Lines 447 to 448 in c533d51

	return build_resnet((imsize..., inchannels), stem, get_layers, block_repeats,
	connection$activation, classifier_fn)

Where connection$activation uses the non-explicit import of PartialFunctions.jl's $ operator

[remi-garcia]

Thank you for finding this and your suggestion of using a simpler AddAct layer type FluxML/Flux.jl#2598 (comment)

I am just trying to experiment with Flux.jl so this is way beyond my capabilities :/