Running issue with cuda error "c10::Error"

[fireworker01]

My environment is Ubuntu20.04, gcc 11.4, CUDA 11.8, cuDNN 8.9.3, OpenCV with opencv_contrib and CUDA 4.8.0, libtorch cxx11-abi-shared-with-deps-2.0.1+cu118.
My compilation passed, but when I ran it, the following error occurred. How can I solve it? I am running the rgbd_dataset_freiburg1_desk of TUM's rgbd dataset.

`-------
Start processing sequence ...
Images in the sequence: 573

First KF:0; Map init KF:0
New Map created with 1132 points
terminate called after throwing an instance of 'c10::Error'
what(): CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)
Exception raised from bgemm at ../aten/src/ATen/cuda/CUDABlas.cpp:209 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x6b (0x7f3e62bf56bb in /home/tros/packages/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0xbf (0x7f3e62bf05ef in /home/tros/packages/libtorch/lib/libc10.so)
frame #2: + 0x30a8a13 (0x7f3e661f5a13 in /home/tros/packages/libtorch/lib/libtorch_cuda.so)
frame #3: + 0x30db8b9 (0x7f3e662288b9 in /home/tros/packages/libtorch/lib/libtorch_cuda.so)
frame #4: at::native::structured_bmm_out_cuda::impl(at::Tensor const&, at::Tensor const&, at::Tensor const&) + 0x76 (0x7f3e66229376 in /home/tros/packages/libtorch/lib/libtorch_cuda.so)
frame #5: + 0x2dc0bbc (0x7f3e65f0dbbc in /home/tros/packages/libtorch/lib/libtorch_cuda.so)
frame #6: + 0x2dc0c73 (0x7f3e65f0dc73 in /home/tros/packages/libtorch/lib/libtorch_cuda.so)
frame #7: at::_ops::bmm::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&) + 0x90 (0x7f3ebbd8a150 in /home/tros/packages/libtorch/lib/libtorch_cpu.so)
frame #8: + 0x3d16218 (0x7f3ebd6b5218 in /home/tros/packages/libtorch/lib/libtorch_cpu.so)
frame #9: + 0x3d16ab6 (0x7f3ebd6b5ab6 in /home/tros/packages/libtorch/lib/libtorch_cpu.so)
frame #10: at::_ops::bmm::call(at::Tensor const&, at::Tensor const&) + 0xdb (0x7f3ebbddf04b in /home/tros/packages/libtorch/lib/libtorch_cpu.so)
frame #11: GaussianKeyframe::computeTransformTensors() + 0x2f7 (0x7f3f2767d6d7 in /home/tros/packages/3DGS_proj/Photo-SLAM/lib/libgaussian_mapper.so)
frame #12: GaussianMapper::run() + 0xb6f (0x7f3f276cc6ef in /home/tros/packages/3DGS_proj/Photo-SLAM/lib/libgaussian_mapper.so)
frame #13: + 0xdf904 (0x7f3e62a62904 in /lib/x86_64-linux-gnu/libstdc++.so.6)
frame #14: + 0x8609 (0x7f3e6294d609 in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #15: clone + 0x43 (0x7f3e62872353 in /lib/x86_64-linux-gnu/libc.so.6)

Aborted (core dumped)`