Allow the installation of docling by GPU.

[rhatdan]

Requested feature

With RamaLama we are attempting to add RAG support using Docling in the PRAGmatic project. RamaLama is using contianer images and installing docling is jumping the size of the install from 1GB to 7 GB.

RamaLama differentiates container images based on GPU type. Would like to have a CPU, Rocm, and Cuda install.

pytorch supports installing with only CPU, ROCM or Cuda, but when I install docling on top, it insists on installing the full pytorch suite, blowing up the size.

Alternatives

...

[PeterStaar-IBM]

@rhatdan Is there a way we can replicate this issue. It looks a bit out of the ordinary that the size of the install goes from 1GB to 7 GB.

[rhatdan]

cat > /tmp/Containerfile
from quay.io/ramalama/ramalama
run pip install docling

podman build -t mydocling /tmp

[rhatdan]

Then compare the container image you built to the quay.io/ramalama/ramalama

[rhatdan]

pip install docling --extra-index-url https://download.pytorch.org/whl/cpu

Helps with the non-cpu case.

Would like to use similar for cuda and rocm installs.

[dolfim-ibm]

It is true that the pypi version of pytorch is very big on linux (it installs tons of nvidia packages).

The Docling package doesn't have a requirement on which pytorch registry is used, but it will depend on pytorch being available.

Did you already try installing the Cuda version with the registry indexes suggested at https://pytorch.org/?