feat: Better cublas support in vLLM container.

[jthomson04]

Related to #4421. Revamped to just to the symlink instead of the full copy.

Summary by CodeRabbit

• Chores
  • Updated container runtime library configuration for optimized initialization and runtime behavior.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

The Dockerfile.vllm runtime stage has been modified to replace explicit copying of CUDA library files (libcublas.so* and libcublasLt.so*) with creation of symbolic links to versioned 12.x libraries, changing the dependency resolution strategy from file copying to symlink creation.

Changes

Cohort / File(s)	Summary
CUDA library linkage update container/Dockerfile.vllm	Replaced COPY commands for libcublas.so* and libcublasLt.so* with RUN ln -s commands creating symlinks to libcublas.so.12 and libcublasLt.so.12 respectively

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

• Single file with homogeneous, repetitive changes (two similar COPY-to-symlink conversions)
• Verify symlink targets (libcublas.so.12 and libcublasLt.so.12) exist in the framework image stage
• Confirm runtime compatibility and behavior with symlinked libraries vs. copied files

Poem

🐰 A rabbit hops through Docker dreams,
Where CUDA libs flow like streams,
From copy to link, a clever dance,
Symlinks now guide library's prance,
Versioned bonds in perfect chant! 📦✨

Pre-merge checks

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is minimal and lacks required template sections (Overview, Details, Where should reviewer start). It only provides a link reference without explaining the changes.	Expand the description to include Overview (brief summary), Details (explanation of symlink changes vs copying), and specific files to review (container/Dockerfile.vllm).

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Better cublas support in vLLM container' directly relates to the main change: replacing library copying with symlinks for improved cublas dependency resolution.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

container/Dockerfile.vllm (1)

220-221: Combine RUN statements to reduce layer count.

Each RUN directive creates a new image layer. Consolidate these two symlink creations into a single RUN statement to reduce the Docker image footprint.

-RUN ln -s /usr/local/cuda/lib64/libcublas.so.12 /usr/local/cuda/lib64/libcublas.so
-RUN ln -s /usr/local/cuda/lib64/libcublasLt.so.12 /usr/local/cuda/lib64/libcublasLt.so
+RUN ln -s /usr/local/cuda/lib64/libcublas.so.12 /usr/local/cuda/lib64/libcublas.so && \
+    ln -s /usr/local/cuda/lib64/libcublasLt.so.12 /usr/local/cuda/lib64/libcublasLt.so

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5fe0476 and 8846ead.

📒 Files selected for processing (1)

• container/Dockerfile.vllm (1 hunks)

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2797
File: container/Dockerfile:437-449
Timestamp: 2025-08-30T20:43:49.632Z
Learning: In the dynamo project's devcontainer setup, the team prioritizes consistency across framework-specific Dockerfiles (like container/Dockerfile, container/Dockerfile.vllm, etc.) by mirroring their structure, even when individual optimizations might be possible, to maintain uniformity in the development environment setup.

Learnt from: grahamking
Repo: ai-dynamo/dynamo PR: 1177
File: container/Dockerfile.vllm:102-105
Timestamp: 2025-05-28T22:54:46.875Z
Learning: In Dockerfiles, when appending to environment variables that may not exist in the base image, Docker validation will fail if you reference undefined variables with ${VARIABLE} syntax. In such cases, setting the environment variable directly (e.g., ENV CPATH=/usr/include) rather than appending is the appropriate approach.

📚 Learning: 2025-08-30T20:43:49.632Z

Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2797
File: container/Dockerfile:437-449
Timestamp: 2025-08-30T20:43:49.632Z
Learning: In the dynamo project's devcontainer setup, the team prioritizes consistency across framework-specific Dockerfiles (like container/Dockerfile, container/Dockerfile.vllm, etc.) by mirroring their structure, even when individual optimizations might be possible, to maintain uniformity in the development environment setup.

Applied to files:

• container/Dockerfile.vllm

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)

• GitHub Check: sglang (amd64)
• GitHub Check: trtllm (amd64)
• GitHub Check: sglang (arm64)
• GitHub Check: trtllm (arm64)
• GitHub Check: operator (arm64)
• GitHub Check: vllm (arm64)
• GitHub Check: operator (amd64)
• GitHub Check: vllm (amd64)
• GitHub Check: Build and Test - dynamo

🔇 Additional comments (1)

container/Dockerfile.vllm (1)

220-221: Symlink targets are present in the CUDA runtime image.

The runtime image extends the base image by adding all the shared libraries from the CUDA toolkit for use with pre-built applications using multiple CUDA libraries. The runtime flavor includes cuBLAS, and the CUDA runtime layer provides the components needed to execute CUDA applications and includes all of the shared libraries. The files libcublas.so.12 and libcublasLt.so.12 are core CUDA runtime shared libraries, not development-only files. The symlinks will successfully resolve at runtime.