Skip to content

amd bug report#77

Open
karenl03 wants to merge 17 commits into
mainfrom
amd-bug-report
Open

amd bug report#77
karenl03 wants to merge 17 commits into
mainfrom
amd-bug-report

Conversation

@karenl03

@karenl03 karenl03 commented Mar 18, 2026

Copy link
Copy Markdown
Collaborator

Description
AMD support for log collector

  • amd-bug-report.sh script mimics nvidia-bug-report for AMD GPUs -- collects ROCm diagnostics, amd-smi metrics, firmware info, etc into compressed .log.gz
  • K8s: AMD Daemonset with correct node affinity, toelrations, and driver namespace to deploy only on AMD GPUs
  • VM: docker and native (--no-docker) installation via crusoe_watch_agent_amd.sh script

Unified log collector

  • Moved log collector from separate k8s/log-collector/ and vm/log-collector/ to common/log-collector/
  • log_collector.py automatically detects K8s vs VM environment and supports both NVIDIA and AMD via GPU_TYPE env var
  • Two docker images: log-collector and amd-log-collector, both built by log-collector-ci (triggered on common/log-collector/*)

Jira
https://crusoe.atlassian.net/browse/CRUSOE-55851

Testing
Tested log collector capabilities on AMD, as well as NVIDIA due to new unified approach
Sample AMD bug report here

  • K8s AMD: amd cluster with MI300x
  • VM AMD: MI300X with Docker and Native installation
  • K8S NVIDIA: ib-commandcenter-cluster and b200 cluster, generate via UI
  • VM NVIDIA: run test script on L40S and A100 across all 5 images and both native/docker installation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant