Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable torch.compile validation on Amazon 2023 #6434

Merged
merged 3 commits into from
Mar 21, 2025

Conversation

atalman
Copy link
Contributor

@atalman atalman commented Mar 17, 2025

Which is currently failing: https://github.com/pytorch/test-infra/actions/runs/13973253199/job/39120358041

Test:

python3 smoke_test.py --package torchonly
torch: 2.7.0+cu126
ATen/Parallel:
	at::get_num_threads() : 8
	at::get_num_interop_threads() : 16
OpenMP 201511 (a.k.a. OpenMP 4.5)
	omp_get_max_threads() : 8
Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications
	mkl_get_max_threads() : 8
Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d)
std::thread::hardware_concurrency() : 16
Environment variables:
	OMP_NUM_THREADS : [not set]
	MKL_NUM_THREADS : [not set]
ATen parallel backend: OpenMP

Skip version check for channel None as stable version is None
Testing smoke_test_conv2d
Testing smoke_test_linalg on cpu
Testing smoke_test_compile for cpu and torch.float16
Testing smoke_test_compile for cpu and torch.float32
Testing smoke_test_compile for cpu and torch.float64
Picked CPU ISA VecAVX2 bit width 256
Testing smoke_test_compile with mode 'max-autotune' for torch.float32

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 17, 2025
Copy link

vercel bot commented Mar 17, 2025

@atalman is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

@atalman atalman changed the title Fix Amazon Linux 2023 validation Fix Amazon Linux 2023 validation. Enabled torch.compile Mar 20, 2025
@atalman
Copy link
Contributor Author

atalman commented Mar 20, 2025

Please note, because of this issue Amazon 2023 its still failing: pytorch/pytorch#149681

Validated with local Amazon 2023 (if numpy is installed test is working)

@seemethere
Copy link
Member

Are these things we recommend people to install when using torch.compile?

Want to level set that we're testing out what a user might actually be installing / have expectations to install

@atalman atalman changed the title Fix Amazon Linux 2023 validation. Enabled torch.compile Enable torch.compile validation on Amazon 2023 Mar 20, 2025
@atalman
Copy link
Contributor Author

atalman commented Mar 20, 2025

Error 1 if Development Tools are not installed:

compiler_info = get_compiler_version_info(get_cpp_compiler())
  File "/usr/local/lib64/python3.9/site-packages/torch/_inductor/cpp_builder.py", line 156, in get_cpp_compiler
    compiler = cpp_compiler_search(search)
  File "/usr/local/lib64/python3.9/site-packages/torch/_inductor/cpp_builder.py", line 102, in cpp_compiler_search
    raise exc.InvalidCxxCompiler
torch._inductor.exc.InductorError: InvalidCxxCompiler: No working C++ compiler found in torch._inductor.config.cpp.cxx: (None, 'g++')

Error 2 if python-devel is not installed:

Testing smoke_test_compile for cpu and torch.float16
/usr/local/lib64/python3.9/site-packages/torch/_inductor/cpp_builder.py:797: UserWarning: Can't find Python.h in /usr/include/python3.9
  warnings.warn(f"Can't find Python.h in {str(include_dir)}")
/usr/local/lib64/python3.9/site-packages/torch/_inductor/cpp_builder.py:797: UserWarning: Can't find Python.h in /usr/include/python3.9
  warnings.warn(f"Can't find Python.h in {str(include_dir)}")
Traceback (most recent call last):
  File "/pytorch/.ci/pytorch/smoke_test/smoke_test.py", line 430, in <module>
    main()
  File "/pytorch/.ci/pytorch/smoke_test/smoke_test.py", line 424, in main
    smoke_test_cuda(
  File "/pytorch/.ci/pytorch/smoke_test/smoke_test.py", line 222, in smoke_test_cuda
    smoke_test_compile("cuda" if torch.cuda.is_available() else "cpu")

@atalman
Copy link
Contributor Author

atalman commented Mar 20, 2025

Same problem with 2.6 release:
Error1:

torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
InvalidCxxCompiler: No working C++ compiler found in torch._inductor.config.cpp.cxx: (None, 'g++')

Error2:

/usr/local/lib64/python3.9/site-packages/torch/_subclasses/functional_tensor.py:275: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:81.)
  cpu = _conversion_method_template(device=torch.device("cpu"))
/usr/local/lib64/python3.9/site-packages/torch/_inductor/cpp_builder.py:799: UserWarning: Can't find Python.h in /usr/include/python3.9
  warnings.warn(f"Can't find Python.h in {str(include_dir)}")
/usr/local/lib64/python3.9/site-packages/torch/_inductor/cpp_builder.py:799: UserWarning: Can't find Python.h in /usr/include/python3.9

Copy link
Contributor

@malfet malfet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please mention the followup issue in the comment, but otherwise LGTM

@atalman
Copy link
Contributor Author

atalman commented Mar 21, 2025

Documentation update is tracked in pytorch/pytorch#149740

@atalman atalman merged commit c1d92bc into pytorch:main Mar 21, 2025
37 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants