Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brew create --python failed with No available formula with the name "[email protected]" #19240

Open
3 tasks done
dz85 opened this issue Feb 5, 2025 · 14 comments
Open
3 tasks done
Labels
bug Reproducible Homebrew/brew bug

Comments

@dz85
Copy link

dz85 commented Feb 5, 2025

brew doctor output

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed casks are deprecated or disabled.
You should find replacements for the following casks:
  todesk

Verification

  • My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
  • I ran brew update twice and am still able to reproduce my issue.
  • This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.4.20-37-g1117ebc
ORIGIN: https://github.com/Homebrew/brew
HEAD: 1117ebc248035ddee17f7838ebb1871e783e53fd
Last commit: 3 hours ago
Branch: master
Core tap HEAD: 2e9d7b588698f6b3cf1ccb7e7d9b5121c0013c4f
Core tap last commit: 63 minutes ago
Core cask tap HEAD: cc234a041f404914379ee599a4b9913fab17cbb2
Core cask tap last commit: 3 hours ago
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: code --wait
HOMEBREW_GITHUB_API_TOKEN: set
HOMEBREW_MAKE_JOBS: 12
HOMEBREW_NO_INSTALL_FROM_API: set
HOMEBREW_SORBET_RUNTIME: set
http_proxy: http://127.0.0.1:1087
https_proxy: http://127.0.0.1:1087
Homebrew Ruby: 3.3.7 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.7/bin/ruby
CPU: dodeca-core 64-bit arm_blizzard_avalanche
Clang: 16.0.0 build 1600
Git: 2.48.1 => /opt/homebrew/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.2-arm64
CLT: 16.2.0.0.1.1733547573
Xcode: 16.2
Rosetta 2: false

What were you trying to do (and why)?

To create a formula for a python app named auto-coder, but failed.

What happened (include all command output)?

First, I tried brew create --python, it failed:

brew create --verbose --set-name auto-coder --set-license Apache-2.0 --python https://files.pythonhosted.org/packages/c3/78/102253fa844920e9c25d928eba36956ebeb65c7dcf89474ec56bcd994b24/auto_coder-0.1.243.tar.gz

==> Downloading https://files.pythonhosted.org/packages/c3/78/102253fa844920e9c25d928eba36956ebeb65c7dcf89474ec56bcd994b24/auto_coder-0.1.243.tar.gz
Already downloaded: /Users/dz/Library/Caches/Homebrew/downloads/cd3a40b4ab748b4e617279215c87cfedf0be2c0d8d7c45f44494a842c651902e--auto_coder-0.1.243.tar.gz
==> Verifying checksum for 'cd3a40b4ab748b4e617279215c87cfedf0be2c0d8d7c45f44494a842c651902e--auto_coder-0.1.243.tar.gz'
Warning: Cannot verify integrity of 'cd3a40b4ab748b4e617279215c87cfedf0be2c0d8d7c45f44494a842c651902e--auto_coder-0.1.243.tar.gz'.
No checksum was provided.
For your reference, the checksum is:
  sha256 "e25645b44b3e36e6d673f61a5e50d198cd3804079d9e955bbb0b449bc05b8da1"
Error: No available formula with the name "[email protected]". Did you mean python-yq, [email protected], [email protected], [email protected], [email protected], [email protected] or [email protected]?

Then I tried to create auto-coder rb file manually:

class AutoCoder < Formula
  include Language::Python::Virtualenv

  desc "AI-powered interactive coding assistant"
  homepage "https://auto-coder.chat/"
  url "https://files.pythonhosted.org/packages/c3/78/102253fa844920e9c25d928eba36956ebeb65c7dcf89474ec56bcd994b24/auto_coder-0.1.243.tar.gz"
  sha256 "e25645b44b3e36e6d673f61a5e50d198cd3804079d9e955bbb0b449bc05b8da1"
  license "Apache-2.0"

  depends_on "[email protected]"

  def install
    venv = virtualenv_install_with_resources
  end

  test do
    system "false"
  end
end

And tried brew update-python-resources to update the resources, it failed again:

brew update-python-resources --verbose --debug auto-coder


/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading auto-coder
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
==> Retrieving PyPI dependencies for "auto-coder==0.1.226"...
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
ERROR: Could not find a version that satisfies the requirement Spire.Doc (from auto-coder) (from versions: none)
ERROR: No matching distribution found for Spire.Doc
Error: Unable to determine dependencies for "auto-coder==0.1.226" because of a failure when running
`/opt/homebrew/opt/[email protected]/libexec/bin/python -m pip install -q --disable-pip-version-check --dry-run --ignore-installed --report=/dev/stdout auto-coder==0.1.226`.
Please update the resources manually.
Error: Kernel.exit
/opt/homebrew/Library/Homebrew/extend/kernel.rb:115:in `exit'
/opt/homebrew/Library/Homebrew/extend/kernel.rb:115:in `odie'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `bind_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `validate_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:442:in `pip_report'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `bind_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `validate_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:340:in `update_python_resources!'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `bind_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `validate_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:39:in `block in run'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:38:in `each'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:38:in `run'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `bind_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `validate_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
/opt/homebrew/Library/Homebrew/brew.rb:95:in `<main>'

Later, i tried all methods on https://docs.brew.sh/Python-for-Formula-Authors, including homebrew-pypi-poet:

poet -f auto-coder

But It didn't go well, some resources missing like this:

...
  resource "dashscope" do
    url ""
    sha256 ""
  end

  resource "google-generativeai" do
    url ""
    sha256 ""
  end

  resource "ray" do
    url ""
    sha256 ""
  end
...

Then I found this: tdsmith/homebrew-pypi-poet#74

I got puzzle, It seems like a loop!

What did you expect to happen?

I expected to create a formula for a python app with the right resources.

Step-by-step reproduction instructions (by running brew commands)

brew create --verbose --set-name auto-coder --set-license Apache-2.0 --python https://files.pythonhosted.org/packages/c3/78/102253fa844920e9c25d928eba36956ebeb65c7dcf89474ec56bcd994b24/auto_coder-0.1.243.tar.gz
brew update-python-resources --verbose --debug auto-coder
@dz85 dz85 added the bug Reproducible Homebrew/brew bug label Feb 5, 2025
@gromgit
Copy link
Contributor

gromgit commented Feb 5, 2025

Error: No available formula with the name "[email protected]". Did you mean python-yq, [email protected], [email protected], [email protected], [email protected], [email protected] or [email protected]?

This is an unfortunate side-effect of #19173. It inserts depends_on "[email protected]" with the expectation that the formula creator would substitute the appropriate Python version, but brew create also checks the formula it generates, hence the error thrown.

If you remove your current auto-coder-rb and do the following:

brew create --verbose --set-name auto-coder --set-license Apache-2.0 --python https://files.pythonhosted.org/packages/c3/78/102253fa844920e9c25d928eba36956ebeb65c7dcf89474ec56bcd994b24/auto_coder-0.1.243.tar.gz
sed -i -e 's/[email protected]/[email protected]/' $(brew --repo homebrew/core)/Formula/a/auto-coder.rb
brew update-python-resources --verbose --debug auto-coder

do you still get any errors aside from the above mentioned one? And are the required auto-coder resources now properly populated in the formula?

@dz85
Copy link
Author

dz85 commented Feb 5, 2025

Still failed:

brew update-python-resources --verbose --debug auto-coder

/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading auto-coder
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
==> Retrieving PyPI dependencies for "auto-coder==0.1.243"...
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
==> Retrieving PyPI dependencies for excluded ""...
==> Getting PyPI info for "aiohappyeyeballs==2.4.4"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/aiohappyeyeballs/2.4.4/json --location --fail
==> Getting PyPI info for "aiohttp==3.11.11"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/aiohttp/3.11.11/json --location --fail
==> Getting PyPI info for "aiohttp-cors==0.7.0"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/aiohttp-cors/0.7.0/json --location --fail
==> Getting PyPI info for "aiosignal==1.3.2"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/aiosignal/1.3.2/json --location --fail
==> Getting PyPI info for "annotated-types==0.7.0"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/annotated-types/0.7.0/json --location --fail
==> Getting PyPI info for "anthropic==0.45.2"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/anthropic/0.45.2/json --location --fail
==> Getting PyPI info for "anyio==4.8.0"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/anyio/4.8.0/json --location --fail
==> Getting PyPI info for "appscript==1.3.0"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/appscript/1.3.0/json --location --fail
==> Getting PyPI info for "astroid==3.3.8"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/astroid/3.3.8/json --location --fail
==> Getting PyPI info for "attrs==25.1.0"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/attrs/25.1.0/json --location --fail
==> Excluding "auto-coder==0.1.243"
==> Getting PyPI info for "azure-cognitiveservices-speech==1.42.0"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/azure-cognitiveservices-speech/1.42.0/json --location --fail
Error: Unable to resolve some dependencies. Please update the resources for "auto-coder" manually.
Error: Kernel.exit
/opt/homebrew/Library/Homebrew/extend/kernel.rb:115:in `exit'
/opt/homebrew/Library/Homebrew/extend/kernel.rb:115:in `odie'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `bind_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `validate_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:361:in `block in update_python_resources!'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:350:in `each'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:350:in `update_python_resources!'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `bind_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `validate_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:39:in `block in run'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:38:in `each'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:38:in `run'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `bind_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/call_validation.rb:282:in `validate_call'
/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11796/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
/opt/homebrew/Library/Homebrew/brew.rb:95:in `<main>'

@dz85
Copy link
Author

dz85 commented Feb 5, 2025

And I tried this:

curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-37-g1117ebc\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.2\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/azure-cognitiveservices-speech/1.42.0/json --location --fail

It responded:

{"info":{"author":"Microsoft","author_email":null,"bugtrack_url":null,"classifiers":["Development Status :: 5 - Production/Stable","Intended Audience :: Developers","License :: Other/Proprietary License","Operating System :: MacOS :: MacOS X","Operating System :: Microsoft :: Windows","Operating System :: POSIX :: Linux","Programming Language :: Python","Topic :: Scientific/Engineering","Topic :: Software Development :: Libraries :: Python Modules"],"description":"\nFor an introduction to this package, have a look at `the quickstart\narticle <https://learn.microsoft.com/azure/cognitive-services/speech-service/get-started-speech-to-text?pivots=programming-language-python>`_.\n\nFor information about the Speech Service, please refer to `its\nwebsite <https://learn.microsoft.com/azure/cognitive-services/speech-service/>`_.\n\nDocumentation\n-------------\n\nAPI documentation for this package can be found `here <https://aka.ms/csspeech/pythonref>`_.\n\nLicense information\n-------------------\n\n- `Microsoft Software License Terms for the Speech SDK <https://aka.ms/csspeech/license/>`_\n- `Third party notices <https://aka.ms/csspeech/toctpn/>`_\n","description_content_type":null,"docs_url":null,"download_url":null,"downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":null,"keywords":null,"license":null,"license_expression":null,"license_files":null,"maintainer":null,"maintainer_email":null,"name":"azure-cognitiveservices-speech","package_url":"https://pypi.org/project/azure-cognitiveservices-speech/","platform":null,"project_url":"https://pypi.org/project/azure-cognitiveservices-speech/","project_urls":null,"provides_extra":null,"release_url":"https://pypi.org/project/azure-cognitiveservices-speech/1.42.0/","requires_dist":null,"requires_python":">=3.7","summary":"Microsoft Cognitive Services Speech SDK for Python","version":"1.42.0","yanked":false,"yanked_reason":null},"last_serial":26929981,"urls":[{"comment_text":"","digests":{"blake2b_256":"861d07fc84ab9590fae9cc66a789d1971a0e3494e605e1787879c3581c5a385a","md5":"d6ceff2996e7977bc696396aa27a79be","sha256":"ad45a18ad6973a4fa2dbd4d71ded3a1a02c4dbbf13696b08f7a16f4156dddce7"},"downloads":-1,"filename":"azure_cognitiveservices_speech-1.42.0-py3-none-macosx_10_14_x86_64.whl","has_sig":false,"md5_digest":"d6ceff2996e7977bc696396aa27a79be","packagetype":"bdist_wheel","python_version":"py3","requires_python":">=3.7","size":7420332,"upload_time":"2025-01-13T22:10:18","upload_time_iso_8601":"2025-01-13T22:10:18.831551Z","url":"https://files.pythonhosted.org/packages/86/1d/07fc84ab9590fae9cc66a789d1971a0e3494e605e1787879c3581c5a385a/azure_cognitiveservices_speech-1.42.0-py3-none-macosx_10_14_x86_64.whl","yanked":false,"yanked_reason":null},{"comment_text":"","digests":{"blake2b_256":"a1727ebe03784b220b9adece692adc31ec6e4a1bac96c3e9d3fef511b5ca08aa","md5":"26eee756dffb4b0d2b7ccbcb744d48db","sha256":"9105a64a9d83044790f4f8c9358b6ea66a7c042cbd67173db303501782e62d3f"},"downloads":-1,"filename":"azure_cognitiveservices_speech-1.42.0-py3-none-macosx_11_0_arm64.whl","has_sig":false,"md5_digest":"26eee756dffb4b0d2b7ccbcb744d48db","packagetype":"bdist_wheel","python_version":"py3","requires_python":">=3.7","size":7277345,"upload_time":"2025-01-13T22:10:24","upload_time_iso_8601":"2025-01-13T22:10:24.230697Z","url":"https://files.pythonhosted.org/packages/a1/72/7ebe03784b220b9adece692adc31ec6e4a1bac96c3e9d3fef511b5ca08aa/azure_cognitiveservices_speech-1.42.0-py3-none-macosx_11_0_arm64.whl","yanked":false,"yanked_reason":null},{"comment_text":"","digests":{"blake2b_256":"83f79241ad7154e554730ea56271e14ad1115c278b26a81eb892eac16fabb480","md5":"f322a0c994487fa351d59a591b903e79","sha256":"90890a147499239f37b0b1a5112c51820b90fa2b5adafa0df4da6cc0c211887f"},"downloads":-1,"filename":"azure_cognitiveservices_speech-1.42.0-py3-none-manylinux1_x86_64.whl","has_sig":false,"md5_digest":"f322a0c994487fa351d59a591b903e79","packagetype":"bdist_wheel","python_version":"py3","requires_python":">=3.7","size":39727186,"upload_time":"2025-01-13T22:10:01","upload_time_iso_8601":"2025-01-13T22:10:01.628198Z","url":"https://files.pythonhosted.org/packages/83/f7/9241ad7154e554730ea56271e14ad1115c278b26a81eb892eac16fabb480/azure_cognitiveservices_speech-1.42.0-py3-none-manylinux1_x86_64.whl","yanked":false,"yanked_reason":null},{"comment_text":"","digests":{"blake2b_256":"fcfdaf607bdfa95306b13fcdeadcd48d28b80b27cc5e3b99e2bde96f6212cd3a","md5":"d8628a6ec47f0bca936e52c6cc1acb33","sha256":"106fbdb165a215cada47d7e95851e0b9d2755a3f2355369bab4915ad001efe89"},"downloads":-1,"filename":"azure_cognitiveservices_speech-1.42.0-py3-none-manylinux2014_aarch64.whl","has_sig":false,"md5_digest":"d8628a6ec47f0bca936e52c6cc1acb33","packagetype":"bdist_wheel","python_version":"py3","requires_python":">=3.7","size":39508123,"upload_time":"2025-01-13T22:10:13","upload_time_iso_8601":"2025-01-13T22:10:13.719217Z","url":"https://files.pythonhosted.org/packages/fc/fd/af607bdfa95306b13fcdeadcd48d28b80b27cc5e3b99e2bde96f6212cd3a/azure_cognitiveservices_speech-1.42.0-py3-none-manylinux2014_aarch64.whl","yanked":false,"yanked_reason":null},{"comment_text":"","digests":{"blake2b_256":"17fb1c998efbfcb1e44f9dc4dbb8b182ea3e5287fdc167aa352aef4685e29435","md5":"b79ab1d92ec2be7f00fa9c06fda65e27","sha256":"7d57218beec24360a8b7ce89755c2c133259e3411c233ef0a659b951e4c4c904"},"downloads":-1,"filename":"azure_cognitiveservices_speech-1.42.0-py3-none-win32.whl","has_sig":false,"md5_digest":"b79ab1d92ec2be7f00fa9c06fda65e27","packagetype":"bdist_wheel","python_version":"py3","requires_python":">=3.7","size":2109807,"upload_time":"2025-01-13T22:09:50","upload_time_iso_8601":"2025-01-13T22:09:50.516872Z","url":"https://files.pythonhosted.org/packages/17/fb/1c998efbfcb1e44f9dc4dbb8b182ea3e5287fdc167aa352aef4685e29435/azure_cognitiveservices_speech-1.42.0-py3-none-win32.whl","yanked":false,"yanked_reason":null},{"comment_text":"","digests":{"blake2b_256":"52bbef7a29f5717cca646be6698d80e542446a6a442be897c8f67bf93551c672","md5":"19d238739167a95d65ae0ff9d726f5d7","sha256":"32076ee03b3b402a2e8841f2c21e5cd54dc3ffbf5af183426344727298c8bbd4"},"downloads":-1,"filename":"azure_cognitiveservices_speech-1.42.0-py3-none-win_amd64.whl","has_sig":false,"md5_digest":"19d238739167a95d65ae0ff9d726f5d7","packagetype":"bdist_wheel","python_version":"py3","requires_python":">=3.7","size":2377971,"upload_time":"2025-01-13T22:09:44","upload_time_iso_8601":"2025-01-13T22:09:44.706563Z","url":"https://files.pythonhosted.org/packages/52/bb/ef7a29f5717cca646be6698d80e542446a6a442be897c8f67bf93551c672/azure_cognitiveservices_speech-1.42.0-py3-none-win_amd64.whl","yanked":false,"yanked_reason":null}],"vulnerabilities":[]}

Is there anything missing in this json?
@gromgit

gromgit added a commit to gromgit/homebrew-brew that referenced this issue Feb 5, 2025
Followup to Homebrew#19173, which causes `brew create` to error out on the literal `[email protected]` dependency.

Related to Homebrew#19240.
@gromgit
Copy link
Contributor

gromgit commented Feb 5, 2025

Is there anything missing in this json?

Sorry, not a clue. @Homebrew/maintainers, any idea?

gromgit added a commit to gromgit/homebrew-brew that referenced this issue Feb 5, 2025
Followup to Homebrew#19173, which causes `brew create` to error out on the literal `[email protected]` dependency.

Related to Homebrew#19240.
gromgit added a commit to gromgit/homebrew-brew that referenced this issue Feb 5, 2025
Followup to Homebrew#19173, which causes `brew create` to error out on the literal `[email protected]` dependency.

Related to Homebrew#19240.
@MikeMcQuaid
Copy link
Member

CC @woodruffw @cho-m who may know about the PyPI JSON?

@cho-m
Copy link
Member

cho-m commented Feb 5, 2025

https://pypi.org/project/azure-cognitiveservices-speech/#files is the issue.

It doesn't have a source distribution. Also has proprietary license1 so unlikely to be open-source / source-available.

update-python-resources is written mainly for homebrew/core usage so it only looks for source distributions or pure python "any" wheels to satisfy our requirements.

Footnotes

  1. https://csspeechstorage.blob.core.windows.net/drop/license_202311.html

@MikeMcQuaid
Copy link
Member

Thanks @cho-m! What, if anything, do you think the fix is here; just a better error message?

@gromgit
Copy link
Contributor

gromgit commented Feb 5, 2025

Looking at the relevant code block:

ohai "Getting PyPI info for \"#{package}\"" if show_info
name, url, checksum = package.pypi_info
# Fail if unable to find name, url or checksum for any resource
if name.blank?
odie "Unable to resolve some dependencies. Please update the resources for \"#{formula.name}\" manually."
elsif url.blank? || checksum.blank?
odie <<~EOS
Unable to find the URL and/or sha256 for the "#{name}" resource.
Please update the resources for "#{formula.name}" manually.
EOS
end

can I suggest that if any of name, url, or checksum is blank, that a commented resource block be added to the formula with whatever info is available? Something like this for the OP's problematic package:

# ERROR: Unable to resolve "azure-cognitiveservices-speech==1.42.0"
# resource "" do
#   url ""
#   sha256 ""
# end

Even if nothing is known about a package, I think it would be good to leave a placeholder resource block identifying the problem package, and continue processing/recording the resources that can be resolved. That way, formula writers aren't left wondering with no resources at all, when a single unresolved dependency crash-halts the entire process.

I'm currently testing a fix that does the above, and it turns out auto-coder requires 215 resources, only two of which (azure-cognitiveservices-speech and ray) need to be resolved by hand, so I personally feel this is the user-friendly way to go.

@woodruffw
Copy link
Member

It doesn't have a source distribution. Also has proprietary license1 so unlikely to be open-source / source-available.

update-python-resources is written mainly for homebrew/core usage so it only looks for source distributions or pure python "any" wheels to satisfy our requirements.

Yep. I think we could definitely improve the error message here, i.e. emit a more specific warning when the resource-ification fails not because the release is missing but because it doesn't provide the kind(s) of files we want. I can look into that.

@woodruffw
Copy link
Member

Very small, but this will at least make the error message in the missing sources case better: #19250

@dz85
Copy link
Author

dz85 commented Feb 6, 2025

Thank you all for your work. And when the users find the resources need to be resolved by hand as update-python-resources which missing the source distributions, WHAT can they do? HOW can they find the right resource code, such as url, sha256 etc.

https://pypi.org/project/azure-cognitiveservices-speech/#files is the issue.

It doesn't have a source distribution. Also has proprietary license1 so unlikely to be open-source / source-available.

update-python-resources is written mainly for homebrew/core usage so it only looks for source distributions or pure python "any" wheels to satisfy our requirements.

Footnotes

  1. https://csspeechstorage.blob.core.windows.net/drop/license_202311.html

gromgit added a commit to gromgit/homebrew-brew that referenced this issue Feb 6, 2025
This is particularly useful for third-party Python formulae that have a ton of resources, not all of which may adhere to homebrew/core's strict policies. See Homebrew#19240 for context.

I've also added logic that ignores `--ignore-errors` on `homebrew/core`, although I personally think this new behavior is also useful for mainline formula creation.

Before: error out on a single non-conforming resource, zero resource blocks added to formula, scary stacktrace.

After: all conforming resources added, all non-conforming resources identified in comments, error message at end, `brew` exits non-zero without scary stacktrace:-
```
% brew update-python-resources --ignore-errors gromgit/test/auto-coder || echo OOPS
==> Retrieving PyPI dependencies for "auto-coder==0.1.243"...
==> Retrieving PyPI dependencies for excluded ""...
==> Getting PyPI info for "aiohappyeyeballs==2.4.4"
[200+ resource lines elided]
==> Getting PyPI info for "zhipuai==2.1.5.20250106"
==> Updating resource blocks
Error: Unable to resolve some dependencies. Please check /opt/homebrew/Library/Taps/gromgit/homebrew-test/Formula/auto-coder.rb for RESOURCE-ERROR comments.
OOPS

% brew cat gromgit/test/auto-coder | ggrep -C10 RESOURCE-ERROR
  license "Apache-2.0"

  depends_on "[email protected]"

  # Additional dependency
  # resource "" do
  #   url ""
  #   sha256 ""
  # end

  # RESOURCE-ERROR: Unable to resolve "azure-cognitiveservices-speech==1.42.0" (no suitable source distribution on PyPI)
  # RESOURCE-ERROR: Unable to resolve "ray==2.42.0" (no suitable source distribution on PyPI)

  resource "aiohappyeyeballs" do
    url "https://files.pythonhosted.org/packages/7f/55/e4373e888fdacb15563ef6fa9fa8c8252476ea071e96fb46defac9f18bf2/aiohappyeyeballs-2.4.4.tar.gz"
    sha256 "5fdd7d87889c63183afc18ce9271f9b0a7d32c2303e394468dd45d514a757745"
  end

  resource "aiohttp" do
    url "https://files.pythonhosted.org/packages/37/4b/952d49c73084fb790cb5c6ead50848c8e96b4980ad806cf4d2ad341eaa03/aiohttp-3.11.12.tar.gz"
    sha256 "7603ca26d75b1b86160ce1bbe2787a0b706e592af5b2504e12caa88a217767b0"
  end
```
@gromgit
Copy link
Contributor

gromgit commented Feb 6, 2025

And when the users find the resources need to be resolved by hand as update-python-resources which missing the source distributions, WHAT can they do? HOW can they find the right resource code, such as url, sha256 etc.

You should probably check first with the module authors, to see if they forgot to provide sources. If no sources will be provisioned, then I seriously doubt the Homebrew maintainers will allow it on the core tap, so you'd have to host it on your own tap, then you can point to the relevant binary-only wheels.

As for filling out the resource stanza, the URLs should be available from the module's download page, and the SHA256 checksums can be calculated trivially:

curl -sL <wheel_URL> | sha256sum -

I don't know if resource blocks support on_<arch> and on_<os> blocks though.

@dz85
Copy link
Author

dz85 commented Feb 7, 2025

But after I tried the newest Homebrew, I'm sorry to say it seems not work:

homebrew (master)> pwd
/opt/homebrew
homebrew (master)> git log --oneline

9e3ae9f3f9 (HEAD -> master, origin/master, origin/HEAD) Merge pull request #19251 from Homebrew/revert-json-v3-logic
...
bd7a3cc1df Merge pull request #19252 from gromgit/update-python-resources/ignore-errors
956b71eeed update-python-resources: add option to ignore errors
...
46d67ba560 utils/pypi: warn when `pypi_info` fails due to missing sources
...
homebrew (master)> cat Library/Homebrew/utils/pypi.rb | grep RESOURCE-ERROR                                                                                                                                                  (base)
        package_errors += "  # RESOURCE-ERROR: Unable to resolve \"#{package}\" (#{package_error})\n"
        (\#\ RESOURCE-ERROR:\ .*\s+)*
      ofail "Unable to resolve some dependencies. Please check #{formula.path} for RESOURCE-ERROR comments."
homebrew (master)> brew update-python-resources --ignore-errors --debug auto-coder || echo OOPS

/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading auto-coder
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
==> Retrieving PyPI dependencies for "auto-coder==0.1.250"...
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::FromNameLoader): loading [email protected]
==> Retrieving PyPI dependencies for excluded ""...
==> Getting PyPI info for "aiohappyeyeballs==2.4.4"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-84-g9e3ae9f\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.3\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/aiohappyeyeballs/2.4.4/json --location --fail
...
==> Excluding "auto-coder==0.1.250"
==> Getting PyPI info for "azure-cognitiveservices-speech==1.42.0"
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.20-84-g9e3ae9f\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.3\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 https://pypi.org/pypi/azure-cognitiveservices-speech/1.42.0/json --location --fail
Error: azure-cognitiveservices-speech exists on PyPI but lacks a suitable source distribution
Error: Unable to resolve some dependencies. Please update the resources for "auto-coder" manually.
Error: Kernel.exit
Warning: Removed Sorbet lines from backtrace!
Rerun with `--verbose` to see the original backtrace
/opt/homebrew/Library/Homebrew/extend/kernel.rb:115:in `exit'
/opt/homebrew/Library/Homebrew/extend/kernel.rb:115:in `odie'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:375:in `block in update_python_resources!'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:360:in `each'
/opt/homebrew/Library/Homebrew/utils/pypi.rb:360:in `update_python_resources!'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:47:in `block in run'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:41:in `each'
/opt/homebrew/Library/Homebrew/dev-cmd/update-python-resources.rb:41:in `run'
/opt/homebrew/Library/Homebrew/brew.rb:95:in `<main>'
OOPS

And my rb file is created by brew create ...:

# Documentation: https://docs.brew.sh/Formula-Cookbook
#                https://rubydoc.brew.sh/Formula
# PLEASE REMOVE ALL GENERATED COMMENTS BEFORE SUBMITTING YOUR PULL REQUEST!
class AutoCoder < Formula
  include Language::Python::Virtualenv

  desc ""
  homepage ""
  url "https://files.pythonhosted.org/packages/e9/08/935554bf82eaf3ae60c9ad3af80a4449c061018d773c27da860cd572364e/auto_coder-0.1.250.tar.gz"
  sha256 "f9a169013ac43ed521ba749931ad07da61c29ded2788944767d541190dadc7a4"
  license "Apache-2.0"

  depends_on "[email protected]"

  # Additional dependency
  # resource "" do
  #   url ""
  #   sha256 ""
  # end

  def install
    virtualenv_install_with_resources
  end

  test do
    # `test do` will create, run in and delete a temporary directory.
    #
    # This test will fail and we won't accept that! For Homebrew/homebrew-core
    # this will need to be a test that verifies the functionality of the
    # software. Run the test with `brew test auto-coder`. Options passed
    # to `brew install` such as `--HEAD` also need to be provided to `brew test`.
    #
    # The installed folder is not in the path, so use the entire path to any
    # executables being tested: `system bin/"program", "do", "something"`.
    system "false"
  end
end

@gromgit

@gromgit
Copy link
Contributor

gromgit commented Feb 7, 2025

It's on purpose:

% brew update-python-resources --help
[...]
      --ignore-errors              Record all discovered resources, even those
                                   that can't be resolved successfully. This
                                   option is ignored for homebrew/core formulae.

Note that last statement. Also see the discussion in #19252.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Reproducible Homebrew/brew bug
Projects
None yet
Development

No branches or pull requests

5 participants