Improving brew livecheck and autobump

[nandahkrishna]

Verification

• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

Provide a detailed description of the proposed feature

• For synced–version packages, check the JSON and loop over all these packages to simultaneously bump them – @bevanjkay
  • See brew bump refuses to bump linked formulae #16383
• Livecheck: adding POST requests with parameters – @samford
  • See livecheck: Add support for POST requests #19233.
• Livecheck for resources: PyPI, RubyGems, etc. formulae should have livecheck return the right dependency versions instead of the latest – @nandahkrishna
  • Add strategies or utils/rubygems.rb like utils/pypi.rb specific to resource-handling
  • Add livecheck blocks to resources in homebrew-core wherever required
• GitHub pre-releases: policy to change livecheck strategy to :github_latest (perhaps upon asking upstream), especially for autobumped formulae – allows us to be cautious
  • How do we avoid rate limit issues? – sharding autobump.txt, GitHub App v/s PAT rate limit, separate token for livecheck?

People involved: @chenrui333 @bevanjkay @Moisan @nandahkrishna @samford

What is the motivation for the feature?

Improving livecheck means improving autobump, which means more automated PRs for version bumps and more happy maintainers, who have more time to work on non-trivial feature implementations.

Instituting a clear policy for :github_latest means that maintainers can stop checking the upstream repository if the newest tag is marked as latest. While this would mean having to check if upstream has stopped marking tags as latest, these checks are easy to automate and the policy means we err on the side of caution, instead of bumping to versions that could be retagged/experimental.

How will the feature be relevant to at least 90% of Homebrew users?

Yes, they get reliable package updates as early as possible, and maintainers have time to deal with more pressing issues than trivial version bumps.

What alternatives to the feature have been considered?

Each of these action items represents a separate feature, some of which have alternatives listed, while others are straightforward.

[samford]

Nanda and I were discussing this earlier but, for the sake of visibility, I'm reworking my local Post strategy branch into a standalone implementation and will create a draft PR once that's done (hopefully later today or tomorrow). The existing implementation was based on the forthcoming options feature and I had been planning to add the Post strategy afterward but it can technically work without it. [For what it's worth, the options feature has been done for quite some time but I've been working on PRs to unblock it (e.g., Utils::Curl changes), so it makes sense to add Post now rather than later.]

There's naturally a bit of overlap with some areas of the options feature but I've come up with some short-term solutions that we can use in the interim time. The required changes aren't terrible and they'll be cleaned up as part of the options PR, so it should be fine.

[samford]

I finished refactoring my Post strategy, had written tests, and while writing documentation I happened to think, "Hmm...but how will this parse a JSON/XML/YAML/etc. response?" That got me thinking that it may be better to handle POST requests as url options (instead of a separate strategy), like we do for the main formula url (e.g., url "...", using: :post, data: ...).

That would allow us to use POST for any strategy that uses curl (via Strategy::page_headers or ::page_content) and we could use the existing strategies in the same way we already do. That's likely the more sustainable solution, as adding parsing logic to the Post strategy would require us to update it for any new format we parse (i.e., adding maintenance burden, more things that can break if we forget).

I went ahead and reimplemented POST support using the aforementioned approach. Thankfully I was able to reuse a lot of the existing logic from my previous implementation (i.e., I had already done the hard work). I created a PR at #19233, so feel free to take a look and test it out.

Edit: POST support has been merged. You can test it out but it's not in a tagged brew release, so we can't use it in formulae/casks yet.