We need a more robust link checking solution #336
Description
Recently, the docs checks have been failing a lot due to the link checker getting rate limited. We use the built-in sphinx link checker currently. It would be fine to keep using it if we can configure it appropriately, or perhaps write a custom extension that inherits its functionality and does what we want. It seems like a more robust solution might be to only check links that have changed in PRs, and check links as a whole on a slower schedule, like a weekly cron job (e.g. using a Github artifact to track which links have already been validated in the last week).
#335 disables link checking for now.