Added Pipeline for scheduled link rot checker

[krishnaduttPanchagnula]

Closes #3635

[Copilot]

Pull Request Overview

This pull request adds a scheduled link rot checker that scans markdown files for links and validates them by making HEAD requests.

• Added a Python script that extracts and processes URLs from markdown files.
• Introduced a GitHub Actions workflow to schedule the execution of the link rot checker.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
hack/lin-rot-checker.py	New script to extract URLs from markdown files and verify links.
.github/workflows/lin-rot-checker.yml	Workflow configuration to run the link rot checker on a schedule.

[Copilot]

Consider adding a timeout to the requests.head call (e.g., requests.head(link, timeout=5)) to prevent the script from hanging on unresponsive links.

Suggested change

	if requests.head(link).status_code==200:
	if requests.head(link, timeout=5).status_code==200:

[jandubois]

I think we don't care about the languages used for external tools, as long as they are easy to install both locally and in CI.

But for tools that should become part of our repo, unless there is a really good reason, they should be written in Go or bash¹.

Footnotes

1. I realize that is somewhat ironic, given that I wrote test-port-forwarding.pl in Perl, but that was ages ago, and I would now argue that it should be written in Go instead, if it didn't already exist. ↩

[krishnaduttPanchagnula]

@jandubois Can you please review and suggest if this cron time works.

[alexandear]

Please fix a typo in the PR's title: "pipline" -> "pipeline". Also, update the title to match implementation (we don't have script anymore).

[AkihiroSuda]

Please squash the commits

[krishnaduttPanchagnula]

@alexandear can we rerun this pipeline. My changes are regarding the new pipeline and should not be affecting any integration tests.

[alexandear]

Currently, the workflow fails when running locally with the help of act:

❯ act -j linkChecker --container-architecture linux/amd64
INFO[0000] Using docker host 'unix:///Users/alexandear/.colima/default/docker.sock', and daemon socket 'unix:///Users/alexandear/.colima/default/docker.sock' 
[Automated Link Health Check/linkChecker] ⭐ Run Set up job
[Automated Link Health Check/linkChecker] 🚀  Start image=node:16-buster-slim
[Automated Link Health Check/linkChecker]   🐳  docker pull image=node:16-buster-slim platform=linux/amd64 username= forcePull=true
[Automated Link Health Check/linkChecker]   🐳  docker create image=node:16-buster-slim platform=linux/amd64 entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[Automated Link Health Check/linkChecker]   🐳  docker run image=node:16-buster-slim platform=linux/amd64 entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[Automated Link Health Check/linkChecker]   🐳  docker exec cmd=[node --no-warnings -e console.log(process.execPath)] user= workdir=
[Automated Link Health Check/linkChecker]   ✅  Success - Set up job
[Automated Link Health Check/linkChecker]   ☁  git clone 'https://github.com/lycheeverse/lychee-action' # ref=82202e5e9c2f4ef1a55a3d02563e1cb6041e5332i
[Automated Link Health Check/linkChecker] Unable to resolve 82202e5e9c2f4ef1a55a3d02563e1cb6041e5332i: reference not found
[Automated Link Health Check/linkChecker] Unable to resolve 82202e5e9c2f4ef1a55a3d02563e1cb6041e5332i: reference not found
[Automated Link Health Check/linkChecker] reference not found
[Automated Link Health Check/linkChecker] ⭐ Run Complete job
[Automated Link Health Check/linkChecker]   ✅  Success - Complete job
[Automated Link Health Check/linkChecker] 🏁  Job failed
Error: reference not found

[alexandear]

Please add a whitespace after # in comments. E.g., should be:

# v4.2.2

[alexandear]

I propose adding this step to output results to a log:

Suggested change

	output: ./lychee/out.md
	output: ./lychee/out.md
	- name: Show Report
	run: cat lychee/out.md

[alexandear]

It makes sense to add these args (inspired https://github.com/open-policy-agent/opa/blob/f72110de200666efeed73887707f8207a767ec9a/.github/workflows/link-checker.yaml):

Suggested change

	with:
	with:
	args: |
	--max-concurrency 1 \
	--no-progress \
	--scheme https \
	--scheme http