Migrate validation to Jekyll hooks (endoflife-date#2859)

Compared to the use of the validate.py script / validate.yaml workflow, this has the advantage of ensuring the validation is always performed. It also gives access to enriched data, which is useful for validated generated content, such as links.

In addition to checks already implemented in the previous 'validate.py' script, this script:

- Check that release dates are not too far in the future to avoid mistakes such as endoflife-date#3281 (max 30 days allowed).
- Check all links are valid (even those in the products description). This check takes time and is not performed by default, but it can be triggered by setting the MUST_CHECK_URLS environment variable to true. To ensure links are regularly verified a new GitHub workflow, check-links.yml, has also been added. It runs every sunday at midnight.

Note that:

- Some links had to be excluded from the link validation because, despite being valid, they are sometime or systematically returning an error.
- A dependency to open-uri (https://github.com/ruby/open-uri) has been added to simplify the validation of HTTP links.
- The priority of the ProductDataEnricher plugin had to be set to normal, to allow hooks to be run before and after enrichment.
- The deploy.sh has been cleaned-up and enriched with the deploy URL replacement originaly done in netlify.toml.

Some alternate URLs has been changed or removed due to an enforcement of the rules regarding permalinks and alternate_urls (no more dots).

Closes endoflife-date#1206, endoflife-date#2709, endoflife-date#3409.

.github/dependabot.yml

@@ -10,6 +10,7 @@ updates:
     directory: "/" # Location of package manifests
     schedule:
       interval: "daily"
+
   # This updates the _data/release-data submodule once a day
   # which itself is updated twice a day.
   - package-ecosystem: "gitsubmodule"

.github/workflows/auto-merge-release-updates.yml

@@ -43,7 +43,6 @@ jobs:
           status_options: '--untracked-files=no'
           commit_author: 'github-actions[bot] <github-actions[bot]@users.noreply.github.com>'
 
-      # TODO: Leave a comment if there is a new major release that we don't match
       - name: Enable auto-merge for Dependabot PRs for release data
         if: ${{contains(steps.metadata.outputs.dependency-names, '_data/release-data')}}
         run: gh pr merge --auto --rebase "$PR_URL"

.github/workflows/check-links.yml

@@ -0,0 +1,23 @@
+name: Check URLs
+
+on:
+  schedule:
+    - cron: '0 0 * * 0' # At 00:00 on Sunday.
+
+jobs:
+  check_urls:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout site
+        uses: actions/checkout@v2
+
+      - name: Setup ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: 3.1
+          bundler-cache: true
+
+      - name: Perform URLs check
+        run: bundle exec jekyll build
+        env:
+          MUST_CHECK_URLS: true

Gemfile

@@ -25,3 +25,6 @@ gem "wdm", "~> 0.1.0" if Gem.win_platform?
 gem "webrick", "~> 1.8"
 
 gem 'icalendar', '~> 2.9'
+
+# Used in product-data-validator to check URLs
+gem "open-uri", "~> 0.3"

Gemfile.lock

@@ -10,6 +10,7 @@ GEM
       public_suffix (>= 2.0.2, < 6.0)
     colorator (1.1.0)
     concurrent-ruby (1.2.2)
+    date (3.3.3)
     em-websocket (0.5.3)
       eventmachine (>= 0.12.9)
       http_parser.rb (~> 0)
@@ -80,6 +81,10 @@ GEM
       racc (~> 1.4)
     nokogiri (1.15.3-x86_64-linux)
       racc (~> 1.4)
+    open-uri (0.3.0)
+      stringio
+      time
+      uri
     pathutil (0.16.2)
       forwardable-extended (~> 2.6)
     posix-spawn (0.3.15)
@@ -96,11 +101,15 @@ GEM
       google-protobuf (~> 3.23)
     sass-embedded (1.64.2-x86_64-linux-gnu)
       google-protobuf (~> 3.23)
+    stringio (3.0.8)
     terminal-table (3.0.2)
       unicode-display_width (>= 1.1.1, < 3)
+    time (0.2.2)
+      date
     tzinfo (2.0.6)
       concurrent-ruby (~> 1.0)
     unicode-display_width (2.4.2)
+    uri (0.12.2)
     webrick (1.8.1)
 
 PLATFORMS
@@ -116,6 +125,7 @@ DEPENDENCIES
   jekyll-timeago
   jemoji
   just-the-docs (~> 0.5.1)
+  open-uri (~> 0.3)
   tzinfo-data
   webrick (~> 1.8)
 

_auto/deploy.sh

@@ -1,15 +1,24 @@
-#!/bin/bash
+#!/bin/bash -e
 
-### Update latest information
-# See https://github.com/endoflife-date/endoflife.date/pull/2081
-git submodule update --remote
+# Display context information
+echo "Current commit: $(git rev-parse HEAD)"
+echo "Ruby version: $(ruby --version)"
+echo "Python version: $(python --version)"
+echo "Jekyll version: $(bundle exec jekyll --version)"
+echo "Deploy URL: $1"
+
+# Update latest product information, see https://github.com/endoflife-date/endoflife.date/pull/2081
 pip install -r requirements.txt
-# If the latest.py script fails,
-# We don't want to raise any errors
-# just undo the changes, and carry on
-if ! python3 _auto/latest.py ; then
-  git checkout -- products/
+git submodule update --remote
+if ! python3 _auto/latest.py ; then # if the latest.py script fails...
+  git checkout -- products/ # ...just undo the changes, and carry on
+fi
+
+# Replace the Deploy URL with the Preview URL
+if [ -n "$1" ]; then
+  echo "Replacing Deploy URL with Preview URL ($1)"
+  sed -i "/url\:/curl\: $1" _config.yml
 fi
 
-# Finally, do a build
-bundle exec jekyll build
+# Finally, do a full build
+bundle exec jekyll build --trace

_config.yml

@@ -71,6 +71,9 @@ defaults:
       path: "products"
     values:
       layout: product
+      alternate_urls: []
+      identifiers: []
+      auto: []
       releaseColumn: true
       releaseColumnLabel: 'Latest'
       releaseDateColumn: false

_plugins/product-data-enricher.rb

@@ -337,6 +337,6 @@ def render_eol_template(template, cycle)
   end
 end
 
-Jekyll::Hooks.register [:pages], :post_init do |page|
+Jekyll::Hooks.register [:pages], :post_init, priority: Jekyll::Hooks::PRIORITY_MAP[:normal] do |page|
   Jekyll::ProductDataEnricher.enrich(page) if Jekyll::ProductDataEnricher.is_product?(page)
 end