Skip to content

Commit

Permalink
CI: switch to GitHub Actions - step 4: test and coverage stage
Browse files Browse the repository at this point in the history
This commit:
* Adds a GH Actions workflow for the CI checks which were previously run on Travis in the `test` and `coverage` stages.
* Removes the, now redundant, `.travis.yml` configuration.
* Updates the `.gitattributes` file.
* Updates the "Build Status" badge in the Readme to use the results from the GH `Test` Actions runs.
* Updates the "Tested on" badge in the Readme to include PHP 8.
* Updates a test file which mentioned Travis.

Notes:
1. Each of these jobs has a `needs` dependency on the previous job to prevent it from starting if the previous job failed.
    While not 100% necessary, this is just an efficiency tweak, being kind to the free service being offered as we know that if linting fails, the tests will fail etc.
2. The builds in the `test` and `coverage` jobs are essentially the same as previously run on Travis, though PHP 8.0 is now fully accounted for and `8.1` is set as `nightly`.
3. Previously, this "stage" would run on all `pull requests` events.
    The current set-up still does so, with one addition: pushes to `master` (merges) will now also use this workflow instead of the quicktest.
    This replaces the full run on tagging a release, meaning that thing will essentially be the same.
4. As there are a couple of jobs which are "allowed to fail" (`experimental` = true), the build status will unfortunately show as "failed", even though all non-experimental jobs have succeeded.
     This is a known issue in GHA: https://github.com/actions/toolkit/issues/399
  • Loading branch information
jrfnl committed Dec 22, 2020
1 parent ff6212e commit 846075c
Show file tree
Hide file tree
Showing 6 changed files with 277 additions and 210 deletions.
1 change: 0 additions & 1 deletion .coveralls.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
coverage_clover: build/logs/clover.xml
json_path: build/logs/coveralls-upload.json
service_name: travis-ci
2 changes: 1 addition & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
/.coveralls.yml export-ignore
/.gitattributes export-ignore
/.gitignore export-ignore
/.travis.yml export-ignore
/.github/ export-ignore
/phpcs.xml.dist export-ignore
/phpdoc.xml.dist export-ignore
/phpunit.xml.dist export-ignore
Expand Down
272 changes: 272 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,272 @@
name: Test

on:
# Run on pushes to `master` and on all pull requests.
push:
branches:
- master
paths-ignore:
- '**.md'
- 'docs/**'
pull_request:
paths-ignore:
- '**.md'
- 'docs/**'

jobs:
#### TEST STAGE ####
test:
runs-on: ubuntu-latest

strategy:
# Keys:
# - risky: Whether to run with tests which check for being in sync with PHPCS.
# - experimental: Whether the build is "allowed to fail".
matrix:
# The GHA matrix works different from Travis.
# You can define jobs here and then augment them with extra variables in `include`,
# as well as add extra jobs in `include`.
# @link https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix
#
# IMPORTANT: test runs shouldn't fail because of PHPCS being incompatible with a PHP version.
# - PHPCS will run without errors on PHP 5.4 - 7.2 on any version.
# - PHP 7.3 needs PHPCS 2.9.2 and 3.3.1+ to run without errors.
# On PHPCS 2.x our tests won't fail though, but on PHPCS 3.x < 3.3.1 they will.
# - PHP 7.4 needs PHPCS 3.5.0+ to run without errors.
# On PHPCS 2.x our tests won't fail though, but on PHPCS 3.x < 3.5.0 they will.
# - PHP 8.0 needs PHPCS 3.5.7+ to run without errors.
#
# The matrix is set up so as not to duplicate the builds which are run for code coverage.
php: ['5.5', '5.6', '7.0', '7.1', '7.2']
phpcs_version: ['2.6.0', '2.9.2', '3.1.0', 'dev-master']
risky: [false]
experimental: [false]

include:
# Complement the builds run in code coverage to complete the matrix and prevent issues
# with PHPCS versions incompatible with certain PHP versions.
- php: '8.0'
phpcs_version: 'dev-master'
risky: false
experimental: false
- php: '8.0'
phpcs_version: '3.5.7'
risky: false
experimental: false

- php: '7.4'
phpcs_version: '3.5.0'
risky: false
experimental: false

- php: '7.3'
phpcs_version: 'dev-master'
risky: false
experimental: false
- php: '7.3'
phpcs_version: '3.3.1'
risky: false
experimental: false
- php: '7.3'
phpcs_version: '2.6.0'
risky: false
experimental: false

- php: '5.4'
phpcs_version: '3.1.0'
risky: false
experimental: false
- php: '5.4'
phpcs_version: '2.9.2'
risky: false
experimental: false

# One extra build to verify issues around PHPCS annotations when they weren't fully accounted for yet.
- php: '7.2'
phpcs_version: '3.2.0'
risky: false
experimental: false

# And a few more to verify various tokenizer issues in older PHPCS versions.
- php: '5.6'
phpcs_version: '2.8.1'
risky: false
experimental: false
- php: '7.0'
phpcs_version: '2.7.1'
risky: false
experimental: false

# Experimental builds. These are allowed to fail.

# Current lowest PHPCS version which _may_ run on PHP 8 is 3.5.0, so don't even try
# to test against older versions.
- php: '8.1'
phpcs_version: 'dev-master'
risky: false
experimental: true

- php: '7.4'
phpcs_version: '4.0.x-dev'
risky: false
experimental: true

# Run risky tests separately.
- php: '7.4'
phpcs_version: '4.0.x-dev'
risky: true
experimental: true

- php: '7.4'
phpcs_version: 'dev-master'
risky: true
experimental: true

name: "Test${{ matrix.phpcs_version == 'dev-master' && matrix.risky == false && ' + Lint' || '' }}: PHP ${{ matrix.php }} - PHPCS ${{ matrix.phpcs_version }}${{ matrix.risky == true && ' (risky)' || '' }}"

continue-on-error: ${{ matrix.experimental }}

steps:
- name: Checkout code
uses: actions/checkout@v2

# On stable PHPCS versions, allow for PHP deprecation notices.
# Unit tests don't need to fail on those for stable releases where those issues won't get fixed anymore.
- name: Setup ini config
id: set_ini
run: |
if [[ "${{ matrix.phpcs_version }}" != "dev-master" && "${{ matrix.phpcs_version }}" != "4.0.x-dev" ]]; then
echo '::set-output name=PHP_INI::error_reporting=E_ALL & ~E_DEPRECATED'
else
echo '::set-output name=PHP_INI::error_reporting=E_ALL'
fi
- name: Install PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
ini-values: ${{ steps.set_ini.outputs.PHP_INI }}
coverage: none

- name: 'Composer: set PHPCS version for tests'
run: composer require --no-update squizlabs/php_codesniffer:"${{ matrix.phpcs_version }}"

- name: 'Composer: conditionally tweak PHPUnit version'
if: ${{ startsWith( matrix.php, '8' ) }}
# Temporary fix - PHPUnit 9.3+ is buggy when used for code coverage, so not allowed "normally".
# For tests which don't run code coverage, we can safely install it for PHP 8 though.
run: composer require --no-update phpunit/phpunit:"^9.3"

# Install dependencies and handle caching in one go.
# @link https://github.com/marketplace/actions/install-composer-dependencies
- name: Install Composer dependencies - normal
if: ${{ matrix.php < 8.1 }}
uses: "ramsey/composer-install@v1"

# For the PHP "nightly", we need to install with ignore platform reqs as not all dependencies allow it yet.
- name: Install Composer dependencies - with ignore platform
if: ${{ matrix.php >= 8.1 }}
uses: "ramsey/composer-install@v1"
with:
composer-options: --ignore-platform-reqs

- name: Lint against parse errors
if: matrix.phpcs_version == 'dev-master' && matrix.risky == false
run: composer lint

- name: Run the unit tests (non-risky)
if: matrix.risky == false
run: vendor/bin/phpunit --no-coverage

- name: Run the unit tests (risky)
if: ${{ matrix.risky }}
# "nothing" is excluded to force PHPUnit to ignore the <exclude> settings in phpunit.xml.dist.
run: vendor/bin/phpunit --no-coverage --group compareWithPHPCS --exclude-group nothing

#### CODE COVERAGE STAGE ####
# N.B.: Coverage is only checked on the lowest and highest stable PHP versions
# and a low/high of each major for PHPCS.
# These builds are left out off the "test" stage so as not to duplicate test runs.
coverage:
# No use running the coverage builds if there are failing test builds.
needs: test

runs-on: ubuntu-latest

strategy:
matrix:
include:
- php: '7.4' # This should be changed to 8.0 when the tests can run on PHPUnit 9.3+.
phpcs_version: 'dev-master'
- php: '7.3'
phpcs_version: '2.9.2'
- php: '5.4'
phpcs_version: 'dev-master'
- php: '5.4'
phpcs_version: '2.6.0'

name: "Coverage: PHP ${{ matrix.php }} - PHPCS ${{ matrix.phpcs_version }}"

steps:
- name: Checkout code
uses: actions/checkout@v2

# On stable PHPCS versions, allow for PHP deprecation notices.
# Unit tests don't need to fail on those for stable releases where those issues won't get fixed anymore.
- name: Setup ini config
id: set_ini
run: |
if [ "${{ matrix.phpcs_version }}" != "dev-master" ]; then
echo '::set-output name=PHP_INI::error_reporting=E_ALL & ~E_DEPRECATED'
else
echo '::set-output name=PHP_INI::error_reporting=E_ALL'
fi
- name: Install PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
ini-values: ${{ steps.set_ini.outputs.PHP_INI }}
coverage: xdebug

- name: 'Composer: adjust dependencies'
run: |
# Set a specific PHPCS version.
composer require --no-update squizlabs/php_codesniffer:"${{ matrix.phpcs_version }}" --no-scripts
- name: Install Composer dependencies - normal
uses: "ramsey/composer-install@v1"

- name: Run the unit tests with code coverage
run: vendor/bin/phpunit

# Uploading the results with PHP Coveralls v1 won't work from GH Actions, so switch the PHP version.
- name: Switch to PHP 7.4
if: ${{ success() && matrix.php != '7.4' }}
uses: shivammathur/setup-php@v2
with:
php-version: 7.4
coverage: none

- name: Install Coveralls
if: ${{ success() }}
run: composer require php-coveralls/php-coveralls:"^2.4.2"

- name: Upload coverage results to Coveralls
if: ${{ success() }}
env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COVERALLS_PARALLEL: true
COVERALLS_FLAG_NAME: php-${{ matrix.php }}-phpcs-${{ matrix.phpcs_version }}
run: vendor/bin/php-coveralls -v -x build/logs/clover.xml

coveralls-finish:
needs: coverage
runs-on: ubuntu-latest

steps:
- name: Coveralls Finished
uses: coverallsapp/github-action@master
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
parallel-finished: true
Loading

0 comments on commit 846075c

Please sign in to comment.