Skip to content

✨ npm-ci-build + README #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Aug 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
root = true

[*]
# Prettier default
charset = utf-8
insert_final_newline = true
end_of_line = lf
max_line_length = 80
trim_trailing_whitespace = true

# Custom
# Use tabs: https://alexandersandberg.com/articles/default-to-tabs-instead-of-spaces-for-an-accessible-first-environment/
indent_style = tab

[*.{yaml,yml}]
# YAML disallows tabs: https://yaml.org/spec/
indent_size = 2
indent_style = space
71 changes: 71 additions & 0 deletions .github/workflows/npm-ci-build~v1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
on:
workflow_call:
inputs:
hasTests:
description: Whether the project has tests. Used to skip test reporting for projects with no tests.
type: boolean
required: false
default: true
projectType:
description: The type of project. Used to customize the CI build process. MUST be one of `application` or `library`.
type: string
required: false
default: library
outputs:
semVer:
description: The SemVer version number of this build (automated with GitVersion).
value: ${{ jobs.NpmCiBuild.outputs.semVer }}

jobs:
NpmCiBuild:
# Workflow names appear as `origin name / workflow name`, so use `npm` for something like `CI Build / npm`
name: npm
runs-on: ubuntu-latest
outputs:
semVer: ${{ steps.GitVersion.outputs.semVer }}
steps:
- name: Validate inputs
uses: actions/github-script@v7
with:
script: |
const { projectType } = ${{ toJson(inputs) }};
if (!["application", "library"].includes(projectType)) {
core.setFailed(`projectType MUST be one of 'application' or 'library', but was '${projectType}'`);
}
core.info("\u001b[1m\u001b[32mSuccess.");

- name: Checkout self
uses: actions/checkout@v4
with:
fetch-depth: 0 # Full depth (not shallow) for GitVersion

- name: Set up GitVersion
uses: gittools/actions/gitversion/[email protected]
with:
versionSpec: 6.x

- name: Execute GitVersion
id: GitVersion
uses: gittools/actions/gitversion/[email protected]
with:
overrideConfig: |
workflow=GitHubFlow/v1
mode=ContinuousDeployment

- name: Set version
run: sed -i 's/0.0.0-gitversion/${{ env.semVer }}/g' package.json

- name: Use Node.js
uses: actions/setup-node@v4
with:
cache: npm
node-version-file: .node-version

- name: Install
run: npm ci --strict-peer-deps

- name: CI build
run: npm run ci-build

# TODO: Test artifacts, using inputs.hasTests
# TODO: Publish pack artifact, using inputs.projectType
202 changes: 202 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
<div align="center">

<h1>GitHub Workflows</h1>

My GitHub workflows. Centralized for reuse.

</div>

**Table of contents**

- [Published workflows](#published-workflows)
- [npm-ci-build](#npm-ci-build)
- [Goal: Universal targets](#goal-universal-targets)
- [Industry observations](#industry-observations)
- [My universal targets](#my-universal-targets)
- [My universal output locations](#my-universal-output-locations)
- [Style conventions](#style-conventions)

## Published workflows

> [!NOTE]
>
> The workflows use file versioning (`name~v1.yaml`) instead of repository tags.
>
> <details><summary>Why?</summary><div>
>
> This provides independent versioning of the workflows while maintaining them in one repository.
> It also allows supporting older versions on the <code>main</code> branch (for example, upgrading all checkout actions).
>
> File versioning uses the tilde (`~`) because GitHub prohibits `@` in workflow file names.
> (GitHub uses `@` for repository references.)
> The tilde still reads well, displays well in URLs, and is not often used in file names.
>
> </div></details>

### npm-ci-build

Job that executes `ci-build` logic for npm packages.

```yaml
jobs:
ci-build:
uses: connorjs/github-workflows/.github/workflows/npm-ci-build~v1.yaml
```

`v1` runs `ci-build` directly and assumes that the underlying package orchestrates the full build correctly.

<details>
<summary>Show inputs, outputs, assumptions, and notes</summary>

#### Inputs

| Name | Default | Type | Description |
|:-------------:|:---------:|:-------:|:-------------------------------------------------------------------------------------------:|
| `hasTests` | `true` | boolean | Whether the project has tests.<br>Used to skip test reporting for projects with no tests. |
| `projectType` | `library` | choice | The type of project: `application` or `library`.<br>Used to customize the CI build process. |

#### Outputs

| Name | Description |
|:--------:|:--------------------------------------------------------------------:|
| `semVer` | The SemVer version number of this build (automated with GitVersion). |

#### Assumptions

The job makes the following assumptions about the repository.

- Uses a `.node-version` file in the root of the repository to set the Node.js version.
- Uses npm (not yarn or pnpm).
- The `package.json` uses `0.0.0-gitversion` for its version (enables GitVersion automation).

#### Notes

The job includes automatic versioning via GitVersion.
It will replace `0.0.0-gitversion` with the correct version.
_Note: While another tool could replace GitVersion, the automatic version string will remain `0.0.0-gitversion`._

Note: The job will ignore any local `GitVersion.yaml`; it configures GitVersion for continuous deployment internally.
Use `+semver:(major|minor)` in commit messages appropriately.
(Patch happens automatically.)

</details>

## Goal: Universal targets

I find that a universal set of targets (also called tasks) simplifies polyglot development.
I observed the benefits with Amazon’s internal build system, and I built similar mechanisms at Total Quality Logistics (TQL).

This repository represents my personal take on the idea, implemented via a consistent set of GitHub workflows.

### Industry observations

This section details my observations from industry tools that I have used.

- Amazon: Standardized install, build, test (unit-test and integration-test) all under release.
(Source: per my memory 18+ months later)

- Gradle: Standardizes check, assemble, and build tasks.
Includes some sort of standardized dependency installation (but I cannot find the task name).
Specific plugins (such as java) extend with their own standard set (compile, javadoc, and more).
(Source: [base plugin](https://docs.gradle.org/current/userguide/base_plugin.html))

- Maven: Standardizes validate, compile, test, package, verify, install, and deploy
(Source: [Maven build lifecycle](https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html))

- .NET (`dotnet`): Restore, build, test, format, run, and publish.
(Source: My limited usage and [dotnet commands](https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet))

- npm: Standardizes install/ci/rebuild, start/stop/restart, test, and many related to prepare and publish.
Many libraries suggest build and lint, but there are others related (compile, tsc, specific linters, and formatters).
(Source: My experience and [npm lifecycle scripts](https://docs.npmjs.com/cli/v10/using-npm/scripts))

- Python: Unsure.
I know of install from pipenv.
I see build, fmt, and test from hatch, install and build from flit, and easy_install and build from Setuptools.

Each of these also had the concept of clean.

### My universal targets

Combining my experiences predominantly with Java, npm, and .NET, I arrive at the following universal targets.
Each underlying build system should be able to hook into these standardized targets.

- `install`: Installs (or restores) dependencies.

- `build`: Builds the project.

- `format`: Formats and lints the project.

- `test`: Runs all tests that do not require a network connection.
Often thought of as unit tests, but can contain network-free integration tests (aka component or module tests).
This should include code coverage.

- `publish`: Publish the library to its package manager.

- `version`: Handles automatic versioning, if desired.

- `clean`: Cleans any build or test artifacts.
SHOULD NOT remove dependencies.

In CI (think code reviews or pull requests), the “build” executes many of these targets.
Using the term “build” here would overload the target name though.
Therefore, I use the term `ci-build` to group all targets that should run during the “CI Build.”

We called this `release` at Amazon, but I have found that folks think `release` means “make the release; actually publish,” which represents the wrong conclusion.
Hence, I chose the term `ci-build`.
_(I also considered terms such as `full-build`, `release-build`, and `ci` or replacing the underlying `build` term with `compile`.
However, others came to the right conclusion with `ci-build`, which reinforces its choice.)_

If I was building my own CLI that wrapped underlying tools with these universal targets, I would include `compile` and `lint` as aliases for `build` and `format`.
If both targets were present, the main name would take precedence.

### My universal output locations

Standardizing the output locations also simplifies polyglot development.
While tooling (GitHub workflows) receives the most benefits, developers can benefit from standardized code coverage output locations.
Standardized outputs can also simplify `.gitignore` configuration.
_(Nit-pick side note: The 100s-lines-long default `.gitignore` files pain me.)_

The standard output locations follow.

- `artifacts` should contain as much output as possible.
This also simplifies clean.

- Dependencies will use underlying tool conventions (example: `node_modules` for npm).

- Publish will use underlying tool conventions where appropriate.
Use `artifacts/dist` to vend transpiled sources.

- Mono-repos will use `$projectName` in the path per artifact type (example: `artifacts/dist/$projectName`).
This preserves output structure semantics between polyrepo (single project repositories) and monorepo use cases.

- Tests will use `artifacts/test-results`.
Prefer JUnit format for the test results file and use the name `test-result.xml`.
Prefer Cobertura format for the code coverage results file and use the name `cobertura.xml`.

In monorepos, each project likely has these files in `artifacts/test-results/$projectName` _AND_ the monorepo workspace aggregate in `artifacts/test-results` (N+1 output files).
Note that tools like [reportgenerator](https://reportgenerator.io) exist to simplify aggregation.

- The HTML output from code coverage should have the name `artifacts/coverage/index.html`.
This differs from `artifacts/test-results` to simplify developers opening the report and to prevent conflicts in monorepos.

## Style conventions

Some notes on how I style (format) my GitHub workflow files.

- The following naming rules ensure seamless script usage, such as in `actions/github-script`.
- Prefer `PascalCase` for job and step IDs.
- Prefer `camelCase` for variable names (inputs and outputs).
- _This means avoid `snake_case`, `kebab-case`, `TRAIN-CASE`, or `SCREAM_CASE` unless a 3rd party uses them._

- Format expressions with inner spaces: `${{ foo.bar }}`.

- Prefer no quotes if YAML does not require them.

- Automatically format files: Use prettier, an editor using .editorconfig, or a similar formatter.

- Start each step with `name`.
- Use sentence case for `name` (with allowed use of Proper Nouns when it helps such as GitVersion).
- Exception: Prefer “CI Build” (instead of “CI build” or “ci build”).

- If a step (or job) would benefit from a longer description, include a YAML comment on the line after `name` (unless GitHub adds proper `description` field).