diff --git a/docs/explanation/testing.md b/docs/explanation/testing.md index 9f9bff5cf..d59477392 100644 --- a/docs/explanation/testing.md +++ b/docs/explanation/testing.md @@ -3,6 +3,8 @@ Charms should have tests to verify that they are functioning correctly. This page describes the types of testing that you should consider. +> See also: [](#set-up-ci) + ## Unit testing Unit tests isolate and validate individual code units (functions, methods, and so on) by mocking Juju APIs and workloads without external interactions. Unit tests are intended to be isolated and fast to complete. These are the tests you would run before committing any code changes. @@ -18,7 +20,7 @@ A charm acts like a function, taking event context (always present), configurati Unit tests focus on mapping these inputs to expected outputs. For example, a unit test could verify a system call, the contents of a file, or the contents of a relation databag. -> See also: {ref}`write-unit-tests-for-a-charm`. +> See also: {ref}`write-unit-tests-for-a-charm` ### Coverage @@ -53,7 +55,7 @@ Interface tests validate charm library behavior against mock Juju APIs, ensuring Interface specifications, stored in {ref}`charm-relation-interfaces `, are contract definitions that mandate how a charm should behave when integrated with another charm over a registered interface. For information about how to create an interface, see {ref}`register-an-interface`. -> See also: {ref}`write-tests-for-an-interface`. +> See also: {ref}`write-tests-for-an-interface` ### Coverage @@ -91,7 +93,7 @@ Integration tests should be focused on a single charm. Sometimes an integration Integration tests typically take significantly longer to run than unit tests. -> See also: {ref}`write-integration-tests-for-a-charm`. +> See also: {ref}`write-integration-tests-for-a-charm` ### Coverage @@ -124,61 +126,3 @@ basepython = python3.10 ### Examples - [Tempo worker integration tests](https://github.com/canonical/tempo-operators/blob/main/worker/tests/integration/test_deploy.py) - -## Continuous integration - -Typically, you want the tests to be run automatically against any PR into your repository's main branch, and potentially trigger a new release whenever the tests succeed. Continuous deployment is out of scope for this page, but we will look at how to set up basic continuous integration. - -Create a file called `.github/workflows/ci.yaml`. For example, to include a `lint` job that runs the `tox` `unit` environment: - -```yaml -name: Tests -on: - push: - branches: - - main - pull_request: - - unit-test: - name: Unit tests - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v6 - - name: Set up uv - uses: astral-sh/setup-uv@7 - - name: Set up tox and tox-uv - run: uv tool install tox --with tox-uv - - name: Run tests - run: tox -e unit -``` - -Integration tests are a bit more complex, because these tests require a Juju controller and a cloud in which to deploy it. The following example uses the [`actions-operator`](https://github.com/charmed-kubernetes/actions-operator) workflow provided by `charmed-kubernetes` to set up `microk8s` and Juju: - -```yaml - integration-test-microk8s: - name: Integration tests (microk8s) - needs: - - lint - - unit-test - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v6 - - name: Setup operator environment - uses: charmed-kubernetes/actions-operator@main - with: - provider: microk8s - channel: 1.32-strict/stable - - name: Run integration tests - # Set a predictable model name so it can be consumed by charm-logdump-action - run: tox -e integration -- --model testing - - name: Dump logs - uses: canonical/charm-logdump-action@main - if: failure() - with: - app: my-app-name - model: testing -``` - -For more actions, documentation, and use cases, see [charming-actions](https://github.com/canonical/charming-actions). diff --git a/docs/howto/index.md b/docs/howto/index.md index b2590bff3..b0dd7cfbc 100644 --- a/docs/howto/index.md +++ b/docs/howto/index.md @@ -43,6 +43,14 @@ Integration tests check that your charm works correctly when deployed to a real Write integration tests for a charm ``` +You should automate your charm's tests using a continuous integration (CI) system. + +```{toctree} +:maxdepth: 1 + +Set up continuous integration for a charm +``` + Juju provides a variety of debugging tools, which Ops integrates with. ```{toctree} diff --git a/docs/howto/manage-charms.md b/docs/howto/manage-charms.md index 84ea26047..71c0a48af 100644 --- a/docs/howto/manage-charms.md +++ b/docs/howto/manage-charms.md @@ -17,8 +17,9 @@ To deploy your charm locally and to run integration tests, you'll also need a Ju > See more: > +> - [Prepare your environment to develop Kubernetes charms](#set-up-your-development-environment) > - [Prepare your environment to develop machine charms](#machine-charm-tutorial-environment) -> - [Prepare a continuous integration environment](#validate-your-charm-with-every-change) +> - [Prepare a continuous integration environment](#set-up-ci-integration) ## Initialise your charm project @@ -70,7 +71,8 @@ and the charm workflow. > - {ref}`write-and-structure-charm-code` > - {ref}`write-unit-tests-for-a-charm` > - {ref}`write-integration-tests-for-a-charm` -> - {ref}`manage-logs` +> - {ref}`set-up-ci` +> - {ref}`log-from-your-charm` > - {ref}`debug-your-charm` The next thing to do is add functionality to your charm. diff --git a/docs/howto/set-up-continuous-integration-for-a-charm.md b/docs/howto/set-up-continuous-integration-for-a-charm.md new file mode 100644 index 000000000..2ae4b3784 --- /dev/null +++ b/docs/howto/set-up-continuous-integration-for-a-charm.md @@ -0,0 +1,113 @@ +--- +myst: + html_meta: + description: Learn how to set up CI for your Juju charm, so that your charm's tests run on every pull request. +--- + +(set-up-ci)= +# How to set up continuous integration for a charm + +```{admonition} Best practice +:class: hint + +The quality assurance pipeline of a charm should be automated using a continuous integration (CI) system. +``` + +This guide demonstrates how to automatically run your charm's tests against any PR into the main branch of your GitHub repository. + +You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing libraries on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries `. + +(set-up-ci-linting-unit)= +## Run linting and unit tests in CI + +Create a file called `.github/workflows/ci.yaml`: + +```yaml +name: Charm tests +on: + push: + branches: + - main + pull_request: + workflow_call: + workflow_dispatch: + +permissions: {} + +jobs: + lint: + name: Linting + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + persist-credentials: false + - name: Set up uv + uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0 + - name: Set up tox and tox-uv + run: uv tool install tox --with tox-uv + - name: Lint the code + run: tox -e lint + + unit: + name: Unit tests + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + persist-credentials: false + - name: Set up uv + uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0 + - name: Set up tox and tox-uv + run: uv tool install tox --with tox-uv + - name: Run unit tests + run: tox -e unit +``` + +(set-up-ci-integration)= +## Run integration tests in CI + +Integration tests require a Juju controller and a cloud in which to deploy your charm. We recommend that you use [Concierge](https://github.com/canonical/concierge) to prepare the CI environment. + +If your charm is a Kubernetes charm, add the following job to `.github/workflows/ci.yaml`: + +```yaml + integration: + name: Integration tests + runs-on: ubuntu-latest + needs: + - unit + steps: + - name: Checkout + uses: actions/checkout@v6 + with: + persist-credentials: false + - name: Set up uv + uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0 + - name: Set up tox and tox-uv + run: uv tool install tox --with tox-uv + - name: Set up Concierge + run: sudo snap install --classic concierge + - name: Set up Juju and charm development tools + run: sudo concierge prepare -p k8s + - name: Pack the charm + # The integration tests don't pack the charm. Instead, they look for a .charm + # file in the project dir (or use CHARM_PATH, if set). + run: charmcraft pack + - name: Run integration tests + run: tox -e integration -- --juju-dump-logs logs + - name: Upload logs + if: ${{ !cancelled() }} + uses: actions/upload-artifact@v7 + with: + name: juju-dump-logs + path: logs +``` + +The option `-p k8s` tells Concierge that we want a cloud managed by Canonical Kubernetes. + +If your charm is a machine charm, use `-p machine` instead. + +The "Upload logs" step assumes that your integration tests use Jubilant together with `pytest-jubilant`. See [How to write integration tests for a charm](#write-integration-tests-for-a-charm-view-juju-logs). diff --git a/docs/howto/write-and-structure-charm-code.md b/docs/howto/write-and-structure-charm-code.md index 19f2881e9..2cca4a227 100644 --- a/docs/howto/write-and-structure-charm-code.md +++ b/docs/howto/write-and-structure-charm-code.md @@ -373,92 +373,4 @@ Notes on best practices for charm development and maintenance can be found acros Configure your continuous integration tooling so that whenever changes are proposed for or accepted into your main branch the `lint`, `unit`, and -`integration` commands are run, and will block merging when failing. - -```{admonition} Best practice -:class: hint - -The quality assurance pipeline of a charm should be automated using a -continuous integration (CI) system. -``` - -If you are using GitHub, create a file called `.github/workflows/ci.yaml`. For -example, to include a `lint` job that runs the `tox` `lint` environment: - -```yaml -name: Tests -on: - workflow_call: - workflow_dispatch: - -jobs: - lint: - name: Lint - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v6 - - name: Set up uv - uses: astral-sh/setup-uv@7 - - name: Set up tox and tox-uv - run: uv tool install tox --with tox-uv - - name: Run linters - run: tox -e lint -``` - -Other `tox` environments can be run similarly; for example unit tests: - -```yaml - unit-test: - name: Unit tests - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v6 - - name: Set up uv - uses: astral-sh/setup-uv@7 - - name: Set up tox and tox-uv - run: uv tool install tox --with tox-uv - - name: Run tests - run: tox -e unit -``` - -Integration tests are a bit more complex, because in order to run those tests, a Juju controller and -a cloud in which to deploy it, is required. This example uses [Concierge](https://github.com/canonical/concierge) to set up -`k8s` and Juju: - -``` - integration-test-k8s: - name: Integration tests (k8s) - needs: - - lint - - unit-test - runs-on: ubuntu-latest - steps: - - name: Install concierge - run: sudo snap install --classic concierge - - name: Install Juju and tools - run: sudo concierge prepare -p k8s - - name: Checkout - uses: actions/checkout@v6 - - name: Set up uv - uses: astral-sh/setup-uv@7 - - name: Set up tox and tox-uv - run: uv tool install tox --with tox-uv - - name: Run integration tests - # Set a predictable model name so it can be consumed by charm-logdump-action - run: tox -e integration -- --model testing - - name: Dump logs - uses: canonical/charm-logdump-action@main - if: failure() - with: - app: my-app-name - model: testing -``` - -```{tip} - -The [charming-actions](https://github.com/canonical/charming-actions) -repository includes actions to ensure that libraries are up-to-date, publish -charms and libraries, and more. -``` +`integration` commands are run, and will block merging when failing. See [](#set-up-ci). diff --git a/docs/index.md b/docs/index.md index 0c6c17e28..d66a1ad90 100644 --- a/docs/index.md +++ b/docs/index.md @@ -44,7 +44,7 @@ For a hands-on introduction to charm development with Ops, try our tutorials: * - **Adding functionality** - [Manage relations](howto/manage-relations) • [Manage configuration](howto/manage-configuration) • [More Juju features](#how-to-guides-managing-features) * - **Testing & CI** - - [Write unit tests](howto/write-unit-tests-for-a-charm) • [Write integration tests](howto/write-integration-tests-for-a-charm) + - [Write unit tests](howto/write-unit-tests-for-a-charm) • [Write integration tests](howto/write-integration-tests-for-a-charm) • [Set up continuous integration](howto/set-up-continuous-integration-for-a-charm) * - **Design & best practices** - [Holistic vs delta charms](explanation/holistic-vs-delta-charms) • [Follow best practices](#follow-best-practices) • [Trace your charm](howto/trace-your-charm) * - **Publishing**