Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 5 additions & 61 deletions docs/explanation/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

Charms should have tests to verify that they are functioning correctly. This page describes the types of testing that you should consider.

> See also: [](#set-up-ci)

## Unit testing

Unit tests isolate and validate individual code units (functions, methods, and so on) by mocking Juju APIs and workloads without external interactions. Unit tests are intended to be isolated and fast to complete. These are the tests you would run before committing any code changes.
Expand All @@ -18,7 +20,7 @@ A charm acts like a function, taking event context (always present), configurati

Unit tests focus on mapping these inputs to expected outputs. For example, a unit test could verify a system call, the contents of a file, or the contents of a relation databag.

> See also: {ref}`write-unit-tests-for-a-charm`.
> See also: {ref}`write-unit-tests-for-a-charm`

### Coverage

Expand Down Expand Up @@ -53,7 +55,7 @@ Interface tests validate charm library behavior against mock Juju APIs, ensuring

Interface specifications, stored in {ref}`charm-relation-interfaces <charm-relation-interfaces>`, are contract definitions that mandate how a charm should behave when integrated with another charm over a registered interface. For information about how to create an interface, see {ref}`register-an-interface`.

> See also: {ref}`write-tests-for-an-interface`.
> See also: {ref}`write-tests-for-an-interface`

### Coverage

Expand Down Expand Up @@ -91,7 +93,7 @@ Integration tests should be focused on a single charm. Sometimes an integration

Integration tests typically take significantly longer to run than unit tests.

> See also: {ref}`write-integration-tests-for-a-charm`.
> See also: {ref}`write-integration-tests-for-a-charm`

### Coverage

Expand Down Expand Up @@ -124,61 +126,3 @@ basepython = python3.10
### Examples

- [Tempo worker integration tests](https://github.com/canonical/tempo-operators/blob/main/worker/tests/integration/test_deploy.py)

## Continuous integration

Typically, you want the tests to be run automatically against any PR into your repository's main branch, and potentially trigger a new release whenever the tests succeed. Continuous deployment is out of scope for this page, but we will look at how to set up basic continuous integration.

Create a file called `.github/workflows/ci.yaml`. For example, to include a `lint` job that runs the `tox` `unit` environment:

```yaml
name: Tests
on:
push:
branches:
- main
pull_request:

unit-test:
name: Unit tests
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Set up uv
uses: astral-sh/setup-uv@7
- name: Set up tox and tox-uv
run: uv tool install tox --with tox-uv
- name: Run tests
run: tox -e unit
```

Integration tests are a bit more complex, because these tests require a Juju controller and a cloud in which to deploy it. The following example uses the [`actions-operator`](https://github.com/charmed-kubernetes/actions-operator) workflow provided by `charmed-kubernetes` to set up `microk8s` and Juju:

```yaml
integration-test-microk8s:
name: Integration tests (microk8s)
needs:
- lint
- unit-test
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Setup operator environment
uses: charmed-kubernetes/actions-operator@main
with:
provider: microk8s
channel: 1.32-strict/stable
- name: Run integration tests
# Set a predictable model name so it can be consumed by charm-logdump-action
run: tox -e integration -- --model testing
- name: Dump logs
uses: canonical/charm-logdump-action@main
if: failure()
with:
app: my-app-name
model: testing
```

For more actions, documentation, and use cases, see [charming-actions](https://github.com/canonical/charming-actions).
8 changes: 8 additions & 0 deletions docs/howto/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,14 @@ Integration tests check that your charm works correctly when deployed to a real
Write integration tests for a charm <write-integration-tests-for-a-charm>
```

You should automate your charm's tests using a continuous integration (CI) system.

```{toctree}
:maxdepth: 1
Set up continuous integration for a charm <set-up-continuous-integration-for-a-charm>
```

Juju provides a variety of debugging tools, which Ops integrates with.

```{toctree}
Expand Down
6 changes: 4 additions & 2 deletions docs/howto/manage-charms.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ To deploy your charm locally and to run integration tests, you'll also need a Ju

> See more:
>
> - [Prepare your environment to develop Kubernetes charms](#set-up-your-development-environment)
> - [Prepare your environment to develop machine charms](#machine-charm-tutorial-environment)
> - [Prepare a continuous integration environment](#validate-your-charm-with-every-change)
> - [Prepare a continuous integration environment](#set-up-ci-integration)

## Initialise your charm project

Expand Down Expand Up @@ -70,7 +71,8 @@ and the charm workflow.
> - {ref}`write-and-structure-charm-code`
> - {ref}`write-unit-tests-for-a-charm`
> - {ref}`write-integration-tests-for-a-charm`
> - {ref}`manage-logs`
> - {ref}`set-up-ci`
> - {ref}`log-from-your-charm`
> - {ref}`debug-your-charm`

The next thing to do is add functionality to your charm.
Expand Down
113 changes: 113 additions & 0 deletions docs/howto/set-up-continuous-integration-for-a-charm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
myst:
html_meta:
description: Learn how to set up CI for your Juju charm, so that your charm's tests run on every pull request.
---

(set-up-ci)=
# How to set up continuous integration for a charm

```{admonition} Best practice
:class: hint

The quality assurance pipeline of a charm should be automated using a continuous integration (CI) system.
```

This guide demonstrates how to automatically run your charm's tests against any PR into the main branch of your GitHub repository.

You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing libraries on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.

(set-up-ci-linting-unit)=
## Run linting and unit tests in CI

Create a file called `.github/workflows/ci.yaml`:

```yaml
name: Charm tests
on:
push:
branches:
- main
pull_request:
workflow_call:
workflow_dispatch:

permissions: {}

jobs:
lint:
name: Linting
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
- name: Set up tox and tox-uv
run: uv tool install tox --with tox-uv
- name: Lint the code
run: tox -e lint

unit:
name: Unit tests
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
- name: Set up tox and tox-uv
run: uv tool install tox --with tox-uv
- name: Run unit tests
run: tox -e unit
```

(set-up-ci-integration)=
## Run integration tests in CI

Integration tests require a Juju controller and a cloud in which to deploy your charm. We recommend that you use [Concierge](https://github.com/canonical/concierge) to prepare the CI environment.

If your charm is a Kubernetes charm, add the following job to `.github/workflows/ci.yaml`:

```yaml
integration:
name: Integration tests
runs-on: ubuntu-latest
needs:
- unit
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
- name: Set up tox and tox-uv
run: uv tool install tox --with tox-uv
- name: Set up Concierge
run: sudo snap install --classic concierge
- name: Set up Juju and charm development tools
run: sudo concierge prepare -p k8s
- name: Pack the charm
# The integration tests don't pack the charm. Instead, they look for a .charm
# file in the project dir (or use CHARM_PATH, if set).
run: charmcraft pack
- name: Run integration tests
run: tox -e integration -- --juju-dump-logs logs
- name: Upload logs
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: juju-dump-logs
path: logs
```

The option `-p k8s` tells Concierge that we want a cloud managed by Canonical Kubernetes.

If your charm is a machine charm, use `-p machine` instead.

The "Upload logs" step assumes that your integration tests use Jubilant together with `pytest-jubilant`. See [How to write integration tests for a charm](#write-integration-tests-for-a-charm-view-juju-logs).
90 changes: 1 addition & 89 deletions docs/howto/write-and-structure-charm-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -373,92 +373,4 @@ Notes on best practices for charm development and maintenance can be found acros

Configure your continuous integration tooling so that whenever changes are
proposed for or accepted into your main branch the `lint`, `unit`, and
`integration` commands are run, and will block merging when failing.

```{admonition} Best practice
:class: hint

The quality assurance pipeline of a charm should be automated using a
continuous integration (CI) system.
```

If you are using GitHub, create a file called `.github/workflows/ci.yaml`. For
example, to include a `lint` job that runs the `tox` `lint` environment:

```yaml
name: Tests
on:
workflow_call:
workflow_dispatch:

jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Set up uv
uses: astral-sh/setup-uv@7
- name: Set up tox and tox-uv
run: uv tool install tox --with tox-uv
- name: Run linters
run: tox -e lint
```

Other `tox` environments can be run similarly; for example unit tests:

```yaml
unit-test:
name: Unit tests
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Set up uv
uses: astral-sh/setup-uv@7
- name: Set up tox and tox-uv
run: uv tool install tox --with tox-uv
- name: Run tests
run: tox -e unit
```

Integration tests are a bit more complex, because in order to run those tests, a Juju controller and
a cloud in which to deploy it, is required. This example uses [Concierge](https://github.com/canonical/concierge) to set up
`k8s` and Juju:

```
integration-test-k8s:
name: Integration tests (k8s)
needs:
- lint
- unit-test
runs-on: ubuntu-latest
steps:
- name: Install concierge
run: sudo snap install --classic concierge
- name: Install Juju and tools
run: sudo concierge prepare -p k8s
- name: Checkout
uses: actions/checkout@v6
- name: Set up uv
uses: astral-sh/setup-uv@7
- name: Set up tox and tox-uv
run: uv tool install tox --with tox-uv
- name: Run integration tests
# Set a predictable model name so it can be consumed by charm-logdump-action
run: tox -e integration -- --model testing
- name: Dump logs
uses: canonical/charm-logdump-action@main
if: failure()
with:
app: my-app-name
model: testing
```

```{tip}

The [charming-actions](https://github.com/canonical/charming-actions)
repository includes actions to ensure that libraries are up-to-date, publish
charms and libraries, and more.
```
`integration` commands are run, and will block merging when failing. See [](#set-up-ci).
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ For a hands-on introduction to charm development with Ops, try our tutorials:
* - **Adding functionality**
- [Manage relations](howto/manage-relations) • [Manage configuration](howto/manage-configuration) • [More Juju features](#how-to-guides-managing-features)
* - **Testing & CI**
- [Write unit tests](howto/write-unit-tests-for-a-charm) • [Write integration tests](howto/write-integration-tests-for-a-charm)
- [Write unit tests](howto/write-unit-tests-for-a-charm) • [Write integration tests](howto/write-integration-tests-for-a-charm) • [Set up continuous integration](howto/set-up-continuous-integration-for-a-charm)
* - **Design & best practices**
- [Holistic vs delta charms](explanation/holistic-vs-delta-charms) • [Follow best practices](#follow-best-practices) • [Trace your charm](howto/trace-your-charm)
* - **Publishing**
Expand Down