docs: add how-to guide about CI by dwilding · Pull Request #2405 · canonical/operator

dwilding · 2026-03-31T01:53:10Z

This PR consolidates our various CI examples and puts them into a how-to guide so they are much easier to find.

I tried to keep the guidance minimal and close to what we currently have. We can build on the guidance in follow-up PRs. But if the examples themselves need tweaking/improving, let's do that in this PR.

Merge this after #2406

docs/howto/manage-charms.md

james-garner-canonical · 2026-03-31T02:23:00Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+
+This guide demonstrates how to automatically run your charm's tests against any PR into the main branch of your GitHub repository.
+
+You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.


I wonder if we should keep endorsing charming-actions.

I think we should not. But I'm not sure whether we should change that in this PR or later on. I feel like it's a separate PR where we figure out for each feature that charming-actions offers what the Charm Tech approved alternative is, and then update our docs (and potentially your spec) accordingly.

There's also the charmlibs bit. There's still value in the various actions for managing Charmhub charmlibs, but I'm not sure we want to promote those in our docs, unless it's in a migration guide or legacy page.

Maybe this should be a roadmap item for 26.10?

It seems to me like it would be simpler to keep the scope of this PR smaller, and focus on consolidating the content more than updating it, except in very straightforward cases.

But also:

Suggested change

You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.

You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing libraries on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.

james-garner-canonical · 2026-03-31T02:24:22Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+
+Integration tests require a Juju controller and a cloud in which to deploy your charm. We recommend that you use [Concierge](https://github.com/canonical/concierge) to prepare the CI environment.
+
+If your charm is a Kubernetes charm, create a file called `.github/workflows/integration-tests.yaml`:


How about we recommend putting everything in a ci.yaml file for now? The integration test job is simple enough to live in the same file as the rest.

If we do that, then we should leave the workflow_dispatch in place.

james-garner-canonical · 2026-03-31T02:24:41Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+  workflow_dispatch:
+
+jobs:
+  integration:


If we put this in the same file, how about adding needs: [lint, unit]?

I don't think integration tests need linting and unit tests to pass in order to run. In CI, I would rather see everything that's going wrong rather than seeing some formatting failure and having to fix that before I can see the integration test failures, rather than seeing all of them at once.

Well maybe just needs: [unit] then if you want to reduce the chance that we're blocking an integration test run that would otherwise succeed or fail in an interesting way?

I personally find it's well worth having to quickly fix bugs that would be caught by lint/unit before the integration tests will run, if only to avoid getting emails 15 minutes later because bad code from an old commit was being packed and deployed in the background and now finally failed due to a missing import or similar that was fixed when unit tests run.

I guess it is a matter of taste though, and arguably a skill issue if you push code that would fail lint/unit, so if we want to stick with Charm Tech's recommendation being to start your integration tests ASAP, it's not a blocker for me.

docs/howto/set-up-continuous-integration-for-a-charm.md

james-garner-canonical · 2026-03-31T02:27:02Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+        run: tox -e integration -- --model testing
+      - name: Dump logs
+        uses: canonical/charm-logdump-action@main
+        if: failure()
+        with:
+          app: your-app
+          model: testing


Maybe we could do this instead?

Would we need to make sure the charm has pytest-jubilant in its integration testing deps?

Yes. I think that's OK, as charm-logdump-action makes some pretty strong assumptions too (microk8s only, models aren't torn down by integration tests). We should document that we're assuming pytest-jubilant integration tests, of course.

The comment link doesn't seem to be working. Trying this link instead #2406 (comment)

tonyandrewmeyer

Thanks! This looks good to me. I think it's best to keep it more-or-less like it is now, and then make adjustments to what we recommend, particularly with regards to external actions/workflows, in subsequent PRs. Partly so that this one stays focused, but partly because I think we haven't answered for ourselves exactly what we're recommending yet.

docs/howto/manage-charms.md

tonyandrewmeyer · 2026-03-31T21:26:18Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+
+This guide demonstrates how to automatically run your charm's tests against any PR into the main branch of your GitHub repository.
+
+You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.


I think we should not. But I'm not sure whether we should change that in this PR or later on. I feel like it's a separate PR where we figure out for each feature that charming-actions offers what the Charm Tech approved alternative is, and then update our docs (and potentially your spec) accordingly.

There's also the charmlibs bit. There's still value in the various actions for managing Charmhub charmlibs, but I'm not sure we want to promote those in our docs, unless it's in a migration guide or legacy page.

Maybe this should be a roadmap item for 26.10?

It seems to me like it would be simpler to keep the scope of this PR smaller, and focus on consolidating the content more than updating it, except in very straightforward cases.

But also:

Suggested change

You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.

You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing libraries on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.

tonyandrewmeyer · 2026-03-31T21:31:52Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+      - main
+  pull_request:
+  workflow_call:
+  workflow_dispatch:


Is this useful for linting and unit tests? I feel like if you want to run those manually, you ought to be able to do that locally.

tonyandrewmeyer · 2026-03-31T21:32:41Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+(set-up-ci-linting-unit)=
+## Create a workflow for linting and unit tests
+
+Create a file called `.github/workflows/tests.yaml`:


Let's Zizmor this file and fix the issues it'll report (like not storing git credentials, default permissions, etc). People will copy it exactly, so we should make it as safe as possible.

tonyandrewmeyer · 2026-03-31T21:33:45Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+
+Integration tests require a Juju controller and a cloud in which to deploy your charm. We recommend that you use [Concierge](https://github.com/canonical/concierge) to prepare the CI environment.
+
+If your charm is a Kubernetes charm, create a file called `.github/workflows/integration-tests.yaml`:


If we do that, then we should leave the workflow_dispatch in place.

tonyandrewmeyer · 2026-03-31T21:34:53Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+  workflow_dispatch:
+
+jobs:
+  integration:


I don't think integration tests need linting and unit tests to pass in order to run. In CI, I would rather see everything that's going wrong rather than seeing some formatting failure and having to fix that before I can see the integration test failures, rather than seeing all of them at once.

tonyandrewmeyer · 2026-03-31T21:36:36Z

docs/howto/set-up-continuous-integration-for-a-charm.md

+        # Set a predictable model name so it can be consumed by charm-logdump-action.
+        run: tox -e integration -- --model testing
+      - name: Dump logs
+        uses: canonical/charm-logdump-action@main


I wonder about recommending this, too. It's a lot more than the Juju logs, but is it something that should be used by all (K8s) charms? If so, should Charm Tech be providing something for this ourselves?

dwilding added 6 commits March 31, 2026 09:49

remove CI content from testing explanation doc

209dbf8

remove CI content from write & structure how-to guide

5463569

add new how-to guide

b3fe881

update homepage and landing page

82d07aa

update "Manage charms"

2874d88

fix link to logging how-to

e86c8e8

dwilding commented Mar 31, 2026

View reviewed changes

docs/howto/manage-charms.md Show resolved Hide resolved

dwilding commented Mar 31, 2026

View reviewed changes

docs/howto/manage-charms.md Show resolved Hide resolved

dwilding requested a review from Copilot March 31, 2026 02:00

Copilot started reviewing on behalf of dwilding March 31, 2026 02:00 View session

This comment was marked as resolved.

Sign in to view

dwilding marked this pull request as ready for review March 31, 2026 02:07

dwilding requested review from james-garner-canonical and tonyandrewmeyer March 31, 2026 02:07

james-garner-canonical mentioned this pull request Mar 31, 2026

docs: update tutorials and how-tos to use pytest-jubilant #2406

Open

james-garner-canonical reviewed Mar 31, 2026

View reviewed changes

dwilding added 2 commits March 31, 2026 10:40

add missing pack

0febefd

add comment about packing

e1690c4

tonyandrewmeyer approved these changes Mar 31, 2026

View reviewed changes


		This guide demonstrates how to automatically run your charm's tests against any PR into the main branch of your GitHub repository.

		You might also want to automatically publish your charm on Charmhub or publish charm libraries on PyPI. [charming-actions](https://github.com/canonical/charming-actions) has some useful GitHub actions for publishing on Charmhub. For guidance about publishing on PyPI, see {external+charmlibs:ref}`How to distribute charm libraries <python-package-distribution-pypi>`.


		Integration tests require a Juju controller and a cloud in which to deploy your charm. We recommend that you use [Concierge](https://github.com/canonical/concierge) to prepare the CI environment.

		If your charm is a Kubernetes charm, create a file called `.github/workflows/integration-tests.yaml`:

Conversation

dwilding commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tonyandrewmeyer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dwilding commented Mar 31, 2026 •

edited

Loading