Issues/3466 Move away from Gitpod #3569

JulianFlesch · 2025-05-14T13:01:17Z

Description

Addresses the suggestions in issue 3466 to move away from gitpod and towards devcontainer for Github codespaces.

PR checklist

This comment contains a description of changes (with reason)
CHANGELOG.md is updated
If you've fixed a bug or added code that should be tested, add tests!
Documentation in docs is updated

… locally for now

…ner.

…tools into issues/3466-gitpod

…r. Update postCreateCommand to match context

JulianFlesch · 2025-05-20T10:10:37Z

Note: This still uses (a newer) gitpod base image and thus also the gitpod user.

.devcontainer/devcontainer.json

nf_core/devcontainer/devcontainer.Dockerfile

codecov · 2025-05-20T10:26:25Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.48%. Comparing base (75d8c83) to head (c5dd2d2).
Report is 15 commits behind head on dev.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ion to handle auth

JulianFlesch · 2025-07-22T15:39:44Z

Ready for review, if you have the time @mashehu @mahesh-panchal ! 🎉

Summary of latest changes:

A way to retry builds for the nf-Core/tools container was added due to nf-test installer download sometimes failing (issue)
Installation of nf-test in the devcontainer image was changed to conda, since conda is needed anyway (commit)
All images are now build for linux/amd64 and linux/arm64 architectures (this requires QEMU and buildx)
Most requirements are installed via (local) features now, with the exception of nf-core tools because it requires the current state of the repo
There are thus two devcontainer configs: everything in .devcontainer/build-devcontainer is for building an image with a specific version of tools and all features installed. The config ./devcontainer/devcontainer.json is used to run it in vscode / codespaces.

Failing Test

To make the failing test work, I assume something in template_features.yml needs to be changed?

mashehu

We also need to these files to the files_changed lint tests. And remove the gitpod.yml file

mashehu · 2025-07-22T17:33:55Z

.devcontainer/build-devcontainer/Dockerfile

+# copy this repo at current revision
+COPY . /root/nfcore-tools/
+
+# install python 3.11, local nf-core tools version, and precommit hooks


Why python 3.11 specifically? The minimal version is 3.9 currently.

mashehu · 2025-07-22T17:34:59Z

.devcontainer/build-devcontainer/devcontainer.json

+
+    "features": {
+        "ghcr.io/devcontainers/features/docker-outside-of-docker:1": {
+            "version": "latest"


Pin this to a specific version please. Renovate will help us update it.

mashehu · 2025-07-22T17:36:17Z

.devcontainer/build-devcontainer/devcontainer.json

+            },
+
+            // Add the IDs of extensions you want installed when the container is created.
+            "extensions": ["ms-python.python", "ms-python.vscode-pylance", "nf-core.nf-core-extensionpack"]


Please add ruff here as well.

mashehu · 2025-07-22T17:39:07Z

.devcontainer/setup.sh

+nextflow self-update
+
+# Update welcome message
+echo "Welcome to the nf-core/tools devcontainer!" > /usr/local/etc/vscode-dev-containers/first-run-notice.txt


Suggested change

echo "Welcome to the nf-core/tools devcontainer!" > /usr/local/etc/vscode-dev-containers/first-run-notice.txt

echo "Welcome to the nf-core devcontainer!" > /usr/local/etc/vscode-dev-containers/first-run-notice.txt

mashehu · 2025-07-22T17:40:48Z

nf_core/pipeline-template/.devcontainer/local-features/apptainer/install.sh

Could we symlink files that are the same? So we save the hassle of keeping everything in sync?

Reducing redundancy sounds good, but I would rather not have symlinks in the template, to make things more simple. If we add them, let's remember to use followlinks?True in Jinja2.

mashehu · 2025-07-22T17:41:41Z

nf_core/pipeline-template/.devcontainer/setup.sh

+#!/usr/bin/env bash
+
+# Update welcome message
+echo "Welcome to the nf-core pipeline devcontainer!" > /usr/local/etc/vscode-dev-containers/first-run-notice.txt


Use jinja variables to customize the welcome message to the current pipeline.

mirpedrol · 2025-07-28T09:41:41Z

The new template file .devcontainer/setup.sh should be skipped by a template feature. We should remove the current gitpod feature and update the codespaces feature to add the new file, in nf_core/pipelines/create/template_features.yml

mirpedrol · 2025-07-28T09:44:30Z

.devcontainer/build-devcontainer/devcontainer.json

+            "version": "21.0.6-ms"
+        },
+        "ghcr.io/rocker-org/devcontainer-features/miniforge:2": {
+            "version": "latest"


the version should also be pinned here

mirpedrol · 2025-07-28T09:51:47Z

nf_core/pipeline-template/.devcontainer/local-features/apptainer/install.sh

Reducing redundancy sounds good, but I would rather not have symlinks in the template, to make things more simple. If we add them, let's remember to use followlinks?True in Jinja2.

JulianFlesch added 6 commits May 14, 2025 11:00

Renaming Dockerfile and location. Changing to building the Dockerfile…

80b0182

… locally for now

change Container user. Change base Image for devcontainer

7a8cebd

Remove installation of apptainer via apt-get because of 404 from ppa

5fd706a

Create unprivileged user

0d6e6a5

Change git workflow for container creation and use the correct contai…

be2996e

…ner.

Update instruction to build locally for testing

53dd172

JulianFlesch changed the title ~~[DRAFT] Issues/3466 gitpod~~ [DRAFT] Issues/3466 Move away from Gitpod May 14, 2025

[automated] Update CHANGELOG.md

dd6ddc3

JulianFlesch mentioned this pull request May 14, 2025

Deprecate gitpod, update devcontainer.json for GitHub codespaces #3466

Open

5 tasks

JulianFlesch added 9 commits May 14, 2025 15:42

Merge branch 'issues/3466-gitpod' of github.com:JulianFlesch/nf-core-…

0c3a21d

…tools into issues/3466-gitpod

More descriptive changelog entry

464d3bd

Change base Image

2ee5fc1

Change back to using gitpod user

776b2c8

Remove comment

a5c6644

WIP: build container locally instead of pulling. Change to gitpod use…

c77a469

…r. Update postCreateCommand to match context

Rename pushed container to nf-core/devcontainer

36e5f7a

Remove gitpod.yml from template

4f8a2c0

Remove .gitpod.yml

c225114

JulianFlesch requested review from mashehu and mahesh-panchal May 20, 2025 10:08

JulianFlesch marked this pull request as ready for review May 20, 2025 10:13

JulianFlesch changed the title ~~[DRAFT] Issues/3466 Move away from Gitpod~~ Issues/3466 Move away from Gitpod May 20, 2025

mashehu requested changes May 20, 2025

View reviewed changes

.devcontainer/devcontainer.json Outdated Show resolved Hide resolved

nf_core/devcontainer/devcontainer.Dockerfile Outdated Show resolved Hide resolved

nf_core/devcontainer/devcontainer.Dockerfile Outdated Show resolved Hide resolved

JulianFlesch and others added 2 commits May 20, 2025 12:13

Merge branch 'dev' into issues/3466-gitpod

56962f2

Remove local container building

f525657

mahesh-panchal reviewed May 20, 2025

View reviewed changes

nf_core/devcontainer/devcontainer.Dockerfile Outdated Show resolved Hide resolved

JulianFlesch added 2 commits May 20, 2025 12:27

Change base image tag

e0e6d97

re-add comments

923d99d

JulianFlesch added 23 commits July 21, 2025 22:31

Add qemu action for multi platform build

99c701c

Change call to setup.sh and make it executable

4316c6d

Test building nfcore-tools image for more platforms

85e2572

Use buildx for nfcore tools image

c8c3db4

Use buildx for multi-platform build and to handle push. Use login-act…

47b8e5d

…ion to handle auth

Fix secrets

ccc72b7

Add targets to buildx

3a53655

Fix build command

365a1fb

Test using variable

5258b94

Fix reference to env variables

0be623a

Fix actions version to sha

adfd85e

Update release workflow for testing

45050e9

Change to installing nf-test via conda in devcontainer

4c758ba

Remove post install commands for nf-test

333bcab

Remove unused local feature

11d1337

Adds retry for tools image build

1234279

Fix retry time

7fee45c

Adds required field

18f51e5

Lower tools image build timeout. Retry on error+timeout (default)

bd96723

Revert testing config

58af5d6

Adapt timeout to have more leeway

16a0a5a

Merge changes from dev

3be9da3

Move changes to 3.4.0dev

bce4da7

JulianFlesch marked this pull request as ready for review July 22, 2025 15:39

JulianFlesch requested review from mashehu and mahesh-panchal July 22, 2025 15:40

mashehu reviewed Jul 22, 2025

View reviewed changes

mirpedrol reviewed Jul 28, 2025

View reviewed changes

	echo "Welcome to the nf-core/tools devcontainer!" > /usr/local/etc/vscode-dev-containers/first-run-notice.txt
	echo "Welcome to the nf-core devcontainer!" > /usr/local/etc/vscode-dev-containers/first-run-notice.txt

Issues/3466 Move away from Gitpod #3569

Are you sure you want to change the base?

Issues/3466 Move away from Gitpod #3569

Conversation

JulianFlesch commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

PR checklist

Uh oh!

JulianFlesch commented May 20, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

JulianFlesch commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of latest changes:

Failing Test

Uh oh!

mashehu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mirpedrol commented Jul 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JulianFlesch commented May 14, 2025 •

edited

Loading

codecov bot commented May 20, 2025 •

edited

Loading

JulianFlesch commented Jul 22, 2025 •

edited

Loading