Skip to content

Add test container based on Alpine image. #953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Add test container based on Alpine image. #953

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ KERNEL_REL_MIN := $(shell echo $(KERNEL_REL) | cut -d'.' -f2)
export KERNEL_REL

# Sysbox image-generation globals utilized during the sysbox's building and testing process.
ifeq ($(IMAGE_BASE_DISTRO),$(filter $(IMAGE_BASE_DISTRO),centos fedora redhat almalinux rocky amzn))
ifeq ($(IMAGE_BASE_DISTRO),$(filter $(IMAGE_BASE_DISTRO),centos fedora redhat almalinux rocky amzn alpine))
IMAGE_BASE_RELEASE := $(shell cat /etc/os-release | grep "^VERSION_ID" | cut -d "=" -f2 | tr -d '"' | cut -d "." -f1)
KERNEL_HEADERS := kernels/$(KERNEL_REL)
else
Expand Down
167 changes: 167 additions & 0 deletions tests/Dockerfile.alpine-3
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
FROM alpine:3.22

# K8s version for k8s-in-docker (i.e., this should be equal or greater than the
# version of K8s running inside the k8s-in-docker container).
ARG k8s_version=v1.28
ARG k8s_version_full=v1.28.2

# CRI-O & crictl version for testing sysbox pods; should match K8s version.
ARG crio_version=v1.28
ARG crictl_version=v1.28.0

# Desired platform architecture to build upon.
ARG sys_arch
ENV SYS_ARCH=${sys_arch}
ARG target_arch
ENV TARGET_ARCH=${target_arch}

RUN apk update && apk add \
alpine-sdk \
acl \
gcc \
musl-dev \
gcompat \
automake \
autoconf \
libtool \
procps-ng procps-compat \
psmisc \
nano \
less \
curl \
sudo \
gawk \
git \
iptables \
jq \
pkgconf \
libaio-dev \
libcap-dev \
libprotobuf \
protobuf \
protobuf-c-dev \
libnl3-dev \
libnet-dev \
libseccomp \
libseccomp-dev \
libseccomp-static \
protobuf-c-compiler \
protobuf-dev \
python3 \
shadow-subids \
kmod \
unzip \
time \
net-tools \
lsb-release-minimal \
wget \
lsof \
iproute2 \
iputils-ping \
ca-certificates \
bc \
openssh \
shellcheck \
gperf \
# sysbox deps
fuse \
rsync \
bash-completion \
attr \
tree \
shadow \
strace \
&& rm -rf /var/cache/apk/* \
&& echo ". /etc/bash_completion" >> /etc/bash.bashrc \
&& ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa \
&& echo " StrictHostKeyChecking accept-new" >> /etc/ssh/ssh_config

# Install Golang
RUN wget https://go.dev/dl/go1.22.6.linux-${sys_arch}.tar.gz && \
tar -C /usr/local -xzf go1.22.6.linux-${sys_arch}.tar.gz && \
/usr/local/go/bin/go env -w GONOSUMDB=/root/nestybox

ENV GOPATH=/go
ENV PATH=$GOPATH/bin:/usr/local/go/bin:$PATH
RUN go env -w GONOSUMDB=/root/nestybox && \
mkdir -p "$GOPATH/src" "$GOPATH/bin" && \
chmod -R 777 "$GOPATH"

# Add a dummy user for the rootless integration tests; needed by the
# `git clone` operations below.
RUN useradd -u1000 -m -d/home/rootless -s/bin/bash rootless

# install bats
RUN cd /tmp \
&& git clone https://github.com/sstephenson/bats.git \
&& cd bats \
&& git reset --hard 03608115df2071fff4eaaff1605768c275e5f81f \
&& ./install.sh /usr/local \
&& rm -rf /tmp/bats

# install protoc compiler for gRPC
RUN if [ "$sys_arch" = "amd64" ] ; then arch_str="x86_64"; \
elif [ "$sys_arch" = "arm64" ]; then arch_str="aarch_64"; \
else echo "Unsupported platform: ${sys_arch}"; exit; fi \
&& curl -LO https://github.com/protocolbuffers/protobuf/releases/download/v3.15.8/protoc-3.15.8-linux-${arch_str}.zip \
&& unzip protoc-3.15.8-linux-${arch_str}.zip -d $HOME/.local \
&& export PATH="$PATH:$HOME/.local/bin" \
&& go install github.com/golang/protobuf/protoc-gen-go@latest \
&& export PATH="$PATH:$(go env GOPATH)/bin"

# Install Docker
RUN apk add --update docker openrc
RUN rc-update add docker boot
ADD https://raw.githubusercontent.com/docker/docker-ce/master/components/cli/contrib/completion/bash/docker /etc/bash_completion.d/docker.sh

# Go Dlv for debugging
RUN go install github.com/go-delve/delve/cmd/dlv@latest

# Install Kubectl for k8s-in-docker integration-testing. Notice that we are explicitly
# stating the kubectl version to download, which should match the K8s release
# deployed in the K8s-in-docker nodes (L2).
RUN cd /tmp && curl -LO "https://dl.k8s.io/release/${k8s_version_full}/bin/linux/amd64/kubectl" \
&& install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl \
&& rm /tmp/kubectl

# Install the upstream CRI-O so we get the cri-o binary plus all config files,
# etc. Later we will replace the binary with our forked Nestybox CRI-O that
# works with Sysbox.
#
# Instructions: https://cri-o.io/
RUN apk update && apk add cri-o


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are missing the installation of the Nestybox forked CRI-O here, as well as crictl and CNIs; similar to:

 168 │ # Build and install the Nestybox CRI-O (for testing deployment of pods with sysbox (aka "sysbox pods"))                                                                                                                                                                  
 169 │ RUN apt-get update && apt-get install -y --no-install-recommends libgpgme-dev \                                                                                                                                                                                          
 170 │     && mkdir -p /tmp/crio-build \                                                                                                                                                                                                                                        
 171 │     && git clone https://github.com/nestybox/cri-o.git /tmp/crio-build/cri-o \                                                                                                                                                                                           
 172 │     && git -C /tmp/crio-build/cri-o checkout -b ${crio_version}-sysbox origin/${crio_version}-sysbox \                                                                                                                                                                   
 173 │     && cd /tmp/crio-build/cri-o && make binaries \                                                                                                                                                                                                                       
 174 │     && mv /usr/bin/crio /usr/bin/crio.orig \                                                                                                                                                                                                                             
 175 │     && cp /tmp/crio-build/cri-o/bin/crio-static /usr/bin/crio \                                                                                                                                                                                                          
 176 │     && rm -rf /tmp/crio-build 
 
 178 │ RUN wget https://github.com/kubernetes-sigs/cri-tools/releases/download/${crictl_version}/crictl-${crictl_version}-linux-${sys_arch}.tar.gz \                                                                                                                            
 179 │     && sudo tar zxvf crictl-${crictl_version}-linux-${sys_arch}.tar.gz -C /usr/local/bin \                                                                                                                                                                               
 180 │     && rm -f crictl-${crictl_version}-linux-${sys_arch}.tar.gz                                                                                                                                                                                                           
 181 │                                                                                                                                                                                                                                                                          
 182 │ # Container CNIs (needed by CRI-O)                                                                                                                                                                                                                                       
 183 │ RUN curl -fsSL https://pkgs.k8s.io/core:/stable:/${k8s_version}/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg \                                                                                                                        
 184 │     && echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/${k8s_version}/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list \                                                                                       
 185 │     && apt-get update \                                                                                                                                                                                                                                                  
 186 │     && apt-get install kubernetes-cni

# Dasel (for yaml, toml, json parsing) (https://github.com/TomWright/dasel)
# Note: manually download Dasel v1 as our testContainerInit script does not yet support Dasel v2.
RUN wget https://github.com/TomWright/dasel/releases/download/v1.27.2/dasel_linux_${sys_arch} && mv dasel_linux_${sys_arch} dasel && chmod +x dasel \
&& mv ./dasel /usr/local/bin/dasel

# K8s.io KinD
RUN go install sigs.k8s.io/[email protected]

# Use the old definition for SECCOMP_NOTIF_ID_VALID in /usr/include/linux/seccomp.h
#
# This is needed because the definition changed in the mainline kernel
# on 06/2020 (from SECCOMP_IOR -> SECCOMP_IOW), and some distros we
# support have picked it up in their latest releases / kernels
# updates. The kernel change was backward compatible, so by using the
# old definition, we are guaranteed it will work on kernels before and
# after the change. On the other hand, if we were to use the new
# definition, seccomp notify would fail when sysbox runs in old
# kernels.
RUN sed -i 's/^#define SECCOMP_IOCTL_NOTIF_ID_VALID[ \t]*SECCOMP_IOW(2, __u64)/#define SECCOMP_IOCTL_NOTIF_ID_VALID SECCOMP_IOR(2, __u64)/g' /usr/include/linux/seccomp.h

# sysbox env
RUN useradd sysbox \
&& mkdir -p /var/lib/sysboxfs

# test scripts
COPY scr/testContainerInit /usr/bin
COPY scr/testContainerCleanup /usr/bin
COPY scr/buildContainerInit /usr/bin
COPY bin/userns_child_exec_${sys_arch} /usr/bin

RUN mkdir -p /root/nestybox
WORKDIR /root/nestybox/sysbox
CMD /bin/bash