Skip to content

Commit 42af2dc

Browse files
harupyBenWilson2
harupy
and
BenWilson2
authored
Add dev container configuration for VSCode (mlflow#6493)
* Codespaces Signed-off-by: harupy <[email protected]> * Use core-requirements.txt Signed-off-by: harupy <[email protected]> * Compile requirements Signed-off-by: harupy <[email protected]> * Improve Signed-off-by: harupy <[email protected]> * fix Signed-off-by: harupy <[email protected]> * Fix DB file name Signed-off-by: harupy <[email protected]> * fix Signed-off-by: harupy <[email protected]> * Add skinny requirements Signed-off-by: harupy <[email protected]> * Use script Signed-off-by: harupy <[email protected]> * Update .devcontainer/README.md Co-authored-by: Ben Wilson <[email protected]> Signed-off-by: Harutaka Kawamura <[email protected]> Signed-off-by: harupy <[email protected]> Signed-off-by: Harutaka Kawamura <[email protected]> Co-authored-by: Ben Wilson <[email protected]>
1 parent 2eb29d1 commit 42af2dc

15 files changed

+776
-3
lines changed

‎.devcontainer/Dockerfile.devcontainer

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
FROM node:16.18 AS ui-builder
2+
3+
WORKDIR /mlflow
4+
COPY mlflow/server/js .
5+
RUN yarn install --silent && yarn build
6+
7+
FROM python:3.8.15
8+
9+
ENV LANG=C.UTF-8
10+
ENV LC_ALL=C.UTF-8
11+
12+
WORKDIR /workspaces/mlflow
13+
COPY --from=ui-builder /mlflow/build ./mlflow/server/js/build
14+
15+
RUN apt update -y \
16+
&& apt install -y sudo \
17+
&& DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC \
18+
sudo -E apt-get install --no-install-recommends -y \
19+
git curl wget jq vim tree zsh openjdk-11-jdk \
20+
# pyenv dependencies
21+
make build-essential libssl-dev zlib1g-dev \
22+
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
23+
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
24+
# Clean up
25+
&& sudo apt clean \
26+
&& rm -rf /var/lib/apt/lists/*
27+
28+
# Install node and yarn for UI development
29+
RUN curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash - \
30+
&& sudo apt install -y nodejs \
31+
&& npm -g install yarn
32+
33+
# Install Oh My Zsh
34+
# fb66b67d683935cd0e4a5282064947a28087a91b is the latest commit as of 2022-11-14
35+
RUN bash -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/fb66b67d683935cd0e4a5282064947a28087a91b/tools/install.sh)"
36+
COPY .devcontainer/codespaces.zsh-theme /root/.oh-my-zsh/custom/themes/codespaces.zsh-theme
37+
ENV ZSH_THEME=codespaces
38+
39+
# Install pyenv
40+
# v2.3.6 is the latest release as of 2022-11-14
41+
RUN git clone --depth 1 --single-branch --branch v2.3.6 https://github.com/pyenv/pyenv.git /root/.pyenv \
42+
&& echo 'eval "$(pyenv init --path)"' >> /root/.zshrc
43+
ENV PYENV_ROOT="/root/.pyenv"
44+
ENV PATH="$PYENV_ROOT/bin:$PATH"
45+
46+
# Install mlflow dependencies
47+
COPY .devcontainer/requirements.txt .
48+
RUN pip install --no-cache-dir -r requirements.txt && rm requirements.txt

‎.devcontainer/Dockerfile.devcontainer.dockerignore

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
**
2+
3+
!.devcontainer/codespaces.zsh-theme
4+
!.devcontainer/requirements.txt
5+
!mlflow/server/js
6+
mlflow/server/js/node_modules

‎.devcontainer/README.md

+72
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# MLflow development container (experimental)
2+
3+
This directory contains a set of files to set up a reproducible and disposable development environment for MLflow in Visual Studio Code using the [Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers).
4+
5+
## Supported features
6+
7+
- Pre-installed tools/packages required for MLflow development.
8+
- Pre-configured VSCode settings and extensions for automatic code formatting and lint check.
9+
- Pre-commit hooks to avoid pushing commits that won't pass the CI checks.
10+
11+
## Prerequisites
12+
13+
- [Visual Studio Code](https://code.visualstudio.com/)
14+
- [Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
15+
- [Docker](https://www.docker.com/)
16+
17+
## Getting started
18+
19+
1. Build the devcontainer image.
20+
21+
```bash
22+
# This command may take a while to finish
23+
DOCKER_BUILDKIT=1 docker build -f .devcontainer/Dockerfile.devcontainer -t ghcr.io/mlflow/mlflow-devcontainer .
24+
25+
# Test the image
26+
docker run --rm -it ghcr.io/mlflow/mlflow-devcontainer zsh
27+
28+
# Alternatively, you can pull the pre-built image from GitHub Container Registry,
29+
# but a GitHub personal access token is required to authenticate to ghcr.io:
30+
echo <GITHUB_TOKEN> | docker login ghcr.io -u <GITHUB_USERNAME> --password-stdin
31+
docker pull ghcr.io/mlflow/mlflow-devcontainer
32+
```
33+
34+
2. Open the MLflow repository on VSCode.
35+
3. Press `Ctrl/Cmd+Shift+P` to launch [the command palette](https://code.visualstudio.com/docs/getstarted/userinterface#_command-palette).
36+
4. Select `Remote-Containers: Reopen in Container`.
37+
5. Once the devcontainer is up and running, launch the command palette again.
38+
6. Select `Terminal: Create New Terminal`.
39+
7. Run the following commands and make sure they run successfully:
40+
41+
```bash
42+
pytest tests/test_version.py
43+
```
44+
45+
## Developing in GitHub Codespaces
46+
47+
You can create the same development environment as your local devcontainer **in a web browser with just a few clicks** using [GitHub Codespaces](https://github.com/features/codespaces). The instructions in [Creating a codespace](https://docs.github.com/en/codespaces/developing-in-codespaces/creating-a-codespace#creating-a-codespace) cover how to set up a codespace.
48+
49+
<img src="./images/codespace.png" width="50%">
50+
51+
## Limitations
52+
53+
The following tools/packages are NOT pre-installed to avoid increasing the image size. They can be manually installed after launching the devcontainer if necessary.
54+
55+
- Python ML packages such as `tensorflow`
56+
- R
57+
- Docker
58+
- conda
59+
60+
## How to update `requirements.txt`
61+
62+
```bash
63+
NAME="mlflow-$(uuidgen)"
64+
docker run \
65+
--name $NAME \
66+
-w /mlflow \
67+
-v $(pwd)/requirements:/mlflow/requirements:ro \
68+
-v $(pwd)/.devcontainer/pip-compile.sh:/mlflow/pip-compile.sh \
69+
python:3.8.15 ./pip-compile.sh
70+
docker cp $NAME:/tmp/requirements.txt .devcontainer/requirements.txt
71+
docker rm -f -v $NAME
72+
```

‎.devcontainer/codespaces.zsh-theme

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Codespaces zsh prompt theme
2+
__zsh_prompt() {
3+
local prompt_username
4+
if [ ! -z "${GITHUB_USER}" ]; then
5+
prompt_username="@${GITHUB_USER}"
6+
else
7+
prompt_username="%n"
8+
fi
9+
PROMPT="%{$fg[green]%}${prompt_username} %(?:%{$reset_color%}➜ :%{$fg_bold[red]%}➜ )" # User/exit code arrow
10+
PROMPT+='%{$fg_bold[blue]%}%(5~|%-1~/…/%3~|%4~)%{$reset_color%} ' # cwd
11+
PROMPT+='$([ "$(git config --get codespaces-theme.hide-status 2>/dev/null)" != 1 ] && git_prompt_info)' # Git status
12+
PROMPT+='%{$fg[white]%}$ %{$reset_color%}'
13+
unset -f __zsh_prompt
14+
}
15+
ZSH_THEME_GIT_PROMPT_PREFIX="%{$fg_bold[cyan]%}(%{$fg_bold[red]%}"
16+
ZSH_THEME_GIT_PROMPT_SUFFIX="%{$reset_color%} "
17+
ZSH_THEME_GIT_PROMPT_DIRTY=" %{$fg_bold[yellow]%}✗%{$fg_bold[cyan]%})"
18+
ZSH_THEME_GIT_PROMPT_CLEAN="%{$fg_bold[cyan]%})"
19+
__zsh_prompt

‎.devcontainer/devcontainer.json

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
// https://code.visualstudio.com/docs/remote/devcontainerjson-reference
3+
"name": "MLflow Development Container",
4+
"dockerComposeFile": "docker-compose.yml",
5+
"workspaceFolder": "/workspaces/mlflow",
6+
"service": "mlflow",
7+
"postCreateCommand": ".devcontainer/postCreate.sh",
8+
"overrideCommand": true,
9+
"settings": {
10+
"terminal.integrated.defaultProfile.linux": "zsh",
11+
"editor.rulers": [80, 100],
12+
"editor.formatOnSave": true,
13+
"python.defaultInterpreterPath": "~/miniconda/envs/mlflow-dev-env/bin/python",
14+
// Disable automatic environment activation because `.zshrc` contains `conda activate mlflow-dev-env`.
15+
"python.terminal.activateEnvironment": false,
16+
"python.analysis.autoImportCompletions": false,
17+
"python.linting.enabled": false,
18+
"python.formatting.provider": "black",
19+
"prettier.prettierPath": "/workspaces/mlflow/mlflow/server/js/node_modules/prettier",
20+
"prettier.configPath": "/workspaces/mlflow/mlflow/server/js/.prettierrc.js"
21+
},
22+
"extensions": [
23+
"dbaeumer.vscode-eslint",
24+
"eamodio.gitlens",
25+
"esbenp.prettier-vscode",
26+
"GitHub.vscode-pull-request-github",
27+
"ms-azuretools.vscode-docker",
28+
"ms-python.python",
29+
"oderwat.indent-rainbow",
30+
"PKief.material-icon-theme",
31+
"ritwickdey.LiveServer",
32+
"shardulm94.trailing-spaces"
33+
]
34+
}

‎.devcontainer/docker-compose.yml

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
version: '3'
2+
services:
3+
mlflow:
4+
image: 'ghcr.io/mlflow/mlflow-devcontainer'
5+
working_dir: /workspaces/mlflow
6+
environment:
7+
MLFLOW_HOME: /workspaces/mlflow
8+
MLFLOW_TRACKING_URI: sqlite:////workspaces/mlflow/mlruns.db
9+
volumes:
10+
- $PWD:/workspaces/mlflow
11+
- build:/workspaces/mlflow/mlflow/server/js/build
12+
13+
volumes:
14+
build:

‎.devcontainer/images/codespace.png

474 KB
Loading

‎.devcontainer/pip-compile.sh

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/usr/bin/env bash
2+
3+
set -ex
4+
5+
pip install pip-tools
6+
7+
cd requirements
8+
pip-compile --verbose \
9+
--output-file /tmp/output.txt \
10+
skinny-requirements.txt \
11+
core-requirements.txt \
12+
doc-requirements.txt \
13+
test-requirements.txt \
14+
lint-requirements.txt
15+
16+
# Add a timestamp at the beginning of the file
17+
echo "# Created at: $(date -u +"%F %T %Z")" | cat - /tmp/output.txt > /tmp/requirements.txt

‎.devcontainer/postCreate.sh

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
#!/bin/bash
2+
3+
# Turn off git status check to improve zsh response speed: https://stackoverflow.com/a/25864063
4+
git config --add oh-my-zsh.hide-status 1
5+
git config --add oh-my-zsh.hide-dirty 1
6+
pip install --no-deps -e .

0 commit comments

Comments
 (0)