-
Notifications
You must be signed in to change notification settings - Fork 0
Added Dockerfile and accessing mdx2 through Docker container #57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
139d286
c4f312e
87f5760
effc361
639e12e
b039712
a41afe8
8d5fa4a
ee992b8
d7b0663
786596e
6e9b65b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| .git | ||
| .github | ||
| .gitignore | ||
| .dockerignore | ||
| .DS_Store | ||
|
|
||
| __pycache__ | ||
| *.py[cod] | ||
| *.egg-info/ | ||
| *.egg | ||
| build/ | ||
| dist/ | ||
| .eggs/ | ||
|
|
||
| .env | ||
| .venv | ||
| env/ | ||
| venv/ | ||
|
|
||
| .mypy_cache/ | ||
| .pytest_cache/ | ||
| .coverage | ||
| htmlcov/ | ||
| .ipynb_checkpoints |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # diffuseproject/mdx2:1.0.0 (https://hub.docker.com/repository/docker/diffuseproject/mdx2/tags/1.0.0/sha256-abfcf7d2feacdf9a207a13c2bb908d62e6f461259f78e82d3bb8983267446291) | ||
|
|
||
| FROM mambaorg/micromamba:1.5.5 AS micromamba_stage | ||
|
|
||
| FROM python:3.10-slim AS final | ||
|
|
||
| RUN apt-get update && apt-get install -y ca-certificates git | ||
|
|
||
| COPY --from=micromamba_stage /bin/micromamba /usr/local/bin/micromamba | ||
|
|
||
| ENV PATH="/usr/local/bin:/opt/conda/bin:$PATH" | ||
| ENV JUPYTER_PORT=8888 | ||
|
|
||
| RUN mkdir -p /opt/conda | ||
|
|
||
| WORKDIR /home/dev | ||
|
|
||
| COPY env.yaml . | ||
|
|
||
| # Use micromamba to create the environment and install packages | ||
| RUN /usr/local/bin/micromamba create -f env.yaml -n mdx2-dev && \ | ||
| /usr/local/bin/micromamba install -y -n mdx2-dev nexpy jupyterlab jupyterlab-h5web dials xia2 wget tar -c conda-forge && \ | ||
| /usr/local/bin/micromamba clean --all --yes | ||
|
|
||
| COPY . . | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want to add a .dockerignore file to prevent unnecessary files (.git, etc.) from going into the image if we decide to publish on Dockerhub?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I created the ".dockerignore" file to reduce the size of the image being pushed to docker hub, but depending on how the volume is mounted, the user can still access the unnecessary files |
||
|
|
||
| # Install dials-extensions pinned to a specific commit (963fe9e458a505a8443c988d45fb8dcb4532768f) for CHESS beamline compatibility | ||
| # Install the local package in editable mode within the environment | ||
| RUN /usr/local/bin/micromamba run -n mdx2-dev pip install git+https://github.com/FlexXBeamline/dials-extensions.git@963fe9e458a505a8443c988d45fb8dcb4532768f | ||
| RUN /usr/local/bin/micromamba run -n mdx2-dev pip install -e . | ||
|
|
||
| EXPOSE 8880-8890 | ||
|
|
||
| CMD /usr/local/bin/micromamba run -n mdx2-dev jupyter lab --ip=0.0.0.0 --port=$JUPYTER_PORT --no-browser --allow-root | ||
|
Check warning on line 34 in .github/Dockerfile
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| name: docker | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| pull_request: | ||
|
|
||
| jobs: | ||
| docker: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Login to Docker Hub | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ vars.DOCKERHUB_USERNAME }} | ||
| password: ${{ secrets.DOCKERHUB_TOKEN }} | ||
|
Comment on lines
+16
to
+20
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Docker Hub login will fail for PRs from forks. Secrets ( 🔧 Proposed fix: Skip login when secrets unavailable - name: Login to Docker Hub
+ if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
uses: docker/login-action@v3
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}And update the build step to only build (not push) when login is skipped: - name: Build and push
uses: docker/build-push-action@v6
with:
- push: ${{ github.ref == 'refs/heads/main' }}
+ push: ${{ github.ref == 'refs/heads/main' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) }}
tags: ${{ steps.tags.outputs.tags }}🤖 Prompt for AI Agents
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not an issue, because right now we want a more rigorous review process of changes being made to the mdx2 repo There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
🧠 Learnings used |
||
|
|
||
| - name: Set up QEMU | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is purpose of setting up QEMU here?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. QEMU is used for multi-platform builds of the docker image. Right now we're supporting linux/amd64 architecture, but if we want o support other platforms like linux/arm64 in the future, we can have a way of building the image on that platform too |
||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Determine Docker tags | ||
| id: tags | ||
| run: | | ||
| if [ "${{ github.ref }}" = "refs/heads/main" ]; then | ||
| echo "tags=${{ vars.DOCKERHUB_USERNAME }}/mdx2:latest" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "tags=${{ vars.DOCKERHUB_USERNAME }}/mdx2:test" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| - name: Build and push | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| file: .github/Dockerfile | ||
| push: ${{ github.ref == 'refs/heads/main' }} | ||
| tags: ${{ steps.tags.outputs.tags }} | ||
|
coderabbitai[bot] marked this conversation as resolved.
coderabbitai[bot] marked this conversation as resolved.
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -44,7 +44,7 @@ Publications describing [ando-lab/mdx2](https://github.com/ando-lab/mdx2): | |
|
|
||
| ### Prerequisites | ||
|
|
||
| For a conda-based installation, you'll need [micromamba](https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html) or equivalent. | ||
| For a conda-based installation, you'll need [micromamba](https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html) or equivalent. The Docker-based installation requires you to install and login to [Docker](https://www.docker.com/get-started/). | ||
|
|
||
| ### User install (conda environment) | ||
|
|
||
|
|
@@ -71,3 +71,24 @@ pip install -e ".[dev]" | |
| ``` | ||
|
|
||
| The last line installs mdx2 in editable mode, with optional development tools including pytest and ruff | ||
|
|
||
| ### Developer Install (Docker container) | ||
|
|
||
| Pull image from Docker Hub and run the container. Search the Docker container logs for the localhost address to access mdx2 within a Jupyter Lab notebook or terminal. | ||
|
|
||
| - Docker image exposes ports 8880-8890 (default: 8888) | ||
| - Platform: linux/amd64 | ||
| ```bash | ||
| git clone https://github.com/diff-use/mdx2.git | ||
| cd mdx2 | ||
|
Comment on lines
+82
to
+83
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Clarify why cloning is needed when Docker already contains mdx2. The Docker image already has mdx2 installed. The 📝 Suggested clarificationAdd context before the commands: Clone the repository to mount your local code into the container for development. This allows you to edit files on your host and see changes reflected inside the container.Or, if cloning is optional: (Optional) Clone the repository if you want to mount local code for development:🤖 Prompt for AI Agents
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The Docker image doesn't have mdx2 installed, so that's why the git clone step is needed There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
That said, the README could still benefit from a brief note explaining that the ### Developer Install (Docker container)
Pull image from Docker Hub and run the container. The repository must be cloned locally so it can be mounted into the container — mdx2 is not pre-installed in the image. Search the Docker container logs for the localhost address to access mdx2 within a Jupyter Lab notebook or terminal.This is just a suggestion to keep the documentation clear. Feel free to word it however fits best! ✏️ Learnings added
|
||
| docker pull diffuseproject/mdx2:1.0.0 | ||
| docker run --platform linux/amd64 --name mdx2_container -it -d -e JUPYTER_PORT={CONTAINER_PORT} -p {HOST_PORT}:{CONTAINER_PORT} -v "$(pwd)":/home/dev diffuseproject/mdx2:1.0.0 | ||
|
coderabbitai[bot] marked this conversation as resolved.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Undefined placeholders Users cannot execute this command without knowing what values to substitute. The Dockerfile sets 🐛 Proposed fix with concrete values and explanationReplace lines 77-86 with: Pull image from Docker Hub and run the container. The Jupyter Lab URL will be printed when you run `docker logs`.
- Default Jupyter port: **8888**
- Platform: `linux/amd64`
- To use a different host port, replace `8888:8888` with `{YOUR_HOST_PORT}:8888`
```bash
git clone https://github.com/diff-use/mdx2.git
cd mdx2
docker pull diffuseproject/mdx2:latest
docker run --platform linux/amd64 --name mdx2_container -d -p 8888:8888 -v "$(pwd)":/home/dev diffuseproject/mdx2:latest
docker logs mdx2_containerVerify each finding against the current code and only fix it if needed. In |
||
| docker logs mdx2_container | ||
| ``` | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| (Optional) Open a shell in the running container: | ||
| ```bash | ||
| docker exec -it mdx2_container /bin/bash | ||
| eval "$(micromamba shell hook --shell bash)" | ||
| micromamba activate 'mdx2-dev' | ||
| ``` | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Default
JUPYTER_PORT(8888) is outside the exposed range (8880-8890).The default
JUPYTER_PORT=8888doesn't fall within theEXPOSE 8880-8890range. Users relying on the default will have Jupyter running on an unexposed port. Either change the default to a port within the exposed range (e.g., 8880) or expand the exposed range to include 8888.🔧 Proposed fix (Option 1: Change default port)
Also applies to: 35-35
🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
8888 is in range (8880-8890)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jlee733, you're absolutely right — 8888 falls within the inclusive range 8880–8890. My original comment was incorrect. Apologies for the noise!