Skip to content

Commit

Permalink
Make deployment better
Browse files Browse the repository at this point in the history
  • Loading branch information
Matej Groman committed May 2, 2023
1 parent 8b6465c commit 5d8afd6
Show file tree
Hide file tree
Showing 8 changed files with 144 additions and 51 deletions.
63 changes: 63 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Docker Compose Environment Variables Example
# =====================================
# --------required variables-----------
# =====================================
NEO4J_PASSWORD=
REDIS_PASSWORD=

# For deployment uisng Azure Containers
# ----------------------
# Name of created Azure Container Registry
ACR_NAME=
# Domain name for the application. It will
# be accessible under
# http://<AZURE_DOMAIN_NAME>.REGION.azurecontainer.io:3000
AZURE_DOMAIN_NAME=depvis

# --------optional variables-----------
# Neo4J Database
# =====================================
NEO4J_HOST=neo4j://neo4j:7687

# Redis Cache
# =====================================
REDIS_HOST=redis-cache
REDIS_PORT=6379

# DepVis Docker Port
# =====================================
# Specify public facing port for Docker container
DEPVIS_PORT=3000

# =====================================
# DepVis other environmental variables
# =====================================

# Vulnerability database
# ----------------------
# Specify default vulnerability database. Currently supported are
# Sonatype OSS
VULN_DB=Sonatype
# or OSV
# VULN_DB=osv

# Sonatype OSS databse authentication token
# ----------------------
# It is not required, but recommended. Wihout token API is rate limited
# Learn how to obtain token here: https://ossindex.sonatype.org/doc/api-token
SONATYPE_OSS_AUTH=

# CORS origin - use when API and frontend are on different host
# ----------------------
# CORS_ORIGIN=*

# GraphQL Server Uri - used for server side requests
# In Docker environment, port is usually diferent for frontend and internal requests
# ----------------------
NEXT_PUBLIC_SERVER_URI=http://localhost:3000

# Excluded nodes Regex Default value - used to separate system nodes from other
# ----------------------
# NEXT_PUBLIC_GRAPH_EXCLUDED_REGEX=.*ystem


4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,9 @@ web_modules/
.env.test.local
.env.production.local
.env.local
!.env.*.example
!.env.production.example
!.env.example


# parcel-bundler cache (https://parceljs.org/)
.cache
Expand Down
52 changes: 45 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Dependency visualization - DepVis

Tool for visualization of open source dependencies and vulnerabilities from Software Bill of Materials (SBOM).
DepVis is a Proof-of-Concept tool for visualization of open source dependencies and vulnerabilities using Software Bill of Materials (SBOM) as the input.

This tool was developed as part of Master's thesis "Visualization of Vulnerabilities in Open Source Software Dependencies" at FI MUNI.
This tool was developed as part of Master's thesis "Visualization of Vulnerabilities in Open Source Software Dependencies" by Matej Groman at FI MUNI.

## Prerequisites

Expand All @@ -11,22 +11,60 @@ This tool was developed as part of Master's thesis "Visualization of Vulnerabili

## Deployment

Deployment is realized using three Docker containers. Following Figure presents the DepVis architecture.

- The Web Client and Web Server is running as one container using [Next.js framework](https://nextjs.org/).
- Data are stored in a [Neo4J Database](https://neo4j.com/) running as second container
- A Redis Cache is used by import queue, realized by third container.

![DepVis Architecture Diagram](./figures/depvis-architecture.png)

To make deployment easier, a `docker-compose.yml` files are used to define complete infrastructure.

### Using Docker locally

- Clone this repository
- _Optional_ Change login credentials for neo4j by editing [docker-compose.yml](./docker-compose.yml)
- Create environment variables file according to sample file in Next.js app - [example](./src/depvis-next/.env.production.example)
- To start all services use `docker-compose up`
- Create environment variables file for docker compose - check [example](./.env.example) for supported variables.
- Minimum required are: `NEO4J_PASSWORD` and `REDIS_PASSWORD`.
- Optionally you can create environment variables file according to sample file in Next.js app - [example](./src/depvis-next/.env.production.example) and edit `docker-compose.yml` accordingly.
- To build all services use `docker-compose build`
- Start all services using `docker-compose up`
- For more details follow installation steps for Next.js app [here](./src/depvis-next/README.md)

### Using Azure containers

It is also possible to deploy the container stack into Azure. Follow [official guide](https://learn.microsoft.com/en-us/azure/container-instances/tutorial-docker-compose) to:

- Clone this repository
- Follow [official guide](https://learn.microsoft.com/en-us/azure/container-instances/tutorial-docker-compose) to create Azure container repository (ACR), publish DepVis image and create new Docker context
- Use [./docker-compose-azure.yml] to deploy containers in Azure (you'll need to update name of the container according to your ACR)
- Create Azure Container Registry
- Login into Azure Container Registry (ACR) using `az acr login --name <acrName>`
- Create environment variables file for docker compose - check [example](./.env.example) for supported variables.
- Minimum required are: `NEO4J_PASSWORD`, `REDIS_PASSWORD` and `ACR_NAME`.
- Build docker image (in default Docker Context) using [Azure Docker Compose file](./docker-compose-azure.yml) and push image to ACR created earlier

```
> docker-compose -f ./docker-compose-azure.yml build
> docker-compose push
```

- Create new Docker context and switch to it using

```
> docker login azure
> docker context create aci depvis
> docker context use depvis
```

- Publish new container stack in new context using `docker compose up`

- See `docker ps` for details

## Repository content

- [sample_bom](./sample_bom/): Contains sample SBOM files for quick testing purposes
- [src/depvis-next](./src/depvis-next/): Next.js web application
- [docker-compose.yml](./docker-compose.yml): Create containers necessary for proper functionality

```
```
30 changes: 16 additions & 14 deletions docker-compose-azure.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,42 +10,44 @@ services:
- neo4j:/data
restart: unless-stopped
environment:
- NEO4J_AUTH=neo4j/${NEO4J_PASSWORD}
- NEO4J_AUTH=neo4j/${NEO4J_PASSWORD:?error}
- NEO4J_apoc_export_file_enabled=true
- NEO4J_apoc_import_file_enabled=true
- NEO4J_apoc_import_file_use__neo4j__config=true
- NEO4JLABS_PLUGINS=["apoc"]
- NEO4J_PLUGINS=["apoc"]
- NEO4J_dbms_security_procedures_unrestricted=apoc.*,algo.*

redis-cache:
image: redis:latest
command: redis-server --requirepass ${REDIS_PASSWORD}
command: redis-server --requirepass ${REDIS_PASSWORD:?error}
restart: unless-stopped
volumes:
- cache:/data
ports:
- ${REDIS_PORT}:6379
- ${REDIS_PORT:-6379}:6379

depvis-next:
depends_on:
- neo4j
- redis-cache
build: ./src/depvis-next
image: depvisacr.azurecr.io/depvis-next:latest
build:
context: ./src/depvis-next/
image: ${ACR_NAME:?error}/depvis-next:latest
ports:
- ${DEPVIS_PORT}:3000
restart: always
domainname: depvis
domainname: ${AZURE_DOMAIN_NAME:-depvis}
environment:
NEO4J_USER: ${NEO4J_USER}
NEO4J_PASSWORD: ${NEO4J_PASSWORD}
NEO4J_HOST: ${NEO4J_HOST}
NEO4J_USER: neo4j
NEO4J_PASSWORD: ${NEO4J_PASSWORD:?error}
NEO4J_HOST: ${NEO4J_HOST:-neo4j://neo4j:7687}
REDIS_HOST: redis-cache
REDIS_PORT: ${REDIS_PORT}
REDIS_PASSWORD: ${REDIS_PASSWORD}
REDIS_PORT: ${REDIS_PORT:-6379}
REDIS_PASSWORD: ${REDIS_PASSWORD:?error}
NEXT_PUBLIC_SONATYPE_OSS_AUTH: ${SONATYPE_OSS_AUTH}
NEXT_PUBLIC_SERVER_URI: ${NEXT_PUBLIC_SERVER_URI}
NEXT_PUBLIC_GRAPH_EXCLUDED_REGEX: ${GRAPH_EXCLUDED_REGEX}
NEXT_PUBLIC_SERVER_URI: ${NEXT_PUBLIC_SERVER_URI:-http://localhost:3000}
VULN_DB: ${VULN_DB:-Sonatype}
CORS_ORIGIN: ${CORS_ORIGIN:-http://localhost:3000}
volumes:
cache:
driver: azure_file
Expand Down
42 changes: 15 additions & 27 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,55 +12,43 @@ services:
- ./runtime/plugins:/var/lib/neo4j/plugins
restart: unless-stopped
environment:
- NEO4J_AUTH=neo4j/${NEO4J_PASSWORD}
- NEO4J_AUTH=neo4j/${NEO4J_PASSWORD:?error}
- NEO4J_apoc_export_file_enabled=true
- NEO4J_apoc_import_file_enabled=true
- NEO4J_apoc_import_file_use__neo4j__config=true
- NEO4JLABS_PLUGINS=["apoc"]
- NEO4J_PLUGINS=["apoc"]
- NEO4J_dbms_security_procedures_unrestricted=apoc.*,algo.*

redis-cache:
image: redis:latest
command: redis-server --requirepass ${REDIS_PASSWORD}
command: redis-server --requirepass ${REDIS_PASSWORD:?error}
restart: unless-stopped
volumes:
- cache:/data
ports:
- ${REDIS_PORT}:6379
links:
- redis-commander

redis-commander:
image: rediscommander/redis-commander:latest
restart: unless-stopped
environment:
REDIS_HOSTS: redis-cache
REDIS_HOST: redis-cache
REDIS_PORT: redis-cache:6379
REDIS_PASSWORD: ${REDIS_PASSWORD}
HTTP_USER: root
HTTP_PASSWORD: root
ports:
- 8081:8081
- ${REDIS_PORT:-6379}:6379

depvis-next:
depends_on:
- neo4j
- redis-cache
build: ./src/depvis-next
build:
context: ./src/depvis-next/
image: depvis-next:latest
ports:
- ${DEPVIS_PORT}:3000
- ${DEPVIS_PORT:-3000}:${DEPVIS_PORT:-3000}
restart: always
environment:
NEO4J_USER: ${NEO4J_USER}
NEO4J_PASSWORD: ${NEO4J_PASSWORD}
NEO4J_HOST: ${NEO4J_HOST}
NEO4J_USER: neo4j
NEO4J_PASSWORD: ${NEO4J_PASSWORD:?error}
NEO4J_HOST: ${NEO4J_HOST:-neo4j://neo4j:7687}
REDIS_HOST: redis-cache
REDIS_PORT: ${REDIS_PORT}
REDIS_PASSWORD: ${REDIS_PASSWORD}
REDIS_PORT: ${REDIS_PORT:-6379}
REDIS_PASSWORD: ${REDIS_PASSWORD:?error}
NEXT_PUBLIC_SONATYPE_OSS_AUTH: ${SONATYPE_OSS_AUTH}
NEXT_PUBLIC_SERVER_URI: ${NEXT_PUBLIC_SERVER_URI}
NEXT_PUBLIC_SERVER_URI: ${NEXT_PUBLIC_SERVER_URI:-http://localhost:3000}
VULN_DB: ${VULN_DB:-Sonatype}
CORS_ORIGIN: ${CORS_ORIGIN:-http://localhost:3000}
volumes:
cache:
driver: local
Binary file added figures/depvis-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion src/depvis-next/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
# Copy .env production variables
COPY .env.production .
COPY *.env.production .

USER nextjs

Expand Down
2 changes: 1 addition & 1 deletion src/depvis-next/apollo/ApolloClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import urlJoin from "url-join";
let httpApolloClient;
let ssrApolloClient;
const ssrHttpLink = createHttpLink({
uri: urlJoin(process.env.NEXT_PUBLIC_SERVER_URI, GraphQLUri),
uri: urlJoin(process.env.NEXT_PUBLIC_SERVER_URI || "", GraphQLUri),
credentials: "same-origin",
fetchOptions: {
mode: "cors",
Expand Down

0 comments on commit 5d8afd6

Please sign in to comment.