-
Notifications
You must be signed in to change notification settings - Fork 7
3. Creating, testing, and deploying preprocessors and handlers
Although other wiki pages cover what preprocessors and handlers are, and general rules concerning docker and server use, this page directly discusses the path from getting code working on your local machine, through testing, and into production. Much of the example text may be McGill-specific, but the general approach is likely valid for anyone creating new preprocessors or handlers.
Juliette created video tutorials that explain server use and preprocessor creation and handler creation and testing. They give a more hands-on walkthrough with examples.
IMPORTANT: To use github from unicorn, you'll need to set up SSH ForwardAgent. See the SSH page! Do not skip this step. If you are doing a live onboarding session, you need to have the repository cloned in your homedir on unicorn BEFORE the start of the onboarding session.
Note: Much of this can be done in docker on your local machine. This would let you create and test locally. For the purposes of illustration, however, we'll be doing everything directly on unicorn. If you want to develop on your local machine, we primarily work with docker on Linux, but some have had success with WSL: Docker for windows. YMMV.
If you are starting from scratch, probably best to start with a minimal template unless there is another that is closer to what you're building: Preprocessor minimal example. We're walking through a preprocessor, but creating/modifying a handler should be similar: Handler minimal example. We'll assume that we're going to modify the hello-preprocessor, so we'll clone and make a new branch. NOTE: We use the name test_preprocessor throughout this tutorial. We recommend, especially if you are participating in a group onboarding with others doing the tutorial at the same time, to please use your userid as a prefix to anything you're creating (e.g., USERID_test_preprocessor) so that we can tell them all apart!
git clone --recurse-submodules [email protected]:Shared-Reality-Lab/IMAGE-server.git
cd IMAGE-server/preprocessors/hello-preprocessor
git checkout -b test_preprocessorNote the Dockerfile that creates an image capable of running your code:
jeffbl@unicorn ~/t/I/p/hello-preprocessor (test_preprocessor)> cat Dockerfile
FROM node:alpine
WORKDIR /usr/src/app
# Apparently splittig this up is good for layers
# Docker images are onions
COPY /preprocessors/hello-preprocessor/package*.json ./
RUN npm ci
COPY /preprocessors/hello-preprocessor/ .
COPY /schemas src/schemas
RUN npm run build
ENV NODE_ENV=production
EXPOSE 8080
USER node
CMD [ "node", "dist/server.js" ]To ensure that data is passed in the proper format without errors, IMAGE relies heavily on JSON schemas, which are a formal statement of exactly what the JSON can contain. These are included using the COPY /schemas src/schemas line of the Dockerfile. If you are building a container manually, this assumes you are doing so from the root of the server repo:
juliette@unicorn:~/Documents/IMAGE-server$ pwd
/home/juliette/Documents/IMAGE-server
juliette@unicorn:~/Documents/IMAGE-server$ docker build -t hello-preprocessor:test -f preprocessors/hello-preprocessor/Dockerfile .
[+] Building 8.9s (12/12) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 390B 0.0s
=> [internal] load metadata for docker.io/library/node:alpine 0.4s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/7] FROM docker.io/library/node:alpine@sha256:ed9736a13b88ba55cbc08c75c9edac8ae7f72840482e4 3.4s
=> => resolve docker.io/library/node:alpine@sha256:ed9736a13b88ba55cbc08c75c9edac8ae7f72840482e4 0.0s
=> => sha256:d433621cbfa16b261a74722998319373dc039606f2467bb0430810596fef0a15 1.39MB / 1.39MB 0.2s
=> => sha256:e27403de326fffb9807b586a4842d6a9548fc3b33b0ba936775e9c4a6809c97e 448B / 448B 0.1s
=> => sha256:ed9736a13b88ba55cbc08c75c9edac8ae7f72840482e40324670b299336680c1 6.41kB / 6.41kB 0.0s
=> => sha256:3dbc5d17cf89e1d8ae6f4a8a562f8a5b5df52b4c7060bfb359de86de6a3ecc3c 1.72kB / 1.72kB 0.0s
=> => sha256:d5b2c58d02546cb936e436e4b27d896b7b3be359e836755a1a86f1696739dbca 6.38kB / 6.38kB 0.0s
=> => sha256:d2e28128a0eb47dce2606af084fb8860d17c44334b95da7a836e007d34c5a785 48.37MB / 48.37MB 0.6s
=> => extracting sha256:d2e28128a0eb47dce2606af084fb8860d17c44334b95da7a836e007d34c5a785 2.5s
=> => extracting sha256:d433621cbfa16b261a74722998319373dc039606f2467bb0430810596fef0a15 0.1s
=> => extracting sha256:e27403de326fffb9807b586a4842d6a9548fc3b33b0ba936775e9c4a6809c97e 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 252.42kB 0.0s
=> [2/7] WORKDIR /usr/src/app 0.1s
=> [3/7] COPY /preprocessors/hello-preprocessor/package*.json ./ 0.0s
=> [4/7] RUN npm ci 2.1s
=> [5/7] COPY /preprocessors/hello-preprocessor/ . 0.0s
=> [6/7] COPY /schemas src/schemas 0.0s
=> [7/7] RUN npm run build 2.0s
=> exporting to image 0.7s
=> => exporting layers 0.7s
=> => writing image sha256:e8c4b79a9bf25d9eb4d4a27776207d31b6a857a048f9c2120d931b6f57eeab99 0.0s
=> => naming to docker.io/library/hello-preprocessor:test 0.0s
juliette@unicorn:~/Documents/IMAGE-server$Normally, unicorn is running all of the unstable built docker images for the project via /var/docker/image/docker-compose.yml, as merged into the main branch. General information can be found in the docker-compose documentation.
The docker-compose.yml file manages services for different environments(test, production) based on profiles. Detailed information on how profiles manage docker services can be found here.
When you want to run additional or different preprocessors (or handlers, or services!) before they get merged you'll use a docker-compose.override.yml in the same directory. This docker-compose.override.yml can also be used to build the container you've modified:
services:
test-preprocessor:
profiles: [test, default] #It is important to have profiles for each service, services without a profile are always enabled
image: "test-preprocessor:test"
build:
context: /home/juliette/Documents/IMAGE-server
dockerfile: /home/juliette/Documents/IMAGE-server/preprocessors/hello-preprocessor/Dockerfile
env_file:
./IMAGE-server/config/express-common.env # This is necessary to just accept larger requests when using express!NB: the docker-compose.override.yml file may already be used by someone else! If that's the case, add your changes to theirs as new keys in YAML rather than deleting what's already there. Then, everyone's overrides will be reflected. However if someone is modifying the same key (e.g., you're both working on test-preprocessor) then you will need to coordinate with them.
You can build your image and start the container at the same time by running docker compose up -d test-preprocessor. Note that this will only rebuild your image if one does not already exist. If you need to rebuild an existing image, then you would run docker compose up --build -d test-preprocessor.
juliette@unicorn:/var/docker/image$ docker compose up -d test-preprocessor
=> [test-preprocessor internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 390B 0.0s
=> [test-preprocessor internal] load metadata for docker.io/library/node:alpine 0.2s
=> [test-preprocessor internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [test-preprocessor 1/7] FROM docker.io/library/node:alpine@sha256:ed9736a13b88ba55cbc08c75c9e 0.0s
=> [test-preprocessor internal] load build context 0.0s
=> => transferring context: 3.75kB 0.0s
=> CACHED [test-preprocessor 2/7] WORKDIR /usr/src/app 0.0s
=> CACHED [test-preprocessor 3/7] COPY /preprocessors/hello-preprocessor/package*.json ./ 0.0s
=> CACHED [test-preprocessor 4/7] RUN npm ci 0.0s
=> CACHED [test-preprocessor 5/7] COPY /preprocessors/hello-preprocessor/ . 0.0s
=> CACHED [test-preprocessor 6/7] COPY /schemas src/schemas 0.0s
=> CACHED [test-preprocessor 7/7] RUN npm run build 0.0s
=> [test-preprocessor] exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:fa33db62d9d3ac5f3018ecc2a92dd4dfe438a41146130b480b20293f578ecb59 0.0s
=> => naming to docker.io/library/test-preprocessor:test 0.0s
=> [test-preprocessor] resolving provenance for metadata file 0.0s
[+] Running 1/1
✔ Container image-test-preprocessor-1 Started 0.2s
juliette@unicorn:/var/docker/image$ Now the test-preprocessor image is running as a container called image-test_preprocessor-1 on unicorn with an internal IP address. We can determine this IP address using the docker inspect command:
juliette@unicorn:/var/docker/image$ docker inspect image-test-preprocessor-1 | grep IPAddress
"SecondaryIPAddresses": null,vim
"IPAddress": "",
"IPAddress": "192.168.240.25",
juliette@unicorn:/var/docker/image$Now we can see what the preprocessor container outputs directly. We'll use helper scripts for IMAGE to do this located in /var/docker/image/bin. The make_request script takes a graphic file and generates an appropriate request. The sendimagereq script sends a request to a URL. Note that our test-preprocessor container runs by default on port 8080.
juliette@unicorn:/var/docker/image$ pwd
/var/docker/image
juliette@unicorn:/var/docker/image$ /var/docker/image/bin/make_request /home/juliette/London_Street_1_920_690_80.jpg | /var/docker/image/bin/sendimagereq - http://192.168.240.25:8080/preprocessor
{"request_uuid":"f24326b3-6250-4666-91aa-903576e8b52f","timestamp":1663176228,"name":"ca.mcgill.a11y.image.hello.preprocessor","data":{"message":"Hello, World!"}}
unicorn:/var/docker/image$ /var/docker/image/bin/make_request /home/juliette/London_Street_1_920_690_80.jpg | /var/docker/image/bin/sendimagereq - http://192.168.240.25:8080/preprocessor | jq
{
"request_uuid": "26a49bc7-8c3e-41dd-ad25-36bbd41eb854",
"timestamp": 1663176454,
"name": "ca.mcgill.a11y.image.hello.preprocessor",
"data": {
"message": "Hello, World!"
}
}
Here we have the raw output from the preprocessor, including a request_uuid, a timestamp, a name indicating that this is the "hello preprocessor" (you would change this for anything that would get into production!), and a data field. The data field contains the outputs of the preprocessor, in this case a message with hello world. In the above example, we use the jq utility to pretty print the output. This is a very powerful tool for manipulating JSON files, but you likely would need to install it manually on your own computer.
In practice, preprocessors aren't called like this, they're made available to the orchestrator using different docker labels. Then the preprocessor will be called along with all other preprocessors when a request is received. First we modify the docker-compose.override.yml file to add these labels. Note that if you're modifying an existing preprocessor, these labels likely exist in the docker-compose.yml file and do not need to be copied.
services:
test-preprocessor:
image: "test-preprocessor:test"
profiles: [test, default]
build:
context: /home/juliette/Documents/IMAGE-server
dockerfile: /home/juliette/Documents/IMAGE-server/preprocessors/hello-preprocessor/Dockerfile
env_file:
- ./IMAGE-server/config/express-common.env
labels:
ca.mcgill.a11y.image.preprocessor: 1
ca.mcgill.a11y.image.port: 8080For more information about these labels, see the section on docker compose configuration for handlers, preprocessors, and services.
Then we can send the same request we sent to our test-preprocessor to the orchestrator. To do this, we need to get its IP address, send the request, and filter the output using jq to only see the section related to the test-preprocessor. Note that the orchestrator runs on port 8080.
juliette@unicorn:/var/docker/image$ pwd
/var/docker/image
juliette@unicorn:/var/docker/image$ docker inspect image-orchestrator-1 | grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "192.168.240.14",
juliette@unicorn:/var/docker/image$ docker compose up -d test-preprocessor # apply the new labels
Recreating image-test-preprocessor-1 ... done
juliette@unicorn:/var/docker/image$ /var/docker/image/bin/make_request /home/juliette/London_Street_1_920_690_80.jpg | /var/docker/image/bin/sendimagereq - http://192.168.240.14:8080/render/preprocess | jq '.preprocessors."ca.mcgill.a11y.image.hello.preprocessor"' -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 851k 100 686k 100 164k 250k 61238 0:00:02 0:00:02 --:--:-- 309k
{
"message": "Hello, World!"
}
juliette@unicorn:/var/docker/image$ The content of data is now available in the output of the preprocessors-only call to the orchestrator (/render/preprocess) under the keys preprocessors and then ca.mcgill.a11y.image.hello.preprocessor, the name that was shown earlier. Each preprocessor being run at once should have a unique name to avoid conflicts.
But why use this docker-compose.override.yml? The /var/docker/image/docker-compose.yml file sets up our test server at https://unicorn.cim.mcgill.ca/image. This means that everything above can actually be condensed down to the following:
juliette@unicorn:/var/docker/image$ pwd
/var/docker/image
juliette@unicorn:/var/docker/image$ /var/docker/image/bin/make_request /home/juliette/London_Street_1_920_690_80.jpg | /var/docker/image/bin/sendimagereq - https://unicorn.cim.mcgill.ca/image/render/preprocess | jq '.preprocessors."ca.mcgill.a11y.image.hello.preprocessor"' -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 851k 100 686k 100 164k 243k 59589 0:00:02 0:00:02 --:--:-- 301k
{
"message": "Hello, World!"
}
juliette@unicorn:/var/docker/image$ The test-preprocessor is now running in a public way! It also would have access to all the other preprocessors running on unicorn at the same time and would be sent on to the handlers if a request was sent to /render rather than /render/preprocess. However, it is still running the original code. We can modify its output by tweaking its source files back in the repository. For this example, we edit preprocessors/hello-preprocessor/src/server.ts and change these lines:
app.post("/preprocessor", (req, res) => {
if (ajv.validate("https://image.a11y.mcgill.ca/request.schema.json", req.body)) {
console.debug("Request validated");
const response = {
"request_uuid": req.body.request_uuid,
"timestamp": Math.round(Date.now() / 1000),
"name": "ca.mcgill.a11y.image.hello.preprocessor",
"data": {
"message": "Hello, World!"
}
};to these:
app.post("/preprocessor", (req, res) => {
if (ajv.validate("https://image.a11y.mcgill.ca/request.schema.json", req.body)) {
console.debug("Request validated");
const response = {
"request_uuid": req.body.request_uuid,
"timestamp": Math.round(Date.now() / 1000),
"name": "ca.mcgill.a11y.image.hello.preprocessor",
"data": {
"message": "My new example message!"
}
};Note that we are changing the data that is reported by the preprocessor. This will ultimately be available under the key ca.mcgill.a11y.image.hello.preprocessor in the output available at /render/preprocess or to handlers. To see our changes reflected we need to rebuild the image and bring up our container again with the new image. Back in the location with our docker-compose.override.yml file:
juliette@unicorn:/var/docker/image$ pwd
/var/docker/image
juliette@unicorn:/var/docker/image$ docker compose up --build -d test-preprocessor
[+] Building 2.3s (13/13) FINISHED docker:default
=> [test-preprocessor internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 390B 0.0s
=> [test-preprocessor internal] load metadata for docker.io/library/node:alpine 0.2s
=> [test-preprocessor internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [test-preprocessor 1/7] FROM docker.io/library/node:alpine@sha256:ed9736a13b88ba55cbc08c75c9e 0.0s
=> [test-preprocessor internal] load build context 0.0s
=> => transferring context: 6.08kB 0.0s
=> CACHED [test-preprocessor 2/7] WORKDIR /usr/src/app 0.0s
=> CACHED [test-preprocessor 3/7] COPY /preprocessors/hello-preprocessor/package*.json ./ 0.0s
=> CACHED [test-preprocessor 4/7] RUN npm ci 0.0s
=> [test-preprocessor 5/7] COPY /preprocessors/hello-preprocessor/ . 0.0s
=> [test-preprocessor 6/7] COPY /schemas src/schemas 0.0s
=> [test-preprocessor 7/7] RUN npm run build 2.0s
=> [test-preprocessor] exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:e3d63f64c522aa593eb69c00cce6f0abd646e45600cec95567c3720a92c0c04b 0.0s
=> => naming to docker.io/library/test-preprocessor:test 0.0s
=> [test-preprocessor] resolving provenance for metadata file 0.0s
WARN[0002] a network with name image exists but was not created for project "image".
Set `external: true` to use an existing network
[+] Running 1/1
✔ Container image-test-preprocessor-1 Started 10.3s
juliette@unicorn:/var/docker/image$ Rerunning the same commands as above will now show the updated message in the preprocessor. Outside of this example, you would make more significant modifications (or create an entirely new preprocessor). The commands you've already run would let you see the direct outputs of the preprocessor, and using the extension would allow you to see how the preprocessor's data is used in the handlers (if any are set up to use that data).
Build your image by running docker compose build $SERVICE_NAME. In this example, docker compose build test-preprocessor. Note that docker compose build by default builds all components listed in docker-compose.yml.
If you're running locally, you won't have all of the unstable modules already running, as they are on unicorn, so you will need to also build any other images you need in order to test, such as the orchestrator.
Once the build is complete, run your container and any containers you need by calling docker compose up -d $LIST_OF_CONTAINERS. You could run the containers with docker run --rm (and must include --rm), but docker-compose will also set up any additional infrastructure needed for IMAGE to work.
A list of container names will be output when started (e.g., image_orchestrator_1).
Determine the IP address of any container you need. On Linux, these addresses can be reached by the host and found using `docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $CONTAINER_NAME
Note for macOS users: On macOS, these addresses are NOT reachable when you are running containers locally (i.e., not on unicorn) and any containers you need to access will need to have its port bound to the host. So to bind port 8080 of the orchestrator to port 8080 of the host, the section of docker-compose.yml will look as follows:
orchestrator:
build:
context: ./orchestrator
dockerfile: Dockerfile
image: "orchestrator:latest"
env_file:
- ./config/express-common.env
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 8080:8080This is only for testing on macOS - these lines for adding port binding should not be committed to the repository! This step is not necessary when you are connected to unicorn, which is a Linux system!
Test if the docker container runs successfully after macOS port binding by either using postman or curl to the endpoint above, e.g., curl -H "Content-Type: application/json" [email protected] http://localhost:3001/preprocessor
To see what is going on, get all the logs from the docker containers with docker-compose logs -f
For a handler, since they run in parallel without dependencies between each-other, you can inject preprocessor output captured via the extension directly, and then check the output. For example, if you already have the preprocessor output in a file called preprocess.json and we want the outputs from a running container called image-photo-audio-handler, you could run:
juliette@unicorn:~$ docker inspect image-photo-audio-handler | grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "192.168.240.12",
juliette@unicorn:~$ /var/docker/image/bin/sendimagereq preprocess.json http://192.168.240.12/handler | jq '.renderings | length' -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1837k 100 1150k 100 686k 672k 401k 0:00:01 0:00:01 --:--:-- 1073k
2
juliette@unicorn:~$ /var/docker/image/bin/sendimagereq preprocess.json http://192.168.240.12/handler | jq '.renderings | map(.type_id)' -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1826k 100 1140k 100 686k 665k 400k 0:00:01 0:00:01 --:--:-- 1065k
[
"ca.mcgill.a11y.image.renderer.Text",
"ca.mcgill.a11y.image.renderer.SegmentAudio"
]
juliette@unicorn:~$ This shows that there are two renderings being returned in this example: one that is plain text, one that is segmented audio. The full output is not shown here, however if you wanted to extract the MP3 file from the second SegmentAudio rendering, you could run the following:
juliette@unicorn:~$ /var/docker/image/bin/sendimagereq preprocess.json http://192.168.240.12/handler | jq -r '.renderings[1].data.audioFile' - | awk -F, '{ print $2 }' | base64 -d - > output.mp3
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1844k 100 1157k 100 686k 672k 398k 0:00:01 0:00:01 --:--:-- 1071k
juliette@unicorn:~$ file output.mp3
output.mp3: MPEG ADTS, layer III, v1, 128 kbps, 48 kHz, JntStereo The command in jq moves into the second rendering and outputs the audioFile encoding as a data URL. The next awk command splits this at the , so only the base64 encoded part is left, not the prefix indicating the file type and encoding. The base64 command then decodes base64 to binary, and that binary file is written to output.mp3. The file command then gives us information on output.mp3, showing that it is in fact a valid MP3 file. By using variations of these commands, it's possible to test handlers in isolation and collect outputs from the command line.
When you are done testing, you need to clean up and restore unicorn to running a clean unstable. If you want to save your override, copy it into your homedir, then in /var/docker/image do a ./restoreunstable. This deletes any override files, rebuilds everything related to IMAGE, and clears out any extra running containers. You should now have a clean running server reflecting what is checked into main.
Note that if you want to clean everything up but still apply your override, you can instead use ./restoreunstable -k which will clean everything up and restore what is in main, but then also keep and apply any overrides.
-
Every morning around 5h00, unicorn is reset to be running the IMAGE system tagged
unstable. This reflects what is checked into main. Any overrides will be deleted! -
Please do not copy the entire docker-compose.yml file, name it
docker-compose.override.yml, then make your changes within it. This has already caused issues, since if you use it again in the future, it will override pretty much everything in the base docker-compose.yml file. This will be impossible to debug and cause random behavior if the underlying docker-compose.yml is updated, since your overrides will create a huge mess. Don't do this! -
If there is already a
docker-compose.override.ymlthere when you need to test, simply edit it and add your changes, then remove them when you are done. Use./restoreunstable -kto clean up when you're done, since you don't want to delete someone else's override file. -
It is considered polite to message in the IMAGE slack
#testingchannel when you're going to be making changes on unicorn, and again when you're done, so that others who are testing know that things may be changing/broken while you're testing. -
To test against with the browser extension, right click the IMAGE extension icon in yoru browser, and go to
options. Make sure the server URL ishttps://unicorn.cim.mcgill.ca/image/ -
Make sure you use the same internal network, or the orchestrator won't pick up your container.
-
Do not run two preprocessors that produce output under the same JSON tags, since this will create a mess.
-
ADVANCED: You can also bring up your own entire stack of containers for testing, running in parallel with the unstable set. This may require making a new path in traefik if you want to run them at a different link (and independently of) the existing
unstableset of containers. Contact Jaydeep or Juliette for help with this, but we'd prefer to avoid doing this unless really necessary... -
docker ps -awill show you all running containers, including their container IDs.glanceswill monitor running containers, and is good to keep active to watch container status. -
There are a number of useful tools in the
/var/docker/image/bin/folder on unicorn, includingmake_request, which creates a request for a graphic file that you can send for testing, and alsogpu_memory_dockerthat shows you how much GPU memory each docker container is using.
Putting features and fixes into production on pegasus is done by Jaydeep or Juliette. Eventually, we plan to release a new version of the extension and the latest server-side components at the end of each sprint, but for now it is more ad-hoc. For the browser extension, this is a manual process done by Jaydeep and Cyan. The process for deploying server-side components into production:
- By default, each component in git is marked
unstableby default when it is merged. - When a component is ready for deployment (including testing!) it should be tagged
latestby the person who owns that component - Juliette or Jaydeep runs
docker compose pull && docker compose up -don pegasus, which fetches and runs all of the components markedlatest, and thelatestversions are live!
Note: If you make a change that will break compatibility with the extension, you need to flag this to Juliette or Jaydeep so it can be managed. If you make another breaking change (e.g., modifying a preprocessor such that it will break other preprocessors or handlers), do not mark it latest until the other components have been updated and everything is tested together.
Since the preprocessors, handlers, and services all work together, when unexpected behavior arises, it can be very difficult to quickly zero in on the root problem. To help with this we require every component to implement solid error checking, as well as message logging. When thinking about this, consider that your code will likely be deployed into production at some point. If something goes wrong, all you will get is the log file, with the information you output. If the system or a GPU runs out of memory, will you know what happened? If the graphic resizing library doesn't handle the type of graphic the user submits, will you know from just looking at the logs? If your component can't return a valid result, will you understand why? If your code hangs, where was it when it stopped? Remember that you may ONLY have the log file, and nothing else, when trying to figure out what happened! Some guidelines (HT Juliette for some of these from the onboarding presentation):
-
Know your log levels and use them
-
When your code moves from one section to another, log that!
-
If anything fails schema validation, log that!
-
If something occurs that means your expected output isn’t present, log that!
-
Make your log messages concise and identifiable
-
Be prepared for exceptions, especially with network requests & GPU tasks. Make sure to try/catch any code that makes significant library calls (e.g., graphic resizing), uses specific devices (e.g., GPUs) or that might take significant time to run (e.g., inferencing, loading a model). Before raising a PR, think through common failure points, and verify your code does the right thing in these cases.
-
Code runs on changing, shared environments over long periods. Assumptions will not hold.
-
Write code that fails gracefully, not code that runs perfectly
-
Preprocessors: logging output should clearly show the major events that occurred. Did it look at previous preprocessor output to decide whether or not to proceeed? Did it change the graphic in some way, like resizing? Did it get an unexpected result back from a library, e.g., when resizing the graphic? If it categorizes the graphic, is it obvious what categories it chose? These things should be obvious from the log!
-
Handlers: How does the hander decide whether it can produce a valid rendering? Are all of the services used returning expected results? Is there data it was expecting, but can't find? What parts of the handler take the most time to run?
IMPORTANT NOTE: For privacy reasons, do NOT log personal data from the query, including the URL of the graphic, or any other information specific to the content itself!
ISSUE: What if I need >12GB of GPU memory, given unicorn's biggest card is only 12GB?
ANSWER: If you really need more than 12GB, that is likely to be problematic in production even though pegasus has a 24GB card. Nonetheless, if you can run a smaller version of the model, or move it to run on CPU (even though slow), you can likely still test on unicorn. If it is simply too big, contact Juliette or Jaydeep about testing (carefully) on pegasus, or other alternatives. If you're building a handler and don't need modifications to the current production preprocessors, get the preprocessor data from production using the Developer mode options available in extension settings, and test with that.
Thanks to Siddharth and Rohan for creating a draft set of instructions, which this incorporates.