Processing Information for Hyp3-gamma

[abradley60]

Hello, I am looking to get some information on the processing environment used to produce backscatter data through the python hyp3-sdk API. i.e. the products produce when the hyp3.submit_rtc_job() function is used. Is there a standard AWS EC2 instance that is spun up? or some standard amount of allocated compute for each scene(e.g. 8CPU, 32GB RAM)?

Our team is comparing some different options of producing RTC data, and want to get a handle on the processing needs if we were to apply at a large scale.

Thanks!

[jhkennedy]

Hi @abradley60, the RTC produces are generated witht the hyp3-gamma plugin:
https://github.com/ASFHyP3/hyp3-gamma

HyP3 uses that plugin's Docker container image to run the RTC workflow; here's the dockerfile:
https://github.com/ASFHyP3/hyp3-gamma/blob/develop/Dockerfile

We use an Ubuntu base image and apt-install the dependencies as described in GAMMA's documentation. If you have a GAMMA license and distribution tarball, you can build the image yourself and run our workflows; plugins are fully independent. The GAMMA distribution is added to the container image on this line:
https://github.com/ASFHyP3/hyp3-gamma/blob/develop/Dockerfile#L40

HyP3 runs hyp3-gamma docker containers on r6id and r6idn AWS EC2 instances of various sizes:
https://github.com/ASFHyP3/hyp3/blob/develop/.github/workflows/deploy-daac.yml#L29

RTC GAMMA jobs require 32 GB of memory, so each job will have 4 CPUs available for the r6id[n] instance families. For the larger instance sizes, we may run multiple jobs on the same instance, but they'll still have 32 GB of memory and 4 CPUs available each.