The current state of the codebase is not ideal: the "venv" package just feels out of place, there are "ONI jobs" that aren't the same as other jobs, despite the fact that loading titles and applying batch patches are both jobs that run ONI commands, every job has a lot of the same boilerplate code where it seems like things could be shared, ... stuff like that. I found testing a bit funky, especially when wanting to do more end-to-end tests without relying on manually standing up the whole NCA + ONI + Agent setup.
I'd like to see the codebase refactored so that the ONI environment is encapsulated in a single object or package, and any tasks that do anything to ONI would be started there, not be in this weird "sort of independent and sort of reliant on other processes" place they are today. Any jobs where ONI will be called should rely very heavily on this package/struct/whatever.
The job runners should largely go away. It's awkward that a job runner can be both simple and low-level (telling ONI to run purge_batch against a given named batch), and then even more low-level and complicated as the title loader which is responsible for generating a file from the input XML. And all jobs that execute commands are currently manually connecting stdout / stderr to streams despite the fact that in many cases that gives a distorted picture of the process. Having raw log data is probably still useful, but it needs more structure. Which sub-process logged which thing? In a batch patch, there are quite a few distinct operations occurring, and debugging gets painful. We use slog for the batch-patch reading / processing, then a combined stdout and stderr for the purge_batch and load_batch operations.
I'd like jobs to have a simple request to the new object. "Load a new title with this XML" or "I want to apply this batch-patch to X and create a new batch Y". The new "thing" (package, struct, whatever) should handle all the details, including temporary files we might need, structuring logs nicely, cleaning up after the process is done, etc.
The commands might need an "initialize" separately from a "run" function so we can validate the job before queueing it up, which is important for giving immediate feedback when there are obvious errors.
The current state of the codebase is not ideal: the "venv" package just feels out of place, there are "ONI jobs" that aren't the same as other jobs, despite the fact that loading titles and applying batch patches are both jobs that run ONI commands, every job has a lot of the same boilerplate code where it seems like things could be shared, ... stuff like that. I found testing a bit funky, especially when wanting to do more end-to-end tests without relying on manually standing up the whole NCA + ONI + Agent setup.
I'd like to see the codebase refactored so that the ONI environment is encapsulated in a single object or package, and any tasks that do anything to ONI would be started there, not be in this weird "sort of independent and sort of reliant on other processes" place they are today. Any jobs where ONI will be called should rely very heavily on this package/struct/whatever.
The job runners should largely go away. It's awkward that a job runner can be both simple and low-level (telling ONI to run
purge_batchagainst a given named batch), and then even more low-level and complicated as the title loader which is responsible for generating a file from the input XML. And all jobs that execute commands are currently manually connecting stdout / stderr to streams despite the fact that in many cases that gives a distorted picture of the process. Having raw log data is probably still useful, but it needs more structure. Which sub-process logged which thing? In a batch patch, there are quite a few distinct operations occurring, and debugging gets painful. We useslogfor the batch-patch reading / processing, then a combined stdout and stderr for thepurge_batchandload_batchoperations.I'd like jobs to have a simple request to the new object. "Load a new title with this XML" or "I want to apply this batch-patch to X and create a new batch Y". The new "thing" (package, struct, whatever) should handle all the details, including temporary files we might need, structuring logs nicely, cleaning up after the process is done, etc.
The commands might need an "initialize" separately from a "run" function so we can validate the job before queueing it up, which is important for giving immediate feedback when there are obvious errors.