Deployment to Lambda readonly docker image

[furyhawk]

When deployed to a readonly docker image, I encountered OSError(30, 'Read-only file system') when instantiating TabularModel. I can't get detail error message from the log. Need some suggestions on which module is creating file.

There was no error when I instantiate SKLearn model in the same env for example.

[manujosephv]

I'm assuming the culprit is ExperimentRunManagerhere.

This ExperimentRunManager creates a temp file on disk to manage experiment versions etc. I checked and it can easily be changed to any other path in code here], but sadly, as of now, it's not exposed in the api.

[manujosephv]

You'll also have ti manage where checkpoints are stored etc. and direct them to a path where you have write access