Improve the CLI experience

[MarinPostma]

The CLI arguments start to become messy, as #616 is a prime example. We should address that issue and start designing a better configuration experience.

I would like the CLI options to be as lean as possible, and most of the configuration to be performed thanks to a configuration file.

The configuration is going to change a lot with multi-tenancy, we should keep that in mind for redesign.