[feature] make list of autodeploy expath packages configurable via conf.xml

[duncdrum]

Is your feature request related to a problem? Please describe.
Users can fine tune xquery modules for a give instances via conf.xml yet they cannot do the same for expath packages, which ship by default. Having a central place where both aspects can be processed via xml tooling would make sense.

Describe the solution you'd like
Control the list of .xars within /autodeploy via a dedicated section in conf.xml. Something like:

<autodeploy>
  <package name='http://www.functx.com' enabled='true'/>
  <package name='http://exist-db.org/apps/dashboard' enabled='false'/>
</autodeploy>

Describe alternatives you've considered
Manually removing files from the directory, using -Dautodeploy=http://exist-db.org/apps/dashboard build flags, all are cumbersome. I see no reason why this data shouldn't be on conf.xml

Additional context
Add any other context or screenshots about the feature request here.

[line-o]

First of all @duncdrum , thank you for sharing the command line flag to control which packages make it into autodeploy. That is valuable information.

Regarding your request: I think it is a useful enhancement for users who do not build from source but use any kind of installer.

[joewiz]

@duncdrum Interesting idea. A few questions:

Am I right in imagining that the conf.xml entry would be auto-generated and auto-inserted when eXist is built?

I can see how this would allow users to remove packages that they don't want installed upon first start-up, but I am a little unclear about the behavior in the following scenarios:

• After first startup, user removes an entry from the <autodeploy>. Upon the next startup, should the package be uninstalled?
• After first startup, user adds an entry that lacks a corresponding .xar in the autodeploy directory. Upon the next startup, should eXist try to fetch the package from the package repo?

I ask because in all other respects, settings in conf.xml affect the next startup, so users may expect the <autodeploy> directive to react similarly.

[duncdrum]

A few questions:

Am I right in imagining that the conf.xml entry would be auto-generated and auto-inserted when eXist is built?

Not sure i understand about the auto-, the canonical place for autodeploy packages would be conf.xml. In the core repo we would specifiy the same set of packages as we do now, and thats what clones or forks have as a starting point.

I can see how this would allow users to remove packages that they don't want installed upon first start-up, but I am a little unclear about the behavior in the following scenarios:

• After first startup, user removes an entry from the <autodeploy>. Upon the next startup, should the package be uninstalled?

Firstly the xar should be removed from the autodeploy folder, the package should then be uninstalled via repo-uninstall. I cannot imagine a scenario where a user modifies the conf.xml to remove a package, but wants to keep it installed. it is however possible to remove a package from autodeploy, and install it manually later. just as with all non default packages.

• After first startup, user adds an entry that lacks a corresponding .xar in the autodeploy directory. Upon the next startup, should eXist try to fetch the package from the package repo?

Yes a custom package set in conf.xml should be downloaded into autodeploy/ which means it will be installed on next startup. We sould basically check if all declared packages in conf.xml have corresponding .xar files in the autodeploy folder.

I ask because in all other respects, settings in conf.xml affect the next startup, so users may expect the <autodeploy> directive to react similarly.

That is the idea, yes . The main user scenario for this, however, is modifying an instance before / whenever it is initially booted.

@line-o the maven command is how i woudl imagine an alternative, i have not tested if this particular command exists, or what it does.

[adamretter]

Hmm... I don't really understand the context/use-case for this (yet)...

1. If the database server has not been started yet for the first time, you can just rename/remove .xar files in autodeploy/ before it is started.
2. If it has been started for the first-time, it's too late to change a config setting for this, as the package has already been deployed.
3. If it is for users that run the installer, I don't think it works as eXist-db is started automatically during install to set the admin password and then is restarted so it is running post-install for the user.

---

Users can fine tune xquery modules for a give instances via conf.xml

I actually think that mechanism is quite bad. You can also override it at Runtime if you know the magic tricks! It gives an illusion of safety rather than a control mechanism.

I had envisaged in future having a much better system of control, whereby individual functions (and/or modules of functions) can be associated with roles, this would allow fine-grained control over who can do what, and get us away from the "only a DBA" can run these arbitrary functions. We had something like that once upon a time based around XACML, but it was far to complex at that time.

[duncdrum]

Hmm... I don't really understand the context/use-case for this (yet)...

CI, Containers, Pupper, Ansible, …

1. If the database server has not been started yet for the first time, you can just rename/remove .xar files in autodeploy/ before it is started.

my point, i can't just run xquery to modify the configuration of my instance, i have to disable modules in one, and then do file-system stuff someplace else.

2. If it has been started for the first-time, it's too late to change a config setting for this, as the package has already been deployed.

just as with other modifications of conf.xml this would require a restart, nothing too late about it.

3. If it is for users that run the installer, I don't think it works as eXist-db is started automatically during install to set the admin password and then is restarted so it is running post-install for the user.

see 2., you could however build your own installer…

Users can fine tune xquery modules for a give instances via conf.xml

I actually think that mechanism is quite bad. You can also override it at Runtime if you know the magic tricks! It gives an illusion of safety rather than a control mechanism.

nothing wrong with overriding at run-time as such, as it makes sense to remove a package from autodeploy, but require it to be manually installed. its that way for all but 5 or 6 packages and there is nothing bad about that either.

I had envisaged in future having a much better system of control, whereby individual functions (and/or modules of functions) can be associated with roles, this would allow fine-grained control over who can do what, and get us away from the "only a DBA" can run these arbitrary functions. We had something like that once upon a time based around XACML, but it was far to complex at that time.

i fail to see the connection, i want my instances to e.g. not have dashboard when they are spun up by a replicator or by a load-balancer. Who can make this kind of modification is a separate question, to how that goal can be easily achieved in the first place.

[adamretter]

CI, Containers, Puppet, Ansible, …

I am still not quite getting it I guess... if I am installing from Puppet or Ansible I would always start with a .tar.gz or Docker image which can be or has been customised already.

I am not necessarily against what you are suggesting, I just can't see a use-case for it yet...

my point, i can't just run xquery to modify the configuration of my instance

You would have to use the File Module to modify the conf.xml from XQuery. That would require DBA, same was as you need DBA to use the File Module to remove a file from the filesystem... No?

see 2., you could however build your own installer…

If you were building your own installer, wouldn't you just customise the autodeploy folder at that point instead?

nothing wrong with overriding at run-time as such

I think we are talking about different things... I am saying that if you remove a Module from conf.xml, then if you "know how" you can still cause it to be loaded and executed at run-time; that should not be possible.