SystemVM template upgrade fails on security hardened management servers

[rajujith]

ISSUE TYPE

• Improvement Request

COMPONENT NAME

Upgrade, systemVM template

CLOUDSTACK VERSION

4.19.1.3

CONFIGURATION

Upgrade from 4.18.2.3 to 4.19.1.3

OS / ENVIRONMENT

EL

SUMMARY

On management servers with security hardening implemented for example umask set to 0027 the systemVM template upgrade fails. Script 'setup-sysvm-tmplt' creates directory with 'sudo' but attempts to find the template without sudo. This fails in the secured operating systems.

[cloud@mgmt1 ~]$ umask 0027
[cloud@mgmt1 ~]$ sudo mkdir -p /tmp/tmp10896605671384965986/template/tmpl/1/8
[cloud@mgmt1 ~]$ find /tmp/tmp10896605671384965986/template/tmpl/1/8 -name '*.ova'
find: ‘/tmp/tmp10896605671384965986/template/tmpl/1/8’: Permission denied

STEPS TO REPRODUCE

1. Set umask to 0027 
2. Upgrade cloudstack from 4.18.2.3 to 4.19.1.3

EXPECTED RESULTS

SystemVM templates are upgraded.

ACTUAL RESULTS

SystemVM templates are not upgraded.

[DaanHoogland]

@rajujith @Pearl1594 , should this be documented as an upgrade requirement (set the umask to 0022 at most), or can we search with sudo as well?

[Pearl1594]

I think it would be nice if we could try and see how we can adapt the template registration script to run on a hardened system. And if that is not straightforward, we can document a workaround that the template needs to be pre-registered before upgrade to prevent the script from running.

[sureshanaparti]

During system vm template registration, currently mount and unmount store is done with sudo.

cloudstack/engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java

Line 150 in df99a29

String cmd = "sudo mount -t nfs";

cloudstack/engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java

Line 89 in df99a29

private static final String UMOUNT_COMMAND = "sudo umount %s";

Without sudo, mount fails with error error: [mount: only root can use "--types" option] (to fix this, appropriate permissions to be set for the cloud user). so, in security hardened system (where umask set to 0027), when a dir created with sudo in setup-sysvm-tmplt script, other cmds (find, ls , df -P, qemu-img) on it without sudo fails.

[cloud@mgmt1 ~]$ umask
0027
[cloud@mgmt1 ~]$ sudo mkdir -p /tmp/test/template/tmpl/1/8
[cloud@mgmt1 ~]$ ls -lrt /tmp/test/template/tmpl/1/8
ls: cannot access '/tmp/test/template/tmpl/1/8': Permission denied
[cloud@mgmt1 ~]$ find /tmp/test/template/tmpl/1/8/ -name '*.ova'
find: '/tmp/test/template/tmpl/1/8/': Permission denied
[cloud@mgmt1 ~]$ df -P /tmp/test/template/tmpl/1/8/
df: /tmp/test/template/tmpl/1/8/: Permission denied
[cloud@mgmt1 ~]$ sudo touch /tmp/test/template/tmpl/1/8/test-template.ova
[cloud@mgmt1 ~]$ qemu-img info /tmp/test/template/tmpl/1/8/test-template.ova
qemu-img: Could not open '/tmp/test/template/tmpl/1/8/test-template.ova': Could not open '/tmp/test/template/tmpl/1/8/test-template.ova': Permission denied

Either these cmds have to be run with sudo in the script, or add the cloudstack service user ('cloud') to sudoers list. What is best fix here, any other idea/thoughts @rajujith @andrijapanicsb @NuxRo @DaanHoogland @rohityadavcloud & others.

[weizhouapache]

Either these cmds have to be run with sudo in the script, or add the cloudstack service user ('cloud') to sudoers list.

IMHO both are needed