(1) N. Saurabh, J. Remmers, D. Kimovski, R. Prodan and J. G. Barbosa, "Semantics-Aware Virtual Machine Image Management in IaaS Clouds," 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Rio de Janeiro, Brazil, 2019, pp. 418-427, doi: 10.1109/IPDPS.2019.00052.
(2) Nishant Saurabh, Shajulin Benedict, Jorge G. Barbosa, Radu Prodan, Expelliarmus: Semantic-centric virtual machine image management in IaaS Clouds, Journal of Parallel and Distributed Computing, Volume 146, 2020, Pages 107-121, ISSN 0743-7315,https://doi.org/10.1016/j.jpdc.2020.08.001.
1. Nishant Saurabh (PhD Researcher,University of Innsbruck, Alpen-Adria Universitat Klagenfurt, Austria)
2. Julian Remmers, (MSc., University of Innsbruck, Austria)
3. Contact: nishant(at)itec.aau.at
Semantic management of Virtual machine images (VMIs). The current implementation of Expelliarmus is tested on Ubuntu 16.04 LTS, however there is no such arbitrary requirement and can be deployed on any Linux based environment with manual build for libguestfs.
1. Expelliarmus Requirements
2. Expelliarmus Installation
3. Dataset (VMIs) preparation for Expelliarmus
4. Experimental workflow of Expelliarmus
5. Evaluation steps for Expelliarmus
1. Python (>=2.7)
2. Python module newtorkx
3. libguestfs-tools (>=1.36.x)
4. python-guestfs
1. $ mkdir Expelliarmus
2. $ cd Expelliarmus
3. $ git clone https://github.com/ExpelliarmusSuperComp/Expelliarmus
4. $ cd ../../
5. $ apt install libguestfs-tools
6. $ apt install libguestfs-dev
7. $ apt install python-guestfs
8. $ wget “download.libguestfs.org/1.36-stable/libguestfs-1.36.13.tar.gz”
9. $ tar -xf libguestfs-1.36.13.tar.gz
10. $ cd libguestfs-1.36.13
11. $ apt-get build-dep libguestfs
12. $ apt install autoconf automake libtool-bin gettext
13. $ ./configure
14. $ make
15. $ cd../Expelliarmus/Expelliarmus/
16. $ python main.py
17. $ Please provide path to libguestfs: ../../libguestfs-1.36.13
18. $ Expelliarmus ready to use : Type “help” to get more details.
1. Error "libguestfs: error: tar_in: write error on directory: ..."
$ Echo dash >/user/lib/x86_64-linux-gnu/guestfs/supermin.d/zz-dash-packages
2. Error "suprmin exited with status 1"
$ chmod 0644 /boot/vmlinuz*
Check http://libguestfs.org/guestfs-building.1.html for more details on building libguestfs
Expelliarmus is tested over synthetic VMIs created using virt-builder. Minimal script to create VMIs for local and Cloud use:
```
#!/bin/bash
# path to libuguestfs
../../libguestfs-1.36.13/run virt-builder ubuntu-16.04 \
-o Image.qcow2 \
--size 4G \
--format qcow2 \
--root-password password:123 \
--edit '/etc/default/grub:s/ ^GRUB_CMDLINE_LINUX_DEFAULT= .*/GRUB_CMDLINE_LINUX_DEFAULT= "console=tty0 console=ttyS0,115200n8"/' \
--run-command update-grub \
--install "dpkg-repack,fakeroot,dpkg-dev" \
--update \
--write /etc/default/locale: $"LANG=\"en_US.UTF-8\"" \
--append /etc/default/locale: $"LANGUAGE=\"en_US\"" \
--firstboot-command 'userdel -r builder && useradd -m -p "" builder ; chage -d 0 builder'
```
1. Save the above script as “vmBuilder.sh”.
2. $ chmod +x vmBuilder.sh
3. source vmBuilder.sh
4. Output: Image.qcow2
On guest OS:
1. $ dd if=/dev/zero of=/mytempfile
2. $ rm -f /mytempfile
To shrink the VMI disk file without compression:
1. $ mv Image.qcow2 Image.qcow2_backup
2. $ qemu-img convert -O qcow2 Image.qcow2_backup Image.qcow2
Copy Image.qcow2 /Expelliarmus/Input/
$ python main.py
Please provide path to libguestfs: /libguestfs-1.36.13
(Expelliarmus): Command line interface
Type "help" to see a list of available commands.
Type "help name" to get a summary about the command "name" and how to use it.
Any path given by the user to specify files or folders has to be relative to the working directory of this program.
(Expelliarmus): help
The following Commands are available. Type "help name" to get a summary about the command "name" and how to use it.
Any path given by the user to specify files or folders has to be relative to the working directory of this program:
list-show information about VMI components currently stored
inspect- inspect VMIs and define main services
decompose- decompose VMIs
reassemble- reassemble VMIs
evaluate- tool to evaluate this program
reset- reset local repository of VMI components
exit- exit program
Step 1: Enter primary software packages name
(Expelliarmus) inspect Input/Image.qcow2
Enter Main Services in format "MS1,MS2,..."
tomcat7,python3
Step 2: Decompose a VMI
(Expelliarmus) decompose Input/
Step 3: Reassemble a VMI
(Expelliarmus) reassemble Image.qcow2
(Expelliarmus) help evaluate
Usage: evaluate [options] { decomposition1 | decomposition2 | reassembly | similarity }
Evaluates the given functionality and saves results in folder "Evaluations".
Functionalities:
decomposition1:
Evaluates the decomposition process exploiting semantic redundancy (i.e. using a local repository).
Option "--path" has to be set to specify a source folder for VMIs (These will only be copied, not manipulated).
decomposition2:
Evaluates the decomposition process without exploiting semantic redundancy (i.e. not using a local repository).
Option "--path" has to be set to specify a source folder for VMIs (These will only be copied, not manipulated).
reassembly:
Evaluates the reassembly process using any VMI present in the local repository.
Option "--path" is ignored.
similarity:
Evaluates the similarity between each VMI in source folder.
Option "--path" has to be set to specify a source folder for VMIs (These will not be manipulated).
Options:
--repetitions=x:
Specify number of repetitions for evaluation (default is 5), not applicable for similarity.
--path=x:
Specify source folder with VMIs for evaluation (ignored for evaluation of reassembly).
Meta Files for all VMIs have to exist. Files in this folder are not manipulated.
Examples for Evaluation:
(Expelliarmus) evaluate --repetitions=5 --path=/Input/ decomposition1
(Expelliarmus) evaluate --repetitions=5 --path=/Input/ decomposition2
(Expelliarmus) evaluate --repetitions=5 reassembly
(Expelliarmus) evaluate --path=/Input/ similarity