Merge pull request #57 from venediktov/docker20170722

vanilla-docker initial release

README.md

@@ -28,14 +28,15 @@ Structure :
    * [boost-dll/](../../tree/master/boost-dll/) -- C++11 in official boost release since version 1.61
    * [CRUD/](../../tree/master/CRUD/) -- C++11 high performance HTTP-restful handlers based on boost.ASIO and CRUD API
    * [jsonv/](../../tree/master/jsonv/) -- DSL mapper of json encoded objects to C++ structures
-   * [parsers/](../../tree/master/jsonv/) -- fast zero copy, zero memory allocation parsers
+   * [parsers/](../../tree/master/parsers/) -- fast zero copy, zero memory allocation parsers
    * [rtb/](../../tree/master/rtb/) -- RTB framework
       * [core/](../../tree/master/rtb/core/) -- generic structures shared in the project ( RTB specific )
       * [common/](../../tree/master/rtb/common) -- generic RTB agnostic structures 
       * [datacache/](../../tree/master/rtb/datacache/) -- IPC data store for fast targeting lookups
       * [exchange/](../../tree/master/rtb/exchange) -- exchange handlers implementation
       * [DSL/](../../tree/master/rtb/DSL) --  DSL formats for jsonv
-    * [examples/](../../tree/master/examples) -- root to our sandbox with examples
+   * [docker/](../../tree/master/docker/) -- vanilla docker files and instructions on how to build and run 
+   * [examples/](../../tree/master/examples) -- root to our sandbox with examples
       * [bidder/](../../tree/master/examples/bidder) -- collection of application specific classes to support targeting
       * [loader/](../../tree/master/examples/loader) -- collection of application specific classes to support campaign loading
       * [campaign/](../../tree/master/examples/campaign) -- add/modify/delete campaign API + UI ( work in progress ) 
@@ -57,16 +58,16 @@ Structure :
 ### Linux \:
 
 ```bash
-$vanilla-rtb> mkdir Release
-$vanilla-rtb> cd Release
-$vanilla-rtb> cmake -DCMAKE_BUILD_TYPE=Release .. -G "Unix Makefiles"
-$vanilla-rtb> gmake -j4 install
+$ mkdir Release
+$ cd Release
+$ cmake -DCMAKE_BUILD_TYPE=Release .. -G "Unix Makefiles"
+$ make -j4 install
 # creating  Debug build
-$vanilla-rtb> cd ..
-$vanilla-rtb> mkdir Debug
-$vanilla-rtb> cd Debug
-$vanilla-rtb> cmake -DCMAKE_BUILD_TYPE=Debug .. -G "Unix Makefiles"
-$vanilla-rtb> gmake -j4 install
+$ cd ..
+$ mkdir Debug
+$ cd Debug
+$ cmake -DCMAKE_BUILD_TYPE=Debug .. -G "Unix Makefiles"
+$ make -j4 install
 ```
 
 ### Windows \:
@@ -90,15 +91,15 @@ To generate an `Xcode` project invoke cmake from an empty build directory with c
 
 ### Mac OS X ( XCode command line tools)
 ```bash
-xcode-select --install
-ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
-brew doctor
-brew install cmake
-brew install boost
-mkdir Release
-cd Release
-cmake -DCMAKE_BUILD_TYPE=Release .. -G "Unix Makefiles"
-make -j4 install
+$ xcode-select --install
+$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+$ brew doctor
+$ brew install cmake
+$ brew install boost
+$ mkdir Release
+$ cd Release
+$ cmake -DCMAKE_BUILD_TYPE=Release .. -G "Unix Makefiles"
+$ make -j4 install
 ```
 
 ### For faster builds invoking multiple make processes  , find number of cores on your system
@@ -109,7 +110,7 @@ Linux command \:
 
 pass it to your make script like this
 
-**gmake -j4 install**
+**make -j4 install**
 
 ### Running examples\:
 - [x] HTTP-Bidder

docker/README.md

@@ -0,0 +1,101 @@
+# VanillaRTB Docker Images
+
+## Structure
+VanillaRTB Docker images have two layers (optionally three) of hierarchy:
+
+* #0.0 VanillaRTB Base - VanillaRTB unified runtime environment
+* #1.1 VanillaRTB Dev (extends #0.0) - VanillaRTB development environment with preinstalled G++, CMake, Boost (selected libraries)
+* #1.2 VanillaRTB Bld (extends #0.0) - VanillaRTB package builder image (!TODO!)
+* #1.3 VanillaRTB Prod (extends #0.0) - VanillaRTB pre-built ready to run package
+* #2.x (extends #1.3) - Specialized container preconfigured to run particular VanillaRTB component or subsystem
+
+## Creating and Updating 
+###( instructions for vanilla-rtb stack contributors only !!!! )
+
+Base package:
+
+```bash
+$ vanilla_env_ver=x.y.z
+$ cd vanilla-rtb/docker
+$ docker build --tag vanillartb/vanilla-base:${vanilla_env_ver} --file vanilla-base.Dockerfile ${PWD}
+$ docker login
+$ docker push vanillartb/vanilla-base:${vanilla_env_ver}
+```
+
+Development environment package : built automatically on hub.docker.com everytime we tag our git push 
+
+```bash
+$ vanilla_env_ver=x.y.z
+$ cd vanilla-rtb/docker
+$ docker build --tag vanillartb/vanilla-dev:${vanilla_env_ver} --file vanilla-dev.Dockerfile ${PWD}
+$ docker tag vanillartb/vanilla-dev:${vanilla_env_ver} vanillartb/vanilla-dev:latest
+$
+$ docker login
+$ docker push vanillartb/vanilla-dev:${vanilla_env_ver} vanillartb/vanilla-dev:latest
+```
+
+Production package :
+
+```bash
+$ vanilla_env_ver=a.b.c
+$ vanilla_ver=x.y.z
+$ cd vanilla-rtb/docker
+$
+$ docker container run -a STDOUT --name vanilla-build-box vanillartb/vanilla-dev:${vanilla_env_ver}
+$ docker cp build-box:/root/pkg/vanilla-rtb/snapshot ./vanilla
+$
+$ docker build --tag vanillartb/vanilla-prod:${vanilla_ver} --file vanilla-prod.Dockerfile ${PWD}
+$ /bin/rm -rf ./vanilla
+$ docker tag vanillartb/vanilla-prod:${vanilla_ver} vanillartb/vanilla-prod:latest
+$
+$ docker login
+$ docker push vanillartb/vanilla-prod:latest
+```
+
+# Running 
+
+```bash
+$ docker run --net=host -it --name vanilla-devbox vanillartb/vanilla-dev:0.0.1
+$ docker run --net=host -it --name vanilla-prodbox vanillartb/vanilla-prod:latest
+```
+# Running in prod with Redis
+```bash
+$ docker run --net=host --name vanilla-redis -d redis
+$ docker run --net=host -it --name vanilla-prodbox vanillartb/vanilla-prod:latest
+```
+
+# Docker maintanance commands 
+```bash
+## see images installed 
+sudo docker image ls
+## remove stored image by id
+sudo docker image rm vanillartb/vanilla-prod
+sudo docker image rm redis
+## remove container to restart with different params
+sudo docker rm vanilla-prodbox
+## stop running container 
+sudo docker stop vanilla-prodbox
+## start known container 
+sudo docker start vanilla-prodbox
+## attach shell to running container
+sudo docker attach vanilla-prodbox
+```
+# Docker performance issues 
+running docker with manual port forwarding command --publish e.g -p8080:9081  has 20-30% performace degradation due 
+to this option utilizing  bridge driver, unless it has to be mapped using -p8080:9081 **avoid for performance issues**! 
+The most performant network solution is **--net=host** ( exposes all container ports to host ) however **vanilla-rtb**  ports in **examples/config.cfg** file may not be open on the cloud/host server therefore you have to think of shipping config with docker container per installation or starting stack with custom command line arguments ( --bidder.port=8080  ) or starting docker with 
+shared host/container file system ( please refer to docker volume -v flag )
+
+```
+$ sudo docker network ls
+NETWORK ID          NAME                DRIVER              SCOPE
+db0004be8eff        bridge              bridge              local
+42cee521962a        host                host                local
+23f65108302b        none                null                local
+```
+#### Disclaimer : on linux most likely all of above docker commads need to preceed with sudo 
+
+## References
+[Debian Releases](https://www.debian.org/releases/)<br>
+[Debian Docker Images](https://store.docker.com/images/debian/)<br>
+[Dockerfile Reference](https://docs.docker.com/engine/reference/builder/)

docker/build-vanilla.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+set -eu
+
+[[ ! -d vanilla-rtb ]] && git clone --recursive https://github.com/venediktov/vanilla-rtb.git
+
+[[ ! -d ../build/vanilla-rtb ]] && mkdir -p ../build/vanilla-rtb
+[[ ! -d ../pkg/vanilla-rtb/snapshot ]] && mkdir -p ../pkg/vanilla-rtb/snapshot
+cd ../build/vanilla-rtb
+
+cmake -G 'Unix Makefiles'\
+    -DCMAKE_BUILD_TYPE=Release \
+    -DCMAKE_INSTALL_PREFIX="$(readlink -f ../../pkg/vanilla-rtb/snapshot)" \
+    "$(readlink -f ../../code/vanilla-rtb)"
+
+make -j$(nproc) -l$(nproc) install
+

docker/vanilla-base.Dockerfile

@@ -0,0 +1,16 @@
+FROM debian:stretch
+LABEL Description="VanillaRTB Base" Vendor="ForkBid" Maintainer="mrbald@github"
+RUN apt-get update && apt-get install -y apt-utils
+RUN apt-get install -y\
+ libboost-atomic1.62.0\
+ libboost-chrono1.62.0\
+ libboost-date-time1.62.0\
+ libboost-filesystem1.62.0\
+ libboost-log1.62.0\
+ libboost-program-options1.62.0\
+ libboost-regex1.62.0\
+ libboost-serialization1.62.0\
+ libboost-system1.62.0\
+ libboost-thread1.62.0
+
+CMD ["bash"]

docker/vanilla-dev.Dockerfile

@@ -0,0 +1,33 @@
+FROM vanillartb/vanilla-base:0.0.1
+LABEL Description="vanilla-rtb Dev" Vendor="ForkBid" Maintainer="mrbald@github"
+RUN apt-get update
+RUN apt-get install -y\
+ cmake\
+ git\
+ 'g++'\
+ vim\
+ libboost-atomic1.62-dev\
+ libboost-chrono1.62-dev\
+ libboost-date-time1.62-dev\
+ libboost-filesystem1.62-dev\
+ libboost-log1.62-dev\
+ libboost-program-options1.62-dev\
+ libboost-regex1.62-dev\
+ libboost-serialization1.62-dev\
+ libboost-system1.62-dev\
+ libboost-thread1.62-dev\
+ libboost-test1.62.0\
+ libboost-test1.62-dev\
+ python
+
+WORKDIR /root/pkg
+WORKDIR /root/build
+WORKDIR /root/code
+
+ADD build-vanilla.sh /root/code
+
+RUN  chmod +x ./build-vanilla.sh && ./build-vanilla.sh
+
+WORKDIR /root/pkg/vanilla-rtb/snapshot/bin
+
+CMD ["bash"]

docker/vanilla-prod.Dockerfile

@@ -0,0 +1,5 @@
+FROM vanillartb/vanilla-base:0.0.1
+LABEL Description="VanillaRTB Prod" Vendor="ForkBid" Maintainer="mrbald@github"
+WORKDIR /root/pkg/vanilla
+ADD vanilla .
+CMD ["bash"]