Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
73be5f3
generators working but incomplete
Jul 17, 2019
1737e2e
all implementations built
Jul 17, 2019
e70f58b
args working for version implementation
Jul 17, 2019
c374491
get request workingggg
Jul 17, 2019
5f063e2
Merge pull request #1 from kryan1622/microservices
kryan1622 Jul 17, 2019
3368e01
working on connector image
Jul 17, 2019
b9e199e
db connect and static website containers up
Jul 18, 2019
c5636ce
Merge pull request #2 from kryan1622/database
kryan1622 Jul 18, 2019
8182541
dockerised server on own container
Jul 18, 2019
0e2ea0f
Merge pull request #3 from kryan1622/docker_notify_nginx
kryan1622 Jul 18, 2019
310c45d
containers running but not communicating with docker-compose
Jul 18, 2019
fdcc616
Merge pull request #4 from kryan1622/docker-compose
kryan1622 Jul 18, 2019
dda3280
database persisting
Jul 18, 2019
36cedf1
endpoints all working
Jul 18, 2019
dc84be5
frontend functions created
Jul 18, 2019
55045bf
Merge pull request #5 from kryan1622/frontend
kryan1622 Jul 18, 2019
b8e64b8
fixed url in javascript
Jul 19, 2019
c99d5ad
begin swarming
Jul 19, 2019
692b9d8
Merge pull request #6 from kryan1622/frontend
kryan1622 Jul 19, 2019
bf94182
added Jenkinsfile
Jul 19, 2019
fdfec2c
Merge pull request #7 from kryan1622/swarm-stack
kryan1622 Jul 19, 2019
c41ed74
added clean stage to Jenkinsfile
Jul 19, 2019
ad9be9d
Merge pull request #8 from kryan1622/swarm-stack
kryan1622 Jul 19, 2019
826f33d
edit Jenkinsfile
Jul 19, 2019
77a3a55
Merge pull request #9 from kryan1622/swarm-stack
kryan1622 Jul 19, 2019
a681cb0
added jenkins mount
Jul 19, 2019
ab2cf9d
Merge pull request #10 from kryan1622/jenkins-container
kryan1622 Jul 19, 2019
546530f
jenkinsfile
Jul 19, 2019
0847c46
Update Jenkinsfile
kryan1622 Jul 19, 2019
4f4fa51
Update Jenkinsfile
kryan1622 Jul 19, 2019
255fa0e
Update docker-compose.yaml
kryan1622 Jul 19, 2019
0881aed
Update Jenkinsfile
kryan1622 Jul 19, 2019
33ac04a
Update docker-compose.yaml
kryan1622 Jul 19, 2019
f2cb908
Update Jenkinsfile
kryan1622 Jul 19, 2019
56d2525
Update Jenkinsfile
kryan1622 Jul 19, 2019
778fc16
Update Jenkinsfile
kryan1622 Jul 19, 2019
3d6d8a7
Update Jenkinsfile
kryan1622 Jul 19, 2019
fd72aa8
Update Jenkinsfile
kryan1622 Jul 19, 2019
5646372
Update Jenkinsfile
kryan1622 Jul 19, 2019
0c310a1
Update Jenkinsfile
kryan1622 Jul 19, 2019
d6eee46
Update Jenkinsfile
kryan1622 Jul 19, 2019
c2373c2
Update Jenkinsfile
kryan1622 Jul 19, 2019
2fc1473
Update Jenkinsfile
kryan1622 Jul 19, 2019
faa6641
Update Jenkinsfile
kryan1622 Jul 19, 2019
3e38e39
edited index to check if pipeline works
Jul 19, 2019
1177f0f
Merge branch 'jenkins-container' of https://github.com/kryan1622/LAFB…
Jul 19, 2019
5a75d98
reverted index - it does work
Jul 19, 2019
25f840c
Merge pull request #11 from kryan1622/jenkins-container
kryan1622 Jul 19, 2019
2d7d330
testing build args
Jul 19, 2019
6ec112c
worked for text_gen so added build args for num and account
Jul 19, 2019
78afbcc
removed args from account to troubleshoot
Jul 19, 2019
d991bec
removed arg from num gen
Jul 19, 2019
2f1ccd0
added args to num gen
Jul 19, 2019
642525c
num edited and build args added again
Jul 19, 2019
8a86fdf
working and reverting num back to version 1
Jul 19, 2019
4e9d6a1
change back
Jul 19, 2019
beb9249
switch back
Jul 19, 2019
602d99f
Merge pull request #12 from kryan1622/implementations
kryan1622 Jul 19, 2019
2502486
readme.md added
hbuck95 Jul 19, 2019
ddfe593
Update docker-compose.yaml
kryan1622 Jul 19, 2019
af4b6a5
Update docker-compose.yaml
kryan1622 Jul 19, 2019
8499279
Update docker-compose.yaml
kryan1622 Jul 21, 2019
cbec65e
readme and installation scripts
ayshamarty Jul 21, 2019
6b4d729
installation scripts
ayshamarty Jul 21, 2019
351e3bd
Update README.md
kryan1622 Jul 21, 2019
304c9ab
Update README.md
kryan1622 Jul 21, 2019
0dcb59b
Update README.md
ayshamarty Jul 21, 2019
9ab49a3
Update README.md
ayshamarty Jul 21, 2019
a793e15
Update README.md
ayshamarty Jul 21, 2019
344f667
Update README.md
ayshamarty Jul 21, 2019
60d219e
Merge branch 'master' into edit-docs
ayshamarty Jul 21, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
pipeline{
agent any
stages{
stage('---build---'){
steps{
sh "docker-compose build"
}
}
stage('---push---'){
steps{
sh "docker-compose push"
}
}
stage('---deploy---'){
steps{
sh "docker stack deploy --compose-file docker-compose.yaml devops"
}
}
}
}


238 changes: 238 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
# Little Anchorage Financial Bank: DevOps Project

In fulfilment of the group DevOps project assignment due Monday week 11 at QA consulting.

## Index
[Brief](#brief)

[Architecture](#architecture)

[Technologies Used](#tech)

[Deployment](#depl)
* [Prerequisites and Installation Guide](#prereq)
* [CI Pipeline](#CI)

[Authors](#auth)

[Acknowledgements](#ack)

<a name="brief"></a>
# The Brief



<a name="architecture"></a>
# Architecture
### Before

### After

<a name="tech"></a>
## Technologies Used

* Mongo - Database
* Node - creating the account generator which includes generating the prize
* Python - creating the number and text generator which generates an account number
* Jenkins - CI Server
* [Git](https://github.com/ayshamarty/SoloProject.git) - VCS
* [Trello](https://trello.com/b/yf6TuPx0/devops-project) - Project Tracking
* Azure - Live Environment
* Docker-Compose - builds the images used to create our containers
* [DockerHub](https://cloud.docker.com/u/keepkarm/repository/list) - registry for storing and updating images used for deployment
* Docker Swarm

<a name="depl"></a>
# Deployment

<a name="prereq"></a>
## Prerequisites
* An Azure virtual machine with Jenkins, Docker and Docker-Compose installed
* At least one other Azure virtual machine with Docker installed
* Access to a Dockerhub registry

## Installing Jenkins, Docker and Docker Compose, and setting up the swarm

### Setting up the project:

1. Create a new resource group:

<p align="center">

az group create –name devops -l uksouth

</p>

2. Create a new virtual machine:

<p align="center">

az vm create -g devops -n ManagerNode --image UbuntuLTS -l uksouth

*repeat this command at least once to create your worker nodes, renaming the machines in the -n tag (eg -n Worker1)*

</p>

3. Clone down the project within your ManagerNode

<p align="center">

git clone https://github.com/kryan1622/LAFB.git

</p>

4. Install docker and jenkins with with the included scripts and follow the instructions

<p align="center">

sh ~/LAFB/installation/docker.sh

*remember to copy this script and run it on your worker nodes*

sh ~/LAFB/installation/jenkins.sh

</p>

### Building the images

1. Build docker images

<p align="center">

docker-compose build

</p>

2. Push docker images to the desired dockerhub account by first logging onto dockerhub within the virtual machine
<p align="center">

docker login

*Enter username and password when requested*

docker-compose push

</p>

### Setting up the swarm

1. Initialise your swarm in the manager node

<p align="center">

docker swarm init

*this will return a command with a unique token which you can run in any number of other virtual machines to set them up as your worker nodes*

</p>

2. Deploy containers with the built images in docker swarm

<p align="center">

docker stack deploy --compose-file docker-compose.yaml devops

</p>

### Setting up Continuous Integration with Jenkins

1. As the Jenkins user, login to a dockerhub account that has access to the registry:

<p align="center">

sudo su jenkins
docker login
*Enter username and password when requested*

</p>

2. Expose port 8080 to access Jenkins externally using the command:

<p align="center">

az vm open-port -g devops -n devopsproject --port 8080 --priority 900

</p>

3. Access Jenkins site using the public ip address of the virtual machine being used with the addition of :8080 at the end. For example:

4. Get password for initial screen in Jenkins using the command:


<p text align="center">

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

</p>

5. On the customize Jenkins page select the option to Install suggested plugins, and create your username and password.

6. Set up the pipeline
1. Create Jenkins pipeline by selecting the new item option. Then name the job and select the pipeline option.

2. Then in the general section select the GitHub project option and copy in the URL below.
https://github.com/kryan1622/LAFB.git

3. Create webhook by selecting the trigger build remotely option in the Build Triggers section and type in:
DEVOPS

4. In the pipeline section, select the "Pipeline Script from SCM" option in "Definition". elect Git in the dropdown menu of SCM and copy in the URL used above into Repository URL.

To complete the webhook go into the settings option within the GitHub repository. Then select the webhook tab and select

http://51.144.95.241:8080/job/jobname/build

Then in the payload URL http://username:password@PublicIP:8080/job/jobname/build?token=TOKEN


<a name="CI"></a>
## CI pipeline
![CI Pipeline](/documentation/CIpipeline.png)

### Overview
The above diagram shows the flow of the continuous integration pipeline.
When a developer makes a change to the application in the source code and pushed to GitHub, the webhook is triggered and the Jenkins pipeline will automatically run.
The pipeline runs the following stages:
#### Build
*Goes through the docker-compose.yaml and rebuilds any images that have been changed*
#### Push
*Pushes changed images to dockerhub*
#### Deploy
*Updates changed containers in the stack without redeploying the entire application or affecting the user experience*

### Switching Implementations
The client asked for three different unique implementations to be included, these were for the prize generator, number generator and account generator. We have provided these and they can be seemlessly switched out for each other.
The images used are as follows:
#### prize generator
keepkarm/account:v1
* on creating an account, generates a prize of £50 or £0 with a 25% probability of winning
keepkarm/account:v2
* on creating an account, generates a prize of £100 or £0 with a 25% probability of winning
#### text generator
keepkarm/text_gen:v1
* generates a random string of three lowercase letters
keepkarm/text_gen:v2
* generates a random string of two uppercase letters
#### number generator
keepkarm/num_gen:v1
* generates a random six digit number
keepkarm/num_gen:v2
* generates a random eight digit number

If the bank's developers want to switch out any of these implementations for the other, all they need to do is edit the docker-compose.yaml file and push it up to git hub. They can swap the implementations for any of the services by changing both version number in the image name, and the number in the build args option. It is crucial to note that that **both numbers must match**.
When the new docker-compose.yaml is pushed to GitHub

## Improvements for the Future
Using a local registry would be helpful if deploying this application continuously. In cases of internet connection failures or dockerhub going down (which is not unlikely), images can still be easily accessed.
We suggest using a registry container to improve redundancy.

<a name="auth"></a>
## Authors

Aysha Marty and Krystal Ryan

<a name="ack"></a>
## Acknowledgements

* QA consulting and our fantastic instructors
* The rest of our wonderful cohort on the programme
1 change: 1 addition & 0 deletions account/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
node_modules
9 changes: 9 additions & 0 deletions account/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM node:10
ARG FILE_VERSION=1
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
COPY routes/account${FILE_VERSION}.js routes/account.js
EXPOSE 5002
ENTRYPOINT [ "node", "server.js" ]
Loading