aussie-1

Order of priority for non-functional requirements:

1. Security. See http://www.infoworld.com/article/2608076/data-center/murder-in-the-amazon-cloud.html
2. Reliability. The site should be up 99.99%
3. Performance. The site should load as fast as possible.
4. Scalability. The site should automatically scale to account for usage peaks, eg Black Friday or DDOS attacks.

Development principles:

• Automation, automation, automation will allow a (very) small team to deliver, improve, operate and maintain the site.

Notes on architecture

Consider this best-practice but rather complicated AWS web app architecture: http://media.amazonwebservices.com/architecturecenter/AWS_ac_ra_web_01.pdf

We are planning to use AWS Elastic Beanstalk to minimize the initial DevOps workload. Our initial architecture will be based more closely on the conventions associated with the Elastic Beanstalk - see http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.concepts.architecture.html and it's sibling pages.

e.g. for now, we will not use S3 and Cloudfront for serving static resources. Instead the web servers will serve up all content, dynamic and static. This is a speed-to-market and development cost/time simplification, and can we undone when we need to benefit from the better performance / lower cost of Cloudfront / S3

We will have a single web app tier on node.js serving up the web-app-private and the web-api which connects to the internet (via the ELB) and to the database, and no separate worker tier (see here for why we might want to split out a worker tier in future: http://www.codingthearchitecture.com/2012/07/20/when_do_you_need_a_3_tier_architecture.html )

Notes on security

• Sessions: Secure session management, including careful use of HTTPS and HTTP for serving content, is critical to maintain user security and confidentiality. See https://www.owasp.org/index.php/Session_Management_Cheat_Sheet and https://stormpath.com/blog/everything-you-ever-wanted-to-know-about-node-dot-js-sessions/

Local setup

• Install Git. Configure git as usual (user.name, user.email, credential.helper)
• Install Vagrant
• Clone the repo: git clone https://github.com/mattbarton/aussie-1.git
• cd to the root of the repo, where the Vagrantfile is. Run vagrant up to (download and) launch the vagrant machine, then vagrant ssh to open a shell on the vagrant machine and run commands on the vagrant machine.

Repo organization

The repo contains several modules. (In future, the web-app-public+web-app-private could be split into a separate repo from the web-api, if there's a good reason to do so. Also, in future we will want to add modules (or repos) for the mobile apps)

web-app-public

This holds all the publicly-accessible, statically-served (ie not dynamically generated by server at time of request) assets, including the public-facing web site pages. The more we can put in here, the faster/cheaper/more performant the site will be, and all these assets can eventually be served by S3/Cloudfront if necessary. Some of these assets require a build chain:

• HTML pages: Use hugo from http://gohugo.io to assemble templates and pieces into complete, static web pages. To run the dev hugo server, use hugo server --buildDrafts --bind="0.0.0.0". You should then be able to view the website from a browser on your host machine at http://localhost:1313/ (in this case, it is necessary to use 0.0.0.0 instead of 127.0.0.1 or localhost for the binding, for it to be accessible from outside the VM) (Note that hugo watch does not work, due to running it in the vagrant environment. This is a known bug in hugo.)
• CSS files: These may require a build compiler if we start using LESS/SASS
• JS files: These may require compilation (if we use Coffeescript etc), uglification and minification
• Image files: These may require optimization
• Gzip of content

TODO After the static pages get built by hugo, they are deployed into the 'public' folder of the node server by the build job

web-app-private

This will contain members-only pages and assets (JS, img, css) and restrict access to them to authorized users. To do this, it must also:

• Handle user account workflow activities (login, logout, password change) (via stormpath APIs)
• Manage active user sessions. The node.js process should be stateless, for scalability reasons, so session data should be held in encrypted signed cookies, or a fast cache like Redis or ElastiCache

web-api

This will hold the web API back-end, served by node.js. This will provide API endpoints that can be called from the web app (potentially, both open endpoints that can be called from the web-app-public pages, and auth-only endpoints that can only be called from the web-app-private pages), and can similarly be called from any mobile apps we build later.

Most of these calls will involve the web-api making calls to the database(s).

Some of functions of the web-api:

• Authenticate user credentials and tokens
• Create and manage user accounts
• Provide data for user dashboard

Vagrant box details

We are using Vagrant to provide a consistent development environment for all team members that has all the necessary tools packaged with it, and is similar to the production environment (if necessary in future, we could use the Vagrant box to spin up the prod instances on EC2, so that dev and prod would be identical).

People generally recommend using the standard Amazon Linux AMI on EC2, which is based on RHEL/CentOS. So we'll use CentOS for the Vagrant box, and version 6.5 still since version 7 is not yet stable, ie chef/centos-6.5 See also: http://thejackalofjavascript.com/vagrant-mean-box/ The box is stored on Atlas as named box mbarton-at-embarkvet/aussie-box-1, see https://atlas.hashicorp.com/mbarton-at-embarkvet/boxes/aussie-box-1 for the commands used to create the box from chef/centos-6.5.