cloudfront-report

Scala code as AWS Lambda which can be triggered automaticlly when there is a new CloudFront log coming to your S3 bucket. Logs will be parsed and index into Amazon Elasticsearch Service and display as a performance report on Kibana.

Currently you can create 3 types of reports on dashboard

• Bad download performance (< 200kb) showing on map
• POP distribution, which can show you edge locations
• Country distribution, which can show you user locations

Deploy AWS Lambda

// build
$ git clone https://github.com/jameswu0629/cloudfront-report.git
$ cd cloudfront-report
$ ./bin/activator assembly

// upload your package to S3
$ aws s3 cp cloudfront-report-x.x-SNAPSHOT.jar s3://[BUCKET_NAME]/source/

// 999999999999 is your account id
$ aws lambda create-function \
--function-name indexLogToES \
--code '{"S3Bucket": "[BUCKET_NAME]","S3Key": "source/cloudfront-report-x.x-SNAPSHOT.jar"}' \
--role arn:aws:iam::999999999999:role/lambda_s3_exec_role \
--handler cn.amazonaws.CloudfrontReport::indexLogToES \
--runtime java8 \
--timeout 300 \
--memory-size 1024

Setup trigger

Elasticsearch mapping

{
	"template": "logstash-*",
	"mappings": {
		"type1": {
			"properties": {
				"created_at": {
					"type": "date",
					"format": "yyyy-MM-dd HH:mm:ss"
				},
				"country": {
					"type": "string",
					"index": "not_analyzed"
				},
				"location": {
					"type": "geo_point"
				},
				"download_speed": {
					"type": "double",
				},
				"edge_location": {
					"type": "string",
					"index": "not_analyzed"
				}
			}
		}
	}
}

Kibana

Settings

TODO

• Use Elasticsearch Bulk API to batch process documents.