Skip to content

Commit

Permalink
Use Express for API routing (#182)
Browse files Browse the repository at this point in the history
  • Loading branch information
Marc authored Feb 23, 2022
1 parent 071b75e commit 8841e0a
Show file tree
Hide file tree
Showing 23 changed files with 1,045 additions and 532 deletions.
14 changes: 10 additions & 4 deletions .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,11 @@
"mocha": true
},
"rules": {
"no-restricted-syntax": ["off", "BinaryExpression[operator='of']"],
"dot-notation": "off",
"no-restricted-syntax": [
"off",
"BinaryExpression[operator='of']"
],
"object-curly-newline": 0,
"semi": [
2,
Expand Down Expand Up @@ -44,7 +48,9 @@
"Regexp": "RegExp",
"promise": "Promise"
},
"requireReturn": true
"requireReturn": true,
"requireParamDescription": false,
"requireReturnDescription": false
}
],
"jsdoc/check-param-names": "error",
Expand All @@ -55,10 +61,10 @@
"jsdoc/require-example": "off",
"jsdoc/require-hyphen-before-param-description": "error",
"jsdoc/require-param": "error",
"jsdoc/require-param-description": "error",
"jsdoc/require-param-description": "off",
"jsdoc/require-param-name": "error",
"jsdoc/require-param-type": "error",
"jsdoc/require-returns-description": "error",
"jsdoc/require-returns-description": "off",
"jsdoc/require-returns-type": "error",
"generator-star-spacing": "off",
"import/no-extraneous-dependencies": "off",
Expand Down
5 changes: 5 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"editor.rulers": [
90
]
}
8 changes: 8 additions & 0 deletions @types/express/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/// <reference types="aws-lambda" />

declare namespace Express {
interface Request {
endpoint?: string
event?: AWSLambda.APIGatewayProxyEvent
}
}
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
Expand All @@ -20,6 +21,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

- Upgrade to Node 14
- Elasticsearch version update 7.9 -> 7.10
- Use Express for API routing

### Removed

Expand Down
55 changes: 9 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,51 +218,19 @@ npm run build-api-docs # TODO: this fails

### Running Locally

The easiest way to run the API server locally is to use an Elasticsearch container running in Docker and `serverless-offline`.

There is a `docker-compose.yml` file to simplify running Elasticsearch locally:
Before the API can be run, Elasticsearch and Localstack need to be running. There is a `docker-compose.yml` file to simplify running Elasticsearch locally:

```sh
docker-compose up -d
```

The API can be run using `serverless-offline`, which will require a `serverless.yml` file. Copy `serverless.yml.example` to `serverless.yml`. You'll also need to edit the `serverless.yml` file to set the `ES_HOST` value.

To do this, first change the value of `ES_HOST` in the serverless.yml file from the default (which dynamically populates it from the resource created upon deploy):

```yml
ES_HOST:
Fn::GetAtt: [ElasticSearchInstance, DomainEndpoint]
```
to instead use a hard-coded string of the Elasticsearch instance URL, e.g.,
The API can then be run with:

```yml
ES_HOST: http://localhost:9200
```
also set the STAC_API_URL explicitly:
```yml
STAC_API_URL: http://localhost:3000/dev
```
Then, use npm `serve` command to run serverless offline:

```
```sh
npm run serve
```

Connect to the sever on http://localhost:3000/dev/
To quickly ingest a collection and item into your deployment, run the following (where `TOPIC_ARN` is the ARN for the SNS topic created by your deployment):
```sh
curl -s https://planetarycomputer.microsoft.com/api/stac/v1/collections/aster-l1t | \
aws sns publish --topic-arn ${TOPIC_ARN} --message file:///dev/stdin
curl -s https://planetarycomputer.microsoft.com/api/stac/v1/collections/aster-l1t/items/AST_L1T_00312272006020322_20150518201805 | \
aws sns publish --topic-arn ${TOPIC_ARN} --message file:///dev/stdin
```
Connect to the sever on <http://localhost:3000/>

## Running Tests

Expand All @@ -284,24 +252,19 @@ npx ava tests/test_es.js --match='foobar*'

### System and Integration Tests

The System and Integration tests use an Elasticsearch server running in Docker and an instance of the API using [Serverless Offline](https://www.npmjs.com/package/serverless-offline).
The System and Integration tests use an Elasticsearch server running in Docker and a local instance of the API.

When the integration tests run, they:
When the system tests run, they:

1. Wait for Elasticsearch to be available
1. Delete all indices from Elasticsearch
1. Add indices and test data to Elasticsearch
1. Move any existing `serverless.yml` file to `serverless.yml.original`
1. Create a `serverless.yml` file to support serverless-offline
1. Build the packages
1. Start an instance of the API using serverless-offline. That API will be available at <http://localhost:3000/dev/>
1. Start an instance of the API. That API will be available at <http://localhost:3000/dev/>
1. Wait for the API to be available
1. Run the integration tests in `./tests/integration/test_*.js`
1. Remove the serverless.yml file
1. Restore `serverless.yml.original` to `serverless.yml`, if necessary
1. Run the integration tests in `./tests/system/test_*.js`
1. Stop the API

Before running the integration tests, make sure to start Elasticsearch using:
Before running the system tests, make sure to start Elasticsearch using:

```sh
docker-compose up -d
Expand Down
46 changes: 0 additions & 46 deletions bin/integration-tests.sh

This file was deleted.

29 changes: 6 additions & 23 deletions bin/system-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,36 +11,19 @@ export AWS_SECRET_ACCESS_KEY='none'
echo "Setting up Elasticsearch"
node ./tests/system/setup-es.js

echo "Configuring serverless-offline"
if [ -e serverless.yml ]; then
mv serverless.yml serverless.yml.original
fi
echo "Starting API"
node ./src/lambdas/api/local.js >/dev/null 2>&2 &
API_PID="$!"

grep -v 'ElasticSearchInstance, DomainEndpoint' serverless.yml.example |\
sed 's/ES_HOST.*/ES_HOST: http:\/\/localhost:9200\n STAC_API_URL: http:\/\/localhost:3000\/dev/' > serverless.yml

echo "Building packages"
npm run build

echo "Starting serverless-offline"
npx serverless offline start >/dev/null 2>&2 &
SERVERLESS_PID="$!"

./bin/wait-for-serverless-offline/run.sh
./bin/wait-for-api/run.sh

echo "Running tests"
set +e
npx ava ./tests/system/test-*.js
TEST_RESULT="$?"
set -e

rm -f serverless.yml

if [ -e serverless.yml.original ]; then
mv serverless.yml.original serverless.yml
fi

echo "Stopping serverless-offline"
kill "$SERVERLESS_PID"
echo "Stopping API"
kill "$API_PID"

exit "$TEST_RESULT"
11 changes: 11 additions & 0 deletions bin/wait-for-api/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/sh

set -e

D=$(dirname "$0")

echo 'Waiting for API'

timeout 60 "${D}/wait.sh"

echo 'API has started'
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/sh

is_serverless_offline_running () {
curl -s -X GET "http://127.0.0.1:3000/dev/" > /dev/null 2>&1
curl -s -X GET "http://127.0.0.1:3000/" > /dev/null 2>&1
}

while ! is_serverless_offline_running; do sleep 1; done
11 changes: 0 additions & 11 deletions bin/wait-for-serverless-offline/run.sh

This file was deleted.

Loading

0 comments on commit 8841e0a

Please sign in to comment.