♻️ Local dev environment improvements (#463)

* Adding a before you begin section

* added install instructions to  before you start still need version check

* Completed changes to before you get started section

* Completed changes to before you get started section

* adding console format to some of them

* playing with formatting

* formatted code in block format

* adding cd to server

* add local redis image

* server: clean up unused css selectors

* clean up mongo deprecations

* add start:local script

* move vscode settings to vscode workspace

* clean up server env file

* reduce logging and add docker check

* move db container away from default mogo port

* update readmes

* fix some readme typos

* switch to correct cypress bin but find child process spawn bug

* test to make sure this still works on travis

* add testing docs and update redis port so we never see a redis v. redis showdown

* clean

* ⬆️ bump postcss version

* add ability to reset local env

* add extra help text in readmes

Co-authored-by: aliza614 <[email protected]>

.gitignore

@@ -31,6 +31,8 @@ config.ts
 .redispid
 .redisuri
 packages/server/cypress/screenshots/**/*(failed).png
+cypress/
+cypress.json
 
 # web integration test
 packages/web/cypress/screenshots/**/*(failed).png

README.md

@@ -6,16 +6,23 @@ UpSwyng is a mobile-ready, digital directory of resources to assist the unhoused
 
 If you have any issues getting started using these directions, please see the [Troubleshooting section](#Troubleshooting).
 
-1. Install the Yarn package manager by following the instructions at
-   [https://yarnpkg.com/lang/en/docs/install](https://yarnpkg.com/lang/en/docs/install)
+### Requirements
 
-2. While in the root of this repo, install all project dependencies by running
+Make sure you have these tools installed before you begin.
 
-   ```console
-   yarn
-   ```
+- Node.js version 14.3
 
-3. Next, try starting try running the web client by entering `yarn workspace @upswyng/web start`. This command will start in on port 3000. For specific setup instructions on the two main projects, see their corresponding READMEs.
+- [Docker](https://docs.docker.com/get-docker/) version 20.10 or above
+
+- [Yarn](https://classic.yarnpkg.com/en/docs/install/#mac-stable)
+
+### Building your local dev environment
+
+1. Clone (or fork and clone) this repository and `cd` into it
+
+2. Make sure docker is running on your machine. You can check by running `docker ps` in your terminal
+
+3. Run `yarn start:local`
 
 ## Project Organization
 
@@ -44,7 +51,7 @@ yarn workspace @upswyng/web start
 To run the same command in _all_ workspaces, use `yarn workspaces`. For instance, to run `test` in all packages use:
 
 ```console
-yarn workspaces test
+yarn workspaces run test
 ```
 
 Typing out the full workspace name can get tedious. You can place aliases in your `.bashrc` or `.profile` to save time:
@@ -55,6 +62,15 @@ alias @uw="yarn workspace @upswyng/web "
 
 Now you only need `@uw start` to start the web dev server.
 
+### Running tests
+
+When you push a branch up to GitHub, [travis](https://docs.travis-ci.com/) will attempt to build the app with your changes and will run our entire test suite to ensure that nothing is broken.
+It's a good idea to run tests locally before pushing, although it is not required.
+
+If you wish to run tests locally, you will need to ensure that `redis-server` is available in your `path`. You can check by running `redis-server`, and you can install it [here](https://redis.io/topics/quickstart) or via `brew install redis` on mac.
+
+Once Redis is installed, run our entire test suite with `yarn test`.
+
 ### Troubleshooting
 
 #### The engine "node" is incompatible with this module.
@@ -65,7 +81,7 @@ When running `yarn` to install project dependencies, you may see a message that
 The engine "node" is incompatible with this module. Expected version "^14.3.x". Got "11.15.0".
 ```
 
-This just means that your machine's current version of node does not match the version required to run the UpSwyng server, 12.14.x. To fix this you'll need to install and which your current version of Node. [nvm](https://itnext.io/nvm-the-easiest-way-to-switch-node-js-environments-on-your-machine-in-a-flash-17babb7d5f1b#d594) is a helpful tool, among others, you could use to easily switch between node versions.
+This just means that your machine's current version of node does not match the version required to run the UpSwyng server, 14.3.x. To fix this you'll need to install and which your current version of Node. [nvm](https://itnext.io/nvm-the-easiest-way-to-switch-node-js-environments-on-your-machine-in-a-flash-17babb7d5f1b#d594) is a helpful tool, among others, you could use to easily switch between node versions.
 
 #### Type Errors
 
@@ -86,6 +102,10 @@ To fix this, run the script:
 yarn build:local-packages
 ```
 
+## Reset
+
+If nothing you try is fixing your issue, you can totally reset your local environment with `yarn reset`. After performing a reset, try `yarn start:local` and see if things work out.
+
 ## Contributing
 
 Pull requests are welcome and encouraged! Please review and follow our [guidelines](CONTRIBUTING.md).

clean.js

@@ -1,14 +1,38 @@
+/**
+ * Script to clean local environment. Usage:
+ * `yarn clean` to clean build artifacts
+ * `yarn clean --reset` to totally reset local environment by removing build artifacts, node modules, and .env files.
+ */
+
 const fs = require("fs");
 const glob = require("glob");
 
+const args = process.argv.slice(2);
+
 const foldersToDelete = [
   "./packages/types/dist",
   "./packages/common/dist",
   "./packages/server/__build__",
   "./packages/web/build",
 ];
 
-const globsToDelete = ["./packages/**/tsconfig*.tsbuildinfo"];
+const globsToDelete = [
+  "./packages/**/tsconfig*.tsbuildinfo",
+  "./packages/web/.env",
+];
+
+const foldersToDeleteForFullReset = [
+  "./node_modules",
+  "./packages/types/node_modules",
+  "./packages/web/node_modules",
+  "./packages/server/node_modules",
+  "./packages/common/node_modules",
+];
+
+const globsToDeleteForFullReset = [
+  "./packages/web/.env",
+  "./packages/server/.env",
+];
 
 function deleteFolderRecursive(path) {
   if (fs.existsSync(path) && fs.lstatSync(path).isDirectory()) {
@@ -24,7 +48,7 @@ function deleteFolderRecursive(path) {
       }
     });
 
-    console.log(`  🗑  Deleting directory "${path}"...`);
+    console.log(`  🗑  Deleting "${path}"...`);
     fs.rmdirSync(path);
   }
 }
@@ -36,9 +60,14 @@ function deleteGlob(globString) {
   });
 }
 
-console.log("🧹  Cleaning build artifacts...");
+console.log("🧹  Cleaning...");
 
 foldersToDelete.forEach(deleteFolderRecursive);
 globsToDelete.forEach(deleteGlob);
 
-console.log("🧼  Successfully cleaned build artifacts.");
+if (args[0] === "--reset") {
+  foldersToDeleteForFullReset.forEach(deleteFolderRecursive);
+  globsToDeleteForFullReset.forEach(deleteGlob);
+}
+
+console.log("🧼  Successfully cleaned.");

docker-compose.yml

@@ -5,7 +5,7 @@ services:
     image: mongo:3.6.20
     restart: always
     ports:
-      - 27017:27017
+      - 27018:27017
     environment:
       MONGO_INITDB_ROOT_USERNAME: root
       MONGO_INITDB_ROOT_PASSWORD: local
@@ -20,3 +20,9 @@ services:
     environment:
       ME_CONFIG_MONGODB_ADMINUSERNAME: root
       ME_CONFIG_MONGODB_ADMINPASSWORD: local
+  redis:
+    image: "redis:alpine"
+    ports:
+      - "6380:6379"
+    environment:
+      - REDIS_REPLICATION_MODE=master

package.json

@@ -83,13 +83,15 @@
     "build:local-packages": "yarn workspace @upswyng/types build && yarn workspace @upswyng/common build",
     "build:server": "yarn build:local-packages && yarn workspace @upswyng/server build",
     "build:web": "yarn build:local-packages && yarn workspace @upswyng/web build",
-    "build:local-db": "docker-compose up --build -d && mongorestore --username upswyng-dev-user --password upswyng123 -d upswyng-dev ./packages/server/localdb/mongodump",
+    "build:local-db": "docker compose up --build -d && mongorestore --port 27018 --username upswyng-dev-user --password upswyng123 -d upswyng-dev ./packages/server/localdb/mongodump",
     "clean": "node clean.js",
+    "reset": "node clean.js --reset && docker compose down -v",
     "heroku-postbuild": "yarn build:web && yarn build:server",
     "lint": "tsc --noEmit && eslint \"packages/**/src/**/*.{js,jsx,ts,tsx}\" --fix",
     "lint:ci": "tsc --noEmit && eslint \"packages/**/src/**/*.{js,jsx,ts,tsx}\" --quiet",
-    "test": "concurrently -n common,server -c yellow,blue \"yarn workspace @upswyng/common test\" \"yarn workspace @upswyng/server test\" && yarn workspace @upswyng/web test",
-    "test:ci": "concurrently -n common,server -c yellow,blue \"yarn workspace @upswyng/common test\" \"yarn workspace @upswyng/server test\" --kill-others-on-fail && yarn workspace @upswyng/web test"
+    "test": "concurrently -n common,server -c yellow,blue \"yarn workspace @upswyng/common test:unit\" \"yarn workspace @upswyng/server test\" && yarn workspace @upswyng/web test",
+    "test:ci": "concurrently -n common,server -c yellow,blue \"yarn workspace @upswyng/common test:unit\" \"yarn workspace @upswyng/server test\" --kill-others-on-fail && yarn workspace @upswyng/web test",
+    "start:local": "./start_local.sh"
   },
   "version": "0.1.0",
   "workspaces": [

packages/common/package.json

@@ -24,7 +24,7 @@
   "name": "@upswyng/common",
   "scripts": {
     "build": "rollup --config",
-    "test": "jest"
+    "test:unit": "jest"
   },
   "types": "dist/index.d.ts",
   "version": "0.1.0"

packages/common/rollup.config.js

@@ -25,6 +25,7 @@ module.exports = {
     // All the other modules besides 'rrule' should be resolved with the default entrypoint priority. (Defaults
     // to ['module', 'main'].) See: https://github.com/rollup/rollup-plugin-node-resolve#usage
     resolve({
+      preferBuiltins: true,
       only: [/^(?!.*rrule).*$/],
       browser: true,
       preferBuiltins: false,

packages/server/.env.example

@@ -1,27 +1,26 @@
-ALGOLIA_APP_ID=<example algolia app id>
-ALGOLIA_INDEX_NAME=<example algolia index name>
-ALGOLIA_WRITE_API_KEY=<example algolia write api key>
-BETA_APP=<boolean: if true, app will only display if 'beta' is in the hostname>
-BOT_USER_ID=<ID of user used to create automated jobs>
+ALGOLIA_APP_ID=
+ALGOLIA_INDEX_NAME=
+ALGOLIA_WRITE_API_KEY=
+BETA_APP= # boolean: if true, app will only display if 'beta' is in the hostname
+BOT_USER_ID= # ID of user used to create automated jobs
 DATABASE_PASSWORD=upswyng123
-DATABASE_SESSION_SECRET=
-DATABASE_URL=mongodb://localhost:27017/upswyng-dev
+DATABASE_SESSION_SECRET=secret formula
+DATABASE_URL=mongodb://localhost:27018/upswyng-dev
 DATABASE_USER=upswyng-dev-user
-OAUTH_FACEBOOK_CLIENT_ID=<example facebook client id>
-OAUTH_FACEBOOK_CLIENT_SECRET=<example facebook secret>
-OAUTH_GOOGLE_CLIENT_ID=<example google client id>
-OAUTH_GOOGLE_CLIENT_SECRET=<example google secret>
-OAUTH_SLACK_CLIENT_ID=<example slack client id>
-OAUTH_SLACK_CLIENT_SECRET=<example slack secret>
+REDIS_URL=redis://localhost:6380 # used for worker queue
+WORKER_JOB_QUEUE_NAME=worker-test-queue
+OAUTH_FACEBOOK_CLIENT_ID=
+OAUTH_FACEBOOK_CLIENT_SECRET=
+OAUTH_GOOGLE_CLIENT_ID=
+OAUTH_GOOGLE_CLIENT_SECRET=
+OAUTH_SLACK_CLIENT_ID=
+OAUTH_SLACK_CLIENT_SECRET=
 OAUTH_SLACK_WORKSPACE_ID=T02GLC0V9 # see docs: https://api.slack.com/docs/sign-in-with-slack
-OPEN_WEATHER_API_KEY=<open_weather_secret>
-REDIS_URL=redis://<host>:<port> # used for worker queue
-SERVER_GOOGLE_CLOUD_API_KEY=<example google cloud api key>
-SERVER_HOST=upswyng.codeforboulder.org:80
+OPEN_WEATHER_API_KEY=
+SERVER_GOOGLE_CLOUD_API_KEY=
+SERVER_HOST=
 SERVER_PROTOCOL=http
 SLACK_BOT_CHANNEL=proj-upswyng
-SLACK_BOT_OAUTH_ACCESS_TOKEN=<https://api.slack.com/apps -> 'OAuth & Permissions Tab' -> 'Bot User OAuth Access Token
-'>
-SLACK_OAUTH_ACCESS_TOKEN=<https://api.slack.com/apps -> 'OAuth & Permissions Tab' -> 'OAuth Access Token'>
+SLACK_BOT_OAUTH_ACCESS_TOKEN= # <https://api.slack.com/apps -> 'OAuth & Permissions Tab' -> 'Bot User OAuth Access Token'>
+SLACK_OAUTH_ACCESS_TOKEN= #<https://api.slack.com/apps -> 'OAuth & Permissions Tab' -> 'OAuth Access Token'>
 SLACK_SIGNING_SECRET=
-WORKER_JOB_QUEUE_NAME=worker-test-queue

packages/server/README.md

@@ -7,6 +7,8 @@ This workspace includes:
 
 ## Setup
 
+> You do not need to follow these steps if you are using `yarn start:local` because everything will already be set up for you
+
 _Note: This project is built upon [Sapper](https://github.com/sveltejs/sapper), which uses a combination of [Svelte](https://svelte.dev/) and [Polka](https://github.com/lukeed/polka) to build and serve the app._
 
 ### 1. Setup Local MongoDB
@@ -34,6 +36,7 @@ For local development, you'll only need to create a copy of `.env.example` and n
 You can do this via your system's file explorer or via the following terminal command while in the repo:
 
 ```console
+cd packages/server
 cp .env.example .env
 ```
 
@@ -63,10 +66,10 @@ This file contains placeholders for various API keys that you may or may not nee
 ### 2. Start the Server
 
 ```console
-yarn dev
+yarn workspace @upswyng/server dev
 ```
 
-Runs the app in the development mode. Open [http:/localhost:3000](http:/localhost:3000) to view it in the browser.
+Runs the server in the development mode. Open [http:/localhost:3000](http:/localhost:3000) to view it in the browser.
 
 If you want to work on the [provider portal](http://localhost:3000/provider), you can log in with this google user:
 

packages/server/cypress/start_memory_redis.js

@@ -34,7 +34,7 @@ child.stdout.on("data", chunk => {
   console.info(`🎉 Started in-memory redis`);
   console.info(`PID:\t\t${pid}`);
   console.info(`Connection URI:\t${connectionString}`);
-  
+
   // got data, maybe look at it to verify that it started up ok
   child.unref(); // let the process close normally
   process.exit(0);

packages/server/cypress/start_memory_redis_impl.js

@@ -10,7 +10,7 @@ const tmpRedis = require("tmp-redis");
 const getPort = require("get-port");
 
 getPort({ port: 6379 }).then(p => {
-  tmpRedis(p, (e, _shutdown) =>{
+  tmpRedis(p, (e, _shutdown) => {
     if (e) throw e;
     console.info(`CONNECTION_STRING=redis://localhost:${p}`);
   });