Merge pull request #7 from MITLibraries/awsorg-cleanup

Repository Rename Cleanup

Author: cabutlermit

.github/workflows/dev_build.yml renamed to .github/workflows/dev-build.yml

@@ -1,4 +1,4 @@
-### This is the Terraform-generated dev-build.yml workflow for the matomo-dev app repository ###
+### This is the Terraform-generated dev-build.yml workflow for the docker-matomo-dev app repository ###
 name: Dev Build and Deploy Fargate Container
 on:
   workflow_dispatch:
@@ -15,5 +15,5 @@ jobs:
     secrets: inherit
     with:
       AWS_REGION: "us-east-1"
-      GHA_ROLE: "matomo-gha-dev"
-      ECR: "matomo-dev"
+      GHA_ROLE: "docker-matomo-gha-dev"
+      ECR: "docker-matomo-dev"

.github/workflows/prod-promote.yml

@@ -1,4 +1,4 @@
-### This is the Terraform-generated prod-promote.yml workflow for the matomo-prod app repository ###
+### This is the Terraform-generated prod-promote.yml workflow for the docker-matomo-prod app repository ###
 name: Prod Promote Fargate Container
 on:
   workflow_dispatch:
@@ -12,7 +12,7 @@ jobs:
     secrets: inherit
     with:
       AWS_REGION: "us-east-1"
-      GHA_ROLE_STAGE: matomo-gha-stage
-      GHA_ROLE_PROD: matomo-gha-prod
-      ECR_STAGE: "matomo-stage"
-      ECR_PROD: "matomo-prod"
+      GHA_ROLE_STAGE: docker-matomo-gha-stage
+      GHA_ROLE_PROD: docker-matomo-gha-prod
+      ECR_STAGE: "docker-matomo-stage"
+      ECR_PROD: "docker-matomo-prod"

.github/workflows/stage-build.yml

@@ -1,4 +1,4 @@
-### This is the Terraform-generated stage-build.yml workflow for the matomo-stage app repository ###
+### This is the Terraform-generated stage-build.yml workflow for the docker-matomo-stage app repository ###
 name: Stage Build and Deploy Fargate Container
 on:
   workflow_dispatch:
@@ -15,5 +15,5 @@ jobs:
     secrets: inherit
     with:
       AWS_REGION: "us-east-1"
-      GHA_ROLE: "matomo-gha-stage"
-      ECR: "matomo-stage"
+      GHA_ROLE: "docker-matomo-gha-stage"
+      ECR: "docker-matomo-stage"

Dockerfile

@@ -9,4 +9,7 @@ COPY ./files/config.ini.php /var/www/html/config/config.ini.php
 # Create mount point for EFS partition
 RUN mkdir -p /mnt/efs
 
+# Move in the "backup persistent files" script
+COPY --chmod=0755 ./files/backup-data.sh /usr/local/bin/backup-data.sh
+
 EXPOSE 80

Makefile

@@ -1,8 +1,8 @@
 .PHONY: help dist-dev publish-dev dist-stage publish-stage
 SHELL=/bin/bash
-### This is the Terraform-generated header for matomo-dev ###
-ECR_NAME_DEV:=matomo-dev
-ECR_URL_DEV:=222053980223.dkr.ecr.us-east-1.amazonaws.com/matomo-dev
+### This is the Terraform-generated header for docker-matomo-dev ###
+ECR_NAME_DEV:=docker-matomo-dev
+ECR_URL_DEV:=222053980223.dkr.ecr.us-east-1.amazonaws.com/docker-matomo-dev
 ### End of Terraform-generated header ###
 
 help: ## Print this message

README.md

@@ -5,15 +5,22 @@ This repository supports MIT Libraries' implementation of [Matomo](https://matom
 ## Dependencies
 
 * Deploying this container to AWS requires that the AWS ECR Repository for the container has been created by the [mitlib-tf-workloads-ecr](https://github.com/MITLibraries/mitlib-tf-workloads-ecr) repository.
+* Running this container in AWS requires the [mitlib-tf-workloads-matomo](https://github.com/mitlibraries/mitlib-tf-workloads-matomo) infrastructure repository.
 * Building this container requires access to [Official Docker Matomo](https://hub.docker.com/_/matomo/).
 
-## Build
+## Dev/Test Build
 
 Run `make dist-dev` to create a new container tagged as `matomo:latest`.
 
-## Deploy
+## Dev/Test Deploy
 
-Run `make publish-dev` to build, tag, and push a container to Dev1 for testing, or open a PR to `main`. Merge a PR to `main` to build, tag, and push the container to Stage-Workloads. After merging the PR to `main`, tag a release to promote to production.
+Run `make publish-dev` to build, tag, and push a container to Dev1 for testing, or open a PR to `main` (a GitHub Action will build, tag, push the container for you).
+
+## Stage Builds and Promotion to Prod
+
+Merge a PR to `main` to build, tag, and push the container to Stage-Workloads. After merging the PR to `main`, tag a release on the `main` branch to promote to production. GitHub Actions in this repo will take care of the build, tag, push to Stage and the copy from Stage to Production.
+
+**Important Note**: There is no automation in GitHub to automatically deploy the new container after it is push to the ECR repository in AWS. At this time, the only method to deploy the updated container is to force a new deployment of the Matomo service via the AWS Console.
 
 ## Implementation notes
 
@@ -24,15 +31,39 @@ For detailed instructions on initial setup, migration, database upgrades, and ap
 "State" for Matomo is managed/stored in at most two locations:
 
 * the `config.ini.php` file
-* the MySQL database
+* the MariaDB database
+
+The `config.ini.php` file contains some core Matomo configuration. There are a couple of sections worth addressing. First, a note about a special plugin we use.
+
+#### The EnvironmentVariables Plugin
+
+We use the [EnvironmentVariables](https://plugins.matomo.org/EnvironmentVariables) plugin that allows us to set configuration values for Matomo via environment variables via the [mitlib-tf-workloads-matomo](https://github.com/mitlibraries/mitlib-tf-workloads-matomo) infrastructure repo.
+
+The current practice is to set the following core configuration information in `config.ini.php` via environment variables. 
+
+* Database connection information
+* SSL/TLS configuration
+* SMTP information
+
+The remainder of the configuration values should be set by modifying the [config.ini.php](./files/config.ini.php) file and building an updated container.
 
-The `config.ini.php` file contains some core Matomo configuration. There are a couple of sections worth addressing:
+#### [Plugins] and [PluginsInstalled]
 
-1. The `[Plugins]` section tells Matomo which plugins to enable on installation. To disable a plugin, simply remove it from this section. Our preference is to modify this section and not the `PluginsInstalled` section, as removing a plugin from `PluginsInstalled` would remove it from the application completely. *Sadly, with the container-based Matomo, it seems to only way to activate a plugin is to use the Matomo web UI*.
-1. The `[mail]` section configures the SMTP server. Matomo uses mailers for several core functions, including password resets and reporting. Some related config values (e.g., `login_password_recovery_email_address`) are defined in the `[General]` section.
-1. The `[General]` section handles other configurations. A few of these settings are covered by the EnvironmentVariables plugin, but the rest are managed in this code.
+The `[Plugins]` section tells Matomo which plugins to enable on installation. To disable a plugin, simply remove it from this section. Our preference is to modify this section and not the `PluginsInstalled` section, as removing a plugin from `PluginsInstalled` would remove it from the application completely. While plugins can get installed in the container build, they are not automatically activated. There are two activation methods that can be used: via the web UI or via the CLI. See [Install a new plugin](https://matomo.org/faq/plugins/faq_21/) in the official Matomo documentation.
 
-By using the [EnvironmentVariables](https://plugins.matomo.org/EnvironmentVariables) plugin for Matomo, we generally don't need to store much in the `config.ini.php` file. Instead, the [mitlib-tf-workloads-matomo](https://github.com/MITLibraries/mitlib-tf-workloads-matomo) code defines the key environment variables for the ECS service, overriding anything in `config.ini.php` and `global.ini.php`.
+Any time plugins are installed/removed, it is imperative to capture a backup of the `config.ini.php` file (via the EFS partition) before launching the updated container and then compare it to the `config.ini.php` file after verifying that the plugin is activated. This is one of the few situations where the Matomo application might update/overwrite the `config.ini.php` file.
+
+#### [Mail]
+
+The `[mail]` section configures the SMTP server. Matomo uses mailers for several core functions, including password resets and reporting. Some related config values (e.g., `login_password_recovery_email_address`) are defined in the `[General]` section.
+
+All of the settings in `[mail]` are managed via environment variables that are set by the [mitlib-tf-workloads-matomo](https://github.com/mitlibraries/mitlib-tf-workloads-matomo) infrastructure repo.
+
+#### [General]
+
+The `[General]` section handles other configurations. A few of these settings are covered by the EnvironmentVariables plugin, but the rest are managed in this code.
+
+#### Other configuration
 
 The database stores the rest of the state of Matomo: all the data itself along with the Superuser credentials and the other user accounts and credentials.
 
@@ -54,6 +85,10 @@ We generally assign the 'View' role to new users, unless they require a higher p
 
 Matomo has built-in two-factor authentication, which we enforce for all accounts. When a user logs in to Matomo for the first time, they will be prompted to configure 2FA. At this time, we are recommending Duo as an authenticator.
 
+#### Recover from lost 2FA
+
+See the official [Recover from lost 2FA](https://matomo.org/faq/how-to/faq_27248) documentation. This allows us to recover the Superuser login if we lose the 2FA information.
+
 ## Additional documentation
 
 * [MIT Libraries dev docs](https://mitlibraries.github.io/guides/misc/matomo.html) - includes information about setting up a website for tracking in Matomo.

docs/HOWTO.md

@@ -0,0 +1,18 @@
+# How-to instructions
+
+Detailed how-to instructions for upgrade scenarios.
+
+## Upgrade Database Engine
+
+The database engine is managed by [mitlib-tf-workloads-matomo](https://github.com/mitlibraries/mitlib-tf-workloads-matomo). In testing, Matomo had no problems with database engine upgrades through different MariaDB versions.
+
+## Update Matomo version
+
+1. Ensure that a backup of the current `config/config.ini.php` exists in the EFS mount.
+1. Publish updated container to ECR.
+1. Deploy updated container for ECS service.
+1. Verify via webUI that the Matomo installation is ready to be upgraded.
+1. SSH (via AWSCLI + Session Manager) to the container and run the upgrade on the CLI (see the [Troubleshooting](./HOWTO-miscellaneous.md) section). It might be possible to do this via the web UI, but there seems to be a timeout issue related to health checks.
+1. Copy updated `config/config.ini.php` to the EFS mount.
+1. Verify that there were no changes to the `config.ini.php` file that need to be captured here.
+1. Relaunch the container.