Merge pull request #11 from MITLibraries/Update-4.13

Update version and clarify documentation

Author: tim-rix

Dockerfile

@@ -1,4 +1,4 @@
-FROM matomo:4.12.0
+FROM matomo:4.13.1
 
 # Add the EnvironmentVariables plugin
 COPY ./files/plugin-EnvironmentVariables-4.0.1/ /var/www/html/plugins/EnvironmentVariables

docs/HowTos/HOWTO-compare-ini-files.md

@@ -0,0 +1,29 @@
+# How-to instructions
+
+Detailed how-to instructions to compare the running copy of config.ini.php to the version in this repo.
+
+## Command line access
+
+1. Log in to the [AWS Console](https://mitlib.awsapps.com/start#/) to retrieve your `Command line or programmatic access` credentials.
+1. Copy the appropriate OS information and paste it into your local terminal.
+1. Log in to the docker container in ECS - See [docker-matomo/docs/HowTos/HOWTO-miscellaneous.md](https://github.com/MITLibraries/docker-matomo/blob/main/docs/HowTos/HOWTO-miscellaneous.md) for instructions.
+1. Run the backup script:
+
+   ```bash
+   /usr/local/bin/backup-data.sh
+   ```
+
+1. Access the EFS mounted copy of config.ini.php.
+
+   ```bash
+   more /mnt/efs/config/config/config.ini.php # Hit space until all content is revealed
+   ```
+
+1. Copy/paste this information into a new file in your local editor and save as dev.ini (or other name of your choice).
+1. Compare the 2 files for any differences.  Ignore any changes in these sections:
+   * [database]
+   * [Tracker]
+   * [mail]
+   * **NB** Comment lines are not copied to the running/backup copy of config.ini.php.
+1. Verify information in the [General] section. **NB** Sometimes the order of various settings will change between Matomo versions
+1. Copy any differences in [Plugins] and [PluginsInstalled] to the repo copy of config.ini.php and save.

docs/HowTos/HOWTO-matomo-upgrade.md

@@ -1,19 +1,41 @@
 # How-to instructions
 
-Detailed how-to instructions for upgrade scenarios.
+Detailed how-to instructions for upgrading Matomo.
 
 ## Upgrade Database Engine
 
 The database engine is managed by [mitlib-tf-workloads-matomo](https://github.com/mitlibraries/mitlib-tf-workloads-matomo). In testing, Matomo had no problems with database engine upgrades through different MariaDB versions.
 
 ## Update Matomo version
 
+These instructions assume you are working in the **dev** environment.  Change to the appropriate <env> tag if you are working in a different environment.
+
 1. Ensure that a backup of the current `config/config.ini.php` exists in the EFS mount.
-1. Publish updated container to ECR.
+   * SSH (via AWSCLI + Session Manager) to the container(see the [Troubleshooting](./HOWTO-miscellaneous.md) section for the AWS CLI connection command).
+   * Run the **backup-data.sh** script `/usr/local/bin/backup-data.sh`
+1. Make any necessary changes to the repo.
+   * For version upgrades, change line 1 in **Dockerfile** to the new version.
+1. Publish the updated container to ECR.
+   * Run `make dist-dev` to create the updated container.
+   * Run `make publish-dev` to push the new container to ECR and tag it as 'latest'.
 1. Deploy updated container for ECS service.
-1. Verify via webUI that the Matomo installation is ready to be upgraded.
-1. SSH (via AWSCLI + Session Manager) to the container and run the upgrade on the CLI (see the [Troubleshooting](./HOWTO-miscellaneous.md) section for the AWS CLI connection command). The database update command is `php /var/www/html/console core:update`.
-1. For the 3.x to 4.x upgrade, there is an additional step to update to the utf8mb4 character set. Once the database upgrade is complete, run `/var/www/html/console core:convert-to-utf8mb4` to update the tables.
-1. Copy updated `config/config.ini.php` to the EFS mount.
+   * Via the AWS Console:
+     * Go to ECS
+     * Click on **matomo-ecs-dev-cluster**
+     * Click on the checkbox in **Services** next to `matomo-ecs-dev-service`
+     * Click  `Update`
+     * On the next page, expand **Deployment options** and choose `Force new deployment`
+     * Click `Update`
+   * Via AWS CLI
+     * `aws ecs update-service --cluster matomo-ecs-dev-cluster --service matomo-ecs-dev-service --force-new-deployment`
+   *Either method takes a few minutes to complete the deployment.  It's easiest to verify completion in the AWS Console.  A green bar will appear at the top of the page stating `Service updated: matomo-ecs-dev-cluster:matomo-ecs-dev-service`
+1. Verify via webUI that the Matomo installation is ready to be upgraded. Ask someone in DEP to assist with this step.
+   * This step is only necessary when a database update is required.
+1. SSH (via AWSCLI + Session Manager) to the container and run the upgrade on the CLI (see the [Troubleshooting](./HOWTO-miscellaneous.md) section for the AWS CLI connection command).
+   * The database update command is `php /var/www/html/console core:update`.
+   * This step is only necessary when there is a required update to the table structure in Matomo listed in the release notes.
+1. Copy the updated `config/config.ini.php` to the EFS mount.
+   * Run the **backup-data.sh** script as in step 1.
 1. Verify that there were no changes to the `config.ini.php` file that need to be captured back in this repo.
-1. Log back in to the webUI to verify that everything is working.
+   * See [compare-ini-files](./HOWTO-compare-ini-files.md)
+1. Log back in to the webUI to verify that everything is working.  Ask someone in DEP to assist with this step.

docs/HowTos/HOWTO-miscellaneous.md

@@ -15,10 +15,36 @@ The container-based Matomo installation requires a walk through of the web UI to
 
 ## Troubleshooting
 
-It **is** possible to SSH into the Matomo container while it is running in ECS:
+It **is** possible to SSH into the Matomo container while it is running in ECS:  **Ensure you have the latest Session Manager plugin installed if you have errors or no output to the following command:**
 
 ```bash
 aws ecs execute-command --region {name-of-the-region} --cluster {name-of-the-cluster} --task {task number} --command "/bin/bash" --interactive
 ```
 
 This can be used for quick checks of the `config.ini.php` file to see if actions in the UI made any modifications. This is also required for doing major version upgrades of Matomo (e.g. from 3.x to 4.x).
+
+To retrieve the **task number** value for the command:
+
+* Open the AWS Dev1 console in your browser
+* Navigate to ECS (Elastic Container Service)
+* Click on `matomo-ecs-dev-cluster` (This is also the cluster name for the above command)
+* Click on the `Tasks` tab
+* Copy the Task number from the list (there should be only one)
+
+OR
+
+```bash
+aws ecs list-clusters | grep matomo
+aws ecs list-tasks --cluster matomo-ecs-dev-cluster (or result from previous command)
+# The task number is the 32 character hex string at the end of the line
+```
+
+## Reset 2-Factor auth
+
+If we ever need to reset the 2FA configuration for a user, this is the command to run on the CLI of the container
+
+```bash
+./console twofactorauth:disable-2fa-for-user --login=yourlogin
+```
+
+See [FAQ_27248](https://matomo.org/faq/how-to/faq_27248/) for more details.

files/backup-data.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
-/bin/cp -r /var/www/html/config /mnt/efs/config
+/bin/cp -r /var/www/html/config/ /mnt/efs/config/
 /bin/cp -r /var/www/html/misc/user /mnt/efs/misc_user
-/bin/cp -r /var/www/html/js /mnt/efs/js
+/bin/cp -r /var/www/html/js/ /mnt/efs/js/
 /bin/cp -r /var/www/html/matomo.js /mnt/efs/matomo.js
 /bin/cp -r /var/www/html/piwik.js /mnt/efs/piwik.js

files/config.ini.php

@@ -11,27 +11,28 @@
 ; maximum number of rows for any of the Referers subtable (search engines by keyword, keyword by campaign, etc.), and Custom variables values
 datatable_archiving_maximum_rows_subtable_referrers = 5000
 
+; maximum number of rows for the User ID report
+datatable_archiving_maximum_rows_userid_users = 5000
+
 ; maximum number of rows for any of the Actions tables (pages, downloads, outlinks)
 datatable_archiving_maximum_rows_actions = 5000
 
 ; maximum number of rows for pages in categories (sub pages, when clicking on the + for a page category)
 datatable_archiving_maximum_rows_subtable_actions = 5000
 
+; maximum number of rows for the Site Search table
+ datatable_archiving_maximum_rows_site_search = 5000
+
 ; maximum number of rows for any of the Events tables (Categories, Actions, Names)
 datatable_archiving_maximum_rows_events = 5000
 
 ; maximum number of rows for sub-tables of the Events tables (eg. for the subtables Categories>Actions or Categories>Names).
 datatable_archiving_maximum_rows_subtable_events = 100
 
-; maximum number of rows for the Site Search table
- datatable_archiving_maximum_rows_site_search = 5000
-
-; maximum number of rows for the User ID report
-datatable_archiving_maximum_rows_userid_users = 5000
-
 [mail]
 
 [Plugins]
+Plugins[] = "CoreVue"
 Plugins[] = "CorePluginsAdmin"
 Plugins[] = "CoreAdminHome"
 Plugins[] = "CoreHome"
@@ -71,7 +72,6 @@
 Plugins[] = "ScheduledReports"
 Plugins[] = "UserCountryMap"
 Plugins[] = "Live"
-Plugins[] = "CustomVariables"
 Plugins[] = "PrivacyManager"
 Plugins[] = "ImageGraph"
 Plugins[] = "Annotations"
@@ -86,7 +86,10 @@
 Plugins[] = "Heartbeat"
 Plugins[] = "Intl"
 Plugins[] = "Tour"
+Plugins[] = "PagePerformance"
+Plugins[] = "CustomDimensions"
 Plugins[] = "MobileAppMeasurable"
+Plugins[] = "CustomVariables"
 Plugins[] = "EnvironmentVariables"
 
 [PluginsInstalled]
@@ -150,3 +153,7 @@
 PluginsInstalled[] = "ProfessionalServices"
 PluginsInstalled[] = "Tour"
 PluginsInstalled[] = "EnvironmentVariables"
+PluginsInstalled[] = "CoreVue"
+PluginsInstalled[] = "MobileAppMeasurable"
+PluginsInstalled[] = "PagePerformance"
+PluginsInstalled[] = "CustomDimensions"