Add event ID and regex features to CWAgent configuration wizard

[Paamicky]

Description of the issue

The configuration wizard does not support the new CWAgent Windows event filtering features (event ID and regex filtering). This limitation creates a poor customer experience since customers cannot use these features while creating their agent configuration through the wizard.

📝 Note: This is a draft. Waiting for event ids and regex feature branches to merge first.

Description of changes

• Add event id and regex options to the agent for use by customers in the configuration wizard.
• Add validation checks for event ID and regex inputs to prevent users from submitting invalid values

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

• Verified feature functionality in wizard interface
• Successfully generated configuration with event id and regex filtering.

Below is the wizard interface for creating a configuration

= Welcome to the Amazon CloudWatch Agent Configuration Manager =
=                                                              =
= CloudWatch Agent allows you to collect metrics and logs from =
= your host and send them to CloudWatch. Additional CloudWatch =
= charges may apply.                                           =

On which OS are you planning to use the agent?

1. linux
2. windows
3. darwin

default choice: [2]:
2
Trying to fetch the default region based on ec2 metadata...
I! imds retry client will retry 1 timesAre you using EC2 or On-Premises hosts?

1. EC2
2. On-Premises

default choice: [1]:
1
Do you want to turn on StatsD daemon?

1. yes
2. no

default choice: [1]:
2
Do you have any existing CloudWatch Log Agent configuration file to import for migration?

1. yes
2. no

default choice: [2]:
2
Do you want to monitor any host metrics? e.g. CPU, memory, etc.

1. yes
2. no

default choice: [1]:
2
Do you want to monitor any customized log files?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor any Windows event log?

1. yes
2. no

default choice: [1]:
1
Windows event log name:
default choice: [System]

Do you want to monitor VERBOSE level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor INFORMATION level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
1
Do you want to monitor WARNING level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor ERROR level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
1
Do you want to monitor CRITICAL level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
1
Do you want to filter by specific Event IDs?

1. yes
2. no

default choice: [1]:
1
Enter Event IDs (comma-separated, e.g., 1001,1002,1003):
4624,4625
Do you want to add regex filters to include/exclude specific events?

1. yes
2. no

default choice: [1]:
1
Filter type:

1. Include (events matching regex)
2. Exclude (events matching regex)

default choice: [1]:
1
Enter regex pattern:
Database.*
Do you want to add another regex filter?

1. yes
2. no

default choice: [1]:
1
Filter type:

1. Include (events matching regex)
2. Exclude (events matching regex)

default choice: [1]:
2
Enter regex pattern:
success
Do you want to add another regex filter?

1. yes
2. no

default choice: [1]:
2
Log group name:
default choice: [System]

Log stream name:
default choice: [{instance_id}]
test
Which log group class would you like to have for this log group?

1. STANDARD
2. INFREQUENT_ACCESS

default choice: [1]:
1
In which format do you want to store windows event to CloudWatch Logs?

1. XML: XML format in Windows Event Viewer
2. Plain Text: Legacy CloudWatch Windows Agent (SSM Plugin) Format

default choice: [1]:
2
Log Group Retention in days

1. -1
2. 3
3. 5
4. 7
5. 14
6. 30
7. 60
8. 90
9. 120
10. 150
11. 180
12. 365
13. 400
14. 545
15. 731
16. 1096
17. 1827
18. 2192
19. 2557
20. 2922
21. 3288
22. 3653

default choice: [1]:
1
Do you want to specify any additional Windows event log to monitor?

1. yes
2. no

default choice: [1]:
2
Do you want the CloudWatch agent to also retrieve X-ray traces?

1. yes
2. no

default choice: [1]:
2
Existing config JSON identified and copied to:  C:\Users\Administrator\AppData\Roaming\Amazon\CloudWatchAgent\etc\backup-configs
Saved config file to config.json successfully.
Current config as follows:
{
        "logs": {
                "logs_collected": {
                        "windows_events": {
                                "collect_list": [
                                        {
                                                "event_format": "text",
                                                "event_ids": [
                                                        4624,
                                                        4625
                                                ],
                                                "event_levels": [
                                                        "INFORMATION",
                                                        "ERROR",
                                                        "CRITICAL"
                                                ],
                                                "event_name": "System",
                                                "filters": [
                                                        {
                                                                "expression": "Database.*",
                                                                "type": "include"
                                                        },
                                                        {
                                                                "expression": "success",
                                                                "type": "exclude"
                                                        }
                                                ],
                                                "log_group_class": "STANDARD",
                                                "log_group_name": "System",
                                                "log_stream_name": "test",
                                                "retention_in_days": -1
                                        }
                                ]
                        }
                }
        }
}
Please check the above content of the config.
The config file is also located at config.json.
Edit it manually if needed.
Do you want to store the config in the SSM parameter store?

1. yes
2. no

default choice: [1]:
2
Please press Enter to exit...

This is the final configuration for the prompt above

{
        "logs": {
                "logs_collected": {
                        "windows_events": {
                                "collect_list": [
                                        {
                                                "event_format": "text",
                                                "event_ids": [
                                                        4624,
                                                        4625
                                                ],
                                                "event_levels": [
                                                        "INFORMATION",
                                                        "ERROR",
                                                        "CRITICAL"
                                                ],
                                                "event_name": "System",
                                                "filters": [
                                                        {
                                                                "expression": "Database.*",
                                                                "type": "include"
                                                        },
                                                        {
                                                                "expression": "success",
                                                                "type": "exclude"
                                                        }
                                                ],
                                                "log_group_class": "STANDARD",
                                                "log_group_name": "System",
                                                "log_stream_name": "test",
                                                "retention_in_days": -1
                                        }
                                ]
                        }
                }
        }
}

For validation, when a customer enters a wrong event ids, they are prompted and those invalid ids are ignored. For regex you will be asked to input another one. Below is the prompt showing the invalids ids were ignored in the final config.

= Welcome to the Amazon CloudWatch Agent Configuration Manager =
=                                                              =
= CloudWatch Agent allows you to collect metrics and logs from =
= your host and send them to CloudWatch. Additional CloudWatch =
= charges may apply.                                           =

On which OS are you planning to use the agent?

1. linux
2. windows
3. darwin

default choice: [2]:
2
Trying to fetch the default region based on ec2 metadata...
I! imds retry client will retry 1 timesAre you using EC2 or On-Premises hosts?

1. EC2
2. On-Premises

default choice: [1]:
1
Do you want to turn on StatsD daemon?

1. yes
2. no

default choice: [1]:
2
Do you have any existing CloudWatch Log Agent configuration file to import for migration?

1. yes
2. no

default choice: [2]:
2
Do you want to monitor any host metrics? e.g. CPU, memory, etc.

1. yes
2. no

default choice: [1]:
2
Do you want to monitor any customized log files?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor any Windows event log?

1. yes
2. no

default choice: [1]:
1
Windows event log name:
default choice: [System]

Do you want to monitor VERBOSE level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor INFORMATION level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor WARNING level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor ERROR level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
2
Do you want to monitor CRITICAL level events for Windows event log System ?

1. yes
2. no

default choice: [1]:
2
Do you want to filter by specific Event IDs?

1. yes
2. no

default choice: [1]:
1
Enter Event IDs (comma-separated, e.g., 1001,1002,1003):
-100,65536,799,543
Warning: Invalid Event ID '-100' ignored
Warning: Invalid Event ID '65536' ignored
Do you want to add regex filters to include/exclude specific events?

1. yes
2. no

default choice: [1]:
1
Filter type:

1. Include (events matching regex)
2. Exclude (events matching regex)

default choice: [1]:
1
Enter regex pattern:
([a-z+
Error: Invalid regex pattern '([a-z+': error parsing regexp: missing closing ]: [a-z+
Filter type:

1. Include (events matching regex)
2. Exclude (events matching regex)

default choice: [1]:
1
Enter regex pattern:
([a-z+])
Do you want to add another regex filter?

1. yes
2. no

default choice: [1]:
2
Log group name:
default choice: [System]

Log stream name:
default choice: [{instance_id}]

Which log group class would you like to have for this log group?

1. STANDARD
2. INFREQUENT_ACCESS

default choice: [1]:
1
In which format do you want to store windows event to CloudWatch Logs?

1. XML: XML format in Windows Event Viewer
2. Plain Text: Legacy CloudWatch Windows Agent (SSM Plugin) Format

default choice: [1]:
2
Log Group Retention in days

1. -1
2. 3
3. 5
4. 7
5. 14
6. 30
7. 60
8. 90
9. 120
10. 150
11. 180
12. 365
13. 400
14. 545
15. 731
16. 1096
17. 1827
18. 2192
19. 2557
20. 2922
21. 3288
22. 3653

default choice: [1]:
1
Do you want to specify any additional Windows event log to monitor?

1. yes
2. no

default choice: [1]:
2
Do you want the CloudWatch agent to also retrieve X-ray traces?

1. yes
2. no

default choice: [1]:
2
Existing config JSON identified and copied to:  C:\Users\Administrator\AppData\Roaming\Amazon\CloudWatchAgent\etc\backup-configs
Saved config file to config.json successfully.
Current config as follows:
{
        "logs": {
                "logs_collected": {
                        "windows_events": {
                                "collect_list": [
                                        {
                                                "event_format": "text",
                                                "event_ids": [
                                                        799,
                                                        543
                                                ],
                                                "event_name": "System",
                                                "filters": [
                                                        {
                                                                "expression": "([a-z+])",
                                                                "type": "include"
                                                        }
                                                ],
                                                "log_group_class": "STANDARD",
                                                "log_group_name": "System",
                                                "log_stream_name": "{instance_id}",
                                                "retention_in_days": -1
                                        }
                                ]
                        }
                }
        }
}
Please check the above content of the config.
The config file is also located at config.json.
Edit it manually if needed.
Do you want to store the config in the SSM parameter store?

1. yes
2. no

default choice: [1]:
2
Please press Enter to exit...

Below is the final config without the invalid ids:

{
        "logs": {
                "logs_collected": {
                        "windows_events": {
                                "collect_list": [
                                        {
                                                "event_format": "text",
                                                "event_ids": [
                                                        799,
                                                        543
                                                ],
                                                "event_name": "System",
                                                "filters": [
                                                        {
                                                                "expression": "([a-z+])",
                                                                "type": "include"
                                                        }
                                                ],
                                                "log_group_class": "STANDARD",
                                                "log_group_name": "System",
                                                "log_stream_name": "{instance_id}",
                                                "retention_in_days": -1
                                        }
                                ]
                        }
                }
        }

Requirements

Before commiting your code, please do the following steps.

1. Run make fmt and make fmt-sh
2. Run make lint

---

Integration Tests

To run integration tests against this PR, add the ready for testing label.

[okankoAMZ]

I think you should re-run make fmt and make lint, except for that LGTM