Skip to content

Commit ee3d295

Browse files
danxie1999Dan Xie
danxie1999
authored and
Dan Xie
committed
Fix the MegaLinter and add pytest for China code
1 parent 8627be4 commit ee3d295

31 files changed

+657
-972
lines changed

‎docs/admin-guide.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -59,9 +59,9 @@ definitions in them as desired.
5959
## adfconfig
6060

6161
The `adfconfig.yml` file resides on the
62-
[management account](#management-account) CodeCommit Repository (in `us-east-1` or `cn-north-1`)
63-
and defines the general high-level configuration for the AWS Deployment
64-
Framework.
62+
[management account](#management-account) CodeCommit Repository
63+
(in `us-east-1` or `cn-north-1`) and defines the general
64+
high-level configuration for the AWS Deployment Framework.
6565

6666
The configuration properties are synced into AWS Systems Manager Parameter
6767
Store and are used for certain orchestration options throughout your
@@ -964,8 +964,8 @@ To determine the current version, follow these steps:
964964
### ADF version you have deployed
965965

966966
To check the current version of ADF that you have deployed, go to the management
967-
account in us-east-1 or cn-north-1. Check the CloudFormation stack output or tag of the
968-
`serverlessrepo-aws-deployment-framework` Stack.
967+
account in us-east-1 or cn-north-1. Check the CloudFormation stack
968+
output or tag of the `serverlessrepo-aws-deployment-framework` Stack.
969969

970970
- In the outputs tab, it will show the version as the `ADFVersionNumber`.
971971
- In the tags on the CloudFormation stack, it is presented as
@@ -985,8 +985,8 @@ releases](https://github.com/awslabs/aws-deployment-framework/releases).
985985
The `serverlessrepo-aws-deployment-framework` stack is updated through this
986986
process with new changes that were included in that release of ADF.
987987

988-
To check the progress in the management account in `us-east-1` or `cn-north-1`, follow these
989-
steps:
988+
To check the progress in the management account in
989+
`us-east-1` or `cn-north-1`, follow these steps:
990990

991991
1. Go to the [CloudFormation
992992
console](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks?filteringStatus=active&filteringText=serverlessrepo-aws-deployment-framework&viewNested=true&hideStacks=false)
@@ -1138,11 +1138,11 @@ Alternatively, you can also perform the update using the AWS CLI.
11381138

11391139
If you wish to remove ADF you can delete the CloudFormation stack named
11401140
`serverlessrepo-aws-deployment-framework` in the management account in
1141-
the `us-east-1` region for global partition deployments; for China deployments
1142-
in `cn-north-1` region. This will remove most resources created by ADF in the management
1143-
account. With the exception of S3 buckets and SSM parameters. If you bootstrapped
1144-
ADF into the management account you need to manually remove the bootstrap stacks
1145-
as well.
1141+
the `us-east-1` region for global partition deployments; for China deployments
1142+
in `cn-north-1` region. This will remove most resources created by ADF
1143+
in the management account. With the exception of S3 buckets and SSM parameters.
1144+
If you bootstrapped ADF into the management account you need to manually
1145+
remove the bootstrap stacks as well.
11461146

11471147
Feel free to delete the S3 buckets, SSM parameters that start with the `/adf`
11481148
prefix, as well as other CloudFormation stacks such as:
@@ -1212,8 +1212,8 @@ The main components to look at are:
12121212
branch for the `aws-deployment-framework-bootstrap` (ADF Bootstrap) repository.
12131213
5. The [CodePipeline execution of the AWS Bootstrap pipeline](https://console.aws.amazon.com/codesuite/codepipeline/pipelines/aws-deployment-framework-bootstrap-pipeline/view?region=us-east-1).
12141214
6. Navigate to the [AWS Step Functions service](https://us-east-1.console.aws.amazon.com/states/home?region=us-east-1#/statemachines)
1215-
in the management account in `us-east-1` or `cn-north-1`. Check the state machines named
1216-
`AccountManagementStateMachine...` and
1215+
in the management account in `us-east-1` or `cn-north-1`. Check the
1216+
state machines named `AccountManagementStateMachine...` and
12171217
`AccountBootstrappingStateMachine...`. Look at recent executions only.
12181218
- When you find one that has a failed execution, check the components that
12191219
are marked orange/red in the diagram.

‎docs/installation-guide.md

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -43,13 +43,13 @@ Ensure you have setup [AWS CloudTrail](https://aws.amazon.com/cloudtrail/)
4343
*(Not the default trail)* in your Management Account that spans **all
4444
regions**, the trail itself can be created in any region. Events [triggered via
4545
CloudTrail](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_incident-response.html)
46-
for AWS Organizations can only be acted upon in the us-east-1 (North Virginia) or `cn-northwest-1`
47-
region.
46+
for AWS Organizations can only be acted upon in the us-east-1 (North Virginia)
47+
or cn-northwest-1 region.
4848

4949
Please use the [AWS CloudTrail
5050
instructions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html)
51-
to configure the CloudTrail in the `us-east-1` or `cn-north-1` region within the AWS
52-
Organizations Management AWS Account.
51+
to configure the CloudTrail in the `us-east-1` or `cn-north-1` region
52+
within the AWS Organizations Management AWS Account.
5353

5454
### 1.2. Enable AWS Organizations API Access
5555

@@ -92,6 +92,7 @@ Please note that building on *Windows* is not supported, please use the
9292
This should return a table that is possibly empty.
9393
- Additionally, running `docker --version` should return version 19 or
9494
later.
95+
<!-- markdown-link-check-disable-next-line -->
9596
- [make](https://www.gnu.org/software/make/)
9697
- To test if it is available, run `make --version`.
9798
This should return 4.3 or later.
@@ -101,6 +102,7 @@ Please note that building on *Windows* is not supported, please use the
101102
- [jq](https://github.com/jqlang/jq)
102103
- To test if it is available, run `jq --version`.
103104
This version should be 1.6 or later.
105+
<!-- markdown-link-check-disable-next-line -->
104106
- [sed](https://www.gnu.org/software/sed/)
105107
- To test if it is available, run `sed --version`.
106108
This should return 4.3 or later.
@@ -287,10 +289,10 @@ or applications into via AWS CodePipeline *(this can be updated later)*.
287289

288290
When deploying ADF for the first time, part of the installation process will
289291
automatically create an AWS CodeCommit repository in the management AWS Account
290-
within the `us-east-1` or `cn-north-1` region. It will also make the initial commit to the
291-
default branch of this repository with a default set of examples that act as a
292-
starting point to help define the AWS Account bootstrapping processes for your
293-
Organization.
292+
within the `us-east-1` or `cn-north-1` region. It will also make the initial
293+
commit to the default branch of this repository with a default set of
294+
examples that act as a starting point to help define the AWS Account
295+
bootstrapping processes for your Organization.
294296

295297
Part of the questions that follow will end up in the initial commit into the
296298
repository. These are passed directly the `adfconfig.yml` file prior to it
@@ -646,8 +648,9 @@ automatically in the background, to follow its progress:
646648

647649
1. Please navigate to the AWS Console in the AWS Management account.
648650
As the stack `serverlessrepo-aws-deployment-framework` completes you can now
649-
open AWS CodePipeline from within the management account in `us-east-1` or
650-
`cn-north-1` and see that there is an initial pipeline execution that started.
651+
open AWS CodePipeline from within the management account in `us-east-1`
652+
or `cn-north-1` and see that there is an initial pipeline
653+
execution that started.
651654

652655
Upon first installation, this pipeline might fail to fetch the source
653656
code from the repository. Click the retry failed action button to try again.

‎docs/user-guide.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -981,7 +981,7 @@ There are five different styles that one could choose from.
981981
- In case the bucket is stored in `us-east-1`, it will return:
982982
`https://s3.amazonaws.com/${bucket}/${key}`
983983
- In case the bucket is stored in `cn-north-1` or `cn-northwest-1`, it will return:
984-
`https://${bucket}.s3.${region}.amazonaws.cn/${key}`
984+
`https://${bucket}.s3.${region}.amazonaws.cn/${key}`
985985
- In case the bucket is stored in any other region, it will return:
986986
`https://s3-${region}.amazonaws.com/${bucket}/${key}`
987987
- `virtual-hosted` style, will return the S3 location using the virtual hosted

‎src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/tests/stubs/slack.py

Lines changed: 84 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -10,94 +10,106 @@
1010
PARTITION = Session().get_partition_for_region(REGION)
1111

1212
if PARTITION == "aws":
13-
test_region = "eu-central-1"
13+
test_region = "eu-central-1"
1414
else:
15-
test_region = "cn-northwest-1"
15+
test_region = "cn-northwest-1"
1616

1717
stub_approval_event = {
18-
'Records': [{
19-
'EventSource': 'aws:sns',
20-
'EventVersion': '1.0',
21-
'EventSubscriptionArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
22-
'Sns': {
23-
'Type': 'Notification',
24-
'MessageId': '1',
25-
'TopicArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
26-
'Subject': 'APPROVAL NEEDED: AWS CodePipeline adf-pipeline-sample-vpc for action Approve',
27-
'Message': '{"region":"{test_region}","consoleLink":"https://console.aws.amazon.com","approval":{"pipelineName":"adf-pipeline-sample-vpc","stageName":"approval-stage-1","actionName":"Approve","token":"fa777887-41dc-4ac4-8455-a209a93c76b9","expires":"2019-03-17T11:08Z","externalEntityLink":null,"approvalReviewLink":"https://console.aws.amazon.com/codepipeline/"}}',
28-
'Timestamp': '3000-03-10T11:08:34.673Z',
29-
'SignatureVersion': '1',
30-
'Signature': '1',
31-
'SigningCertUrl': f'https://sns.{test_region}.amazonaws.com/SimpleNotificationService',
32-
'UnsubscribeUrl': f'https://sns.{test_region}.amazonaws.com',
33-
'MessageAttributes': {}
18+
"Records": [
19+
{
20+
"EventSource": "aws:sns",
21+
"EventVersion": "1.0",
22+
"EventSubscriptionArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
23+
"Sns": {
24+
"Type": "Notification",
25+
"MessageId": "1",
26+
"TopicArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
27+
"Subject": "APPROVAL NEEDED: AWS CodePipeline adf-pipeline-sample-vpc for action Approve",
28+
"Message": '{"region":"{test_region}","consoleLink":"https://console.aws.amazon.com","approval":{"pipelineName":"adf-pipeline-sample-vpc","stageName":"approval-stage-1","actionName":"Approve","token":"fa777887-41dc-4ac4-8455-a209a93c76b9","expires":"2019-03-17T11:08Z","externalEntityLink":null,"approvalReviewLink":"https://console.aws.amazon.com/codepipeline/"}}',
29+
"Timestamp": "3000-03-10T11:08:34.673Z",
30+
"SignatureVersion": "1",
31+
"Signature": "1",
32+
"SigningCertUrl": f"https://sns.{test_region}.amazonaws.com/SimpleNotificationService",
33+
"UnsubscribeUrl": f"https://sns.{test_region}.amazonaws.com",
34+
"MessageAttributes": {},
35+
},
3436
}
35-
}]
37+
]
3638
}
3739

38-
stub_approval_event['Records'][0]['Sns']['Message'] = re.sub(r"{test_region}", test_region, stub_approval_event['Records'][0]['Sns']['Message'])
40+
stub_approval_event["Records"][0]["Sns"]["Message"] = re.sub(
41+
r"{test_region}", test_region, stub_approval_event["Records"][0]["Sns"]["Message"]
42+
)
3943

4044
stub_bootstrap_event = {
41-
'Records': [{
42-
'EventSource': 'aws:sns',
43-
'EventVersion': '1.0',
44-
'EventSubscriptionArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
45-
'Sns': {
46-
'Type': 'Notification',
47-
'MessageId': '1',
48-
'TopicArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
49-
'Subject': 'AWS Deployment Framework Bootstrap',
50-
'Message': 'Account 1111111 has now been bootstrapped into banking/production',
51-
'Timestamp': '3000-03-10T11:08:34.673Z',
52-
'SignatureVersion': '1',
53-
'Signature': '1',
54-
'SigningCertUrl': f'https://sns.{test_region}.amazonaws.com/SimpleNotificationService',
55-
'UnsubscribeUrl': f'https://sns.{test_region}.amazonaws.com',
56-
'MessageAttributes': {}
45+
"Records": [
46+
{
47+
"EventSource": "aws:sns",
48+
"EventVersion": "1.0",
49+
"EventSubscriptionArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
50+
"Sns": {
51+
"Type": "Notification",
52+
"MessageId": "1",
53+
"TopicArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
54+
"Subject": "AWS Deployment Framework Bootstrap",
55+
"Message": "Account 1111111 has now been bootstrapped into banking/production",
56+
"Timestamp": "3000-03-10T11:08:34.673Z",
57+
"SignatureVersion": "1",
58+
"Signature": "1",
59+
"SigningCertUrl": f"https://sns.{test_region}.amazonaws.com/SimpleNotificationService",
60+
"UnsubscribeUrl": f"https://sns.{test_region}.amazonaws.com",
61+
"MessageAttributes": {},
62+
},
5763
}
58-
}]
64+
]
5965
}
6066

6167
stub_failed_pipeline_event = {
62-
'Records': [{
63-
'EventSource': 'aws:sns',
64-
'EventVersion': '1.0',
65-
'EventSubscriptionArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
66-
'Sns': {
67-
'Type': 'Notification',
68-
'MessageId': '1',
69-
'TopicArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
70-
'Subject': None,
71-
'Message': '{"version":"0","id":"1","detail-type":"CodePipeline Pipeline Execution State Change","source":"aws.codepipeline","account":"2","time":"3000-03-10T11:09:38Z","region":"{test_region}","resources":["arn:aws:codepipeline:{test_region}:999999:adf-pipeline-sample-vpc"],"detail":{"pipeline":"adf-pipeline-sample-vpc","execution-id":"1","state":"FAILED","version":9.0}}',
72-
'Timestamp': '2019-03-10T11:09:49.953Z',
73-
'SignatureVersion': '1',
74-
'Signature': '2',
75-
'SigningCertUrl': f'https://sns.{test_region}.amazonaws.com/SimpleNotificationService',
76-
'UnsubscribeUrl': f'https://sns.{test_region}.amazonaws.com',
77-
'MessageAttributes': {}
68+
"Records": [
69+
{
70+
"EventSource": "aws:sns",
71+
"EventVersion": "1.0",
72+
"EventSubscriptionArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
73+
"Sns": {
74+
"Type": "Notification",
75+
"MessageId": "1",
76+
"TopicArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
77+
"Subject": None,
78+
"Message": '{"version":"0","id":"1","detail-type":"CodePipeline Pipeline Execution State Change","source":"aws.codepipeline","account":"2","time":"3000-03-10T11:09:38Z","region":"{test_region}","resources":["arn:aws:codepipeline:{test_region}:999999:adf-pipeline-sample-vpc"],"detail":{"pipeline":"adf-pipeline-sample-vpc","execution-id":"1","state":"FAILED","version":9.0}}',
79+
"Timestamp": "2019-03-10T11:09:49.953Z",
80+
"SignatureVersion": "1",
81+
"Signature": "2",
82+
"SigningCertUrl": f"https://sns.{test_region}.amazonaws.com/SimpleNotificationService",
83+
"UnsubscribeUrl": f"https://sns.{test_region}.amazonaws.com",
84+
"MessageAttributes": {},
85+
},
7886
}
79-
}]
87+
]
8088
}
8189

8290
stub_failed_bootstrap_event = {
83-
'Records': [{
84-
'EventSource': 'aws:sns',
85-
'EventVersion': '1.0',
86-
'EventSubscriptionArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
87-
'Sns': {
88-
'Type': 'Notification',
89-
'MessageId': '1',
90-
'TopicArn': f'arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example',
91-
'Subject': 'Failure - AWS Deployment Framework Bootstrap',
92-
'Message': '{"Error":"Exception","Cause":"{\\"errorMessage\\": \\"CloudFormation Stack Failed - Account: 111 Region: {test_region} Status: ROLLBACK_IN_PROGRESS\\", \\"errorType\\": \\"Exception\\", \\"stackTrace\\": [[\\"/var/task/wait_until_complete.py\\", 99, \\"lambda_handler\\", \\"status))\\"]]}"}',
93-
'Timestamp': '2019-03-10T11:09:49.953Z',
94-
'SignatureVersion': '1',
95-
'Signature': '2',
96-
'SigningCertUrl': f'https://sns.{test_region}.amazonaws.com/SimpleNotificationService',
97-
'UnsubscribeUrl': f'https://sns.{test_region}.amazonaws.com',
98-
'MessageAttributes': {}
91+
"Records": [
92+
{
93+
"EventSource": "aws:sns",
94+
"EventVersion": "1.0",
95+
"EventSubscriptionArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
96+
"Sns": {
97+
"Type": "Notification",
98+
"MessageId": "1",
99+
"TopicArn": f"arn:{PARTITION}:sns:{test_region}:9999999:adf-pipeline-sample-vpc-PipelineSNSTopic-example",
100+
"Subject": "Failure - AWS Deployment Framework Bootstrap",
101+
"Message": '{"Error":"Exception","Cause":"{\\"errorMessage\\": \\"CloudFormation Stack Failed - Account: 111 Region: {test_region} Status: ROLLBACK_IN_PROGRESS\\", \\"errorType\\": \\"Exception\\", \\"stackTrace\\": [[\\"/var/task/wait_until_complete.py\\", 99, \\"lambda_handler\\", \\"status))\\"]]}"}',
102+
"Timestamp": "2019-03-10T11:09:49.953Z",
103+
"SignatureVersion": "1",
104+
"Signature": "2",
105+
"SigningCertUrl": f"https://sns.{test_region}.amazonaws.com/SimpleNotificationService",
106+
"UnsubscribeUrl": f"https://sns.{test_region}.amazonaws.com",
107+
"MessageAttributes": {},
108+
},
99109
}
100-
}]
110+
]
101111
}
102112

103-
stub_failed_bootstrap_event['Records'][0]['Sns']['Message'] = re.sub(r"{test_region}", test_region, stub_failed_bootstrap_event['Records'][0]['Sns']['Message'])
113+
stub_failed_bootstrap_event["Records"][0]["Sns"]["Message"] = re.sub(
114+
r"{test_region}", test_region, stub_failed_bootstrap_event["Records"][0]["Sns"]["Message"]
115+
)

‎src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/tests/stubs/stub_iam.py

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,27 +9,25 @@
99
PARTITION = Session().get_partition_for_region(REGION)
1010

1111
if PARTITION == "aws":
12-
test_region = "eu-west-1"
12+
test_region = "eu-west-1"
1313
else:
14-
test_region = "cn-northwest-1"
14+
test_region = "cn-northwest-1"
1515

1616
"""
1717
Stubs for testing iam.py
1818
"""
1919

2020
get_role_policy = {
21-
'RoleName': 'string',
22-
'PolicyName': 'string',
23-
'PolicyDocument': {
21+
"RoleName": "string",
22+
"PolicyName": "string",
23+
"PolicyDocument": {
2424
"Version": "2012-10-17",
2525
"Statement": [
2626
{
2727
"Sid": "KMS",
2828
"Effect": "Allow",
2929
"Action": ["iam:ChangePassword"],
30-
"Resource": (
31-
f"arn:{PARTITION}:kms:{test_region}:111111111111:key/existing_key"
32-
),
30+
"Resource": (f"arn:{PARTITION}:kms:{test_region}:111111111111:key/existing_key"),
3331
},
3432
{
3533
"Sid": "S3",
@@ -44,8 +42,8 @@
4442
"Sid": "AssumeRole",
4543
"Effect": "Allow",
4644
"Action": "sts:AssumeRole",
47-
"Resource": ['something'],
45+
"Resource": ["something"],
4846
},
49-
]
50-
}
51-
}
47+
],
48+
},
49+
}

‎src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/china-support/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Copyright Amazon.com Inc. or its affiliates.
2+
# SPDX-License-Identifier: MIT-0
3+
4+
# pylint: skip-file

‎src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/china-support/china-forward-function/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Copyright Amazon.com Inc. or its affiliates.
2+
# SPDX-License-Identifier: MIT-0
3+
4+
# pylint: skip-file

0 commit comments

Comments
 (0)