Update statefulset lab (#1063)

* add package json file

* Update statefulSet lab

* EBS-CSI-DRIVER is now installed using HELM
* using IRSA for the driver
* update the number of mysql pods due to t3 machines instead of m5
* update pictures
* Cosmetic changes based on linter
* Except for statefulset, move from downloading YAML file to create them
  inline
* delete all unused files

Co-authored-by: Frederic Medery <[email protected]>

Author: fmedery

.gitignore

@@ -7,3 +7,4 @@ terraform.tfstate
 terraform.tfstate.backup
 .DS_Store
 *.swp
+.vscode

content/beginner/170_statefulset/_index.md

@@ -10,9 +10,10 @@ tags:
 # Stateful containers using StatefulSets
 
 [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods, suitable for applications that require one or more of the following.
+
 * Stable, unique network identifiers
 * Stable, persistent storage
 * Ordered, graceful deployment and scaling
 * Ordered, automated rolling updates
 
-In this Chapter, we will review how to deploy MySQL database using `StatefulSet` and `Amazon Elastic Block Store` (EBS) as [PersistentVolume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). The example is a MySQL single leader topology with multiple followers running asynchronous replication.
+In this Chapter, we will review how to deploy MySQL database using `StatefulSet` and `Amazon Elastic Block Store` (EBS) as [PersistentVolume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). The example is a MySQL single leader topology with a follower running asynchronous replication.

content/beginner/170_statefulset/cleanup.md

@@ -4,21 +4,35 @@ date: 2018-08-07T08:30:11-07:00
 weight: 40
 ---
 ```sh
+export EBS_CSI_POLICY_NAME="Amazon_EBS_CSI_Driver"
+export EBS_CSI_POLICY_ARN=$(aws --region ${AWS_REGION} iam list-policies --query 'Policies[?PolicyName==`'${EBS_CSI_POLICY_NAME}'`].Arn' --output text)
+
+kubectl delete \
+  -f ${HOME}/environment/ebs_statefulset/mysql-statefulset.yaml \
+  -f ${HOME}/environment/ebs_statefulset/mysql-services.yaml \
+  -f ${HOME}/environment/ebs_statefulset/mysql-configmap.yaml \
+  -f ${HOME}/environment/ebs_statefulset/mysql-storageclass.yaml
+
 # Delete the mysql namespace 
 kubectl delete namespace mysql
 
-# Detach the IAM Amazon_EBS_CSI_Driver policy from your worker node instance profile.
-export EBS_CNI_POLICY_NAME="Amazon_EBS_CSI_Driver"
-export EBS_CNI_POLICY_ARN=$(aws --region ${AWS_REGION} iam list-policies --query 'Policies[?PolicyName==`'${EBS_CNI_POLICY_NAME}'`].Arn' --output text)
+# Uninstall the aws-ebs-csi-driver
+helm -n kube-system uninstall aws-ebs-csi-driver
 
-aws iam detach-role-policy \
-  --region ${AWS_REGION} \
-  --policy-arn ${EBS_CNI_POLICY_ARN} \
-  --role-name ${ROLE_NAME}
+# Delete the service account
+eksctl delete iamserviceaccount \
+  --cluster eksworkshop-eksctl \
+  --namespace kube-system \
+  --name ebs-csi-controller-irsa \
+  --wait
 
 # Delete the IAM Amazon_EBS_CSI_Driver policy
 aws iam delete-policy \
   --region ${AWS_REGION} \
-  --policy-arn ${EBS_CNI_POLICY_ARN}
+  --policy-arn ${EBS_CSI_POLICY_ARN}
+
+cd ${HOME}/environment
+rm -rf ${HOME}/environment/ebs_statefulset
 ```
-## Congratulation! You've finished the StatefulSets lab.
+
+## Congratulation! You've finished the StatefulSets lab

content/beginner/170_statefulset/configmap.files/mysql-configmap.yml

@@ -4,32 +4,26 @@ date: 2018-08-07T08:30:11-07:00
 weight: 10
 ---
 
-#### Introduction
-[ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) allow you to decouple configuration artifacts and secrets from image content to keep containerized applications portable. Using ConfigMap, you can independently control MySQL configuration. 
+## Introduction
+
+[ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) allow you to decouple configuration artifacts and secrets from image content to keep containerized applications portable. Using ConfigMap, you can independently control MySQL configuration.
+
+## Create the mysql Namespace
 
-#### Create the mysql Namespace
 We will create a new `Namespace` called `mysql` that will host all the components.
+
 ```sh
 kubectl create namespace mysql
 ```
 
-#### Create ConfigMap
-Run the following commands to download the `ConfigMap`.
-```sh
-cd ~/environment/templates
-wget https://eksworkshop.com/beginner/170_statefulset/configmap.files/mysql-configmap.yml
+## Create ConfigMap
 
-```
+Run the following commands to download the `ConfigMap`.
 
-Check the configuration of mysql-configmap.yml file.
 ```sh
-cat ~/environment/templates/mysql-configmap.yml
-```
+cd ${HOME}/environment/ebs_statefulset
 
-The `ConfigMap` stores master.cnf, slave.cnf and passes them when initializing leader and follower pods defined in StatefulSet:
-* **master.cnf** is for the MySQL leader pod which has binary log option (log-bin) to provides a record of the data changes to be sent to follower servers.
-* **slave.cnf** is for follower pods which have super-read-only option.
-{{< output >}}
+cat << EoF > ${HOME}/environment/ebs_statefulset/mysql-configmap.yaml
 apiVersion: v1
 kind: ConfigMap
 metadata:
@@ -43,14 +37,19 @@ data:
     [mysqld]
     log-bin
   slave.cnf: |
-    # Apply this config only on follower.
+    # Apply this config only on followers.
     [mysqld]
     super-read-only
-{{< /output >}}
+EoF
+```
+
+The `ConfigMap` stores `master.cnf`, `slave.cnf` and passes them when initializing leader and follower pods defined in StatefulSet:
+
+* **master.cnf** is for the MySQL leader pod which has binary log option (log-bin) to provides a record of the data changes to be sent to follower servers.
+* **slave.cnf** is for follower pods which have super-read-only option.
 
 Create "mysql-config" `ConfigMap`.
+
 ```sh
-kubectl create -f ~/environment/templates/mysql-configmap.yml
+kubectl create -f ${HOME}/environment/ebs_statefulset/mysql-configmap.yaml
 ```
-
-{{%attachments title="Related files" pattern=".yml"/%}}

content/beginner/170_statefulset/configmap.md

@@ -4,79 +4,108 @@ title: "Amazon EBS CSI Driver"
 date: 2020-02-23T13:57:00-08:00
 weight: 4
 ---
+## About Container Storage Interface (CSI)
+
+[The Container Storage Interface](https://github.com/container-storage-interface/spec/blob/master/spec.md) (CSI) is a standard for exposing arbitrary block and file storage systems to containerized workloads on Container Orchestration Systems (COs) like Kubernetes.
+
+Using CSI third-party storage providers can write and deploy plugins exposing new storage systems in Kubernetes without ever having to touch the core Kubernetes code.
 
 ## About the Amazon EBS CSI Driver
 
-On Amazon EKS, the open-source [EBS Container Storage Interface (CSI)
-driver](https://github.com/kubernetes-sigs/aws-ebs-csi-driver) is used to manage
-the attachment of Amazon EBS block storage volumes to Kubernetes Pods.
+The [Amazon Elastic Block Store (Amazon EBS) Container Storage Interface (CSI) driver](https://github.com/kubernetes-sigs/aws-ebs-csi-driver) provides a CSI interface that allows Amazon Elastic Kubernetes Service (Amazon EKS) clusters to manage the lifecycle of Amazon EBS volumes for persistent volumes.
+
+This topic shows you how to deploy the Amazon EBS CSI Driver to your Amazon EKS cluster and verify that it works.
 
 ## Configure IAM Policy
 
-The CSI driver is deployed as set of Kubernetes Pods. These Pods must have
-permission to perform EBS API operations, such as creating and deleting volumes,
-and attaching volumes to the EC2 worker nodes that comprise the cluster.
+The CSI driver is deployed as set of Kubernetes Pods. These Pods must have permission to perform EBS API operations, such as creating and deleting volumes, and attaching volumes to the EC2 worker nodes that comprise the cluster.
 
 First, let's download the policy JSON document, and create an IAM Policy from it:
 
 ```sh
-mkdir ~/environment/ebs_csi_driver
-cd ~/environment/ebs_csi_driver
-curl -sSL -o ebs-csi-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-ebs-csi-driver/v0.4.0/docs/example-iam-policy.json
-
 export EBS_CSI_POLICY_NAME="Amazon_EBS_CSI_Driver"
 
+mkdir ${HOME}/environment/ebs_statefulset
+cd ${HOME}/environment/ebs_statefulset
+
+# download the IAM policy document
+curl -sSL -o ebs-csi-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-ebs-csi-driver/master/docs/example-iam-policy.json
+
+# Create the IAM policy
 aws iam create-policy \
   --region ${AWS_REGION} \
   --policy-name ${EBS_CSI_POLICY_NAME} \
-  --policy-document file://ebs-csi-policy.json
+  --policy-document file://${HOME}/environment/ebs_statefulset/ebs-csi-policy.json
 
+# export the policy ARN as a variable
 export EBS_CSI_POLICY_ARN=$(aws --region ${AWS_REGION} iam list-policies --query 'Policies[?PolicyName==`'$EBS_CSI_POLICY_NAME'`].Arn' --output text)
 ```
 
 ## Configure IAM Role for Service Account
 
-Next, we'll ask `eksctl` to create an IAM Role that contains the IAM Policy we
-created, and associate it with a Kubernetes Service Account called
-`ebs-csi-controller-irsa` that will be used by the CSI Driver:
+You can associate an IAM role with a Kubernetes service account. This service account can then provide AWS permissions to the containers in any pod that uses that service account. With this feature, you no longer need to provide extended permissions to the Amazon EKS node IAM role so that pods on that node can call AWS APIs.
+
+We'll ask `eksctl` to create an IAM Role that contains the IAM Policy we just created, and associate it with a Kubernetes Service Account called `ebs-csi-controller-irsa` that will be used by the CSI Driver:
 
 ```sh
-eksctl utils associate-iam-oidc-provider --region=$AWS_REGION --cluster=eksworkshop-eksctl --approve
+# Create an IAM OIDC provider for your cluster
+eksctl utils associate-iam-oidc-provider \
+  --region=$AWS_REGION \
+  --cluster=eksworkshop-eksctl \
+  --approve
 
-eksctl create iamserviceaccount --cluster eksworkshop-eksctl \
+# Create a service account
+eksctl create iamserviceaccount \
+  --cluster eksworkshop-eksctl \
   --name ebs-csi-controller-irsa \
   --namespace kube-system \
   --attach-policy-arn $EBS_CSI_POLICY_ARN \
   --override-existing-serviceaccounts \
   --approve
 ```
 
-## Deploy EBS CSI Driver
+## Deploy the Amazon EBS CSI Driver
 
-Finally, we can deploy the driver.
+Finally, we can deploy the driver using helm.
 
-First, we'll need to download a few files.  Run:
+{{% notice note %}}
+If Helm is not installed, [click here the instruction](/beginner/060_helm/helm_intro/install/)
+{{% /notice %}}
 
 ```sh
-cd ~/environment/ebs_csi_driver
-for file in kustomization.yml deployment.yml attacher-binding.yml provisioner-binding.yml; do
-  curl -sSLO https://raw.githubusercontent.com/aws-samples/eks-workshop/main/content/beginner/170_statefulset/ebs_csi_driver.files/$file
-done
+# add the aws-ebs-csi-driver as a helm repo
+helm repo add aws-ebs-csi-driver https://kubernetes-sigs.github.io/aws-ebs-csi-driver
+
+# search for the driver
+helm search  repo aws-ebs-csi-driver
 ```
 
-To complete the deployment:
+output
+{{< output >}}
+NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
+aws-ebs-csi-driver/aws-ebs-csi-driver   0.9.8           0.9.0           A Helm chart for AWS EBS CSI Driver
+{{< /output >}}
 
 ```sh
-kubectl apply -k ~/environment/ebs_csi_driver
+helm upgrade --install aws-ebs-csi-driver \
+  --version=0.9.8 \
+  --namespace kube-system \
+  --set serviceAccount.controller.create=false \
+  --set serviceAccount.snapshot.create=false \
+  --set enableVolumeScheduling=true \
+  --set enableVolumeResizing=true \
+  --set enableVolumeSnapshot=true \
+  --set serviceAccount.snapshot.name=ebs-csi-controller-irsa \
+  --set serviceAccount.controller.name=ebs-csi-controller-irsa \
+  aws-ebs-csi-driver/aws-ebs-csi-driver
+
+kubectl -n kube-system rollout status deployment ebs-csi-controller
 ```
 
+Output
+
 {{< output >}}
-serviceaccount/ebs-csi-controller-sa created
-clusterrole.rbac.authorization.k8s.io/ebs-external-attacher-role created
-clusterrole.rbac.authorization.k8s.io/ebs-external-provisioner-role created
-clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-attacher-binding created
-clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-provisioner-binding created
-deployment.apps/ebs-csi-controller created
-daemonset.apps/ebs-csi-node created
-csidriver.storage.k8s.io/ebs.csi.aws.com created
+Waiting for deployment "ebs-csi-controller" rollout to finish: 0 of 2 updated replicas are available...
+Waiting for deployment "ebs-csi-controller" rollout to finish: 1 of 2 updated replicas are available...
+deployment "ebs-csi-controller" successfully rolled out
 {{< /output >}}