Deploy Azure Kubernetes Cluster using Terraform

bySystem Engineer -
2

 




Terraform ??

 Terraform is an Infrastructure as code tool that allows developers and operations teams to automate how they provision their infrastructure.

IaC(Infrastructure as code) offers quite a few advantages.

  1. Because your infrastructure is now represented as code it is testable
  2. Your environments are now very much reproducible
  3. You can now track changes to your infrastructure over time with a version control system like Git
  4. Deployments are faster,because you interact with the cloud provider less.




Project setup

Prerequisites:

 

1.az login

 

To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code HZ76HBPXS to authenticate.[0m
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "ce1a0091-119f-4bwd04-8917-6170bd20e243",
    "id": "95798dc6-679a-4a59-914qwde-6d0d3bf79774",
    "isDefault": true,
    "managedByTenants": [],
    "name": "Free Trial",
    "state": "Enabled",
    "tenantId": "ce1a0091-119f-4b04-8917-6170bd20e243",
    "user": {
      "name": "@hotmail.com",
      "type": "user"
    }
  }
]

2. git clone https://github.com/nandlalyadav57/terra_aks_webapp.git

Modules??

Modules in Terraform are a way to separate your configuration so each module can handle a specific task. Sure we could just dump all of our configuration in main.tf but that makes things clunky and less portable.

 

Now lets take a look at the cluster folder in modules directory.

modules/cluster
├── cluster.tf
└── variables.tf

 

image

 

 

Need to run terraform init. This would download any plugins that the Azure provider depends on.

3.terraform init

Created a variable for it earlier. if you have an ssh key pair you can skip this step $ ssh-keygen -t rsa -b 4096

Export the key into an environment variable.

4. export TF_VAR_ssh_key=$( cat ~/.ssh/id_rsa.pub)

Before we actually create our infrastructure its always a good idea to see what exactly Terraform would be creating luckily Terraform has a command for that

5. terraform plan



If every thing looks good we can apply our configuration using:

6. terraform apply





Once the resources have been provisioned head over to your azure dashboard a look Go to your newly build AKS Cluster (Check details by clicking on connect from Azure Dashboard)

7. az account set --subscription 95798dc6-679a-4a59-914e-6dsdgfsd74

8. aks get-credentials --resource-group kubernetes-resource-group --name yourcluster

9.kubectl get nodes -o wide

Now Let's deploy a simple web application which is using redis

10. kubectl create namespace myvote

11. kubectl apply -f azure-vote.yaml -n myvote

or

kubectl apply -f  https://raw.githubusercontent.com/nandlalyadav57/terra_aks_webapp/main/azure-vote.yaml?token=AIAXGZX4NW7UPQUMDVTWWNLASTZEE -n myvote

kubectl get all -n myvote

To manually change the number of pods in the azure-vote-front deployment, use the kubectl scale command. The following example increases the number of front-end pods to 5:

12. kubectl scale --replicas=5 deployment/azure-vote-front -n myvote

kubectl autoscale command to autoscale the number of pods in the azure-vote-front deployment. If average CPU utilization across all pods exceeds 50% of their requested usage

kubectl autoscale deployment azure-vote-front --cpu-percent=50 --min=3 --max=10

 

Some of the Outputs:

 

kubectl apply -f azure-vote.yaml -n myvote


deployment.apps/azure-vote-back created
service/azure-vote-back created
deployment.apps/azure-vote-front created
service/azure-vote-front created

 

 

 

kubectl get all -n myvote
NAME                                    READY   STATUS              RESTARTS   AGE
pod/azure-vote-back-5f8bd8b-4kbwr       1/1     Running             0          44s
pod/azure-vote-front-7797fb8f5d-v7kx2   0/1     ContainerCreating   0          42s

NAME                       TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
service/azure-vote-back    ClusterIP      10.0.26.191    <none>           6379/TCP       44s
service/azure-vote-front   LoadBalancer   10.0.161.190   52.234.160.220   80:32478/TCP   42s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/azure-vote-back    1/1     1            1           45s
deployment.apps/azure-vote-front   0/1     1            0           43s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/azure-vote-back-5f8bd8b       1         1         1       46s
replicaset.apps/azure-vote-front-7797fb8f5d   1         1         0       44s
]0;root@localhost:~/1/terra_aks_webapp[root@localhost terra_aks_webapp]# kubectl get all -n myvote
NAME                                    READY   STATUS    RESTARTS   AGE
pod/azure-vote-back-5f8bd8b-4kbwr       1/1     Running   0          2m2s
pod/azure-vote-front-7797fb8f5d-v7kx2   1/1     Running   0          2m

NAME                       TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
service/azure-vote-back    ClusterIP      10.0.26.191    <none>           6379/TCP       2m2s
service/azure-vote-front   LoadBalancer   10.0.161.190   52.234.160.220   80:32478/TCP   2m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/azure-vote-back    1/1     1            1           2m3s
deployment.apps/azure-vote-front   1/1     1            1           2m1s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/azure-vote-back-5f8bd8b       1         1         1       2m4s
replicaset.apps/azure-vote-front-7797fb8f5d   1         1         1       2m2s
]0;root@localhost:~/1/terra_aks_webapp[root@localhost terra_aks_webapp]# kubectl get all -n myvote
NAME                                    READY   STATUS    RESTARTS   AGE
pod/azure-vote-back-5f8bd8b-4kbwr       1/1     Running   0          2m27s
pod/azure-vote-front-7797fb8f5d-v7kx2   1/1     Running   0          2m25s

NAME                       TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
service/azure-vote-back    ClusterIP      10.0.26.191    <none>           6379/TCP       2m26s
service/azure-vote-front   LoadBalancer   10.0.161.190   52.234.160.220   80:32478/TCP   2m24s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/azure-vote-back    1/1     1            1           2m28s
deployment.apps/azure-vote-front   1/1     1            1           2m26s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/azure-vote-back-5f8bd8b       1         1         1       2m28s
replicaset.apps/azure-vote-front-7797fb8f5d   1         1         1       2m26s


 

[root@localhost ~]# kubectl scale --replicas=5 deployment/azure-vote-front
get all -n myvote
NAME                                    READY   STATUS              RESTARTS   AGE
pod/azure-vote-back-5f8bd8b-8rmxw       1/1     Running             0          4m21s
pod/azure-vote-front-7797fb8f5d-8582k   1/1     Running             0          3s
pod/azure-vote-front-7797fb8f5d-cjpwm   1/1     Running             0          3s
pod/azure-vote-front-7797fb8f5d-hfjnw   0/1     ContainerCreating   0          3s
pod/azure-vote-front-7797fb8f5d-lp6rk   1/1     Running             0          4m19s
pod/azure-vote-front-7797fb8f5d-nccvv   0/1     ContainerCreating   0          3s

NAME                       TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)        AGE
service/azure-vote-back    ClusterIP      10.0.176.83   <none>         6379/TCP       4m21s
service/azure-vote-front   LoadBalancer   10.0.171.45   20.81.24.212   80:31157/TCP   4m19s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/azure-vote-back    1/1     1            1           4m23s
deployment.apps/azure-vote-front   3/5     5            3           4m21s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/azure-vote-back-5f8bd8b       1         1         1       4m23s
replicaset.apps/azure-vote-front-7797fb8f5d   5         5         3       4m21s

 

 

 

 

 

 

 

 

 

 

 

 

 

 

# kubectl autoscale deployment azure-vote-front --cpu-percent=50 --min=3 --max=10
horizontalpodautoscaler.autoscaling/azure-vote-front autoscaled

 

 

 

 

 

Once you are done it's as easy as running terraform destroy to tear down all the resources you have just provisioned.

13. Terraform Destroy

You learnt :

  • Why Infrastructure as code is important is important
  • The basics of HCL(Hashicorp configuration language)
  • How to provision a kubernetes cluster with terraform

Manually scale pods

When the Azure Vote front-end and Redis instance were deployed in previous tutorials, a single replica was created. To see the number and state of pods in your cluster, use the kubectl get command as follows:

Console
kubectl get pods

The following example output shows one front-end pod and one back-end pod:

Output
NAME                               READY     STATUS    RESTARTS   AGE
azure-vote-back-2549686872-4d2r5   1/1       Running   0          31m
azure-vote-front-848767080-tf34m   1/1       Running   0          31m

To manually change the number of pods in the azure-vote-front deployment, use the kubectl scale command. The following example increases the number of front-end pods to 5:

Console
kubectl scale --replicas=5 deployment/azure-vote-front

Run kubectl get pods again to verify that AKS successfully creates the additional pods. After a minute or so, the pods are available in your cluster:

Console
kubectl get pods

                                    READY     STATUS    RESTARTS   AGE
azure-vote-back-2606967446-nmpcf    1/1       Running   0          15m
azure-vote-front-3309479140-2hfh0   1/1       Running   0          3m
azure-vote-front-3309479140-bzt05   1/1       Running   0          3m
azure-vote-front-3309479140-fvcvm   1/1       Running   0          3m
azure-vote-front-3309479140-hrbf2   1/1       Running   0          15m
azure-vote-front-3309479140-qphz8   1/1       Running   0          3m

Autoscale pods

Kubernetes supports horizontal pod autoscaling to adjust the number of pods in a deployment depending on CPU utilization or other select metrics. The Metrics Server is used to provide resource utilization to Kubernetes, and is automatically deployed in AKS clusters versions 1.10 and higher. To see the version of your AKS cluster, use the az aks show command, as shown in the following example:

Azure CLI
az aks show --resource-group myResourceGroup --name myAKSCluster --query kubernetesVersion --output table

 Note

If your AKS cluster is less than 1.10, the Metrics Server is not automatically installed. Metrics Server installation manifests are available as a components.yaml asset on Metrics Server releases, which means you can install them via a url. To learn more about these YAML definitions, see the Deployment section of the readme.

Example installation:

Console
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml

To use the autoscaler, all containers in your pods and your pods must have CPU requests and limits defined. In the azure-vote-front deployment, the front-end container already requests 0.25 CPU, with a limit of 0.5 CPU. These resource requests and limits are defined as shown in the following example snippet:

YAML
resources:
  requests:
     cpu: 250m
  limits:
     cpu: 500m

The following example uses the kubectl autoscale command to autoscale the number of pods in the azure-vote-front deployment. If average CPU utilization across all pods exceeds 50% of their requested usage, the autoscaler increases the pods up to a maximum of 10 instances. A minimum of 3 instances is then defined for the deployment:

Console
kubectl autoscale deployment azure-vote-front --cpu-percent=50 --min=3 --max=10

Alternatively, you can create a manifest file to define the autoscaler behavior and resource limits. The following is an example of a manifest file named azure-vote-hpa.yaml.

YAML
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: azure-vote-back-hpa
spec:
  maxReplicas: 10 # define max replica count
  minReplicas: 3  # define min replica count
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: azure-vote-back
  targetCPUUtilizationPercentage: 50 # target CPU utilization

---

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: azure-vote-front-hpa
spec:
  maxReplicas: 10 # define max replica count
  minReplicas: 3  # define min replica count
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: azure-vote-front
  targetCPUUtilizationPercentage: 50 # target CPU utilization

Use kubectl apply to apply the autoscaler defined in the azure-vote-hpa.yaml manifest file.

Console
kubectl apply -f azure-vote-hpa.yaml

To see the status of the autoscaler, use the kubectl get hpa command as follows:

kubectl get hpa

NAME               REFERENCE                     TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
azure-vote-front   Deployment/azure-vote-front   0% / 50%   3         10        3          2m

After a few minutes, with minimal load on the Azure Vote app, the number of pod replicas decreases automatically to three. You can use kubectl get pods again to see the unneeded pods being removed.

Manually scale AKS nodes

If you created your Kubernetes cluster using the commands in the previous tutorial, it has two nodes. You can adjust the number of nodes manually if you plan more or fewer container workloads on your cluster.

The following example increases the number of nodes to three in the Kubernetes cluster named myAKSCluster. The command takes a couple of minutes to complete.

Azure CLI
az aks scale --resource-group myResourceGroup --name myAKSCluster --node-count 3

When the cluster has successfully scaled, the output is similar to following example:

Output
"agentPoolProfiles": [
  {
    "count": 3,
    "dnsPrefix": null,
    "fqdn": null,
    "name": "myAKSCluster",
    "osDiskSizeGb": null,
    "osType": "Linux",
    "ports": null,
    "storageProfile": "ManagedDisks",
    "vmSize": "Standard_D2_v2",
    "vnetSubnetId": null
  }
Tags:

2 Comments

Previous Post Next Post