Activity:Create an EC2 instance & Install packages using Ansible

How to Create an EC2 Instance & Install Packages Using Ansible – Full DevOps Automation Guide

If you're managing AWS infrastructure, manually spinning up EC2 instances and installing software is a recipe for configuration drift and human error. In this comprehensive tutorial, I'll show you how to automate the entire process using Ansible - from provisioning AWS resources to configuring your instances with the exact packages you need. You'll learn how to create EC2 instances, manage security groups, handle SSH keys, and install software packages - all through declarative Ansible playbooks that can be version-controlled and reused across your organization.

This approach gives you several advantages over manual provisioning or even CloudFormation: better error handling, idempotency, multi-cloud support (though we're focusing on AWS here), and the ability to manage both infrastructure and configuration in one workflow. By the end of this guide, you'll have a production-ready Ansible setup that can provision and configure EC2 instances with just one command.

Prerequisites

Before we begin, ensure you have the following set up:

• A Linux or macOS machine with Ansible 2.9+ installed (tested with Ansible 2.10)
• Python 3.6+ with pip installed

• AWS account with appropriate permissions

• Basic familiarity with YAML and Ansible playbook structure
• An existing SSH key pair in your AWS region (we'll use "n.pem" in examples)

Let's verify your Ansible installation:

ansible --version
ansible 2.10.7
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0]

Step 1: Install Required Python Packages

Ansible uses the AWS SDK (boto3) to interact with AWS services. First, we need to install the necessary Python packages:

sudo apt-get update
sudo apt-get install python3-pip -y
pip3 install boto3 botocore

Verify the installation:

python3 -c "import boto3; print(boto3.__version__)"
1.26.96

Important security note: Never hardcode AWS credentials in your playbooks. We'll use IAM roles or environment variables instead. The credentials shown in the original post are for demonstration only and should be immediately rotated if they were real.

Step 2: Set Up AWS Credentials Securely

Create an IAM user with the following policies:

• AmazonEC2FullAccess
• AmazonVPCFullAccess
• AmazonRDSFullAccess (if you plan to create databases later)

For production, consider using more restrictive policies. Here's how to set up your credentials:

# Option 1: Environment variables (recommended)
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-east-1

# Option 2: AWS credentials file
mkdir -p ~/.aws
cat > ~/.aws/credentials < ~/.aws/config <

Verify your AWS credentials are working:

aws sts get-caller-identity
{
    "UserId": "AIDASAMPLEUSERID",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/ansible-user"
}

Step 3: Create the Ansible Playbook Structure

Let's create a well-organized playbook structure. We'll use a main playbook that includes separate files for different components:

mkdir -p ansible-ec2-demo/{group_vars,roles/{ec2_provision,instance_config}}
touch ansible-ec2-demo/{site.yml,group_vars/all,roles/ec2_provision/tasks/main.yml,roles/instance_config/tasks/main.yml}

Here's our main playbook (site.yml):

- name: Provision EC2 infrastructure
  hosts: localhost
  connection: local
  gather_facts: false
  roles:
    - ec2_provision

- name: Configure instances
  hosts: launched
  become: true
  gather_facts: true
  roles:
    - instance_config

Step 4: Create EC2 Instance with Ansible

Let's build the EC2 provisioning role. First, define our variables in group_vars/all:

---
# AWS Configuration
aws_region: us-east-1
vpc_subnet_id: subnet-afeab7f3
security_group: launch-wizard-1

# EC2 Configuration
instance_type: t2.micro
ami_id: ami-07d0cf3af28718ef8  # Ubuntu 20.04 LTS in us-east-1
keypair: n
instance_count: 1
instance_tags:
  Name: Demo
  Environment: dev
  Role: web

Now, create the EC2 provisioning tasks in roles/ec2_provision/tasks/main.yml:

- name: Create EC2 instance
  amazon.aws.ec2_instance:
    key_name: "{{ keypair }}"
    instance_type: "{{ instance_type }}"
    image_id: "{{ ami_id }}"
    wait: true
    region: "{{ aws_region }}"
    count: "{{ instance_count }}"
    network:
      assign_public_ip: true
      subnet_id: "{{ vpc_subnet_id }}"
    security_group: "{{ security_group }}"
    tags: "{{ instance_tags }}"
    volumes:
      - device_name: /dev/sda1
        ebs:
          volume_size: 8
          delete_on_termination: true
  register: ec2

- name: Add new instance to host group
  add_host:
    hostname: "{{ item.private_ip_address }}"
    groups: launched
    ansible_user: ubuntu
    ansible_ssh_private_key_file: "~/.ssh/{{ keypair }}.pem"
  loop: "{{ ec2.instances }}"

- name: Wait for SSH to be available
  wait_for:
    host: "{{ item.public_ip_address }}"
    port: 22
    delay: 10
    timeout: 300
    state: started
  loop: "{{ ec2.instances }}"

- name: Display instance information
  debug:
    msg: "Instance {{ item.tags.Name }} created with IP {{ item.public_ip_address }}"
  loop: "{{ ec2.instances }}"

Key improvements over the original:

• Using the newer amazon.aws.ec2_instance module instead of the deprecated ec2 module
• Better variable organization with group_vars
• Proper volume configuration
• More robust SSH waiting logic
• Better instance tagging

Step 5: Configure SSH Access

The original playbook had a manual SSH key copy step. Let's improve this by using Ansible's built-in SSH key management:

Add this to your EC2 provisioning role (roles/ec2_provision/tasks/main.yml):

- name: Ensure SSH directory exists
  file:
    path: ~/.ssh
    state: directory
    mode: '0700'

- name: Copy SSH public key to instance
  authorized_key:
    user: ubuntu
    state: present
    key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
  delegate_to: "{{ item.public_ip_address }}"
  loop: "{{ ec2.instances }}"

This approach is more secure and idempotent than the original shell command.

Step 6: Install Required Packages on the Instance

Now let's create the instance configuration role. First, ensure Python is installed (required for Ansible to manage the instance):

In roles/instance_config/tasks/main.yml:

- name: Ensure Python is installed
  raw: test -e /usr/bin/python3 || (apt -y update && apt install -y python3)
  changed_when: false

- name: Gather facts
  setup:

- name: Update apt package index
  apt:
    update_cache: yes
    cache_valid_time: 3600

- name: Install required packages
  apt:
    name:
      - apache2
      - ntp
      - python3-pip
      - software-properties-common
    state: present

- name: Ensure services are running
  service:
    name: "{{ item }}"
    state: started
    enabled: yes
  loop:
    - apache2
    - ntp

Step 7: Create a Security Group with Ansible

Let's add security group creation to our playbook. First, add these variables to group_vars/all:

security_group_name: web-sg
security_group_description: "Security group for web servers"
security_group_rules:
  - proto: tcp
    from_port: 22
    to_port: 22
    cidr_ip: 0.0.0.0/0
  - proto: tcp
    from_port: 80
    to_port: 80
    cidr_ip: 0.0.0.0/0
  - proto: tcp
    from_port: 443
    to_port: 443
    cidr_ip: 0.0.0.0/0

Then add this task to your EC2 provisioning role (roles/ec2_provision/tasks/main.yml):

- name: Create security group
  amazon.aws.ec2_group:
    name: "{{ security_group_name }}"
    description: "{{ security_group_description }}"
    region: "{{ aws_region }}"
    rules: "{{ security_group_rules }}"
    rules_egress:
      - proto: all
        cidr_ip: 0.0.0.0/0
  register: sg

- name: Update security group variable
  set_fact:
    security_group: "{{ sg.group_id }}"

Step 8: Run the Complete Playbook

Now that we have all components, let's run our playbook:

cd ansible-ec2-demo
ansible-playbook site.yml

Sample output:

PLAY [Provision EC2 infrastructure] ********************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [ec2_provision : Create security group] ***********************************
changed: [localhost]

TASK [ec2_provision : Create EC2 instance] *************************************
changed: [localhost]

TASK [ec2_provision : Add new instance to host group] **************************
changed: [localhost] => (item={'instance_id': 'i-0123456789abcdef0', ...})

TASK [ec2_provision : Wait for SSH to be available] ****************************
ok: [localhost] => (item={'public_ip_address': '54.165.123.45', ...})

TASK [ec2_provision : Display instance information] ****************************
ok: [localhost] => (item={'tags': {'Name': 'Demo', ...}}) => {
    "msg": "Instance Demo created with IP 54.165.123.45"
}

PLAY [Configure instances] *****************************************************

TASK [Gathering Facts] *********************************************************
ok: [54.165.123.45]

TASK [instance_config : Ensure Python is installed] ****************************
ok: [54.165.123.45]

TASK [instance_config : Gather facts] ******************************************
ok: [54.165.123.45]

TASK [instance_config : Update apt package index] ******************************
changed: [54.165.123.45]

TASK [instance_config : Install required packages] ****************************
changed: [54.165.123.45]

TASK [instance_config : Ensure services are running] ***************************
changed: [54.165.123.45] => (item=apache2)
changed: [54.165.123.45] => (item=ntp)

PLAY RECAP *********************************************************************
54.165.123.45              : ok=6    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
localhost                  : ok=5    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Common Pitfalls and Troubleshooting

1. Permission Errors

If you see errors like UnauthorizedOperation, check:

• Your IAM user has the correct policies attached
• Your AWS credentials are correct and not expired
• You're using the correct AWS region

Verify with:

aws iam list-attached-user-policies --user-name ansible-user

2. SSH Connection Failures

If Ansible can't connect to your instance:

• Check the security group allows inbound SSH (port 22) from your IP
• Verify the instance has a public IP (check assign_public_ip: true)
• Ensure your SSH key is correct and permissions are set properly:

chmod 600 ~/.ssh/n.pem

3. Python Not Found on Instance

Some minimal AMIs don't include Python by default. The raw module task we added handles this, but if you still see issues:

ansible launched -i inventory -m raw -a "apt update && apt install -y python3" -u ubuntu --private-key ~/.ssh/n.pem

4. Module Not Found Errors

If you see module amazon.aws.ec2_instance not found:

ansible-galaxy collection install amazon.aws

5. Instance Limit Exceeded

If you get InstanceLimitExceeded errors:

• Check your AWS account limits
• Terminate unused instances
• Request a limit increase from AWS support

How to Verify Your Setup

1. Verify EC2 Instance

Check in the AWS Console or via CLI:

aws ec2 describe-instances --filters "Name=tag:Name,Values=Demo" --query "Reservations[].Instances[].{Instance:InstanceId,State:State.Name,IP:PublicIpAddress}"

2. Verify Security Group

aws ec2 describe-security-groups --group-names web-sg --query "SecurityGroups[].IpPermissions[]"

3. Verify Package Installation

SSH into your instance and check:

ssh -i ~/.ssh/n.pem ubuntu@54.165.123.45
apache2 -v
ntpq -p

4. Verify Services

systemctl status apache2
systemctl status ntp

5. Verify Web Server

From your local machine:

curl http://54.165.123.45

You should see the Apache default page.

Key Takeaways

• Infrastructure as Code: By using Ansible playbooks, you've converted manual EC2 provisioning into repeatable, version-controlled code that can be shared across your team.
• Security Best Practices: We implemented proper credential management (never hardcoding keys), least-privilege IAM policies, and secure SSH key distribution.
• Idempotency: The playbook can be run multiple times without creating duplicate resources or causing unintended changes.
• Modular Design: By separating provisioning and configuration into different roles, we've created a maintainable structure that can be extended for other use cases.
• Verification Steps: We included comprehensive verification steps to ensure each component works as expected before moving to the next phase.

FAQ

1. How do I use a different AMI or region?

Update the ami_id and aws_region variables in group_vars/all. Find the latest Ubuntu AMI for your region with:

aws ec2 describe-images --owners 099720109477 --filters "Name=name,Values=ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*" "Name=state,Values=available" --query "sort_by(Images, &CreationDate)[-1].ImageId" --output text

2. How can I make the playbook more secure?

Several improvements you can make:

• Use AWS IAM roles instead of access keys (attach a role to your Ansible control node)
• Restrict security group rules to specific CIDR ranges
• Use Ansible Vault to encrypt sensitive variables
• Implement tag-based resource management for better cost tracking

Example of using Ansible Vault:

ansible-vault encrypt_string 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY' --name 'aws_secret_access_key'

3. How do I scale this to multiple instances?

Update the instance_count variable and add load balancer configuration:

- name: Create load balancer
  amazon.aws.elb_classic_lb:
    name: web-lb
    state: present
    region: "{{ aws_region }}"
    zones:
      - us-east-1a
      - us-east-1b
    listeners:
      - protocol: http
        load_balancer_port: 80
        instance_port: 80
    health_check:
      ping_protocol: http
      ping_port: 80
      ping_path: "/index.html"
      response_timeout: 5
      interval: 30
      unhealthy_threshold: 2
      healthy_threshold: 2
    instance_ids: "{{ ec2.instance_ids }}"

4. How do I add custom user data or cloud-init scripts?

Add the user_data parameter to your EC2 instance task:

- name: Create EC2 instance
  amazon.aws.ec2_instance:
    # ... existing parameters ...
    user_data: |
      #!/bin/bash
      echo "Hello from user data!" > /tmp/user-data.log
      apt-get update
      apt-get install -y nginx

This approach gives you complete control over your AWS infrastructure using Ansible, combining the power of infrastructure as code with configuration management in a single workflow. The playbook we've created can serve as a foundation for more complex deployments, including multi-tier applications, auto-scaling groups, and database provisioning.

🛒 Recommended gear on Amazon

Disclosure: some links above are affiliate links — if you buy through them I may earn a small commission at no extra cost to you. Thanks for supporting the channel!