AWX in Action

Dynamic Inventories: Letting the Cloud Build Your Inventory

When you have a number of hosts that change dynamically, such as an auto-scaling group in AWS, Kubernetes nodes, and so on, IP addresses and hostnames are subject to change and are not known in advance. In this case, it's not practical to update the inventory file manually every time a host is added or removed. To solve this problem, you can use dynamic inventories.

Dynamic inventories are programs that generate inventory files on the fly. They can be written in any language and can interact with any data source. The most common ones target cloud providers like AWS, Azure, Google Cloud, and OpenStack.

To understand how this special type of inventory works, we'll start by creating a new inventory called SolaraDynamicInventory. It will be empty when first created.

After creating the inventory, click on it and then click on the Sources tab. Click on the Add button to see the list of available sources. These are some of the available sources:

• Amazon Web Services EC2
• Google Compute Engine
• Microsoft Azure Resource Manager
• VMware vCenter
• Red Hat Satellite 6
• Red Hat Insights
• OpenStack
• Red Hat Virtualization
• Red Hat Ansible Automation Platform

We're going to walk through an example using the AWS EC2 source. Suppose that the Solara organization runs a number of EC2 VMs on AWS that should be managed by AWX. We'll use the AWS EC2 source to populate SolaraDynamicInventory with those EC2 instances. The flow for other cloud providers is similar: most of the time, you provide credentials for the provider, and the source fetches the list of instances and their metadata.

Create a new AWS IAM user with the following permissions: AmazonEC2ReadOnlyAccess (this policy allows the user to list all EC2 instances in the account). Create an access key for the user and save the Access Key ID and Secret Access Key.

These are the steps to follow using the AWS CLI (you can use the AWS web console if you prefer):

# Create the IAM user
aws iam create-user --user-name awx-ec2-user

# Attach the AmazonEC2ReadOnlyAccess policy to the user
aws iam attach-user-policy --user-name awx-ec2-user --policy-arn arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess

# Create an access key for the user
aws iam create-access-key --user-name awx-ec2-user > /tmp/awx-ec2-user.json

Extract the Access Key ID and Secret Access Key from the JSON file for later use:

# Print the Access Key ID
echo $(jq -r '.AccessKey.AccessKeyId' /tmp/awx-ec2-user.json)

# Print the Secret Access Key
echo $(jq -r '.AccessKey.SecretAccessKey' /tmp/awx-ec2-user.json)

Copy these values and create a new credential in AWX. Click on Credentials, then click on the Add button. Select the Amazon Web Services type and provide a name for the credentials. Let's call it AWS and paste the Access Key ID and Secret Access Key in the appropriate fields. Assign the credentials to the Solara organization.

We'll create two Ansible-managed EC2 nodes on AWS. We're going to tag the first machine with the following tags:

• Name: "mysql-machine"
• Role: "mysql"

We're also going to tag the second machine with the following tags:

• Name: "web-machine"
• Role: "web"

First, create an SSH key pair. The following command creates a key pair and saves the private key to a file:

aws ec2 create-key-pair \
    --key-name MyKeyPair \
    --region $AWS_REGION \
    --query 'KeyMaterial' --output text > MyKeyPair.pem

Change "MyKeyPair" to a name of your choice. The private key will be saved as "MyKeyPair.pem". Make sure to set the correct permissions for the key file:

chmod 400 MyKeyPair.pem

To manage the machines, Ansible needs to connect to them using SSH. To do this, we need to copy the public key to the control node if it hasn't been done already.

scp MyKeyPair.pem root@$AWX1_PUBLIC_IP:~/.ssh/

Now, proceed with AWS and create a security group that allows SSH access from all IP addresses:

aws ec2 create-security-group \
  --group-name MySecurityGroup \
  --description "Security group for SSH access"

Modify the security group to allow inbound SSH traffic (port 22) from all IP addresses:

aws ec2 authorize-security-group-ingress \
  --group-name MySecurityGroup \
  --protocol tcp \
  --port 22 \
  --cidr 0.0.0.0/0

If you want to allow SSH access only from AWX, you should adapt the CIDR block to the IP address of the AWX machine and any other execution nodes.

We're going to use Ubuntu 24.04 on both machines, so start by finding the AMI ID for Ubuntu 24.04 on Ubuntu's cloud images.

Export the AMI ID to an environment variable, as well as the AWS region you're using (e.g., eu-west-1):

export AMI_ID=[THE_AMI_ID]
export AWS_REGION=[THE_AWS_REGION]

In my case, I'm using the region eu-west-1 and the AMI ID is ami-00983e8a26e4c9bd9. This is how the export command looks:

export AMI_ID=ami-09a477b0776212c5c
export AWS_REGION=eu-west-1

Find a subnet, as well as a security group ID by name (or export SUBNET_ID, and SG_ID manually):

# Common flags reused across all AWS calls
region_args="--region $AWS_REGION --output text"

# Find a VPC, or create a default one if none exist.
export VPC_ID=$(aws ec2 describe-vpcs --query 'Vpcs[0].VpcId' $region_args)
if [ -z "$VPC_ID" ] || [ "$VPC_ID" = "None" ]; then
    aws ec2 create-default-vpc --region "$AWS_REGION"
    export VPC_ID=$(aws ec2 describe-vpcs --query 'Vpcs[0].VpcId' $region_args)
fi

# Find a subnet in that VPC.
export SUBNET_ID=$(aws ec2 describe-subnets \
    --filters "Name=vpc-id,Values=$VPC_ID" \
    --query 'Subnets[0].SubnetId' $region_args)

# Find the security group ID by name, or create it if missing.
export SG_ID=$(aws ec2 describe-security-groups \
    --filters "Name=group-name,Values=MySecurityGroup" "Name=vpc-id,Values=$VPC_ID" \
    --query 'SecurityGroups[0].GroupId' $region_args)

# Check whether a security group called MySecurityGroup exists in the VPC; 
# if not, it creates one and opens port 22 (SSH) to the world.
if [ -z "$SG_ID" ] || [ "$SG_ID" = "None" ]; then
    export SG_ID=$(aws ec2 create-security-group \
        --group-name MySecurityGroup \
        --description "AWX lab security group" \
        --vpc-id $VPC_ID \
        --region $AWS_REGION \
        --query 'GroupId' --output text)

    aws ec2 authorize-security-group-ingress \
        --group-id $SG_ID \
        --protocol tcp --port 22 --cidr 0.0.0.0/0 \
        --region $AWS_REGION
fi

Set the shared launch arguments once:

# Common arguments reused for every instance launch
common_args="--image-id $AMI_ID --count 1 --instance-type t2.micro \
  --key-name MyKeyPair --security-group-ids $SG_ID \
  --subnet-id $SUBNET_ID --region $AWS_REGION"

Launch the MySQL machine using the following command:

aws ec2 run-instances $common_args \
  --tag-specifications \
  'ResourceType=instance,Tags=[{Key=Name,Value=mysql-machine},{Key=Role,Value=mysql}]'

Launch the web machine using the following command:

aws ec2 run-instances $common_args \
  --tag-specifications \
  'ResourceType=instance,Tags=[{Key=Name,Value=web-machine},{Key=Role,Value=web}]'

You can check the status of the instances using the following command (install jq if not already installed using apt-get install jq):

aws ec2 describe-instances --region $AWS_REGION | \
jq '.Reservations[].Instances[] | {
      Name: (if (.Tags[]? | select(.Key == "Name")) then 
                (.Tags[] | select(.Key == "Name") | .Value) 
             else 
                "N/A" 
             end),
      State: .State.Name, 
      IP: (if .PublicIpAddress then 
              .PublicIpAddress 
           else 
              "N/A" 
           end)
    }'

You should see something like this:

{
  "Name": "mysql-machine",
  "State": "running",