Python for DevOps: Getting Started with Boto3

1. What is Boto3?

Boto3 is the AWS SDK for Python. It allows us to write Python programs that communicate with AWS services such as Amazon EC2, Amazon S3, IAM, CloudWatch, Lambda, DynamoDB, and many others.

In DevOps, this is very useful because instead of performing every task manually in the AWS Console, we can automate AWS operations from Python. For example, a script can list EC2 instances, upload a file to S3, create an IAM resource, read CloudWatch information, or check which AWS identity is currently being used.

import boto3

After importing Boto3, we normally create a client or a resource for the AWS service we want to work with.

ec2 = boto3.client("ec2")
s3 = boto3.resource("s3")

Boto3 and Botocore

Boto3 is built on top of Botocore. A simple way to understand the relationship is:

Botocore handles much of the low-level work required to communicate with AWS. It knows how to build AWS API requests, sign them with credentials, send them to AWS, and turn the responses back into Python data structures.

Boto3 builds on top of that foundation and gives Python developers convenient features such as clients, resources, sessions, collections, waiters, and paginators.

Botocore is also used by AWS command-line tooling. In particular, AWS CLI version 1 is built using Botocore.

Source code: boto/boto3

Botocore source code: boto/botocore

2. Understanding a Boto3 Session

A Boto3 session stores configuration that Boto3 can use when it connects to AWS. This can include credentials, the AWS Region, and the AWS profile that should be used.

session = boto3.session.Session()

This creates a Session object. If we do not specify a profile name or credentials here, Boto3 follows its normal credential and configuration lookup process. In a typical local setup, that often means using the default AWS profile when one is configured.

What resources are available?

session = boto3.session.Session()

print(session.get_available_resources())

Example output:

['cloudformation', 'cloudwatch', 'dynamodb', 'ec2', 'glacier', 'iam', 's3', 'sns', 'sqs']

This is a short list because a Boto3 resource is a higher-level, object-oriented interface. Only some AWS services have this resource interface.

For example, with an S3 resource we can work with Python objects such as buckets and objects instead of only calling low-level API operations.

s3 = session.resource("s3")

for bucket in s3.buckets.all():
    print(bucket.name)

What services are available?

print(session.get_available_services())

This returns a much larger list. These are the AWS services for which Botocore/Boto3 can create low-level clients in the installed version of the SDK.

A shortened example might look like:

['accessanalyzer', 'acm', 'apigateway', 'athena', 'autoscaling',
 'bedrock', 'cloudformation', 'cloudwatch', 'dynamodb', 'ec2',
 'ecr', 'ecs', 'eks', 'iam', 'lambda', 'logs', 'rds', 's3',
 'secretsmanager', 'sns', 'sqs', 'ssm', 'sts', ...]

The exact list can change as AWS adds services and as newer versions of Boto3 and Botocore are released. That is why it is usually better to run the command in your own environment instead of memorizing the list.

Resources vs Services

Client or Resources

import boto3

ec2 = boto3.resource('ec2')

for instance in ec2.instances.all():
    print(instance.id, instance.state['Name'])

ec2_client = boto3.client('ec2')

response = ec2_client.describe_instances()

for reservation in response['Reservations']:
    for instance in reservation['Instances']:
        print(instance['InstanceId'], instance['State']['Name'])

3. Check Which AWS Identity Boto3 Is Using

Before running automation against AWS, one of the most useful checks is to confirm which AWS identity your Python program is using. This is especially important when you have multiple AWS accounts or profiles.

sts = boto3.client("sts")

identity = sts.get_caller_identity()
print(identity)

Here, we create a low-level client for AWS Security Token Service (STS) and call get_caller_identity(). This operation tells us the identity associated with the credentials currently being used.

A simplified response looks like this:

{
    'UserId': 'AIDASAMPLEUSERID',
    'Account': '123456789012',
    'Arn': 'arn:aws:iam::123456789012:user/example-user',
    'ResponseMetadata': {...}
}

Understanding the output

For day-to-day DevOps troubleshooting, the three fields we usually care about most are UserId, Account, and Arn. ResponseMetadata is generally more useful when debugging an API request.

A useful point about STS GetCallerIdentity is that AWS does not require explicit permission for this operation. That makes it a convenient identity check when troubleshooting credentials.

A cleaner way to print the important fields

import boto3

sts = boto3.client("sts")
identity = sts.get_caller_identity()

print("Account:", identity["Account"])
print("ARN:", identity["Arn"])
print("User ID:", identity["UserId"])

4. Default Session vs Custom Session

This becomes very important when your laptop or automation environment contains more than one AWS profile.

Default session

session = boto3.session.Session()

Here we create a session without specifying a profile. Boto3 uses its normal configuration and credential lookup process. If your environment is configured to use the default profile, this session will normally use that profile.

You will often use this when your machine or runtime has only one intended AWS identity, or when the identity is already provided by the environment, such as an IAM role on EC2, ECS, or another AWS runtime.

Custom session using a named profile

session_custom = boto3.session.Session(profile_name="abc")

Here we explicitly tell Boto3 to use the AWS profile named abc. Boto3 looks for that profile in the shared AWS configuration and credential files.

A local credentials file might look like this:

[default]
aws_access_key_id = ...
aws_secret_access_key = ...

[abc]
aws_access_key_id = ...
aws_secret_access_key = ...

Now the two sessions can use different AWS identities:

default_session = boto3.session.Session()
abc_session = boto3.session.Session(profile_name="abc")

default_sts = default_session.client("sts")
abc_sts = abc_session.client("sts")

print("Default:", default_sts.get_caller_identity()["Arn"])
print("ABC profile:", abc_sts.get_caller_identity()["Arn"])

Simple comparison

The key idea is that a Session is a configuration context. Once we have a session, the clients and resources created from that session inherit its credentials, Region, and related configuration.

5. Practical DevOps Example: Verify the Account Before Automation

Imagine a script that is about to stop EC2 instances or modify infrastructure. Before doing anything destructive, we can first confirm the AWS account and identity being used.

import boto3

session = boto3.session.Session(profile_name="abc")
sts = session.client("sts")

identity = sts.get_caller_identity()

print("AWS Account:", identity["Account"])
print("AWS Identity:", identity["Arn"])

# After verification, create the service client
ec2 = session.client("ec2")

This is a good DevOps habit because the same laptop may contain credentials for development, staging, and production accounts. Checking the identity first reduces the chance of running automation against the wrong account.

6. Key Takeaways

• Boto3 is the AWS SDK for Python and is commonly used to automate AWS from Python scripts.
• Boto3 is built on Botocore, which handles much of the low-level AWS API communication.
• A Boto3 Session stores configuration such as credentials, Region, and profile information.
• get_available_resources() shows the smaller set of higher-level Boto3 resource interfaces.
• get_available_services() shows the much larger set of services that can be accessed through low-level clients.
• STS get_caller_identity() is one of the easiest ways to verify which AWS account, user, or role your script is using.
• Use Session() when you want normal/default credential resolution, and Session(profile_name="abc") when you explicitly want a named AWS profile.

References

• Boto3 Documentation
• Boto3 Sessions
• Boto3 Credentials
• AWS STS GetCallerIdentity
• Boto3 GitHub Repository
• Botocore GitHub Repository