AUTOMATE SNAPSHOTS FOR YOUR EC2 VOLUMES

This is a bash script which works perfectly for AMAZON Linux.

Please make sure to have port 443 opened as it communicates with the aws API to make request calls and also appropriate IAM Role.

This works only if it is deployed on the instance which needs to be backed up. Or you can manually define the MY_INSTANCE_ID variable.

The Linux cron offers multiple possibilities to schedule your scripts, for example: hourly, daily, weekly, monthly … or custom. I’ve chosen to schedule execution of our scripts on a daily basis. If you’re using AWS Linux, look at /etc/anacrontab and notice that all scripts in the /etc/crontab directory will run daily.

Backup Script.

#!/bin/bash
#source /etc/profile <– I have commented that as it is pulling the aws cred’s from a env variable. –>
#set -x o  <– if you need a VERBOSE output you can uncomment this. –>
if [ -z $AWS_REGION ];
then
AWS_REGION=eu-west-1
fi

#
# Get instance-id from metadata
#
MY_INSTANCE_ID=`wget -q -O- http://169.254.169.254/latest/meta-data/instance-id`

#
# Get active/in-use volume-id for current instance-id
#
aws ec2 describe-volumes –region $AWS_REGION –filters Name=attachment.instance-id,Values=$MY_INSTANCE_ID | awk ‘{ print $9 }’ > /backup/volumelist
sync

VOLUME_LIST=`cat /backup/volumelist`
#
# Create snapshot
#
echo “Create EBS Volume Snapshot – Process started at $(date +%m-%d-%Y-%T)”
echo ”
echo $VOLUME_LIST
echo ‘—————–‘

DATE=$(date +’%Y%m%d_%H%M’)

for volume in $(echo $VOLUME_LIST); do
DESC=’EBS_Automated_Snapshot_’$DATE
echo ‘Creating snapshot for volume: $volume with description: $DESC’
echo ”
echo $volume
aws ec2 create-snapshot –region $AWS_REGION –volume-id $volume –description $DESC
echo ”

# Describe the snapshot just created ##OPTIONAL##
#ec2-describe-snapshots -region $AWS_REGION –filter tag-value=$DESC

#####CLOUWATCH CONFIG####OPTIONAL#####

CLOUDWATCH_OPTS=”–namespace <Name of the snapshot custom log >–dimensions InstanceId=$MY_INSTANCE_ID”
aws cloudwatch put-metric-data –metric-name “Name of the metric” –value “0” –unit “Count” $CLOUDWATCH_OPTS
done

echo “******* Ran backup @ $(date)”
echo ‘Completed’

exit 0

I have also included a cloudwatch script which sends a value of “1” each time it runs a backup.

Retention Script

Please define the RETENTION value below as how many days you want it to be.

#!/bin/sh
#source /etc/profile
if [ -z $AWS_REGION ];
then
AWS_REGION=eu-west-1
fi
#set -x o
#
# Get instance-id from metadata
#
MY_INSTANCE_ID=`wget -q -O- http://169.254.169.254/latest/meta-data/instance-id`
#
# Retention in days
###Define the retention value below for how many ever days you want to keep the snapshot.”
RETENTION=’14’

# Dates
datecheck_7d=`date +%Y-%m-%d –date “$RETENTION days ago”`
datecheck_s_7d=`date –date=”$datecheck_7d” +%s`
datenow=`date +%Y-%m-%d-%H:%M:%S`

#
# Get active/in-use volume-id for current instance-id
#
aws ec2 describe-volumes –region $AWS_REGION –filters Name=attachment.instance-id,Values=$MY_INSTANCE_ID | awk ‘{ print $9 }’ > /backup/volumelist
sync

VOLUME_LIST=`cat /backup/volumelist`

#
# Analyzing snapshot
#
echo ”
echo $VOLUME_LIST
echo ‘—————–‘

for volume in $(echo $VOLUME_LIST); do
echo ‘Analyzing snapshot(s) for volume: $volume’
echo ”
aws ec2 describe-snapshots –region $AWS_REGION –filters Name=volume-id,Values=”$volume” > /tmp/${volume}-snapshots
num_snapshots=`cat /tmp/${volume}-snapshots | wc -l`
echo ‘Found $num_snapshots snapshot(s) for volume $volume to be analyzed’

if (( $num_snapshots > 0 ));
then
while read line
do
snapshot_name=`echo $line | awk ‘{print $2}’`
snapshot_id=`echo $line | awk ‘{print $6}’`
datecheck_old=`echo $line | awk ‘{print $7}’ | awk -F ‘T’ ‘{print $1}’`
datecheck_s_old=`date –date=”$datecheck_old” +%s`

# Check if snapshot is older than retention days
if (( $datecheck_s_old <= $datecheck_s_7d ));
then
echo ‘Deleting snapshot $snapshot_name … older than $RETENTION days’
aws ec2 delete-snapshot –region $AWS_REGION –snapshot-id $snapshot_id
else
echo ‘Snapshot $snapshot_name OK!’
fi
done < /tmp/${volume}-snapshots
else
echo ”
echo ‘### no snapshots available for volume $volume’
fi

done

echo ‘******* Ran retention check @ $(date)’
echo ‘Completed’

exit 0

Creating a cronjob for it to run everday at 10pm.

Open /etc/crontab or crontab -e  and add this.

0 22 * * * /backup/backup.sh >> /backup/backuplog.log 2>&1
0 22 * * * /backup/retention.sh >> /backup/retentionslog.log 2>&1

Thank you , hope this was helpful.

Credits:

Nishanth Ponukumatla

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s