Cloud Architectural Diagrams

I have discovered this new simple tool for drawing architectural diagrams for aws cloud environment. The sample diagram is below and the tool i used is free and the link is

cloudcraft - Web App Reference Architecture


Nishanth Ponukumatla




This is a bash script which works perfectly for AMAZON Linux.

Please make sure to have port 443 opened as it communicates with the aws API to make request calls and also appropriate IAM Role.

This works only if it is deployed on the instance which needs to be backed up. Or you can manually define the MY_INSTANCE_ID variable.

The Linux cron offers multiple possibilities to schedule your scripts, for example: hourly, daily, weekly, monthly … or custom. I’ve chosen to schedule execution of our scripts on a daily basis. If you’re using AWS Linux, look at /etc/anacrontab and notice that all scripts in the /etc/crontab directory will run daily.

Backup Script.

#source /etc/profile <– I have commented that as it is pulling the aws cred’s from a env variable. –>
#set -x o  <– if you need a VERBOSE output you can uncomment this. –>
if [ -z $AWS_REGION ];

# Get instance-id from metadata
MY_INSTANCE_ID=`wget -q -O-`

# Get active/in-use volume-id for current instance-id
aws ec2 describe-volumes –region $AWS_REGION –filters Name=attachment.instance-id,Values=$MY_INSTANCE_ID | awk ‘{ print $9 }’ > /backup/volumelist

VOLUME_LIST=`cat /backup/volumelist`
# Create snapshot
echo “Create EBS Volume Snapshot – Process started at $(date +%m-%d-%Y-%T)”
echo ”
echo ‘—————–‘

DATE=$(date +’%Y%m%d_%H%M’)

for volume in $(echo $VOLUME_LIST); do
echo ‘Creating snapshot for volume: $volume with description: $DESC’
echo ”
echo $volume
aws ec2 create-snapshot –region $AWS_REGION –volume-id $volume –description $DESC
echo ”

# Describe the snapshot just created ##OPTIONAL##
#ec2-describe-snapshots -region $AWS_REGION –filter tag-value=$DESC


CLOUDWATCH_OPTS=”–namespace <Name of the snapshot custom log >–dimensions InstanceId=$MY_INSTANCE_ID”
aws cloudwatch put-metric-data –metric-name “Name of the metric” –value “0” –unit “Count” $CLOUDWATCH_OPTS

echo “******* Ran backup @ $(date)”
echo ‘Completed’

exit 0

I have also included a cloudwatch script which sends a value of “1” each time it runs a backup.

Retention Script

Please define the RETENTION value below as how many days you want it to be.

#source /etc/profile
if [ -z $AWS_REGION ];
#set -x o
# Get instance-id from metadata
MY_INSTANCE_ID=`wget -q -O-`
# Retention in days
###Define the retention value below for how many ever days you want to keep the snapshot.”

# Dates
datecheck_7d=`date +%Y-%m-%d –date “$RETENTION days ago”`
datecheck_s_7d=`date –date=”$datecheck_7d” +%s`
datenow=`date +%Y-%m-%d-%H:%M:%S`

# Get active/in-use volume-id for current instance-id
aws ec2 describe-volumes –region $AWS_REGION –filters Name=attachment.instance-id,Values=$MY_INSTANCE_ID | awk ‘{ print $9 }’ > /backup/volumelist

VOLUME_LIST=`cat /backup/volumelist`

# Analyzing snapshot
echo ”
echo ‘—————–‘

for volume in $(echo $VOLUME_LIST); do
echo ‘Analyzing snapshot(s) for volume: $volume’
echo ”
aws ec2 describe-snapshots –region $AWS_REGION –filters Name=volume-id,Values=”$volume” > /tmp/${volume}-snapshots
num_snapshots=`cat /tmp/${volume}-snapshots | wc -l`
echo ‘Found $num_snapshots snapshot(s) for volume $volume to be analyzed’

if (( $num_snapshots > 0 ));
while read line
snapshot_name=`echo $line | awk ‘{print $2}’`
snapshot_id=`echo $line | awk ‘{print $6}’`
datecheck_old=`echo $line | awk ‘{print $7}’ | awk -F ‘T’ ‘{print $1}’`
datecheck_s_old=`date –date=”$datecheck_old” +%s`

# Check if snapshot is older than retention days
if (( $datecheck_s_old <= $datecheck_s_7d ));
echo ‘Deleting snapshot $snapshot_name … older than $RETENTION days’
aws ec2 delete-snapshot –region $AWS_REGION –snapshot-id $snapshot_id
echo ‘Snapshot $snapshot_name OK!’
done < /tmp/${volume}-snapshots
echo ”
echo ‘### no snapshots available for volume $volume’


echo ‘******* Ran retention check @ $(date)’
echo ‘Completed’

exit 0

Creating a cronjob for it to run everday at 10pm.

Open /etc/crontab or crontab -e  and add this.

0 22 * * * /backup/ >> /backup/backuplog.log 2>&1
0 22 * * * /backup/ >> /backup/retentionslog.log 2>&1

Thank you , hope this was helpful.


Nishanth Ponukumatla

Build your own secure mail server on the cloud using Amazon Web Services.

Build your own secure mail server on the cloud using Amazon Web Services.

Services needed by amazon in order to setup the initial layout:

EC2 ( Elastic Load balancing)




Applications needed by Ec2 Instances:

Ubuntu 14.4 ( Base OS  for all the ec2 instances)





Here is the architecture of the mail server.


Build your own secure mail server on the cloud using Amazon Web Services

Setting up the MySQL RDS | Postfix server.

First we  launch an RDS instance , I used mysql as it integrates better with postfix.


  1. Select Multi-AZ deployment as it will be more Highly Available.2
  1. Make it a publicly NOT available


3.After launching the RDS instance , make sure to note the “ENDPOINT”  of the mysql RDS instance.

Setting up the Postfix Server.

1.I chose a R3.large Ubuntu 14.4 instance , as it is high on networking performance and has enhanced networking enabled.

The link above gives us a detailed explanation on how to setup a postfix server with mysql backend.

In the instructions in the link above he uses a locally setup mysql server, where as in our case we are using a RDS instance hence where ever he uses to define the mysql server , we replace it with the “HOSTNAME” of the RDS instance which we had noted down earlier.

I am specifying on the hostname as we are using a private RDS instance and the IP ADDRESS will change periodically by amazon.

Once you setup your postfix server we have little more modification, in order to setup relaying.

Relay Configuration

Since we are using a relay server we have to add the following to the postfix file.


enable_original_recipient = no

relayhost = []:587

smtp_sasl_auth_enable = yes

smtp_sasl_security_options = noanonymous

smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

smtp_use_tls = yes

smtp_tls_security_level = encrypt

smtp_tls_note_starttls_offer = yes

  As we can see a it is pointing towards a  “sasl_passwd” file where we are supposed to load credentials.

Please do not mistake these credentials with your regular iam credentials , we are supposed to generate them using amazon SES. You can do it by following the steps below.

  1. Open amazon SES
  2. Open Smtp Settings and select Create My Smtp Credentials
  1. After clicking on create you will have your SES SMTP credentials
  2. Copy access ID and secret Key Id into the SASL_Passwd file.
  3. If the sasl_passwd file doesn’t exist please create it.
  4. On your mail server, open thecf file. On many systems, this file resides in the/etc/postfix folder.
  5. Comment out the following line of thecf file by putting a # in front of it:

 -o smtp_fallback_relay=

Save and close the file.

  1. Edit the/etc/postfix/sasl_passwd  If the file does not exist, create it. Add the following lines to the file, replacing USERNAME and PASSWORD with your SMTP user name and password. If Postfix cannot authenticate with the Amazon SES SMTP endpoint because the hostname does not match, try adding the additional line specified in Amazon SES SMTP Issues.


Use your SMTP user name and password, not your AWS access key ID and secret access key. Your SMTP credentials and your AWS credentials are not the same. For information about how to obtain your SMTP credentials, see Obtaining Your Amazon SES SMTP Credentials.


Save and close the sasl_passwd file.

  1. At a command prompt, issue the following command to create a hashmap database file containing your SMTP credentials.

sudo postmap hash:/etc/postfix/sasl_passwd

  1. (Optional but recommended) Remove the/etc/postfix/sasl_passwd
  2. (Optional but recommended) The/etc/postfix/sasl_passwd and /etc/postfix/sasl_passwd.dbfiles you created in the previous steps are not encrypted. Because these files contain your SMTP credentials, it is a good idea to use the following commands to change the owner to root and set permissions to restrict access to the files as much as possible. (Note that if you deleted /etc/postfix/sasl_passwd in the previous step, you should omit it from the commands below.)

sudo chown root:root /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db

sudo chmod 0600 /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db

  1. Tell Postfix where to find the CA certificate (needed to verify the Amazon SES server certificate). You could use a self-signed certificate or you could use default certificates as follows:

If running on the Amazon Linux AMI:

sudo postconf -e ‘smtp_tls_CAfile = /etc/ssl/certs/ca-bundle.crt’

If running on Ubuntu Linux:

sudo postconf -e ‘smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt’

  1. When you have finished updating the configuration, stop and start Postfix by typing the following at the command line:

sudo postfix stop

sudo postfix start

  1. Send a test email by typing the following at a command line, pressing Enter after each line. Note that you must with your “From” email address, which you must have previously verified with Amazon SES. Replace with your “To” address. If your account is still in the sandbox, the “To” address must also be verified. Also note that the final line is a single period.

sendmail -f


Subject: Test

This email was sent through Amazon SES!


  1. Check your inbox for the email. If the message was not delivered, check your Junk box, and then check your system’s mail log (typically/var/log/maillog) for errors. For example, you will get an “Email address not verified” error if you have not verified the “From” address that follows “-f” on the command line.

Now, once the postfix server is setup we are done with 80% of the work.

Setting up the Proxy Servers.

The main reason of setting up the proxy server is , it Que’s the incoming mails before it actually hits the mail server. This helps us , if in case our “postfix server” is down for any reason , the incoming mail will be present in the proxy server que , this helps us in a lot of ways as there is not loss of mail.

  1. I chose a m3.medium for proxy servers , as I get a average traffic (~10,000 mails per day)
  2. Please do realize this is only for the incoming mail.
  3. After launching your instance, install latest nginx version (nginx/1.8.0) with the mail module.
  4. You can type apt-get install nginx* and it will show you all the nginx modules and you can choose the mail module.
  5. After install nginx with mail module , open /etc/nginx/nginx.conf
  6. Paste the following configuration , and modify accordingly.

user www-data;

worker_processes  1;

error_log  logs/error.log  info;

events {

  worker_connections  1024;

  multi_accept on;


mail {



        auth_http localhost:8008/auth-smtppass.php;


        server {

                listen <ipaddr of the current server>:25;

                protocol smtp;

                timeout 5s;

                proxy on;

                xclient off;

                smtp_auth none;



http {

   log_format main

                ‘$remote_addr – $remote_user [$time_local] ‘

                ‘”$request” $status $bytes_sent ‘

                ‘”$http_referer” “$http_user_agent” ‘


server {


                server_name localhost;

                access_log /var/log/nginx/localhost.access_log main;

                error_log /var/log/nginx/localhost.error_log info;


                root /var/www/localhost/htdocs;


                location ~ .php$ {

                        add_header Auth-Server <ipaddr of the mailserver>;

                        add_header Auth-Port 25;

                        return 200;




  1. Please make sure to change whatever is highlighted in red to its respective IPaddress’s
  2. After loading the configuration do a “ nginx –t ” if the configuration is okay.
  3. Start the nginx server.

You can launch the second proxy server by following the same steps above.

Setting up the load balancer.

  1. Create an external facing load balancer and add the two Reverse Proxy instance’s to the load balancer.
  2. Make sure to make the load balancer listen on port 25 and forward requests to the proxy server on port 25.


  1. Once the load balancer is created please take the DNS NAME info , which is present in the description of the load balancer.

Setting up Route 53.

Create a MX record for your domain and point the MX record to the External load balancer which we created in the previous step.

We are all done !

You can start testing it by sending and receiving some emails.

To make it more redundant, you can deploy the same setup in other region as Ireland , Frankfurt etc..

And you can do a dns failover in the route53 , which would take around 300 seconds to switch to the other region which could result in a loss of mail sometimes.

You can set it up in any region depending on the edge data or whichever region has the most traffic.


Joel Nishanth Ponukumatla