Amazon ec2 presents a true virtual computing environment, allowing you to use web service interfaces to requisition machines for use, load them with your custom application environment, manage your network’s access permissions, and run your image using as many or few systems as you desire.

Pre-requisites :

1)Account for amazon S3 and ec2 .
2)Command-line tool (ec2-ami-tool).The AMI Tools are a set of command line utilities for creating,bundling and uploading AMIs to S3 storage where they can be used by ec2.

a). Download ec2-ami-tool from the below mentioned URL:
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351
Select rpm for linux and zip for windows.

b). Installation : To install the AMI Tools unzip the installation file into a suitable installation directory
such as ‘/usr/local’:

unzip ec2-ami-tools-X.X-XXXX.zip -d
This will create the directory ‘/ec2-ami-tools-X.X-XXXX’
where X.X-XXXX represents the release and build numbers.
Before running the utilities the ec2_HOME environment variable needs to be set:

export ec2_HOME=/ec2-ami-tools-X.X-XXXX

Running and authorizing an instance :

1). Sign up for Amazon S3 & ec2 .
2). Create a work directory and there download and unzip Command-Line Tools(ec2-ami-tools.zip) .
3). Create a new X.509 Certificate to generate a PEM encoded signed X.509 certificate (cert-*.pem) and an unencrypted, PEM encoded RSA private key that corresponds to the X.509 certificate (pk-*.pem) and save the certificate and private key in the work directory.
4). Setup environment variables by creating a batch file with the following configuration :

set JAVA_HOME =
set EXTRACT_DIR =
set ec2_HOME = \
set PATH = \bin
set ec2_PRIVATE_KEY = \PRIVATE_KEY_NAME
set ec2_CERT = \CAERTIFICATE_KEY_NAME

copy this batch file onto your work directory and execute it.

5). Run the command > “ec2-describe-images” : it will return a list of available images.
(on a mac you must set JAVA_HOME by doing: export JAVA_HOME=/home/foobar/jdk1.6.0 )

Output will be something like this :
IMAGE ami-68ae4b01 ec2-public-images/fedora-core4-base.manifest 206029621532 available public
Description of the above said command :

Record Type Identifier – IMAGE
Image Identifier – ami-68ae4b01
Manifest Location – ec2-public-images/fedora-core4-base.manifest
User Identifier(Who has registered the image) – 206029621532
Image Status – available
Visibility – public

6). Create a Keypair : “ec2-add-keypair ”

In order to launch an AMI we need a named key pair. The name of the key pair is specified in the web service call that launches the instance and the private key is used by ssh to authenticate for the same.

Use this private key that is hence created, save it to a file called id_rsa-gsg-keypair.
Now set it’s permissions to be rw——- (chmod 600 id_rsa-gsg-keypair).And copy this keypair file onto your work directory

7). Create a group : “ec2-add-group group_name -d ‘description_of_group’ ”

Security is important for any machine hooked up to any network at any time .Amazon’s ec2 cloud has a built-in firewall capability that is easy to setup. It is based on the concept of security groups. A security group is simply a group of access rules that apply to any instance which is a member of that group.

8). Create and Authorize a Security group to HTTP :
Follow the below mentioned commands for opening specific ports for ssh & http respectively.
ec2-authorize default -p 22 (ssh)
ec2-authorize default -p 80 (and http)

9). Now we have done the basics, here is the final step to get an instance up and running on Cloud.
Create an instance of your machine : ec2-run-instances ami-xxxxxxxx -k gsg-keypair

Where ami-xxxxxxxxx is the id of the machine you used to as in step 5 with ec2-describe-images.If this step failed and reported creation of the instance something like this as seen below :

So output will something like this :

INSTANCE i-xxxxxxxx ami-xxxxxxxx gsg-keypair

Description of the o/p of the above command :

Output Type Identifier – INSTANCE
Instance ID – i-xxxxxxxx
AMI ID – ami-xxxxxxxx
Key name – gsg-keypair
Hence ,wait for it,

Using ec2-describe-instances you can check the status of your instance, it will display the desired details as mentioned below:

INSTANCE i-xxxxxxxx ami-xxxxxxxx domU-##-##-##-##-##-##.usma1.compute.amazonaws.com running gsg-keypair

Description of the o/p of the above command :

Output Type Identifier – INSTANCE
Instance ID – i-xxxxxxxx
AMI ID – ami-xxxxxxxx
Public DNS name – domU-##-##-##-##-##-##.usma1.compute.amazonaws.com
Instance state – running
Key name – gsg-keypair

10). Url to access your application installed on ec2 instance is : http://publicDNSname/appname

For example : http://domU-##-##-##-##-##-##.usma1.compute.amazonaws.com/mworks

You can ssh into the machine using the key you created in step 6

ssh -i id_rsa-gsg-keypair root@domU-##-##-##-##-##-##.usma1.compute.amazonaws.com

and user will view the o/p as seen below :

__| __|_ )
_| ( /
___|\___|___|

Welcome to an ec2 Public Image 🙂

S3(Simple storage service) steps & configuration :

How to take the backup of your hosted instance(s) as an Private Image for storing it on Amazon S3(Simple Storage Service)

Taking backup of instance means storing the complete instance as an image in S3(Simple Storage Service),for this first we need to create a folder called ec2 on your instance with the following details:

1) ec2-api-tools-1.2-13740
2) keypair_name.ppk
3) keypair_file
4) setup.sh(batch file for setting the Environment variables)
5) certificate
6) private key

one more batch file Setup.sh is required(For bundling up the instance as an image and upload onto S3 in a Bucket) .
Setup.sh should contain following details:
#!bin/sh

export JAVA_HOME = java_home_location
export EXTRACT_DIR = location_of_ec2_folder
export ec2_HOME = EXTRACT_DIR/ec2-api-tools-1.2-13740
export PATH = ec2_HOME/bin
export ec2_PRIVATE_KEY = EXTRACT_DIR/private-key
export ec2_CERT = EXTRACT_DIR/certificate

After setting up the environment variables , provide all permissions using this command – chmod +x setup.sh .And then execute this .sh file as mentioned below:
> ./setup.sh.

If it doesn’t execute properly then it may be possible that it is not in proper unix format, hence execute the below mentioned command:
> dos2unix setup.sh setup.sh

the above said command formats your .sh file in proper unix format.Now we can execute all ec2-api commands hereafter on the hosted instance.

We need one more called jSh3ll(The jSh3ll is a Java based command shell for managing your Amazon S3 objects) on the hosted instance. And one more batch_file.sh will contain following details:

#!/bin/sh
echo “Starting to script”
current_date=`date ‘+%Y-%m-%d-%H-%M-‘`
export JAVA_HOME = jdk location
export EXTRACT_DIR = location_of_ec2_folder
export ec2_HOM = EXTRACT_DIR/ec2-api-tools-1.2-13740
export PATH = ec2_HOME/bin/JAVA_HOME/bin
export ec2_PRIVATE_KEY= EXTRACT_DIR/private_key
export ec2_CERT = EXTRACT_DIR/certificate

export jShellDir=/usr/ami/jSh3ll/(location of jSh3ll)
s3Dir=$jShellDir$current_date
s3inputCommands=$s3Dir”s3commands.txt”
s3output=$s3Dir”s3output.txt”

echo “bucket “$current_date”myappS(bucket_name)” > $s3inputCommands
echo “createbucket private” >> $s3inputCommands
echo “quit” >> $s3inputCommands

cat $s3inputCommands

$JAVA_HOME/bin/java -jar $jShellDir/dist/jSh3ll.jar -h s3.amazonaws.com -u AWS_ACCESS_KEY -p AWS_SECRET ACCESS KEY -i $s3inputCommands -o
$s3Dir”s3output.txt”
echo “create bucket output”
cat $s3Dir”s3output.txt”
echo “after creating bucket: “‘date’

currentDir=/usr/ami/
amiDir=/mnt/backup/
#mkdir “/usr/ami/backups/”$current_date
dbDir=”/usr/ami/backups/”$current_date”/mysql”
#mkdir $dbDir

mkdir “/mnt/backup/”$current_date

/usr/local/bin/ec2-bundle-vol -r i386 -d “/mnt/backup/”$current_date -c /usr/ami/ec2/CERITIFICATE_NAME -k /usr/ami/ec2/PRIVATE_KEY_NAME -u ec2
_USERID -a -p $current_date”myapp(BUCKET_NAME)”

echo “after bundling: “`date`

/usr/local/bin/ec2-upload-bundle -b $current_date”myapp” -m “/mnt/backup/”$current_date”/”$current_date”myapp.manifest.xml” -a AWS_ACCESS_KEY -s
AWS_SECRET ACCESS KEY

echo “after uploading: “`date
ec2-register $current_date”myapp”/$current_date”myapp.manifest.xml”//This command register whole data as an image(private)

This script will bundle the instance as an image and upload it in S3 in a BUCKET with unique name.

One great thing about Amazon S3 is that all uploaded files are available as URLs from a web browser (or any application that can read a URL). The format of the URL is as follows:

http://.s3.amazonaws.com/

Recovering data from S3 :
For this we should go to jetS3t directory there we will see configs/synchronize.properties.Provide AWS
ACCESS KEY ,JAVA_HOME and AWS SECRET ACCESS KEY,something like this in it.

####################################
# Synchronize application properties
#
# This file must be available on the
# classpath when Synchronize is run
####################################

# AWS Access Key (required)

accesskey=01G4VJZ4VPV73ZXV10G2

# AWS Secret Key (required)
secretkey=5hl8jETZD7wvAsS5VDuQgfiTyuafExGgKbbGFRE4

export JAVA_HOME=/usr/jdk/jdk1.6.0_04

# Access Control List setting to apply to uploads, must be one of: PRIVATE, PUBLIC_READ,
PUBLIC_READ_WRITE
# The ACL setting defaults to PRIVATE if this setting is missing.
acl=PRIVATE

# Password used when encrypting/decrypting files. Only required when the –crypto setting is used.
# password=

Create a folder in which you want to store the data,after this go to jetS3t/bin directory and execute the below command.
> ./synchronize.sh –keepfiles DOWN bucketname foldername.

Now the data is being stored in the desired folder.When you will execute ec2-describe-images then will return back with the private image which user has created and using the desired image we can easily launch/host a new instance.

Useful Link:http://jets3t.s3.amazonaws.com/applications/synchronize.html