Associating IAM roles to the IAM instance profile

We were getting some weird error when trying to create a simple EMR cluster using an AWS admin account with all access that was recently created.

Terminated with errorsFailed to provision ec2 instances because 'IAM Instance Profile "arn:aws:iam::...:instance-profile/EMR_EC2_DefaultRole" has no associated IAM Roles (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: ..)'

We have reported this to AWS, and still waiting for a decent reply (already got some canned response, which was useless). Let me know if you have encountered this before, and how did you resolve this.

In the mean time, we are trying to work-around this by allowing all EC2 actions and Elastic MapReduce actions to the admin account. Given below is the set of actions that we performed.

Go to:
https://console.aws.amazon.com/iam/home?region=us-west-2#policies

Select:
Create Your Own Policy

Give a name:  emrAllPolicy
(any name is fine)
Description:  Give all access to emr 
(anything)
Policy Document:
{
    "Version": "2012-10-17",
    "Statement": [
       {
           "Effect": "Allow",
           "Resource": "*",
           "Action": [
               "cloudwatch:*",
               "dynamodb:*",
               "ec2:*",
               "elasticmapreduce:*",
               "kinesis:CreateStream",
               "kinesis:DeleteStream",
               "kinesis:DescribeStream",
               "kinesis:GetRecords",
               "kinesis:GetShardIterator",
               "kinesis:MergeShards",
               "kinesis:PutRecord",
               "kinesis:SplitShard",
               "rds:Describe*",
               "s3:*",
               "sdb:*",
               "sns:*",
               "sqs:*"
           ]
       }
    ]
}

Validate policy and create policy using the relevant options.

Once the above policy is created,
Go to:
https://console.aws.amazon.com/iam/home?region=us-west-2#roles/EMR_EC2_DefaultRole

Click attach policy, search for the policy you just created (if it does not show up, feel free to refresh your browser and retry from the attach policy step again).

Once you found the policy, click and attach it to the IAM instance profile (EMR_EC2_DefaultRole) above, using "Attach Policy" option.

Retry the failed operation.

You may also use the IAM Policy Simulator (by using Simulate Policy option) to simulate all the actions in relevant services to confirm that they are successful in performing those actions, quickly.

This is a related read.

Hive on Amazon EMR

Hive on Amazon EMR-4.0.0 is pretty handy, as EMR comes with Hadoop 2.6.0, Hive 1.0.0, Pig 0.14.0, and optionally other tools pre-configured.

Connection can be established to EMR Hive by,

con = DriverManager.getConnection("jdbc:hive2://ec2XXXX.compute-1.amazonaws.com:10000/default","hadoop", "");

The below error is common, and due to the mismatch of the version of Hive Client and Hive server. Note that Hive Server in EMR is 1.0.0. Make sure to have your client same version. Not a later version to avoid the below error.

17:59:22.298 [main] ERROR edu.emory.bmi.datacafe.hdfs.HiveConnector - SQL Exception in writing to Hive Table: patients java.sql.SQLException: Could not establish connection to jdbc:hive2://ec2-54-82-17-142.compute-1.amazonaws.com:10000/default: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:594) ~[hive-jdbc-1.2.1.jar:1.2.1] at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:192) ~[hive-jdbc-1.2.1.jar:1.2.1] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) ~[hive-jdbc-1.2.1.jar:1.2.1] at java.sql.DriverManager.getConnection(DriverManager.java:571) ~[?:1.7.0_40] at java.sql.DriverManager.getConnection(DriverManager.java:215) ~[?:1.7.0_40] at edu.emory.bmi.datacafe.hdfs.HiveConnector.writeToHive(HiveConnector.java:68) ~[datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.hdfs.HiveConnector.writeToWarehouse(HiveConnector.java:88) [datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.hdfs.HiveConnector.writeDataSourcesToWarehouse(HiveConnector.java:50) [datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.impl.main.Initiator.initiate(Initiator.java:71) [datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.impl.main.Initiator.main(Initiator.java:39) [datacafe-server-1.0-SNAPSHOT.jar:?] Caused by: org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[libthrift-0.9.2.jar:0.9.2] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71) ~[libthrift-0.9.2.jar:0.9.2] at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:156) ~[hive-service-1.2.1.jar:1.2.1] at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:143) ~[hive-service-1.2.1.jar:1.2.1] at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:583) ~[hive-jdbc-1.2.1.jar:1.2.1] ... 9 more 17:59:22.318 [main] INFO edu.emory.bmi.datacafe.hdfs.HiveConnector - Successfully written the output to the file, slices 17:59:22.331 [main] ERROR edu.emory.bmi.datacafe.hdfs.HiveConnector - SQL Exception in writing to Hive Table: slices java.sql.SQLException: Could not establish connection to jdbc:hive2://ec2-54-82-17-142.compute-1.amazonaws.com:10000/default: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:594) ~[hive-jdbc-1.2.1.jar:1.2.1] at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:192) ~[hive-jdbc-1.2.1.jar:1.2.1] at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) ~[hive-jdbc-1.2.1.jar:1.2.1] at java.sql.DriverManager.getConnection(DriverManager.java:571) ~[?:1.7.0_40] at java.sql.DriverManager.getConnection(DriverManager.java:215) ~[?:1.7.0_40] at edu.emory.bmi.datacafe.hdfs.HiveConnector.writeToHive(HiveConnector.java:68) ~[datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.hdfs.HiveConnector.writeToWarehouse(HiveConnector.java:88) [datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.hdfs.HiveConnector.writeDataSourcesToWarehouse(HiveConnector.java:50) [datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.impl.main.Initiator.initiate(Initiator.java:71) [datacafe-server-1.0-SNAPSHOT.jar:?] at edu.emory.bmi.datacafe.impl.main.Initiator.main(Initiator.java:39) [datacafe-server-1.0-SNAPSHOT.jar:?] Caused by: org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[libthrift-0.9.2.jar:0.9.2] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71) ~[libthrift-0.9.2.jar:0.9.2] at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:156) ~[hive-service-1.2.1.jar:1.2.1] at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:143) ~[hive-service-1.2.1.jar:1.2.1] at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:583) ~[hive-jdbc-1.2.1.jar:1.2.1] ... 9 more

Concerns of the public cloud and how PaaS helps mitigating them..

Using cloud model to deploy the applications has become a major trend in the recent years. Developers deploy their applications on top of the infrastructure as a service providers, considering its advantages.

Information outside

Software as a Service (SaaS) providers rely on the Infrastructure as a Service providers for their hardware requirements, in many cases. Having a private cloud set up on their own servers is yet another choice, where they deploy their SaaS solutions in their own servers. Hybrid cloud setup comprises of the best of both the worlds - private and public clouds.

Cloud outages and Server unavailability

When hosting your applications over a simple Infrastructure as a Service, you will have to consider the IaaS provided as you would consider your local single hard drive. Redundancy should be considered to mitigate the risk for the availability, as you would, in case of the local hardware in place. Recent outage faced with the Amazon infrastructure has invoked many cloud developers to think more in terms of developing so as to withstand the failures. What is your backup plan to ensure the availability during such outages or how are you going to handle the situation when an infrastructure or the platform provider leaves the business altogether? You have to have proper migration plan and backups for that. Multiple availability zones and having lesser dependency over the infrastructure or the platform should be considered.

Security Issues

In the cloud, you are on your own to find the ways of securing your applications from the attacks, though the infrastructure vendors have their measures in protecting the data, platform or the applications deployed on top of them. Security vulnerabilities may be higher in IaaS, than in a local hard-drive. When using IaaS in its purest form, customer has the responsibility to implement his own security solutions for his control objectives. Here Platform as a Service comes handy, where it takes care of the security solutions that are common to all the software applications. Hence the Software-as-a-Service (SaaS) developer doesn't need to bother about the recurring security and availability issues for each of the software being deployed on top of the infrastructure. Rather, a platform handles all the issues that should be taken care of, and let the software as a service developer focus solely on the application itself, than considering mitigating the issues that might occur due to the inherent concerns of the cloud.

Privacy and legal issues

Data of your business are legally bound, and care should be taken such that deploying it on cloud will not compromise any legal requirements. In most of the cases, infrastructure providers have the control over your information, and legal and state entities of the host country may have control over your information even without your knowledge, which is highly unlikely, when you are having your data in your own data servers, private cloud, or local network of computers.

Vendor lock-in

On the other hand, coding for a platform minimizes the vendor lock-in than coding for a particular infrastructure. However we should also note the risk of a vendor lock-in by the PaaS provider, where they require proprietary service interfaces or development languages. Platforms should adhere to the standards such that the migration between multiple platforms won't be of much of a task for the SaaS developers, unlike migrating between the infrastructures.

Flexibility

Rapidly evolving applications often require more flexibility in the PaaS offerings. Platform-as-a-Service should provide more flexibility than an Infrastructure-as-a-Service can provide.

Let's look at an example Platform-as-a-Service, and see how does that provides the required flexibility. WSO2 Carbon incorporates the accepted standards as a lean middleware platform, and as WSO2 Stratos is WSO2 Carbon platform as a service, it has the advantages inherent to the award winning WSO2 Carbon. Above all, WSO2 Carbon as well as WSO2 Stratos is open source, and free to extend, without any hidden fees for licensing, hence this eliminates the fear of being locked before trying, while providing the required flexibility to suit the needs of the sophisticated SaaS developers in the enterprise.

Loss of Control / Freedom

As we move towards the cloud, the cloud providers decide themselves in many cases, what are the services to offer or what services will be compatible with their offering. Unlike in servers in-house, as you move your data centers and servers to be on cloud, you almost lose the control over them to the cloud providers, not to mention, the privacy and security risks that may occur due to that. This leads to the third party controlling your applications.

The vendor of the IaaS or the platform has control over the applications provided or supported, or even whether they allow the external developers to write new applications to their infrastructure or platform. A platform that facilitates the extensibility of the cloud comes to the rescue in this place. If the platform supports installing or deploying multiple software applications or configure and update the existing applications, that would mitigate the above said short coming for a considerable extent.

Infrastructure expertise

When software are developed considering the infrastructure in mind, they often require the expertise for the particular infrastructure provider, let it be Amazon EC2 Cloud or Rackspace. Migrating for another infrastructure provider will need code level changes in many cases. While consuming the fruits of the scalability and the pay-as-you-go model of the cloud, this includes an additional overhead from the software application developers' point of view. Platform as a Service providers invest on this, by delegating the need to code for the underline infrastructure to themselves, away from the SaaS developers.

More Portability

While some of the PaaS such as Windows Azure rely completely on their own infrastructure, platforms including WSO2 Stratos are designed to function on any of the infrastructure. In this case, as the infrastructure level issues related to the portability are handled by the PaaS layer, the SaaS developers are freed from the burden of coding for multiple infrastructures. This is analogous to the well-known programming challenge - coding for multiple operating system APIs.

An extra bit of work

Those existing applications that work amazingly well on the standard computers or the servers are to be work on the new cloud model. Are they cloud-ready? In other words, can they work over the cloud, as they worked in the non-cloud environment? Ideally, making an application that was written without the cloud in mind needs some extra bit of work to make it utilize the cloud. Coding a new application having cloud in mind is a different consideration altogether, where SaaS model comes. But porting the applications to a cloud infrastructure needs the know-hows on elasticity, load balancing, and auto-scaling, which come as the fruits of the cloud. Platform as a Service vendors promise a smooth deployment of your existing applications into their platform, with lesser or no effort.

Amazon ELB HTTPS Stickiness

Stickiness is needed in most of the complex applications when scaled up in cloud. Say one user logs in. Now his information should be attached to him. If not, each time, he would be routed to a random instance, which will not identify him. Attaching a session id would provide the stickiness, where a particular user can be forwarded to the particular instance. 

I tried using amazon elb-api commands using terminal to enable stickiness for the elastic load balancer. it gave error when trying to apply the stickiness for the port 443, and works fine for 80.

Create Stickiness Policy
pradeeban@pradeeban:~/pem$ elb-create-app-cookie-stickiness-policy wso2cloud1-as -p my-app-cookie-lb-policy -c jsessionid -K KEY.pem -C CERT.pem
OK-Creating App Stickiness Policy

Setting the Policy to the Listener - Fails for port 443
pradeeban@pradeeban:~/pem$ elb-set-lb-policies-of-listener wso2cloud1-as --lb-port 443 --policy-names my-app-cookie-lb-policy -K KEY.pem -C CERT.pem
elb-set-lb-policies-of-listener: Service error: aws:Client.InvalidConfigurationRequest AWSRequestId:4b2c7fc3-4fbf-11e0-a778-d17858cabec6

Setting the Policy to the Listener - Works for port 80
pradeeban@pradeeban:~/pem$ elb-set-lb-policies-of-listener wso2cloud1-as --lb-port 80 --policy-names my-app-cookie-lb-policy -K KEY.pem -C CERT.pem
OK-Setting Policies

So it seemed, it was an issue with https/stickiness from the amazon's end. It was not giving any error when tried with amazon management console nevertheless, but simply not allowing to enable stickiness for the listener with the tcp protocol.

It works for --lb-port 80 --lb-port 443, but not for --lb-port 443 --lb-port 80 (since it picks the first entry only, it seems).

Tried the same with elb-create-lb-cookie-stickiness-policy (elb controlled stickiness) and the results was same.

So I felt, https/stickiness is not working for both application based or amazon elb based cookies, while it works for http.


But Amazon announced in Oct, 2010 that they have started supporting HTTPS stickiness.


I reported this in AWS forum, but later found that it (Setting stickiness policy) works. What we had to change was the protocol to https from tcp for the port 443.

--listener "lb-port=443,instance-port=9443,protocol=https"

HTTPS/Stickiness is fine with Amazon's, but not TCP/Stickiness. The TCP/Stickiness issue has already been discussed in another thread [1] in AWS forums. Hence we resolved the issue, by changing the protocol from TCP to HTTPS for the port 443. However we also noted that aiCache provides web application HTTPS acceleration and stickiness for TCP too.

[1] Service error when setting LB policies of listener
[2] Elastic Load Balancing with Sticky Sessions
[3] Amazon Makes the Cloud Sticky
[4] jmeter + amazon ec2 + load balancing (elb)
[5] How can I force "non-sticky" connections for my ELB? 
[6] Load Balancer (ELB) - port forwarding on the load balancer itself
[7] "Sticky connection" on ELB for https?  
[8] Using ELB to Serve Multiple Domains Over SSL on EC2 for Giggles and Unicorns
[9] Amazon Simple Monthly Calculator

Amazon Autoscaling ~ Issue uploading payload?

We tried to upload a payload.zip for the autoscaled load balanced system, with the startup-parameters for our Application Server (using the commands given below). There seems to be some issue for the current Amazon Autoscaling API, that prevents us from uploading the user-data-file. We have reported the issue in the AWS forum, and awaiting their reply. :)

Creating a launch config
pradeeban@pradeeban:~/pem$ as-create-launch-config autoscalelcapp --image-id ami-xxxxxxxx --instance-type m1.large --user-data-file /tmp/payload.zip --key "keypair" --group "default" -K KEY.pem -C CERT.pem

Updating Auto Scaling Group
pradeeban@pradeeban:~/pem$ as-update-auto-scaling-group autoscleasg1 --availability-zones us-east-1c --launch-configuration autoscalelcas --min-size 1 --max-size 5 -K KEY.pem -C CERT.pem

However we found that, it works when the payload is sent as a string using the param --user-data, instead of --user-data, so we used that to get appserver running, load balanced and auto scaled.

Launch Config
pradeeban@pradeeban:~/pem$ as-create-launch-config autoscalelcas2 --image-id ami-xxxxxxxx --instance-type m1.large --user-data  "AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxx,AWS_SECRET_ACCESS_KEY=xxxxxxxxxxx,AMI_ID=ami-xxxxxxxx,ELASTIC_IP=xx.xx.xxx.xxx,PRODUCT_MODIFICATIONS_PATH_S3=s3://wso2-stratos-conf-1.0.0/appserver/,COMMON_MODIFICATIONS_PATH_S3=s3://wso2-stratos-conf-1.0.0/stratos/,PRODUCT_PATH_S3=s3://wso2-stratos-products-1.0.0,PRODUCT_NAME=wso2stratos-as-1.0.0,SERVER_NAME=appserver.cloud.wso2.com,HTTP_PORT=9763,HTTPS_PORT=9443,STARTUP_DELAY=0" -K KEY.pem -C CERT.pem 
OK-Created launch config

Since this works, we are happy to proceed passing the payload as a string, instead of sending it as a zip file. Relevant products will be taken from the S3 buckets and run using the script, hence producing the autoscaled appserver instances.

When smartness of the word processors overtake you.. ;)

There were two times smart completion features have overtaken me. I was unable to expose the first experience, as it would violate someone else's privacy; so here comes the second experience. But this time a pretty boring one. :D

I was trying get a description of a load balancer I created. I have all the commands written down in a file. I opened it using a word processor and copied it to the terminal.

pradeeban@pradeeban:~/pem$ elb-describe-lbs autoscalelb –headers -K KEY.pem -C CERT.pem

elb-describe-lbs:  Service error: LoadBalancer name cannot contain characters that are not

 letters, or digits or the dash.

 AWSRequestId:6634cf73-423d-11e0-97ad-fd607d01edca

I was confused, where did I use an invalid character in the name (which was indeed just "autoscalelb"). After a few minutes, I figured out, it was my smart word processor, which had replaced "--headers" with "–headers" (Smart hyphenation!). :D

Auto Scaling With Amazon EC2 - II

We created an Amazon ELB as discussed here. Let's see more of it now.

Appserver
The load balancer we have created listens on the port 80 and 443 and forwards requests to 9763 and 9443. Say now we need to delete the listeners.

elb-delete-lb-listeners autoscalelb --lb-ports 80 443 -K KEY.pem -C CERT.pem  
    Warning: Deleting a LoadBalancer listener can lead to service disruption to
    any customers connected to the LoadBalancer listener. Are you sure you want
    to delete this LoadBalancer listener? [Ny]N
elb-delete-lb-listeners:  User stopped the execution of elb-delete-lb-listeners.

(Providing 'N' stops the action. You can proceed deleting the listener, by giving 'y' as the response.)

You can also create more listeners

elb-create-lb-listeners autoscalelb --headers --listener "lb-port=8280,instance-port=9763,protocol=http" --listener "lb-port=8243,instance-port=9443,protocol=tcp" -K KEY.pem -C CERT.pem

OK-Creating LoadBalancer Listener

The load balancer now listens on the port 8280 and 8243 and forwards requests to 9763 and 9443.

Now what will happen if you forcefully try to kill the instances initiated by the ELB? It will create one more identical instance immediately, handling the failover case.


Fail over

pradeeban@pradeeban:~/pem$ as-describe-auto-scaling-groups autoscleasg -K KEY.pem -C CERT.pem

Initially,

AUTO-SCALING-GROUP  autoscleasg  autoscalelc  us-east-1c  autoscalelb  1  10  1

INSTANCE  i-xxxxxxxx  us-east-1c  InService  Healthy  autoscalelc

When we killed the instance.

AUTO-SCALING-GROUP  autoscleasg  autoscalelc  us-east-1c  autoscalelb  1  10  1

INSTANCE  i-xxxxxxxx  us-east-1c  Terminating  Unhealthy  autoscalelc

INSTANCE  i-yyyyyyyy  us-east-1c  Pending      Healthy    autoscalelc

After a few seconds,

AUTO-SCALING-GROUP  autoscleasg  autoscalelc  us-east-1c  autoscalelb  1  10  1

INSTANCE  i-yyyyyyyy  us-east-1c  InService  Healthy  autoscalelc

Sounds cool.. So how to terminate the instances that ELB creates?


Shall we try deleting?

pradeeban@pradeeban:~/pem$ as-delete-auto-scaling-group autoscleasg -K KEY.pem -C CERT.pem

    Are you sure you want to delete this AutoScalingGroup? [Ny]y

as-delete-auto-scaling-group:  Service error: You cannot delete an AutoScalingGroup while there are instances

 still in the group.  AWSRequestId:aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa

Min = Max = 0
ok! So let's stop auto-scaling process by setting the minimum and maximum to zero.

pradeeban@pradeeban:~/pem$ as-update-auto-scaling-group autoscleasg --min-size 0 --max-size 0 -K KEY.pem -C CERT.pem

OK-Updated AutoScalingGroup

pradeeban@pradeeban:~/pem$ as-describe-auto-scaling-groups autoscleasg -K KEY.pem -C CERT.pem

AUTO-SCALING-GROUP  autoscleasg  autoscalelc  us-east-1c  autoscalelb  0  0  0

INSTANCE  i-yyyyyyyy  us-east-1c  InService  Healthy  autoscalelc

Delete
Now let's try to delete the auto-scaling group once more -- yes, as we have set it to zero, it should be possible now.

pradeeban@pradeeban:~/pem$ as-delete-auto-scaling-group autoscleasg -K KEY.pem -C CERT.pem

  
    Are you sure you want to delete this AutoScalingGroup? [Ny]y

as-delete-auto-scaling-group:  Service error: You cannot delete an AutoScalingGroup while there are scaling activities in progress for that group.

 AWSRequestId:bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbb

oh.. Let's wait a minute for that scaling down activity to finish!
Again!

pradeeban@pradeeban:~/pem$ as-delete-auto-scaling-group autoscleasg -K KEY.pem -C CERT.pem

          

    Are you sure you want to delete this AutoScalingGroup? [Ny]y

OK-Deleted AutoScalingGroup

Done!

Now, let's check once more whether we have deleted it properly.. ;)

pradeeban@pradeeban:~/pem$ as-describe-auto-scaling-groups autoscleasg -K KEY.pem -C CERT.pem

No AutoScalingGroups found

Yes, we have deleted the Amazon ELB along with the scaling group of nodes it created.  (We are sinners! ;))

Auto Scaling With Amazon EC2

Creating an auto scaled system using an Amazon load balancer is an interesting task that I did recently. We have an Amazon EC2 image with WSO2 Application Server installed. Creating an image with WSO2 WSAS installed is described here. 

Amazon EC2 API Tools

You will need Amazon EC2 API tools to create the image yourself. You can install it using "sudo apt-get install ec2-api-tools" in debian based operating systems, or you can download it from Amazon S3. These tools provide a client interface to the Amazon EC2 web service, to register and launch instance and more.

Configure

pradeeban@pradeeban:~/Downloads$ ec2-run-instances -K KEY.pem  -C CERT.pem

Instance Details

You can get the instance details using the InstanceID i-xxxxxxxx you get above.

ec2-describe-instances -K KEY.pem -C CERT.pem  i-xxxxxxxx or get the details of all the instances by,

ec2-describe-instances -K KEY.pem -C CERT.pem

[Providing the relevant public key and cert, KEY.pem and CERT.pem.]


Load Balancing with Auto Scaling
Now we come to the interesting part. That is auto scaling the Amazon EC2 Image with the load.

Download and set up Elastic Load Balancing API tools
Download it here, extract, and set the path up appropriately.

export AWS_ELB_HOME=/home/pradeeban/program/ElasticLoadBalancing-1.0.11.1

export PATH=$PATH:$AWS_ELB_HOME/bin

ELB Quick Reference Card
Downloading and setting up Auto Scaling API tools
Download it here, extract, and set the path up appropriately.

export AWS_AUTO_SCALING_HOME=/home/pradeeban/programs/AutoScaling-1.0.33.1

export PATH=$PATH:$AWS_AUTO_SCALING_HOME/bin

Creating a Load Balancer

pradeeban@pradeeban:~/Downloads$ elb-create-lb  autoscalelb --headers --listener "lb-port=80,instance-port=9763,protocol=http" --listener "lb-port=443,instance-port=9443,protocol=tcp" --availability-zones us-east-1c -K KEY.pem -C CERT.pem
DNS_NAME  DNS_NAME
DNS_NAME  autoscalelb-1316227031.us-east-1.elb.amazonaws.com

Describe ELB
elb-describe-lbs autoscalelb -K KEY.pem -C CERT.pem
LOAD_BALANCER  autoscalelb  autoscalelb-1316227031.us-east-1.elb.amazonaws.com  2011-01-28T09:40:54.750Z

Register instances with the load balancer
elb-register-instances-with-lb autoscalelb --instances i-xxxxxxxx -K KEY.pem -C CERT.pemINSTANCE_ID  i-xxxxxxxx

Configuring a health check
pradeeban@pradeeban:~/Downloads$ elb-configure-healthcheck  autoscalelb --headers --target "TCP:9763" --interval 5 --timeout 3 --unhealthy-threshold 2 --healthy-threshold 2 -K KEY.pem -C CERT.pem
HEALTH_CHECK  TARGET    INTERVAL  TIMEOUT  HEALTHY_THRESHOLD  UNHEALTHY_THRESHOLD
HEALTH_CHECK  TCP:9763  5         3        2                  2


 
Creating an AutoScaled System

Launching configuration for Amazon EC2 framework to launch new Amazon instances.
pradeeban@pradeeban:~/Downloads$ as-create-launch-config autoscalelc --image-id ami-xxxxxxxx --instance-type m1.large -K KEY.pem -C CERT.pem
OK-Created launch config


You can choose the instance type (m1.small, m1.large, and m1.xlarge) based on the requirements.

Creating Auto Scaling Group
pradeeban@pradeeban:~/Downloads$ as-create-auto-scaling-group autoscleasg --availability-zones us-east-1c --launch-configuration autoscalelc --min-size 1 --max-size 10 --load-balancers autoscalelb -K KEY.pem -C CERT.pem
OK-Created AutoScalingGroup

Describe auto scaling groups
pradeeban@pradeeban:~/Downloads$ as-describe-auto-scaling-groups autoscleasg -K KEY.pem -C CERT.pem
AUTO-SCALING-GROUP  autoscleasg  autoscalelc  us-east-1c  autoscalelb  1  10  1


Configuring a trigger with start actions according to the load.
pradeeban@pradeeban:~/Downloads$ as-create-or-update-trigger autoscaletrigger --auto-scaling-group autoscleasg --namespace "AWS/ELB" --measure Latency --statistic Average --dimensions "LoadBalancerName=autoscalelb" --period 60 --lower-threshold 0.5 --upper-threshold 1.2 --lower-breach-increment=-1 --upper-breach-increment 1 --breach-duration 120 -K KEY.pem -C CERT.pem
DEPRECATED: This command is deprecated and included only to facilitate migration to the new trigger mechanism.  You should use this command for migration purposes only.
OK-Created/Updated trigger

measure

You can choose the measure, based on your auto-scaling requirements. Let it be CPUUtilization, Latency, or Load. You will have to choose this wisely based on the application types, let them be CPU-intense applications, or huge applications, or applications that consume too much time. 

Now you notice as-create-or-update-trigger is depreciated. You can use scale up and scale down policies, along with the cloud-watch tools as described below instead!

Amazon CloudWatch API Tools
Downloading and Setting up 
Download it here, extract, and set the path up appropriately, to monitor the AWS cloud resources.

export AWS_CLOUDWATCH_HOME=/home/pradeeban/programs/CloudWatch-1.0.9.5

export PATH=$PATH:$AWS_CLOUDWATCH_HOME/bin

Now we have to define the scale up and scale down policies on scaling up and down the system based on the load, along with the monitoring.

Scale-up Policy
pradeeban@pradeeban:~/pem$ as-put-scaling-policy MyScaleUpPolicy1 --auto-scaling-group autoscleasg1 --adjustment=1 --type ChangeInCapacity --cooldown 300 -K KEY.pem -C CERT.pem
arn:aws:autoscaling:us-east-1:xxxxxxxxxxxxxx:scalingPolicy:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:autoScalingGroupName/autoscleasg1:policyName/MyScaleUpPolicy1

pradeeban@pradeeban:~/pem$ mon-put-metric-alarm MyHighCPUAlarm1 --comparison-operator GreaterThanThreshold --evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" --period 600 --statistic Average --threshold 80 --alarm-actions arn:aws:autoscaling:us-east-1:xxxxxxxxxxxxxx:scalingPolicy:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:autoScalingGroupName/autoscleasg1:policyName/MyScaleUpPolicy1 --dimensions "AutoScalingGroupName=autoscleasg" -K KEY.pem -C CERT.pem
OK-Created Alarm

Scale Down Policy
pradeeban@pradeeban:~/pem$ as-put-scaling-policy MyScaleDownPolicy1 --auto-scaling-group autoscleasg1 --adjustment=-1 --type ChangeInCapacity --cooldown 300 -K KEY.pem -C CERT.pem
arn:aws:autoscaling:us-east-1:xxxxxxxxxxxxxx:scalingPolicy:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:autoScalingGroupName/autoscleasg1:policyName/MyScaleDownPolicy1

pradeeban@pradeeban:~/pem$ mon-put-metric-alarm MyLowCPUAlarm --comparison-operator LessThanThreshold --evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" --period 600 --statistic Average --threshold 40 --alarm-actions arn:aws:autoscaling:us-east-1:xxxxxxxxxxxxxx:scalingPolicy:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:autoScalingGroupName/autoscleasg1:policyName/MyScaleDownPolicy1 --dimensions "AutoScalingGroupName=autoscleasg" -K KEY.pem -C CERT.pem
OK-Created Alarm

Now the auto scaling group gets an instance, as the minimum number of instances in the auto scaled system has been set to 1,
pradeeban@pradeeban:~/Downloads$ as-describe-auto-scaling-groups autoscleasg -K KEY.pem -C CERT.pem
AUTO-SCALING-GROUP  autoscleasg  autoscalelc  us-east-1c  autoscalelb  1  10  1
INSTANCE  i-xxxxxxxx  us-east-1c  InService  Healthy  autoscalelc

Once the elastic load balancer is set fine and triggered, it starts new nodes or remove the existing nodes according to the load. Following these steps, the system can be load balanced with autoscaling.

Load Balanced Instances' Health
Initially,

pradeeban@pradeeban:~/pem$ elb-describe-instance-health autoscalelb –headers -K KEY.pem -C CERT.pem
INSTANCE_ID  INSTANCE_ID  STATE      DESCRIPTION  REASON-CODE
INSTANCE_ID  i-xxxxxxxx   InService  N/A          N/A

Later with the load, you may see at least a new instance.

pradeeban@pradeeban:~/pem$ elb-describe-instance-health autoscalelb –headers -K KEY.pem -C CERT.pem
INSTANCE_ID  INSTANCE_ID  STATE      DESCRIPTION  REASON-CODE
INSTANCE_ID  i-xxxxxxxx   InService  N/A          N/A

INSTANCE_ID  i-yyyyyyyy   InService  Active Instance

After a few failed attempts, an instance will be marked as 'OutOfService' with the reason, 'Instance has failed at least the UnhealthyThreshold number of health checks consecutively.' or so.

Some interesting read
Setting up an elastic load balancer
Automatically scaling a LAMP application in the cloud
Monitoring performance with Amazon CloudWatch
Examining Load Average - Choosing the metric