Sunday 16 November 2014

More detail on the capacity script

More detail on the capacity script

Now that we've successfully got the script up and running, its time to work on the capacity model.

Its working with the simplest of measurements, number of instances. But that doesnt necessarily apply realistically to private clouds, as vms come in different sizes. So now we are going to add more detail to the capacity calculations. So now, we are going to count the cpus deployed. If we go over the 'capacity ceiling' again we'll deploy to azure.

So - in AWS - CPUs are based on the instancetype.

Heres the thing though - for some reason - AWS doesnt specify a setting in the instancetype which correlates in each case to the number of cpus.

Take t2.medium size instances for example. They have 2 cpus.
m3.medium has only the one.

So - looks like we're using powershell hash tables.

Theres a bit of information here: http://www.computerperformance.co.uk/powershell/powershell_hashtable.htm#Example_1:_Simple_PowerShell_Hashtables_

Havent used hash tables before, but to start i'm going to set the $ec2instanceinfo hash table variable like this:

#This script determines the number of CPUs deployed in AWS
#$ec2instanceinfo is the info from aws about the vcpus deployed
#to each instancetype

Clear-Variable $ec2instanceinfo

$ec2instanceinfo = @{
"t2.micro"=1;"t2.small"=1;"t2.medium"=2;
"m3.medium"=1;"m3.large"=2;"m3.xlarge"=4;
"m3.2xlarge"=8;"c3.large"=2;"c3.xlarge"=4;
"c3.2xlarge"=8;"c3.4xlarge"=16;"c3.8xlarge"=32;
"g2.2xlarge"=8;"r3.large"=2;"r3.xlarge"=4;
"r3.2xlarge"=8;"r3.4xlarge"=16;"r3.8xlarge"=32;
"i2.xlarge"=4;"i2.2xlarge"=8;"i2.4xlarge"=16;
"i2.8xlarge"=32;"hs1.8xlarge"=16}

$ec2instanceinfo

That returns me with a table which looks like this:

Name                           Value                                            
----                           -----                                            
m3.xlarge                      4                                                
t2.micro                       1                                                
r3.8xlarge                     32                                               
m3.large                       2                                                
c3.xlarge                      4                                                
c3.8xlarge                     32                                               
t2.small                       1                                                
g2.2xlarge                     8                                                
i2.2xlarge                     8                                                
m3.2xlarge                     8                                                
t2.medium                      2                                                
r3.xlarge                      4                                                
c3.large                       2                                                
m3.medium                      1                                                
r3.2xlarge                     8                                                
c3.2xlarge                     8                                                
r3.large                       2                                                
i2.xlarge                      4                                                
hs1.8xlarge                    16                                               
c3.4xlarge                     16                                               
i2.4xlarge                     16                                               
i2.8xlarge                     32                                               
r3.4xlarge                     16

I now have a table with the number of cpus per instance type.

So now - i have to compare the instancetype of each instance deployed in AWS to a value on this table, and i'm going to add the results to the $ec2numbercpus variable. That will give me the full count of the cpus.

So how the hell do i do this?

First i need to get the list of ec2instanceids, to allow me to run the Get-EC2InstanceAttribute command, which will give me the instance types. I'll need to loop through each instance to get the number of cpus for that particular instance, and then count them up each time into a variable.


$instances = Get-EC2Instance `
             |%{ $_.RunningInstance } `
             | Select-Object InstanceId

Now I have the instanceids, I can run Get-EC2InstanceAttribute against these instance ids, and each time i'll get the instance type (which i'm putting in the $instance_type variable)

Then i can compare that variable to the hash table from above, to get the number of cpus that are in use. Heres the loop where i cycle through the instances counting the cores:

foreach ($i in $instances.InstanceId)
{

$instance_type = Get-EC2InstanceAttribute -InstanceId $i -Attribute instanceType

$ec2numbercpus = $ec2numbercpus + $ec2instanceinfo[$instance_type.InstanceType]


}


So thats it. My core count in AWS is now held in the variable $ec2numbercpus. From this I can go back to my instance count script and use this new capacity counter and apply that. Heres my cpu counter script in its totality:

#This script determines the number of CPUs deployed in AWS
#$ec2instanceinfo is the info from aws about the vcpus deployed
#to each instancetype

#Clearing out any variables in case they were in use somewhere else

Clear-Variable ec2instanceinfo
Clear-Variable Filter
Clear-Variable ec2numbercpus
Clear-Variable numberAWSInstances

#This Variable ec2instanceinfo has the cpu details for each ec2 instancetype

$ec2instanceinfo = @{
"t2.micro"=1;"t2.small"=1;"t2.medium"=2;
"m3.medium"=1;"m3.large"=2;"m3.xlarge"=4;
"m3.2xlarge"=8;"c3.large"=2;"c3.xlarge"=4;
"c3.2xlarge"=8;"c3.4xlarge"=16;"c3.8xlarge"=32;
"g2.2xlarge"=8;"r3.large"=2;"r3.xlarge"=4;
"r3.2xlarge"=8;"r3.4xlarge"=16;"r3.8xlarge"=32;
"i2.xlarge"=4;"i2.2xlarge"=8;"i2.4xlarge"=16;
"i2.8xlarge"=32;"hs1.8xlarge"=16}

#Need to make sure we're doing the right region:

Set-DefaultAWSRegion -Region us-west-2

#$numberAWSInstances = Get-EC2Instance -region us-west-2

#Get the list of instances, and add them to a table - along with some other info
#had to reference this bit from
#http://stackoverflow.com/questions/18090022/how-to-access-list-value-in-get-ec2instances-runninginstance-method

$instances = Get-EC2Instance `
             |%{ $_.RunningInstance } `
             | Select-Object InstanceId
           

#This bit loops through each instanceid returned in the instances variable
#and returns the number of cpus in each instance, by referencing the number
#of cpus against the instancetype held in the hash table above.
#It then counts up the total number of cpus deployed and leaves that in the
#variable $ec2numbercpus

foreach ($i in $instances.InstanceId)
{

$instance_type = Get-EC2InstanceAttribute -InstanceId $i -Attribute instanceType

$ec2numbercpus = $ec2numbercpus + $ec2instanceinfo[$instance_type.InstanceType]


}

echo 'Number of Cpus in use is' $ec2numbercpus


Savage. 

We are nearly there!!

One final blog post this weekend - cleaning up the script

Richie




No comments:

Post a Comment