How to manage Amazon Machine Images with the .NET Amazon SDK Part 1: starting an image instance

If you have an access to Amazon Web Services (AWS) EC2 then you can manage Amazon Machine Images (AMI) in the cloud using this screen:

In case you are not familiar with AMIs then here‘s a short summary.

Amazon has created SDKs in for several different programming languages, such as Java, Python, Ruby, C# etc. by which you can manage the servers, AMIs etc. in an elegant way in code. You can check out the various packages on the AWS homepage:

In this post I’ll concentrate on the .NET package in a console application: how to start, monitor and shut down the AMI instances. Note that if you don’t have an Amazon account then it will be difficult to test the provided code samples. You will need both your Access Key ID and your Secret Access Key in order to communicate with AWS through the SDK.

Demo

In the demo I’ll concentrate on showing the functionality and ignore best practices such as SOLID, IoC containers, patterns, DRY etc. in order to eliminate the “noise”. I’ll put all code within Program.cs. It’s up to you how you organise it in your code later.

Open Visual Studio 2012 and create a new Console application. Add a reference to the AWS SDK using NuGet:

We’ll put the access keys in app.config:

<configuration>
	<appSettings>
		<add key="AmazonAccessKeyId" value="accesskeyid"/>
		<add key="AmazonSecretAccessKey" value="secretaccesskey"/>
	</appSettings>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
</configuration>

We’ll insert a simple method to retrieve the necessary AWS credentials as follows:

private static BasicAWSCredentials GetAmazonCredentials()
{
	string secretAccessKey = ConfigurationManager.AppSettings["AmazonSecretAccessKey"];
	string accessKeyId = ConfigurationManager.AppSettings["AmazonAccessKeyId"];
	BasicAWSCredentials basicAwsCredentials = new BasicAWSCredentials(accessKeyId, secretAccessKey);
	return basicAwsCredentials;
}

You’ll need to add a reference to the System.Configuration dll. BasicAWSCredentials is located in the Amazon namespace so you’ll need to reference that as well.

We’ll also need a HTTP client object which will communicate with AWS. This is represented by the AmazonEC2Client object. Now, if you log onto the AWS EC2 management web site then you’ll see that by default the region called US East 1 is selected:

The AmazonEC2Client object has a constructor where you don’t set the selected region in which case the selected region defaults to US East 1 – North Virginia, just like when you log onto the EC2 manager page and the page doesn’t remember what your previous selected region was. This object can be used to send region-independent queries as well, such as finding all available AWS regions, which we’ll look at in a second. Before that we’ll need a method to construct the AmazonEC2Client object:

private static AmazonEC2Client GetAmazonClient(string selectedAmazonRegionEndpoint)
{
	AmazonEC2Client amazonClient = null;
	if (string.IsNullOrEmpty(selectedAmazonRegionEndpoint))
	{
		amazonClient = new AmazonEC2Client(GetAmazonCredentials());
	}
	else
	{
		AmazonEC2Config amazonConfig = new AmazonEC2Config();
		amazonConfig.ServiceURL = "http://" + selectedAmazonRegionEndpoint;
		amazonClient = new AmazonEC2Client(GetAmazonCredentials(), amazonConfig);
	}
	return amazonClient;
}

You see that if you don’t specify a region we return a client specifying our credentials but not the region. Otherwise we provide the region using the AmazonEC2Config object. We’ll see how this section is used in a while, don’t worry about it yet.

Now we have the client object ready so let’s try to find the available regions in Amazon:

private static List<Region> GetAmazonRegions()
{
	AmazonEC2Client amazonEc2Client = GetAmazonClient(null);
	try
	{
		DescribeRegionsRequest describeRegionsRequest = new DescribeRegionsRequest();
		DescribeRegionsResponse describeRegionsResponse = amazonEc2Client.DescribeRegions(describeRegionsRequest);
		DescribeRegionsResult describeRegionsResult = describeRegionsResponse.DescribeRegionsResult;
		List<Region> regions = describeRegionsResult.Region;
		return regions;
	}
	catch
	{
		throw;
	}
}

You’ll see Request and Response objects quite a lot throughout the .NET SDK and this is a good example. We get hold of the list of regions using the request-response pattern. We’ll print the available regions in a separate method:

private static void PrintAmazonRegions(List<Region> regions)
{
	for (int i = 0; i < regions.Count; i++)
	{
		Region region = regions[i];
		Console.WriteLine(string.Format("{0}: Display name: {1}, http endpoint: {2}", i + 1, region.RegionName, region.Endpoint));
	}
}

Let’s connect our methods in Main as follows:

static void Main(string[] args)
{
	List<Region> amazonRegions = GetAmazonRegions();
	PrintAmazonRegions(amazonRegions);
	Console.ReadKey();
}

Run the programme and you should get a list of regions like this:

So far so good! The next step is to find the available AMIs in the selected region. Before we do that let’s alter the existing code so that the user needs to pick a region. Add the following method to read the selected menu point of the user:

private static int GetSelectedRegionOfUser(List<Region> amazonRegions)
{
	Console.Write("Select a region: ");
	string selection = Console.ReadLine();
	int selectableMin = 1;
	int selectableMax = amazonRegions.Count;
	int selectedMenuPoint;
	bool validFormat = int.TryParse(selection, out selectedMenuPoint);
	while (!validFormat || (selectedMenuPoint < selectableMin || selectedMenuPoint > selectableMax))
	{
		Console.WriteLine("Invalid input.");
		Console.Write("Select a region: ");
		selection = Console.ReadLine();
		validFormat = int.TryParse(selection, out selectedMenuPoint);
	}

	return selectedMenuPoint;
}

The revised Main method looks as follows:

static void Main(string[] args)
{
	List<Region> amazonRegions = GetAmazonRegions();
	PrintAmazonRegions(amazonRegions);
	int usersChoice = GetSelectedRegionOfUser(amazonRegions);
        Region selectedRegion = amazonRegions[usersChoice - 1];
	Console.ReadKey();
}

So now we have the selected region. It’s time to look for a suitable AMI in that region. The easiest way to retrieve the list of available machines is the following method:

private static List<Amazon.EC2.Model.Image> GetSuitableImages(Region selectedRegion)
{
	AmazonEC2Client amazonEc2client = GetAmazonClient(selectedRegion.Endpoint);
	try
	{
		DescribeImagesRequest imagesRequest = new DescribeImagesRequest();
		DescribeImagesResponse imagesResponse = amazonEc2client.DescribeImages(imagesRequest);
		DescribeImagesResult imagesResult = imagesResponse.DescribeImagesResult;
		List<Amazon.EC2.Model.Image> images = imagesResult.Image;
		return images;
	}
	catch
	{
		throw;
	}
}

We send in the selected endpoint to the GetAmazonClient method. If you recall then this method will put the selected endpoint into the constructor of the AmazonEC2Client object thereby overriding the default US East 1 region. We then use the request-response objects to retrieve the AMIs from the selected endpoint. However, in its present form the method will return ALL available machines, meaning all public ones and any other that your account may have permission to use. That list is way too long so I recommend that you do not run this method without filtering. You can filter based on the properties of the AMI, e.g. the owner code or the current state. If you are looking for AMIs that belong to a certain owner then you’ll need the code of that owner:

List<String> owners = new List<string>();
owners.Add(ConfigurationManager.AppSettings["AmiSavOwnerId"]);
owners.Add(ConfigurationManager.AppSettings["AmiGclOwnerId"]);
owners.Add(ConfigurationManager.AppSettings["NewAmiOwnerId"]);
imagesRequest.Owner = owners;

As you see it’s possible to fill up a list of strings with the owner IDs which will be assigned to the Owner property of the DescribeImagesRequest object. Go through the available properties of this object to see what other filtering possibilities exist. If you don’t find a ready made property then you can still try the Filter object. Here we’ll filter the AMIs according to their current state:

Filter availabilityFilter = new Filter();
availabilityFilter.Name = "state";
List<String> filterValues = new List<string>();
filterValues.Add("available");
availabilityFilter.Value = filterValues;
List<Filter> filters = new List<Filter>();
filters.Add(availabilityFilter);
imagesRequest.Filter = filters;

It looks a bit cumbersome for a bit of filtering but it goes like this: you define the AMI property by which you want to filter the results in the Name property of the Filter object. Each Filter key can have multiple values hence you need to assign a list of strings. I’m only interested in those AMIs whose ‘state’ property has the value ‘available’. This list of strings will be assigned to the Value property of the Filter object. Then we add this specific filter to the list of filters of the request. So our revised method looks as follows:

private static List<Amazon.EC2.Model.Image> GetSuitableImages(Region selectedRegion)
{
	AmazonEC2Client amazonEc2client = GetAmazonClient(selectedRegion.Endpoint);
	try
	{
		DescribeImagesRequest imagesRequest = new DescribeImagesRequest();

		List<String> owners = new List<string>();
		owners.Add(ConfigurationManager.AppSettings["AmiSavOwnerId"]);
		owners.Add(ConfigurationManager.AppSettings["AmiGclOwnerId"]);
		owners.Add(ConfigurationManager.AppSettings["NewAmiOwnerId"]);
		imagesRequest.Owner = owners;

		Filter availabilityFilter = new Filter();
		availabilityFilter.Name = "state";
		List<String> filterValues = new List<string>();
		filterValues.Add("available");
		availabilityFilter.Value = filterValues;
		List<Filter> filters = new List<Filter>();
		filters.Add(availabilityFilter);
		imagesRequest.Filter = filters;

		DescribeImagesResponse imagesResponse = amazonEc2client.DescribeImages(imagesRequest);
		DescribeImagesResult imagesResult = imagesResponse.DescribeImagesResult;
		List<Amazon.EC2.Model.Image> images = imagesResult.Image;
		return images;
	}
	catch
	{
		throw;
	}
}

The following method will print the images in the Console:

private static void PrintAmis(List<Amazon.EC2.Model.Image> images)
{
	Console.WriteLine(Environment.NewLine);
	Console.WriteLine("Images in the selected region:");
	Console.WriteLine("------------------------------");
	for (int i = 0; i < images.Count; i++)
	{
		Image image = images[i];
		Console.WriteLine(string.Format("{0}: image location: {1}, architecture: {2}", i + 1, image.ImageLocation, image.Architecture));
	}
}

Note that I selected the ImageLocation and Architecture properties of the Image object but feel free to discover all other properties that you can extract from it. The extended Main method looks like this:

static void Main(string[] args)
{
	List<Region> amazonRegions = GetAmazonRegions();
	PrintAmazonRegions(amazonRegions);
	int usersChoice = GetSelectedRegionOfUser(amazonRegions);
	Region selectedRegion = amazonRegions[usersChoice - 1];
	List<Amazon.EC2.Model.Image> imagesInRegion = GetSuitableImages(selectedRegion);
	PrintAmis(imagesInRegion);
	Console.ReadKey();
}

Run the application and if everything goes well then you may see an output similar to the following:

The names of the AMIs will of course be different in your case. We’re now ready to start an image. First let’s get the user’s choice:

private static int GetSelectedRegionOfUser(List<Region> amazonRegions)
{
	Console.Write("Select a region: ");
	string selection = Console.ReadLine();
	int selectableMin = 1;
	int selectableMax = amazonRegions.Count;
	int selectedMenuPoint;
	bool validFormat = int.TryParse(selection, out selectedMenuPoint);
	while (!validFormat || (selectedMenuPoint < selectableMin || selectedMenuPoint > selectableMax))
	{
		Console.WriteLine("Invalid input.");
		Console.Write("Select a region: ");
		selection = Console.ReadLine();
		validFormat = int.TryParse(selection, out selectedMenuPoint);
	}

	return selectedMenuPoint;
}

Main:

static void Main(string[] args)
{
	List<Region> amazonRegions = GetAmazonRegions();
	PrintAmazonRegions(amazonRegions);
	int usersChoice = GetSelectedRegionOfUser(amazonRegions);
	Region selectedRegion = amazonRegions[usersChoice - 1];
	List<Amazon.EC2.Model.Image> imagesInRegion = GetSuitableImages(selectedRegion);
	PrintAmis(imagesInRegion);
	int usersImageChoice = GetSelectedImageOfUser(imagesInRegion);
	Image selectedImage = imagesInRegion[usersImageChoice - 1];
	Console.ReadKey();
}

The code to launch one image instance looks as follows:

private static List<RunningInstance> LaunchImage(Image selectedImage, Region selectedRegion)
{
	AmazonEC2Client amazonEc2client = GetAmazonClient(selectedRegion.Endpoint);
	try
	{
		RunInstancesRequest runInstanceRequest = new RunInstancesRequest();
		runInstanceRequest.ImageId = selectedImage.ImageId;
		runInstanceRequest.InstanceType = "m1.large";
		runInstanceRequest.MinCount = 1;
		runInstanceRequest.MaxCount = 1;
		runInstanceRequest.SecurityGroup = new List<string>() { ConfigurationManager.AppSettings["AmazonSecurityGroupName"] };
		runInstanceRequest.DisableApiTermination = false;

		RunInstancesResponse runInstancesResponse = amazonEc2client.RunInstances(runInstanceRequest);
		RunInstancesResult runInstancesResult = runInstancesResponse.RunInstancesResult;
		Reservation reservation = runInstancesResult.Reservation;
	        List<RunningInstance> runningInstances = reservation.RunningInstance;
		return runningInstances;

	}
	catch
	{
		throw;
	}
}

As before we set the region in the AmazonEC2Client constructor. We then construct the RunInstanceRequest object: we set the selected image ID, the instance type – in this case a large instance -, the number of instances to start – we only want 1 -, and the security group name. We finally determine that we want to be able to terminate the image instance using the API. We then send the request to the AWS API and get a Reservation object back which includes the list of image instances we have started. If only 1 instance was requested then this list will only contain a single element.

Here’s the revised Main method:

static void Main(string[] args)
{
	List<Region> amazonRegions = GetAmazonRegions();
	PrintAmazonRegions(amazonRegions);
	int usersChoice = GetSelectedRegionOfUser(amazonRegions);
	Region selectedRegion = amazonRegions[usersChoice - 1];
	List<Amazon.EC2.Model.Image> imagesInRegion = GetSuitableImages(selectedRegion);
	PrintAmis(imagesInRegion);
	int usersImageChoice = GetSelectedImageOfUser(imagesInRegion);
	Image selectedImage = imagesInRegion[usersImageChoice - 1];
	List<RunningInstance> launchedInstances = LaunchImage(selectedImage, selectedRegion);
	Console.ReadKey();
}

Make your selections in the console. If everything went fine then you’ll see the instance starting up in the AWS console:

In case you don’t see the instance starting up then it may be because you’re not viewing the same region as you selected in the console app. Make sure to select the same region in the AWS manager:

We’re doing good so far. The next step will be to monitor the status of the machine – pending, started etc. and to terminate it. This will be the topic of the next post.