Return a default value from a Map in Java 8

Consider the following Employee class:

public class Employee
{
    private UUID id;
    private String name;
    private int age;

    public Employee(UUID id, String name, int age)
    {
        this.id = id;
        this.name = name;
        this.age = age;
    }
        
    public UUID getId()
    {
        return id;
    }

    public void setId(UUID id)
    {
        this.id = id;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }    
    
    public int getAge()
    {
        return age;
    }

    public void setAge(int age)
    {
        this.age = age;
    }
}

Let’s put some Employee objects into a hash map:

Map<Integer, Employee> employeeMap = new HashMap<>();
employeeMap.put(1, new Employee(UUID.randomUUID(), "Elvis", 50));
employeeMap.put(2, new Employee(UUID.randomUUID(), "Marylin", 18));
employeeMap.put(3, new Employee(UUID.randomUUID(), "Freddie", 25));
employeeMap.put(4, null);
employeeMap.put(5, new Employee(UUID.randomUUID(), "Mario", 43));
employeeMap.put(6, new Employee(UUID.randomUUID(), "John", 35));
employeeMap.put(7, new Employee(UUID.randomUUID(), "Julia", 55));

Note the null value for key 4. Let’s also define a default Employee object:

Employee defaultEmployee = new Employee(UUID.fromString("00000000-0000-0000-0000-000000000000"), "", -1);

Java 8 includes a new method called “getOrDefault” on the Map interface. It accepts a key, like the “get” method on the Map, but it also accepts a default object that will be returned if the key does not exist.

Can you guess what the below code will return?

Employee employee = employeeMap.getOrDefault(4, defaultEmployee);

“employee” will be null of course, as key 4 exists and its value is null. However, if you simply call the “get” method with 4 as the key input then you don’t know exactly how to interpret the null result: does 4 exist as key in the map and its value is null or does the key 4 not exist at all in the map? With getOrDefault returning 0 in this case you can be 100% sure that a null response is unambiguous.

Let’s see what the below bit of code returns:

Employee employee = employeeMap.getOrDefault(12, defaultEmployee);

This time it returns the default employee as the key 12 does not exist in the map.

View all posts related to Java here.

How to merge two Maps in Java 8

The Map interface has been extended with the “merge” function in Java 8. Let’s see an example on how to use it.

Consider the following Empolyee class:

public class Employee
{
    private UUID id;
    private String name;
    private int age;

    public Employee(UUID id, String name, int age)
    {
        this.id = id;
        this.name = name;
        this.age = age;
    }
        
    public UUID getId()
    {
        return id;
    }

    public void setId(UUID id)
    {
        this.id = id;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }    
    
    public int getAge()
    {
        return age;
    }

    public void setAge(int age)
    {
        this.age = age;
    }
}

Let’s say that we have the following two maps where the key is an indicator of the employees’ performance and the value is the list of employees that fall into that category:

List<Employee> averageMapOne = new ArrayList<>();
averageMapOne.add(new Employee(UUID.randomUUID(), "Elvis", 50));
averageMapOne.add(new Employee(UUID.randomUUID(), "Marylin", 18));
List<Employee> poorMapOne = new ArrayList<>();
poorMapOne.add(new Employee(UUID.randomUUID(), "Mario", 43));
poorMapOne.add(new Employee(UUID.randomUUID(), "John", 35));
List<Employee> excellentMapOne = new ArrayList<>();
excellentMapOne.add(new Employee(UUID.randomUUID(), "Julia", 55));
List<Employee> okMapOne = new ArrayList<>();
okMapOne.add(new Employee(UUID.randomUUID(), "Nick", 43));
okMapOne.add(new Employee(UUID.randomUUID(), "Richard", 61));

Map<String, List<Employee>> employeeMapOne = new HashMap<>();
employeeMapOne.put("average", averageMapOne);
employeeMapOne.put("poor", poorMapOne);
employeeMapOne.put("excellent", excellentMapOne);
employeeMapOne.put("OK", okMapOne);

List<Employee> averageMapTwo = new ArrayList<>();
averageMapTwo.add(new Employee(UUID.randomUUID(), "Lotta", 52));
averageMapTwo.add(new Employee(UUID.randomUUID(), "Eva", 42));
averageMapTwo.add(new Employee(UUID.randomUUID(), "Mark", 24));
List<Employee> poorMapTwo = new ArrayList<>();
poorMapTwo.add(new Employee(UUID.randomUUID(), "Anna", 20));
List<Employee> excellentMapTwo = new ArrayList<>();
excellentMapTwo.add(new Employee(UUID.randomUUID(), "Bertil", 28));
excellentMapTwo.add(new Employee(UUID.randomUUID(), "Cecilia", 36));
excellentMapTwo.add(new Employee(UUID.randomUUID(), "Edit", 21));

Map<String, List<Employee>> employeeMapTwo = new HashMap<>();

employeeMapTwo.put("average", averageMapTwo);
employeeMapTwo.put("poor", poorMapTwo);
employeeMapTwo.put("excellent", excellentMapTwo);

Our goal is to merge map 2 into map 1 in way that all employees that fall into the same category will have all employees in a joined list. E.g. “average” will have 5 employees: Elvis, Marylin, Lotta, Eva and Mark.

The solution requires an understanding of the Java Stream API.

Here’s an explanation of the below bit of code:

  • We open a stream on the entry set of the first map
  • Then we iterate through the map using the forEach method
  • We pass a lambda expression into forEach where we want to do “something” with each entry in the entry set
  • This “something” is the actual merge operation of map 2
  • The merge operation accepts a key and a value which will be the key and value of map 1
  • Merge also accepts a BiFunction which will be used in case the key already exists in map 2
  • In this function we pass in the lists from map 1 and map 2, add the contents of map 1 into map 2 and return the new list

Here’s the code that will do the trick:

employeeMapOne.entrySet().stream()
        .forEach(entry -> employeeMapTwo.merge(entry.getKey(), entry.getValue(),
                (listTwo, listOne) ->
                        {
                            listOne.addAll(listTwo);
                            return listOne;
                        }));

View all posts related to Java here.

Compressing and decompressing strings with BZip2 in .NET C#

There are times when you need to return a large text from a web service. The large text will then need to be handled by the recipient. In order to reduce the size of the message you can combine two simple techniques:

  • Compress the string value with a compression algorithm, such as BZip2
  • Base64 encode the resulting byte array

You will be able to send the base 64 encoded compressed string over the wire.

You’ll need to import the following NuGet package to use BZip2:

sharpziplib nuget

This is how you can compress a string and base 64 encode it:

string largeUncompressedText = "<root><value size=\"xxl\">This is a large text</value></root>";
string largeCompressedText = string.Empty;
using (MemoryStream source = new MemoryStream(Encoding.UTF8.GetBytes(largeUncompressedText)))
{
	using (MemoryStream target = new MemoryStream())
	{
		BZip2.Compress(source, target, true, 4096);
		byte[] targetByteArray = target.ToArray();
		largeCompressedText = Convert.ToBase64String(targetByteArray);
	}
}

The variable “largeCompressedText” can be sent to a listener who will be able to read it as follows:

byte[] largeCompressedTextAsBytes = Convert.FromBase64String(largeCompressedText);
using (MemoryStream source = new MemoryStream(largeCompressedTextAsBytes))
{
	using (MemoryStream target = new MemoryStream())
	{
		BZip2.Decompress(source, target, true);
		string uncompressedString = Encoding.UTF8.GetString(target.ToArray());
		Console.WriteLine(uncompressedString);
	}
}

The example is not perfect in a sense that largeCompressedText will be bigger than the actual source string but you’ll see the benefits with much larger texts.

View all posts related to string and text operations here.

Using Amazon DynamoDb with the AWS.NET API Part 3: table operations

Introduction

In the previous post we started coding our DynamoDb demo application. In particular we saw how to extract information about tables in DynamoDb.

In this post we’ll look at various table operations, more specifically how to create, delete and update tables in DynamoDb.

Open the demo application we started working on previously and let’s get to it!

Creating tables

We can set the properties of our table through the CreateTableRequest object. This is where we set the table name, the provisioned read and write throughput, the key schema and their attribute definitions, and the secondary indexes. The following code sets all of these except the secondary keys.

Insert the following method into DynamoDbDemoService.cs:

public void CreateNewTableDemo(string tableName)
{
	try
	{
		using (IAmazonDynamoDB client = GetDynamoDbClient())
		{				
			CreateTableRequest createTableRequest = new CreateTableRequest();
			createTableRequest.TableName = tableName;
			createTableRequest.ProvisionedThroughput = new ProvisionedThroughput() { ReadCapacityUnits = 1, WriteCapacityUnits = 1 };
			createTableRequest.KeySchema = new List<KeySchemaElement>()
			{
				new KeySchemaElement()
				{
					AttributeName = "Name"
					, KeyType = KeyType.HASH
				},
				new KeySchemaElement()
				{
					AttributeName = "Birthdate"
					, KeyType = KeyType.RANGE
				}
			};
			createTableRequest.AttributeDefinitions = new List<AttributeDefinition>()
			{
				new AttributeDefinition(){AttributeName = "Name", AttributeType = ScalarAttributeType.S}
				, new AttributeDefinition(){AttributeName = "Birthdate", AttributeType = ScalarAttributeType.S}
			};
			CreateTableResponse createTableResponse = client.CreateTable(createTableRequest);

			TableDescription tableDescription = createTableResponse.TableDescription;
			Debug.WriteLine(string.Format("Table {0} creation command sent to Amazon. Current table status: {1}", tableName, tableDescription.TableStatus));

			String tableStatus = tableDescription.TableStatus.Value.ToLower();
			while (tableStatus != "active")
			{
				Debug.WriteLine(string.Format("Table {0} not yet active, waiting...", tableName));
				Thread.Sleep(2000);
				DescribeTableRequest describeTableRequest = new DescribeTableRequest(tableName);
				DescribeTableResponse describeTableResponse = client.DescribeTable(describeTableRequest);
				tableDescription = describeTableResponse.Table;
				tableStatus = tableDescription.TableStatus.Value.ToLower();
				Debug.WriteLine(string.Format("Latest status of table {0}: {1}", tableName, tableStatus));
			}

			Debug.WriteLine(string.Format("Table creation loop exited for table {0}, final status: {1}", tableName, tableStatus));
		}
	}
	catch (AmazonDynamoDBException exception)
	{
		Debug.WriteLine(string.Concat("Exception while creating new DynamoDb table: {0}", exception.Message));
		Debug.WriteLine(String.Concat("Error code: {0}, error type: {1}", exception.ErrorCode, exception.ErrorType));
	}
}

You’ll recognise the Hash and Range key types from the first post of this series. The KeySchemaElement object lets you define the name and type of the key. The AttributeDefinition will further let you define the schema of the keys.

We set up a compound key that consists of “Name” and “Birthdate”. We’ll insert some people-related objects in the next post where each record will have a compound key of Name and Birthdate, i.e. the name and birthday properties will uniquely identify each record. You’ll note the attribute type of S in the AttributeDefinition object. S stands for String. We saw earlier that DynamoDb supports 3 data types: strings, numbers and binary data. Numbers are represented by ScalarAttributeType.N and binary types by ScalarAttributeType.B. As dates are not supported by DynamoDb we’ll go with strings instead.

After constructing the CreateTableRequest object we call the CreateTable method of the DynamoDb client. The CreateTableResponse includes a TableDescription object which helps us read the initial status of the table. It should be “creating” at first. We then enter a loop where we continuously check the status of the new table until it reaches status “active”.

Let’s call the method from Main as follows:

DynamoDbDemoService service = new DynamoDbDemoService();
service.CreateNewTableDemo("a-demo-table");

If all goes well then you should see output similar to the following:

Table a-demo-table creation command sent to Amazon. Current table status: CREATING
Table a-demo-table not yet active, waiting…
Latest status of table a-demo-table: creating
Table a-demo-table not yet active, waiting…
Latest status of table a-demo-table: creating
Table a-demo-table not yet active, waiting…
Latest status of table a-demo-table: active
Table creation loop exited for table a-demo-table, final status: active

The table is up and running in the DynamoDB GUI as well:

Demo table created visible in Amazon DynamoDb UI

It’s obviously not allowed to have two tables with the same name. Try running the same code again and you should get an exception message:

Exception while creating new DynamoDb table: {0}Table already exists: a-demo-table
Error code: {0}, error type: {1}ResourceInUseExceptionUnknown

Updating a table

You can update a table through the UpdateTableRequest object. You can update the read and write throughput values and the global secondary indexes, nothing else. The following method demonstrates the usage of the object:

public void UpdateTableDemo(string tableName)
{
	try
	{
		using (IAmazonDynamoDB client = GetDynamoDbClient())
		{
			UpdateTableRequest updateTableRequest = new UpdateTableRequest();
			updateTableRequest.TableName = tableName;
			updateTableRequest.ProvisionedThroughput = new ProvisionedThroughput() { ReadCapacityUnits = 2, WriteCapacityUnits = 2 };					
			UpdateTableResponse updateTableResponse = client.UpdateTable(updateTableRequest);
			TableDescription tableDescription = updateTableResponse.TableDescription;
			Debug.WriteLine(string.Format("Update table command sent to Amazon for table {0}, status after update: {1}", tableName
				, tableDescription.TableStatus));
		}			
	}
	catch (AmazonDynamoDBException exception)
	{
		Debug.WriteLine(string.Concat("Exception while updating DynamoDb table: {0}", exception.Message));
		Debug.WriteLine(String.Concat("Error code: {0}, error type: {1}", exception.ErrorCode, exception.ErrorType));
	}
}

Run the method like this:

DynamoDbDemoService service = new DynamoDbDemoService();
//service.UpdateTableDemo("a-demo-table");

Here’s the Debug output:

Update table command sent to Amazon for table a-demo-table, status after update: UPDATING

…and the table has been updated with the new throughput values:

Demo table update visible in Amazon DynamoDb UI

Deleting a table

Deleting a table is as trivial as providing the name of the table to the DeleteTableRequest object and sending it to AWS by way of the DynamoDb client:

public void DeleteTableDemo(string tableName)
{
	try
	{
		using (IAmazonDynamoDB client = GetDynamoDbClient())
		{
			DeleteTableRequest deleteTableRequest = new DeleteTableRequest(tableName);
			DeleteTableResponse deleteTableResponse = client.DeleteTable(deleteTableRequest);
			TableDescription tableDescription = deleteTableResponse.TableDescription;
			TableStatus tableStatus = tableDescription.TableStatus;
			Debug.WriteLine(string.Format("Delete table command sent to Amazon for table {0}, status after deletion: {1}", tableName
				, tableDescription.TableStatus));
		}
	}
	catch (AmazonDynamoDBException exception)
	{
		Debug.WriteLine(string.Concat("Exception while deleting DynamoDb table: {0}", exception.Message));
		Debug.WriteLine(String.Concat("Error code: {0}, error type: {1}", exception.ErrorCode, exception.ErrorType));
	}
}

Run the method as follows:

DynamoDbDemoService service = new DynamoDbDemoService();
service.DeleteTableDemo("a-demo-table");

The table won’t be deleted immediately but have the status “deleting” at first before the table is finally deleted:

Delete table command sent to Amazon for table a-demo-table, status after deletion: DELETING

Here’s what it looks like in the DynamoDb GUI:

Demo table deleting status in Amazon DynamoDb UI

We’ll explore how to insert records into a DynamoDb table in the next post.

View all posts related to Amazon Web Services and Big Data here.

Compressing and decompressing files with BZip2 in .NET C#

BZip2 is yet another data compression algorithm, similar to GZip and Deflate. There’s no native support for BZip2 (de)compression in .NET but there’s a NuGet package provided by icsharpcode.net.

You’ll need to import the following NuGet package to use BZip2:

sharpziplib nuget

You can compress a file as follows:

FileInfo fileToBeZipped = new FileInfo(@"c:\bzip2\logfile.txt");
FileInfo zipFileName = new FileInfo(string.Concat(fileToBeZipped.FullName, ".bz2"));
using (FileStream fileToBeZippedAsStream = fileToBeZipped.OpenRead())
{
	using (FileStream zipTargetAsStream = zipFileName.Create())
	{
		try
		{
			BZip2.Compress(fileToBeZippedAsStream, zipTargetAsStream, true, 4096);
		}
		catch (Exception ex)
		{
			Console.WriteLine(ex.Message);
		}
	}
}

…and this is how you can decompress the resulting bz2 file again:

using (FileStream fileToDecompressAsStream = zipFileName.OpenRead())
{
	string decompressedFileName = @"c:\bzip2\decompressed.txt";
	using (FileStream decompressedStream = File.Create(decompressedFileName))
	{
		try
		{
			BZip2.Decompress(fileToDecompressAsStream, decompressedStream, true);
		}
		catch (Exception ex)
		{
			Console.WriteLine(ex.Message);
		}
	}
}

Read all posts dedicated to file I/O here.

How to compress and decompress files with Deflate in .NET C#

We saw the usage of the GZipStream object in this post. GZipStream follows the GZip compression algorithm which is actually based on DEFLATE and includes some headers. As a result GZip files are somewhat bigger than DEFLATE files.

Just like with GZip, DEFLATE compresses a single file and does not hold multiple files in a zip archive fashion. It is represented by the DeflateStream object and is used in much the same way as a GZipStream. The example code is in fact almost identical.

This is how to compress a file:

FileInfo fileToBeDeflateZipped = new FileInfo(@"c:\deflate\logfile.txt");
FileInfo deflateZipFileName = new FileInfo(string.Concat(fileToBeDeflateZipped.FullName, ".cmp"));

using (FileStream fileToBeZippedAsStream = fileToBeDeflateZipped.OpenRead())
{
	using (FileStream deflateZipTargetAsStream = deflateZipFileName.Create())
	{
		using (DeflateStream deflateZipStream = new DeflateStream(deflateZipTargetAsStream, CompressionMode.Compress))
		{
			try
			{
				fileToBeZippedAsStream.CopyTo(deflateZipStream);
			}
			catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

…and here’s how you can decompress a file:

using (FileStream fileToDecompressAsStream = deflateZipFileName.OpenRead())
{
	string decompressedFileName = @"c:\deflate\decompressed.txt";
	using (FileStream decompressedStream = File.Create(decompressedFileName))
	{
		using (DeflateStream decompressionStream = new DeflateStream(fileToDecompressAsStream, CompressionMode.Decompress))
		{
			try
			{
				decompressionStream.CopyTo(decompressedStream);
			}
			catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

Read all posts dedicated to file I/O here.

Using Amazon DynamoDb with the AWS.NET API Part 2: code beginnings

Introduction

In the previous post we went through the basics of Amazon DynamoDb. It is Amazon’s take on NoSql where you can store unstructured data in the cloud. We talked about primary keys and available data types. We also created and deleted our first table.

In this post we’ll install the .NET SDK and start building some test code.

Note that we’ll be concentrating on showing and explaining the technical code examples related to AWS. We’ll ignore software principles like SOLID and layering so that we can stay focused. It’s your responsibility to organise your code properly. There are numerous posts on this blog that take up topics related to software architecture.

Installing the SDK

If you already have the .NET AWS SDK installed then you can ignore the installation bit of this section. You’ll only need to create a new project in Visual Studio.

The Amazon .NET SDK is available through NuGet. Open Visual Studio 2012/2013 and create a new C# console application called DynamoDbDemo. The purpose of this application will be to demonstrate the different parts of the SDK around DynamoDb. In reality the DynamoDb handler could be any type of application:

  • A website
  • A Windows/Android/iOS app
  • A Windows service
  • etc.

…i.e. any application that’s capable of sending HTTP/S requests to a service endpoint. We’ll keep it simple and not waste time with view-related tasks.

Install the following NuGet package:

AWS SDK NuGet package

Preparations

We cannot just call the services within the AWS SDK without proper authentication. This is an important reference page to handle your credentials in a safe way. We’ll the take the recommended approach and create a profile in the SDK Store and reference it from app.config.

This series is not about AWS authentication so we won’t go into temporary credentials but later on you may be interested in that option too. Since we’re programmers and it takes a single line of code to set up a profile we’ll go with the programmatic options. Add the following line to Main:

Amazon.Util.ProfileManager.RegisterProfile("demo-aws-profile", "your access key id", "your secret access key");

I suggest you remove the code from the application later on in case you want to distribute it. Run the application and it should execute without exceptions. Next open app.config and add the appSettings section with the following elements:

<appSettings>
        <add key="AWSProfileName" value="demo-aws-profile"/>
</appSettings>

First demo: reading tables information

We’ll put all our test code into a separate class. Insert a cs file called DynamoDbDemoService. We’ll need a method to build a handle to the service which is of type IAmazonDynamoDB:

private IAmazonDynamoDB GetDynamoDbClient()
{
	return new AmazonDynamoDBClient(RegionEndpoint.EUWest1);
}

Note that we didn’t need to provide our credentials here. They will be extracted automatically using the profile name in the config file. You may need to adjust the region according to your preferences or what you selected in the previous part.

Let’s first find out what table names we have in DynamoDb. This is almost a trivial task:

public List<string> GetTablesList()
{
	using (IAmazonDynamoDB client = GetDynamoDbClient())
	{
		ListTablesResponse listTablesResponse = client.ListTables();
		return listTablesResponse.TableNames;
	}
}

Notice the return type of “ListTables()”, ListTablesResponse. Request and Response objects are abound in the Amazon SDK, this is one example. There’s actually an overload of ListTables which accepts a ListTablesRequest object. The ListTablesRequest object allows you to set a limit on the number of table names returned:

public List<string> GetTablesList()
{
	using (IAmazonDynamoDB client = GetDynamoDbClient())
	{
		ListTablesRequest listTablesRequest = new ListTablesRequest();
		listTablesRequest.Limit = 5;
		ListTablesResponse listTablesResponse = client.ListTables(listTablesRequest);
		return listTablesResponse.TableNames;
	}
}

Let’s call this from Main:

static void Main(string[] args)
{
	DynamoDbDemoService service = new DynamoDbDemoService();
	List<string> dynamoDbTables = service.GetTablesList();
}

If you followed through the previous post or if you already have tables in DynamoDb then the above bit of code should return at least one table name.

So now we can retrieve all table names but we also want to find out some details on each table. The following method will do just that:

public void GetTablesDetails()
{
	List<string> tables = GetTablesList();
	using (IAmazonDynamoDB client = GetDynamoDbClient())
	{
		foreach (string table in tables)
		{
			DescribeTableRequest describeTableRequest = new DescribeTableRequest(table);
			DescribeTableResponse describeTableResponse = client.DescribeTable(describeTableRequest);
			TableDescription tableDescription = describeTableResponse.Table;
			Debug.WriteLine(string.Format("Printing information about table {0}:", tableDescription.TableName));
			Debug.WriteLine(string.Format("Created at: {0}", tableDescription.CreationDateTime));
			List<KeySchemaElement> keySchemaElements = tableDescription.KeySchema;
			foreach (KeySchemaElement schema in keySchemaElements)
			{
				Debug.WriteLine(string.Format("Key name: {0}, key type: {1}", schema.AttributeName, schema.KeyType));
			}
			Debug.WriteLine(string.Format("Item count: {0}", tableDescription.ItemCount));
			ProvisionedThroughputDescription throughput = tableDescription.ProvisionedThroughput;
			Debug.WriteLine(string.Format("Read capacity: {0}", throughput.ReadCapacityUnits));
			Debug.WriteLine(string.Format("Write capacity: {0}", throughput.WriteCapacityUnits));
			List<AttributeDefinition> tableAttributes = tableDescription.AttributeDefinitions;
			foreach (AttributeDefinition attDefinition in tableAttributes)
			{
				Debug.WriteLine(string.Format("Table attribute name: {0}", attDefinition.AttributeName));
				Debug.WriteLine(string.Format("Table attribute type: {0}", attDefinition.AttributeType));
			}
			Debug.WriteLine(string.Format("Table size: {0}b", tableDescription.TableSizeBytes));
			Debug.WriteLine(string.Format("Table status: {0}", tableDescription.TableStatus));
			Debug.WriteLine("====================================================");
					
		}
	}
}

We can extract the details of a table through the DescribeTableRequest object which is passed into the DescribeTable method. The DescribeTable method returns a DescribeTableResponse object which in turn includes another object of type TableDescription. TableDescription holds a number of properties that describe a table in DynamoDb in the selected region. The above code extracts the following properties:

  • Creation date
  • The key schema, i.e. if the primary key is of type Hash or a composite key of Hash and Range
  • The number of records in the table
  • The provisioned read and write throughput
  • The table attribute names and their types, i.e. string, number or binary
  • The table size in bytes
  • The table status, i.e. if it’s Active, Creating or Deleting

The TableDescription method also includes properties to check the global and local secondary indexes but I’ve ignored them in the demo.

You can call the above method from Main as follows:

static void Main(string[] args)
{
	DynamoDbDemoService service = new DynamoDbDemoService();
	service.GetTablesDetails();
}

Here’s an example output:

Printing information about table Application:
Created at: 2014-10-28 09:53:57
Key name: Id, key type: HASH
Item count: 9
Read capacity: 1
Write capacity: 1
Table attribute name: Id
Table attribute type: S
Table size: 123b
Table status: ACTIVE

You can see that the table attribute name Id is the same as the Key called Id. Attribute type “S” means String – we’ll go through these types in other posts of this series.

In the next post we’ll create a new table and insert records into it in code.

View all posts related to Amazon Web Services and Big Data here.

Concatenate strings with the StringJoiner class in Java 8

Java 8 introduces a new object which enables you to join individual strings: the StringJoiner.

The StringJoiner has two overloads. The simpler one accepts a delimiter:

StringJoiner sj = new StringJoiner(" | ");
sj.add("Hello").add("my").add("dear").add("world!");

System.out.println(sj.toString());

This prints the following:

Hello | my | dear | world!

Note how the StringJoiner was smart enough not put the delimiter after the last string.

In case you don’t need any delimiter then you can just pass in an empty string:

StringJoiner sj = new StringJoiner("");
sj.add("Hello ").add("my").add(" dear").add(" world!");

This will print Hello my dear world! accordingly.

This simpler overload of StringJoiner can be called indirectly using the String.join static method:

String res = String.join(" | ", "Hello", "my", "dear", "world");

The String.join method has another version where you can pass in an iterable class such as an array or array list of strings instead of specifying the elements one by one like above.

The other overload of StringJoiner allows you to specify an opening and ending string to encapsulate the concatenated string:

StringJoiner sj = new StringJoiner(" | ", "-=", "=-");
sj.add("Hello").add("my").add("dear").add("world!");

The result looks as follows:

-=Hello | my | dear | world!=-

We saw an example of Collectors.joining in this post on the Stream API. The joining method uses a StringJoiner behind the scenes.

View all posts related to Java here.

Exploring a directory with the Java 8 Stream API

We saw an example of using the Java 8 Stream API in File I/O in this post. We saw how the Files object was enhanced with the lines() method to open a line reader stream to a text file.

There are other enhancements related to streams that make is simple to explore a directory on your hard drive. The following code example will collect all folders and files within the c:\gitrepos folder and add them to an ArrayList:

Path gitReposFolderPath = Paths.get("c:\\gitrepos");
gitReposFolderPath.toFile().getName();
try (Stream<Path> foldersWithinGitReposStream = Files.list(gitReposFolderPath))            
{
    List<String> elements = new ArrayList<>();
    foldersWithinGitReposStream.forEach(p -> elements.add(p.toFile().getName()));            
    System.out.println(elements);
}
catch (IOException ioe)
{

}

I got the following output:

[cryptographydotnet, dotnetformsbasedmvc5, entityframeworksixdemo, owinkatanademo, signalrdemo, singletondemoforcristian, text.txt, webapi2demo, windowsservicedemo]

The code returns both files and folders one level below the top directory, i.e. the “list” method does not dive into the subfolders. I put a text file into the folder – text.txt – just to test whether in fact all elements are returned.

Say you only need files – you can use the filter method:

foldersWithinGitReposStream.filter(p -> p.toFile().isFile()).forEach(p -> elements.add(p.toFile().getName())); 

This will only collect text.txt.

Let’s try something slightly more complex. We’ll organise the elements within the directory into a Map of Boolean and List of Paths. The key indicates whether the group of files are directories or not. We can use the collect method that we saw in this post:

try (Stream<Path> foldersWithinGitReposStream = Files.list(gitReposFolderPath))            
{
    Map<Boolean, List<Path>> collect = foldersWithinGitReposStream.collect(Collectors.groupingBy(p -> p.toFile().isDirectory()));
    System.out.println(collect);
}

This prints the following:

{false=[c:\gitrepos\text.txt], true=[c:\gitrepos\cryptographydotnet, c:\gitrepos\dotnetformsbasedmvc5, c:\gitrepos\entityframeworksixdemo, c:\gitrepos\owinkatanademo, c:\gitrepos\signalrdemo, c:\gitrepos\singletondemoforcristian, c:\gitrepos\webapi2demo, c:\gitrepos\windowsservicedemo]}

So we successfully grouped the paths.

As mentioned above the “list” method goes only one level deep. The “walk” method in turn digs deeper and extracts sub-directories as well:

try (Stream<Path> foldersWithinGitReposStream = Files.walk(gitReposFolderPath))
{
    List<String> elements = new ArrayList<>();
    foldersWithinGitReposStream.filter(p -> p.toFile().isFile()).forEach(p -> elements.add(p.toFile().getAbsolutePath()));
    System.out.println(elements);
}

We can also instruct the walk method to go n levels down with an extra integer argument:

try (Stream<Path> foldersWithinGitReposStream = Files.walk(gitReposFolderPath, 3))

View all posts related to Java here.

How to compress and decompress files with GZip in .NET C#

You have probably seen compressed files with the “gz” extension. These are files that hold a single compressed file according to the GZIP specifications.

GZip files are represented by the GZipStream object in .NET. It’s important to note that the GZip format doesn’t support adding multiple files to the same .gz file. If you need to insert multiple files in a GZip file then you’ll need to create a “tar” file first which bundles the individual files and then compresses the tar file itself. The result will be a “.tar.gz” file. At present tar files are not supported in .NET. They are supported by the ICSharpCode SharpZipLib library available here. We’ll look at tar files in another post soon.

With that in mind let’s see how a single file can be gzipped:

FileInfo fileToBeGZipped = new FileInfo(@"c:\gzip\logfile.txt");
FileInfo gzipFileName = new FileInfo(string.Concat(fileToBeGZipped.FullName, ".gz"));
			
using (FileStream fileToBeZippedAsStream = fileToBeGZipped.OpenRead())
{
	using (FileStream gzipTargetAsStream = gzipFileName.Create())
        {
		using (GZipStream gzipStream = new GZipStream(gzipTargetAsStream, CompressionMode.Compress))
		{
			try
			{
				fileToBeZippedAsStream.CopyTo(gzipStream);
			}
			catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

…and this is how you can decompress the gz file:

using (FileStream fileToDecompressAsStream = gzipFileName.OpenRead())
{
	string decompressedFileName = @"c:\gzip\decompressed.txt";
	using (FileStream decompressedStream = File.Create(decompressedFileName))
	{
		using (GZipStream decompressionStream = new GZipStream(fileToDecompressAsStream, CompressionMode.Decompress))
		{
			try
			{
				decompressionStream.CopyTo(decompressedStream);
			}
         		catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

Read all posts dedicated to file I/O here.

Elliot Balynn's Blog

A directory of wonderful thoughts

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Once Upon a Camayoc

ARCHIVED: Bite-size insight on Cyber Security for the not too technical.