Packing and unpacking files using Zip archives in .NET

We’ve looked at a couple of (de)compression techniques available in .NET in previous posts, see link below. Here we’ll look at how to compress multiple files into well-known ZIP files using .NET.

.NET4.5 has added native support for ZIP files, though you need to add the following library to reach the new functions:

New system compression dll

Say you’d like to compress all files within a folder:

DirectoryInfo filesToBeZipped = new DirectoryInfo(@"c:\zip\start");
FileInfo zipFileName = new FileInfo(@"c:\zip\zipped.zip");
ZipFile.CreateFromDirectory(filesToBeZipped.FullName, zipFileName.FullName);

…and this how you can unzip it:

DirectoryInfo extractTo = new DirectoryInfo(@"c:\zip\unzip");
ZipFile.ExtractToDirectory(zipFileName.FullName, extractTo.FullName);

The above code examples will zip and unzip all files in the directory. However, there are times when you’d like to access the individual files in the ZIP archive or add new files to an existing zip file. For this you’ll need to add one more library reference:

New system compression dll for zip archive

The following code will create the zip file and then look at each archived file one by one. If a file is larger than a certain limit then it’s extracted to a special folder:

DirectoryInfo filesToBeZipped = new DirectoryInfo(@"c:\zip\start");
FileInfo zipFileName = new FileInfo(@"c:\zip\zipped.zip");
ZipFile.CreateFromDirectory(filesToBeZipped.FullName, zipFileName.FullName);
DirectoryInfo extractTo = new DirectoryInfo(@"c:\zip\unzip_individual");
if (!extractTo.Exists)
{
	extractTo.Create();
}
using (ZipArchive zipArchive = ZipFile.OpenRead(zipFileName.FullName))
{
	foreach (ZipArchiveEntry zipArchiveEntry in zipArchive.Entries)
	{
		Console.WriteLine(zipArchiveEntry.FullName);
		if (zipArchiveEntry.Length > 100)
		{
			zipArchiveEntry.ExtractToFile(Path.Combine(extractTo.FullName, zipArchiveEntry.FullName));
		}
	}
}

Here’s how you can add a existing file to an existing ZIP archive:

FileInfo zipFileName = new FileInfo(@"c:\zip\zipped.zip");
FileInfo newFileToBeAdded = new FileInfo(@"c:\temp\result.txt");
using (FileStream zipToBeExtended = new FileStream(zipFileName.FullName, FileMode.Open))
{
	using (ZipArchive zipArchive = new ZipArchive(zipToBeExtended, ZipArchiveMode.Update))
	{
		ZipArchiveEntry newEntry = zipArchive.CreateEntryFromFile(newFileToBeAdded.FullName, "result-zipped.txt");
	}
}

This code will add “newFileToBeAdded” to the existing zip archive and name it “result-zipped.txt”.

You can also create a new zip archive entry and add content to it on the fly:

FileInfo zipFileName = new FileInfo(@"c:\zip\zipped.zip");
using (FileStream zipToBeExtended = new FileStream(zipFileName.FullName, FileMode.Open))
{
	using (ZipArchive zipArchive = new ZipArchive(zipToBeExtended, ZipArchiveMode.Update))
	{
		ZipArchiveEntry newZipEntryOnTheFly = zipArchive.CreateEntry("new-result.txt");
		using (StreamWriter streamWriter = new StreamWriter(newZipEntryOnTheFly.Open()))
		{
			streamWriter.WriteLine("Hello from the brand new zip archive entry!");
		}
	}
}

Read all posts dedicated to file I/O here.

Compressing and decompressing files with BZip2 in .NET C#

BZip2 is yet another data compression algorithm, similar to GZip and Deflate. There’s no native support for BZip2 (de)compression in .NET but there’s a NuGet package provided by icsharpcode.net.

You’ll need to import the following NuGet package to use BZip2:

sharpziplib nuget

You can compress a file as follows:

FileInfo fileToBeZipped = new FileInfo(@"c:\bzip2\logfile.txt");
FileInfo zipFileName = new FileInfo(string.Concat(fileToBeZipped.FullName, ".bz2"));
using (FileStream fileToBeZippedAsStream = fileToBeZipped.OpenRead())
{
	using (FileStream zipTargetAsStream = zipFileName.Create())
	{
		try
		{
			BZip2.Compress(fileToBeZippedAsStream, zipTargetAsStream, true, 4096);
		}
		catch (Exception ex)
		{
			Console.WriteLine(ex.Message);
		}
	}
}

…and this is how you can decompress the resulting bz2 file again:

using (FileStream fileToDecompressAsStream = zipFileName.OpenRead())
{
	string decompressedFileName = @"c:\bzip2\decompressed.txt";
	using (FileStream decompressedStream = File.Create(decompressedFileName))
	{
		try
		{
			BZip2.Decompress(fileToDecompressAsStream, decompressedStream, true);
		}
		catch (Exception ex)
		{
			Console.WriteLine(ex.Message);
		}
	}
}

Read all posts dedicated to file I/O here.

How to compress and decompress files with Deflate in .NET C#

We saw the usage of the GZipStream object in this post. GZipStream follows the GZip compression algorithm which is actually based on DEFLATE and includes some headers. As a result GZip files are somewhat bigger than DEFLATE files.

Just like with GZip, DEFLATE compresses a single file and does not hold multiple files in a zip archive fashion. It is represented by the DeflateStream object and is used in much the same way as a GZipStream. The example code is in fact almost identical.

This is how to compress a file:

FileInfo fileToBeDeflateZipped = new FileInfo(@"c:\deflate\logfile.txt");
FileInfo deflateZipFileName = new FileInfo(string.Concat(fileToBeDeflateZipped.FullName, ".cmp"));

using (FileStream fileToBeZippedAsStream = fileToBeDeflateZipped.OpenRead())
{
	using (FileStream deflateZipTargetAsStream = deflateZipFileName.Create())
	{
		using (DeflateStream deflateZipStream = new DeflateStream(deflateZipTargetAsStream, CompressionMode.Compress))
		{
			try
			{
				fileToBeZippedAsStream.CopyTo(deflateZipStream);
			}
			catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

…and here’s how you can decompress a file:

using (FileStream fileToDecompressAsStream = deflateZipFileName.OpenRead())
{
	string decompressedFileName = @"c:\deflate\decompressed.txt";
	using (FileStream decompressedStream = File.Create(decompressedFileName))
	{
		using (DeflateStream decompressionStream = new DeflateStream(fileToDecompressAsStream, CompressionMode.Decompress))
		{
			try
			{
				decompressionStream.CopyTo(decompressedStream);
			}
			catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

Read all posts dedicated to file I/O here.

Exploring a directory with the Java 8 Stream API

We saw an example of using the Java 8 Stream API in File I/O in this post. We saw how the Files object was enhanced with the lines() method to open a line reader stream to a text file.

There are other enhancements related to streams that make is simple to explore a directory on your hard drive. The following code example will collect all folders and files within the c:\gitrepos folder and add them to an ArrayList:

Path gitReposFolderPath = Paths.get("c:\\gitrepos");
gitReposFolderPath.toFile().getName();
try (Stream<Path> foldersWithinGitReposStream = Files.list(gitReposFolderPath))            
{
    List<String> elements = new ArrayList<>();
    foldersWithinGitReposStream.forEach(p -> elements.add(p.toFile().getName()));            
    System.out.println(elements);
}
catch (IOException ioe)
{

}

I got the following output:

[cryptographydotnet, dotnetformsbasedmvc5, entityframeworksixdemo, owinkatanademo, signalrdemo, singletondemoforcristian, text.txt, webapi2demo, windowsservicedemo]

The code returns both files and folders one level below the top directory, i.e. the “list” method does not dive into the subfolders. I put a text file into the folder – text.txt – just to test whether in fact all elements are returned.

Say you only need files – you can use the filter method:

foldersWithinGitReposStream.filter(p -> p.toFile().isFile()).forEach(p -> elements.add(p.toFile().getName())); 

This will only collect text.txt.

Let’s try something slightly more complex. We’ll organise the elements within the directory into a Map of Boolean and List of Paths. The key indicates whether the group of files are directories or not. We can use the collect method that we saw in this post:

try (Stream<Path> foldersWithinGitReposStream = Files.list(gitReposFolderPath))            
{
    Map<Boolean, List<Path>> collect = foldersWithinGitReposStream.collect(Collectors.groupingBy(p -> p.toFile().isDirectory()));
    System.out.println(collect);
}

This prints the following:

{false=[c:\gitrepos\text.txt], true=[c:\gitrepos\cryptographydotnet, c:\gitrepos\dotnetformsbasedmvc5, c:\gitrepos\entityframeworksixdemo, c:\gitrepos\owinkatanademo, c:\gitrepos\signalrdemo, c:\gitrepos\singletondemoforcristian, c:\gitrepos\webapi2demo, c:\gitrepos\windowsservicedemo]}

So we successfully grouped the paths.

As mentioned above the “list” method goes only one level deep. The “walk” method in turn digs deeper and extracts sub-directories as well:

try (Stream<Path> foldersWithinGitReposStream = Files.walk(gitReposFolderPath))
{
    List<String> elements = new ArrayList<>();
    foldersWithinGitReposStream.filter(p -> p.toFile().isFile()).forEach(p -> elements.add(p.toFile().getAbsolutePath()));
    System.out.println(elements);
}

We can also instruct the walk method to go n levels down with an extra integer argument:

try (Stream<Path> foldersWithinGitReposStream = Files.walk(gitReposFolderPath, 3))

View all posts related to Java here.

How to compress and decompress files with GZip in .NET C#

You have probably seen compressed files with the “gz” extension. These are files that hold a single compressed file according to the GZIP specifications.

GZip files are represented by the GZipStream object in .NET. It’s important to note that the GZip format doesn’t support adding multiple files to the same .gz file. If you need to insert multiple files in a GZip file then you’ll need to create a “tar” file first which bundles the individual files and then compresses the tar file itself. The result will be a “.tar.gz” file. At present tar files are not supported in .NET. They are supported by the ICSharpCode SharpZipLib library available here. We’ll look at tar files in another post soon.

With that in mind let’s see how a single file can be gzipped:

FileInfo fileToBeGZipped = new FileInfo(@"c:\gzip\logfile.txt");
FileInfo gzipFileName = new FileInfo(string.Concat(fileToBeGZipped.FullName, ".gz"));
			
using (FileStream fileToBeZippedAsStream = fileToBeGZipped.OpenRead())
{
	using (FileStream gzipTargetAsStream = gzipFileName.Create())
        {
		using (GZipStream gzipStream = new GZipStream(gzipTargetAsStream, CompressionMode.Compress))
		{
			try
			{
				fileToBeZippedAsStream.CopyTo(gzipStream);
			}
			catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

…and this is how you can decompress the gz file:

using (FileStream fileToDecompressAsStream = gzipFileName.OpenRead())
{
	string decompressedFileName = @"c:\gzip\decompressed.txt";
	using (FileStream decompressedStream = File.Create(decompressedFileName))
	{
		using (GZipStream decompressionStream = new GZipStream(fileToDecompressAsStream, CompressionMode.Decompress))
		{
			try
			{
				decompressionStream.CopyTo(decompressedStream);
			}
         		catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}
	}
}

Read all posts dedicated to file I/O here.

5 ways to write to a file with C# .NET

It’s a common scenario that you need to write some content to a file. Here you see 5 ways to achieve that.

1. Using a FileStream:

private void WriteToAFileWithFileStream()
{
	using (FileStream target = File.Create(@"c:\mydirectory\target.txt"))
	{
		using (StreamWriter writer = new StreamWriter(target))
		{
			writer.WriteLine("Hello world");
		}
	}
}

This will create a new file or overwrite an existing one.

Write and WriteLine have asynchronous versions as well: WriteAsync and WriteLineAsync. If you don’t know how to use async methods that return a Task then start here.

The following variation will append the text to an existing file or create a new one:

using (FileStream target = File.Open(@"c:\mydirectory\target2.txt", FileMode.Append, FileAccess.Write))
{
	using (StreamWriter writer = new StreamWriter(target))
	{
		writer.WriteLine("Hello world");
	}
}

2. Using a StreamWriter directly – to be used for text files:

private static void WriteToAFileWithStreamWriter()
{
	using (StreamWriter writer = File.CreateText(@"c:\mydirectory\target.txt"))
	{
		writer.WriteLine("Bye world");
	}
}

This will create a new file or overwrite an existing one.

The following variation will append the text to an existing file or create a new one:

using (StreamWriter writer = File.AppendText(@"c:\mydirectory\target.txt"))
{
	writer.WriteLine("Bye world");
}

3. Using File:

private static void WriteToAFileWithFile()
{
	File.WriteAllText(@"c:\mydirectory\target.txt", "Hello again World");
}

There’s File.WriteAllBytes as well if you have the contents as a byte array. There’s also a File.AppendAllText method to append to an existing file or create a new one if it doesn’t exist.

4. Using a MemoryStream first:

private static void WriteToAFileFromMemoryStream()
{
	using (MemoryStream memory = new MemoryStream())
	{
		using (StreamWriter writer = new StreamWriter(memory))
		{
			writer.WriteLine("Hello world from memory");
			writer.Flush();
			using (FileStream fileStream = File.Create(@"c:\mydirectory\memory.txt"))
			{
				memory.WriteTo(fileStream);
			}
		}
	}
}

5. Using a BufferedStream:

private static void WriteToAFileFromBufferedStream()
{
	using (FileStream fileStream = File.Create(@"c:\mydirectory\buffered.txt"))
	{
		using (BufferedStream buffered = new BufferedStream(fileStream))
		{
			using (StreamWriter writer = new StreamWriter(buffered))
			{
				writer.WriteLine("Hello from buffered.");
			}
		}
	}
}

Read all posts dedicated to file I/O here.

4 ways to read the contents of a file with C# .NET

.NET supports a number of ways to read from a file. Here come 4 of them.

1. Using FileStream:

private void ReadFileWithFileStream()
{
	using (FileStream fileStream = File.Open(@"c:\mydirectory\source.txt", FileMode.Open, FileAccess.Read))
	{
		using (StreamReader streamReader = new StreamReader(fileStream))
		{
			Console.Write(streamReader.ReadToEnd());
		}
	}
}

2. Using StreamReader:

private void ReadFileWithStreamReader()
{
	using (StreamReader reader = File.OpenText(@"c:\mydirectory\source.txt"))
	{
		Console.Write(reader.ReadToEnd());
	}
}

StreamReader.ReadToEnd has an awaitable version ReadToEndAsync which you can use to read from the file asynchronously. If you don’t know what this means then start here.

3. Using File:

private void ReadFileWithFile()
{
	Console.WriteLine(File.ReadAllText(@"c:\mydirectory\source.txt"));
}

4. Using BufferedStream:

private void ReadFileWithBufferedStream()
{
	using (FileStream fileStream = File.Open(@"c:\mydirectory\source.txt", FileMode.Open, FileAccess.Read))
	{
		using (BufferedStream bufferedStream = new BufferedStream(fileStream))
		{
			using (StreamReader streamReader = new StreamReader(bufferedStream))
			{
				Console.Write(streamReader.ReadToEnd());
			}
		}
	}
}

Read all posts dedicated to file I/O here.

Reading text files using the Stream API in Java 8

We discussed the Java 8 Stream API thoroughly on this blog starting here. We mostly looked at how the API is applied to MapReduce operations to analyse data in a stream.

The same API can be applied to File I/O. Java 8 adds a new method called “lines” to the BufferedReader object which opens a Stream of String. From then on it’s just standard Stream API usage to filter the lines in the file – and perform other operations on them in parallel such as filtering out the lines that you don’t need.

Here’s an example how you can read all lines in a file:

String filename = "c:\\logs\\log.txt";
File logFile = new File(filename);
try (BufferedReader reader = new BufferedReader(new FileReader(logFile));)
{
    StringBuilder fileContents = new StringBuilder();
    Stream<String> fileContentStream = reader.lines();
    fileContentStream.forEach(l -> fileContents.append(l).append(System.lineSeparator()));
    System.out.println(fileContents.toString());
}
catch (IOException ioe)
{

}

We simply append each line in the stream to a StringBuilder.

In my case the log.txt file has the following contents:

hello
this is a line
next line
this is another line
…and this is yet another line
goodbye

As we’re dealing with the Stream API the usual Map, Filter and Reduce methods are all available to be performed on the text. E.g. what if you’re only interested in those lines that start with “this”? Easy:

fileContentStream.filter(l -> l.startsWith("this"))
                    .forEach(l -> fileContents.append(l).append(System.lineSeparator()));

The StringBuilder will now only hold the following lines:

this is a line
this is another line

You can also use the Path and Files objects that were introduced in Java 7. The Files object too was extended with a method to get hold of the Stream object. The below example is equivalent to the above:

Path logFilePath = Paths.get("C:\\logs\\log.txt");
try (Stream<String> fileContentStream = Files.lines(logFilePath))            
{
    StringBuilder fileContents = new StringBuilder();
    fileContentStream.filter(l -> l.startsWith("this"))
            .forEach(l -> fileContents.append(l).append(System.lineSeparator()));
    System.out.println(fileContents.toString());
}
catch (IOException ioe)
{

}

View all posts related to Java here.

How to partially read a file with C# .NET

Say you have a large file with a lot of text in it and you need to find a particular bit. One way could be to read the entire text into memory and search through it. Another, more memory-friendly solution is to keep reading the file line by line until the search term has been found.

Suppose you have a text file with the following random English content:

—————–
Unfeeling so rapturous discovery he exquisite. Reasonably so middletons or impression by terminated. Old pleasure required removing elegance him had. Down she bore sing saw calm high. Of an or game gate west face shed. no great but music too old found arose.

Offered say visited elderly and. Waited period are played family man formed. He ye body or made on pain part meet. You one delay nor begin our folly abode. By disposed replying mr me unpacked no. As moonlight of my resolving unwilling.

Inquietude simplicity terminated she compliment remarkably few her nay. The weeks are ham asked jokes. Neglected perceived shy nay concluded. Not mile draw plan snug next all. Houses latter an valley be indeed wished merely in my. Money doubt oh drawn every or an china. Visited out friends for expense message set eat.
—————-

Also suppose that you’d like to find the first occurrence of “formed”. The following code will do just that:

private static void ReadFilePartially()
{
	using (StreamReader streamReader = File.OpenText(@"c:\mydirectory\source.txt"))
	{
		String searchString = "formed";
		bool searchStringFound = false;

		while (!searchStringFound && !streamReader.EndOfStream)
		{
			string line = streamReader.ReadLine();
			if (line.Contains(searchString))
			{
				Console.WriteLine("Search term {0} found in the following line:\n{1}", searchString, line);
				searchStringFound = true;
			}
		}
	}
}

Read all posts dedicated to file I/O here.

Monitor the file system with FileSystemWatcher in C# .NET Part 4: other features

In the first three parts of this mini-series – starting here – we looked at how to subscribe to the events raised by a FileSystemWatcher object. FileSystemWatcher has a couple more properties that can be interesting.

Consider the following code:

static void Main(string[] args)
{
	RunVariousFeaturesExample();
	Console.ReadKey();
}

private static void RunVariousFeaturesExample()
{
	FileSystemWatcher watcher = new FileSystemWatcher();
	watcher.Path = @"c:\mydirectory";
	watcher.Filter = "*.txt";
	watcher.IncludeSubdirectories = true;
	watcher.NotifyFilter = NotifyFilters.Security | NotifyFilters.Size;
	watcher.Changed += watcher_Changed;
	watcher.EnableRaisingEvents = true;
}

static void watcher_Changed(object sender, FileSystemEventArgs e)
{
	Console.WriteLine("A change occurred in the monitored directory. Change type: {0}, file name: {1}", e.ChangeType, e.Name);
}

We set the path to be monitored as usual. We can filter the file names to be monitored. Here we’re interested in text files only. We also specify that we want to monitor all subfolders of the “Path” directory through the IncludeSubdirectories property. With NotifyFilter we can further refine the cases when we want to be notified of a file change. Here we want to be notified if either the file size changes or the security properties have been updated. Then we subscribe to the generic Changed event which is raised in case an existing file changes.

In the below example I’ve updated the security settings of a file and it successfully raised the Changed event:

File change monitored by filesystemwatcher

Read all posts dedicated to file I/O here.

Elliot Balynn's Blog

A directory of wonderful thoughts

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Once Upon a Camayoc

Bite-size insight on Cyber Security for the not too technical.

%d bloggers like this: