Using the BlockingCollection for thread-safe producer-consumer scenarios in .NET Part 1

We’ve looked at the 4 concurrent, i.e. thread-safe collections available in .NET:

• ConcurrentDictionary
• ConcurrentQueue
• ConcurrentStack
• ConcurrentBag

We also mentioned that 3 of these, i.e. the concurrent queue, bag and stack implement the generic IProducerConsumer interface. They can be used in producer-consumer scenarios where one or more threads write to a work queue. They are the producers in the setup. One or more other threads then read from the same work queue in order to perform some task on them. These are the consumers. When the consumer takes an item from the work queue then that item is also removed from it so that it’s not processed more than once. If you need to build a scenario like that then you’ll obviously need thread-safe collections that can synchronise access to them in a reliable manner.

We haven’t seen before how a producer-consumer scenario can be built from scratch using the available classes in the System.Collections.Concurrent namespace. We’ll do that in this and the next couple of posts.

You could build a multi-threaded producer-consumer application with ConcurrentQueue/Bag/Stack alone if you wish. Note that most often it will be a queue as you’ll probably want to handle the items in the order they’ve come in.

However, you won’t know exactly when a thread has added a new element to a concurrent queue so the consumer will need to check constantly if there’s anything to be processed. That would probably happen in some endless while-loop where you check for new items and let the thread sleep for some time before the next loop starts. It would be easier to only act upon a new element just when it has been added to the work queue. There’s no straightforward way to achieve that with the collections we’ve looked at so far.

This is where another class in the System.Collections.Concurrent namespace enters the scene: the generic BlockingCollection of T. We’ll start investigating it in the next post.

View the list of posts on the Task Parallel Library here.