Using the Redis NoSql database with .NET Part 5: the sorted set data type

Introduction

In the previous post we looked at the set data type in Redis. Sets are similar to list but they only store unique strings. Duplicates are discarded and the items are case sensitive. This means that we can add “monday” and “MONDAY” in the same set. Another difference compared to lists is that sets are not guaranteed to hold the items in the same order as they were entered. Redis has commands to perform a couple of common set-related actions such as calculating the union, difference and intersection of two or more sets. We also looked at two techniques to extract the key names from the Redis database: KEY and SCAN where SCAN is a cursor based iterator and is the recommended method since it’s a lighter operation for the database.

In this post we’ll first talk a little bit about persistence to disk in Redis. Then we’ll move onto a discussion of the sorted set data type.

A bit on persistence

By now we know that Redis stores its data in memory but also offers persistent storage. The persistence related options are set in a configuration file. We installed the Redis Windows service before and we specified a specific configuration file in one of the commands during the process: redis.windows-service.conf. It is a text file in the Redis root folder which can be opened in a text editor like Notepad++. The file contains a large number of settings but for this discussion we can concentrate on the section called SNAPSHOTTING. The configuration file is well documented and provides a lot of default values as well so that we don’t need to spend any time on configuration before testing Redis. The following persistence intervals are set by default:

save 900 1
save 300 10
save 60 10000

…where the first integer is the number of seconds and the second integer is the number of changes. Here’s how the configuration file explains these values:

In the example below the behaviour will be to save:

after 900 sec (15 min) if at least 1 key changed
after 300 sec (5 min) if at least 10 keys changed
after 60 sec if at least 10000 keys changed

It’s important to note that there is some “danger zone” where new data records may be lost if the Redis server goes down since the values are not persisted immediately.

Another interesting setting is where the key-value pairs are stored on disk:

# The filename where to dump the DB
dbfilename dump.rdb

If you check the Redis folder there will be a file called dump.rdb. This is where the data is persisted. You can open it with a text editor, the file is somewhat human-readable. The key and value entries are shown in plain text but the other instructions are encoded.

Sorted sets

Originally I wanted to discuss sorted sets in the previous post about sets. I thought sorted sets would be the same as sets where the values are sorted. However, sorted sets are more than that in Redis. Each value consists of two parts: a value name and a score which is the basis for the sorting. Examples include scores that participants received in some competition, the amount of water consumed during various agricultural activities or the distance between a specific city and a range of others. We’ll take this last example and build a short sorted set: we’ll add the distance in kilometres between Budapest, the capital of Hungary and a number of other European capitals. I’ll use this website to get the numbers correct.

The command to add a new value to a sorted set is ZADD which accepts the score and the value name:

ZADD distance-from-budapest 320 “Belgrade”

We can add multiple score-value pairs to a sorted set at once:

ZADD distance-from-budapest 899 “Kiev” 930 “Minsk” 442 “Prague” 545 “Warsaw” 161 “Bratislava” 380 “Ljubljana” 643 “Bucharest” 299 Zagreb 214 “Vienna” 1244 “Paris” 688 “Berlin” 1317 “Stockholm” 1379 “Tallinn” 1012 “Copenhagen” 1569 “Moscow”

We can add an existing value and its score will be updated. Say we entered an incorrect score and want to update it…:

ZADD distance-from-budapest 445 “Prague”

Prague will have an updated score of 445.

ZCARD returns the number of items in a sorted set:

ZCARD distance-from-budapest

…which gives 16.

ZCOUNT accepts two integers and returns the number of items whose scores are within the lower and upper values:

ZCOUNT distance-from-budapest 400 1000

…which gives 6, i.e. 6 capitals lie within 400 and 1000 kilometres from Budapest.

If we want to know which 6 then we can use the ZRANGEBYSCORE command:

ZRANGEBYSCORE distance-from-budapest 400 1000

…which returns the following list:

1) “Prague”
2) “Warsaw”
3) “Bucharest”
4) “Berlin”
5) “Kiev”
6) “Minsk”

If we need the scores as well then we have to add the WITHSCORES argument:

ZRANGEBYSCORE distance-from-budapest 400 1000 WITHSCORES

…and the scores will be returns as a second output:

1) “Prague”
2) “445”
3) “Warsaw”
4) “545”
5) “Bucharest”
6) “643”
7) “Berlin”
8) “688”
9) “Kiev”
10) “899”
11) “Minsk”
12) “930”

Since the set is sorted the capitals are also listed in increasing order.

ZRANGE provides index based access where the lower and upper index limits are both inclusive:

ZRANGE distance-from-budapest 0 2

1) “Bratislava”
2) “Vienna”
3) “Zagreb”

ZRANGE also accepts the WITHSCORES extension:

ZRANGE distance-from-budapest 0 2 WITHSCORES

1) “Bratislava”
2) “161”
3) “Vienna”
4) “214”
5) “Zagreb”
6) “299”

ZRANK returns the index position of a value:

ZRANK distance-from-budapest “Paris”

…which gives 12 which is the 13th element in the sorted set since the first index is 0.

ZREVRANK does the same as ZRANK but from the tail of the set:

ZREVRANK distance-from-budapest “Paris”

…giving 4, i.e. Paris is the 4th most distant capital in the set.

ZINCRBY increments the score of a member by a certain value. Normally cities do not change their physical position on Earth but let’s just try it for demo purposes:

ZINCRBY distance-from-budapest 40 “Berlin”

…i.e. we moved Berlin by 40 kilometres. The function returns the new score, i.e. 728 in this case. Let’s move Berlin back with a negative value:

ZINCRBY distance-from-budapest -40 “Berlin”

…it’s back at 688.

We can even use ZINCRBY to add a new member. If the provided member is not found then it’s added to the set and its initial value will be the increment:

ZINCRBY distance-from-budapest 909 “Vilnius”

…which adds Vilnius to the set with 909 as the score. This is not really how ZINCRBY is meant to be used but it’s good to be aware of this side effect.

ZREM removes a member:

ZADD distance-from-budapest 100 “nosuchplace”
ZREM distance-from-budapest “nosuchplace”

We can read the score of an individual member using ZSCORE:

ZSCORE distance-from-budapest “Warsaw”

…which gives 545.

There’s one remaining data type in Redis: hashes, a.k.a maps or dictionaries. We’ll look at them in the next post.

You can view all posts related to data storage on this blog here.