Introduction to CouchDB with .NET part 14: list functions in design documents

Introduction

In the previous post we looked at validation functions in CouchDB design documents. A validation function is used for data validation when inserting a new document or updating an existing one. It is called automatically upon an insert or update operation. We saw how to write a function that checks particular properties in the incoming document and throws an exception if it violates some business rules. We can also check whether the user executing the modification has a certain username or is in a user role, i.e. whether they are authorised to perform the action.

In this post we’ll look at lists in design documents.

Lists

List functions operate in conjunction with views. In other words they are not independent objects. A list function takes the default output of a view and enhances it with a custom function which is the list function we have provided. Lists do not change anything in the data set, they only enhance the output of the view they are applied on. Lists iterate through each row in the data set produced by the view and add new information to the row based on the row values in the loop. They can also turn the view content into a HTML response for fancy reports.

In preparation for the demo let’s create a new database called “players” with the following four documents:

{
  "name": "John",
  "yearOfBirth": 1980,
  "scoreBase": 100
}

{  
  "name": "Jane",
  "yearOfBirth": 1987,
  "scoreBase": 110
}

{  
  "name": "Mary",
  "yearOfBirth": 1989,
  "scoreBase": 95
}

{
  "name": "Peter",
  "yearOfBirth": 1988,
  "scoreBase": 105
}

Remember that a list function cannot work without a view so next we’ll create a design document with a view:

{
  "_id": "_design/enhanced-views",
  "views": {
    "default": {
      "map": "function (doc) { emit (doc.name, doc.yearOfBirth) }"
    }
  }
}

After all we’ve seen about views this piece of code should be easy to follow. We can call it as follows:

GET http://localhost:5984/players/_design/enhanced-views/_view/default

Here’s the output:

{
  "total_rows": 4,
  "offset": 0,
  "rows": [
    {
      "id": "3031afee194a8e4295c021221a0130a1",
      "key": "Jane",
      "value": 1987
    },
    {
      "id": "3031afee194a8e4295c021221a012b31",
      "key": "John",
      "value": 1980
    },
    {
      "id": "3031afee194a8e4295c021221a013327",
      "key": "Mary",
      "value": 1989
    },
    {
      "id": "3031afee194a8e4295c021221a014306",
      "key": "Peter",
      "value": 1988
    }
  ]
}

Note also the total_rows and offset properties. They constitute the head section of the view which can be accessed when a list is invoked through a function parameter. The other list parameter gives access to the HTTP request details when the list is invoked through a URL. Just like views are invoked through an API call, lists are also accessed in the same way. We’ll see in a bit how the list function accepts these two parameters where the the HTTP request parameter is the more interesting one.

Basic demo

The iteration will be a while-loop through a built-in function called getRow(). We’ll see how it works in a very simple first step. Update the design document to the following:

{
  "_id": "_design/enhanced-views",
  "_rev": "your revision ID",
  "views": {
    "default": {
      "map": "function (doc) { emit (doc.name, doc.yearOfBirth) }"
    }
  },
  "lists": {
    "age": "function(header, request){while (row=getRow()) {var currentYear = new Date().getFullYear(); var age = currentYear - row.value;row['age'] = age; send(JSON.stringify(row));}}"
  }
}

Lists are represented by the “lists” section in a design document. We have a list function called “age” which calculates the age of each player and adds it to the rows of the view. Here’s the anatomy of the list function:

• A list function has two parameters: the view header and the HTTP request. We mentioned both of them above. Our first example doesn’t use them at all. We’ll see a couple of examples later on.
• The iteration begins in the while loop which loops through each row in the view output
• Within the loop we get the current year and calculate the age
• We access the age by “row.value” where “row” is the current row in the loop. Check the JSON output of the view above. The rows are contained within the “rows” JSON property as an array. Each array element has the fields “id”, “key” and “value”. So “row.value” in the loop refers to the “value” property of the elements in the “rows” section of the view. By the same fashion we could access row.id and row.key as well.
• We add the age to a new row called “age” and send back a JSONified version of the row so that it will be correctly formatted and inserted into the JSON output of the view

Here’s how we can call a list:

GET /_design/{design-document-id}/_list/{list-name}/{view-name}

The view name is the view that the list will operate on.

In our example it looks like the following:

GET http://localhost:5984/players/_design/enhanced-views/_list/age/default

…which produces the following output:

{"id":"3031afee194a8e4295c021221a0130a1","key":"Jane","value":1987,"age":30}{"id":"3031afee194a8e4295c021221a012b31","key":"John","value":1980,"age":37}{"id":"3031afee194a8e4295c021221a013327","key":"Mary","value":1989,"age":28}{"id":"3031afee194a8e4295c021221a014306","key":"Peter","value":1988,"age":29}

We have a new JSON property age with the calculated age of each player. That’s a great start I would say!

What if we want to access any property in the documents, not just the keys and values of the view? We can achieve that by including the full documents in the view by the include_docs flag. Recall that we used it as follows:

GET http://localhost:5984/players/_design/enhanced-views/_view/default?include_docs=true

…which adds a “doc” property to each view row. Here’s an example:

"doc":{"_id":"3031afee194a8e4295c021221a0130a1","_rev":"2-b33111a9f7ef28152180425970150cbb","name":"Jane","yearOfBirth":1987,"scoreBase":110}

We can include the same flag when calling the list and access the document with “row.doc.some-property-of-document”.

Here’s our age list function with accessing the yearOfBirth property:

function(header, request){while (row=getRow()) {var currentYear = new Date().getFullYear(); var age = currentYear - row.doc.yearOfBirth ;row['age'] = age; send(JSON.stringify(row));}}

…and here’s how to execute it:

GET http://localhost:5984/players/_design/enhanced-views/_list/age/default?include_docs=true

Now we get all the documents as well. Here’s an example, I won’t copy the entire output:

{"id":"3031afee194a8e4295c021221a0130a1","key":"Jane","value":1987,"doc":{"_id":"3031afee194a8e4295c021221a0130a1","_rev":"2-b33111a9f7ef28152180425970150cbb","name":"Jane","yearOfBirth":1987,"scoreBase":110},"age":30}

The request object

The request object in the list function is very rich. Instead of me listing everything there’s an excellent reference on this page.

Add the following list function after “age” :

"request-demo": "function (header, request) {while (row=getRow()){ var headers = JSON.stringify(request.headers); var method = request.method; var query = JSON.stringify(request.query); row['headers'] = headers; row['method'] = method; row['query'] = query; send(JSON.stringify(row));}}"

Don’t forget to put a comma after the “age” function otherwise Fauxton won’t save it. Also, it won’t tell you what is failing unfortunately.

Here’s the JS function for better visibility:

function (header, request) {
	while (row=getRow()){ 
		var headers = JSON.stringify(request.headers); 
		var method = request.method; 
		var query = JSON.stringify(request.query); 
		row['headers'] = headers; 
		row['method'] = method; 
		row['query'] = query; 
		send(JSON.stringify(row));
	}
}

We read out the headers, the web method and the query parameters for each row. So each will be identical, which is not very clever, but it’s fine to just play around and see what the request parameter contains. Execute the following URL in a web browser:

GET http://localhost:5984/players/_design/enhanced-views/_list/request-demo/default?someParam=10

Here’s a shortened output is a single row:

{
	"id": "3031afee194a8e4295c021221a0130a1",
	"key": "Jane",
	"value": 1987,
	"headers": "{\"Accept\":\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\" ... other headers ignored}",
	"method": "GET",
	"query": "{\"someParam\":\"10\"}"
}

The query parameter is especially useful if you want to use that value for an extra calculation like int the following list function:

 "use-param": "function(header, request) {while (row = getRow()) { var extraScore = request.query.extraScore; var finalScore = row.doc.scoreBase; if (extraScore) { finalScore += parseInt(extraScore);} row['finalScore'] = finalScore; send(JSON.stringify(row)); }}"

Here’s the JS function in a more readable format:

function(header, request) {
	while (row = getRow()) { 
		var extraScore = request.query.extraScore;       
		var finalScore = row.doc.scoreBase; 
		if (extraScore) { 
			finalScore += parseInt(extraScore);
		} 
		row['finalScore'] = finalScore; 
		send(JSON.stringify(row)); 
	}
}

We read the value of the query parameter “extraScore” and add it to the base score to produce a final score. Here’s how we can call this list:

GET http://localhost:5984/players/_design/enhanced-views/_list/use-param/default?extraScore=23&include_docs=true

..and here’s an example output:

{
	"id": "3031afee194a8e4295c021221a0130a1",
	"key": "Jane",
	"value": 1987,
	"doc": {
		"_id": "3031afee194a8e4295c021221a0130a1",
		"_rev": "2-b33111a9f7ef28152180425970150cbb",
		"name": "Jane",
		"yearOfBirth": 1987,
		"scoreBase": 110
	},
	"finalScore": 133
}

Producing HTML markup from a list

With lists we can even emit HTML markup. This documentation page includes a good starting example, let’s see how it works. Here’s the list function:

"try-html": "function(head, req){ start({'headers': { 'Content-Type': 'text/html' }}); send('<html><body><table>'); send('<tr><th>ID</th><th>Key</th><th>Value</th></tr>'); while(row=getRow()){ send(''.concat( '<tr>', '<td>' + toJSON(row.id) + '</td>','<td>' + toJSON(row.key) + '</td>','<td>' + toJSON(row.value) + '</td>', '</tr>' ));} send('</table></body></html>');}"

…and here’s the formatted JS function:

function(head, req){
    start({
        'headers': {
            'Content-Type': 'text/html'
        }
    });
    send('<html><body><table>');
    send('<tr><th>ID</th><th>Key</th><th>Value</th></tr>')
    while(row = getRow()){
        send(''.concat(
            '<tr>',
            '<td>' + toJSON(row.id) + '</td>',
            '<td>' + toJSON(row.key) + '</td>',
            '<td>' + toJSON(row.value) + '</td>',
            '</tr>'
        ));
    }
    send('</table></body></html>');
}

We start with the headers and the build up the HTML body step by step using multiple send statements. You’ll probably recognise that we’re building a simple table with the ID, key and value of each row in the view. Here’s how to call it:

GET http://localhost:5984/players/_design/enhanced-views/_list/try-html/default

We now have our view data in a basic tabular format:

You’re probably beginning to see that lists can be used to built larger HTML reports as well.

Read the next part here.

You can view all posts related to data storage on this blog here.