Web security in MVC4 .NET4.5 with C#: mass assignment aka overposting

January 31, 2013 2 Comments

This blog post will describe what mass assignment means and how you can protect your MVC4 web site against such an attack. Note that mass assignment is also called overposting.

Before we go into the details let’s set up our MVC4 project. Start Visual Studio 2012 and create an MVC4 internet application with .NET4.5 as the underlying framework.

Locate the Model folder and add a class called Customer. Add the following properties to it:

public class Customer
    {
        public String Name { get; set; }
        public int Age { get; set; }
        public String Address { get; set; }
        public int AccountBalance { get; set; }
    }

Let’s say that our business rules say that every new customer starts with an account balance of zero so the AccountBalance property should have its default value of 0 when creating a new customer.

Add a new controller to the Controllers folder called CustomerController. In the Add Controller window select the Empty MVC controller template. The only action you’ll see in this controller at first is “Index”.

In this controller we’ll simulate fetching our customers from the database. We of course do not have a database and it’s a very bad idea to read directly from the database in a controller but that’s not relevant for our discussion. Add the following method underneath the Index action:

private List<Customer> GetCustomersFromDb()
        {
            return new List<Customer>()
            {
                new Customer(){Address ="Chicago",
                    Age = 30, Name = "John", AccountBalance = 1000}
                , new Customer(){Address = "New York", 
                    Age = 35, Name = "Jill", AccountBalance = 500}
                , new Customer(){Address = "Stockholm",
                    Age = 25, Name = "Andrew", AccountBalance = 800}
            };
        }

This code is quite clear I suppose.

Right-click the Index action and select “Add view” from the context menu. In the Add View windows set the View name to “Index”, check the “Create a strongly-typed view” checkbox and select Customer as the Model class as follows:

Values in Add View to Customer

The corresponding view file Index.cshtml will be created in the Views/Customer folder. Open that file and modify its model declaration to use a List of Customer instead of a single customer as follows:

@model List<MassAssignment.Models.Customer>

In this view we only want to list our customers in an unordered list:

@model List<MassAssignment.Models.Customer>

@{
    ViewBag.Title = "Index";
}

<h2>Customers</h2>

<ul>
    @foreach (var customer in Model)
    {
        <li>Name: @customer.Name</li>
        <li>Address: @customer.Address</li>
        <li>Age: @customer.Age</li>
        <li>Account balance: @customer.AccountBalance</li>
    }
</ul>

We’d like to have a link to this view so open _Layout.cshtml in the Views/Shared folder and locate the ‘nav’ tag. The navigation will already have the following three links:

<li>@Html.ActionLink("Home", "Index", "Home")</li>
<li>@Html.ActionLink("About", "About", "Home")</li>
<li>@Html.ActionLink("Contact", "Contact", "Home")</li>

Add one more link to the list as follows:

<li>@Html.ActionLink("Customers", "Index", "Customer")</li>

Now run the application and click the Customers link on the opening screen. You should be directed to the View showing our customers as follows:

List of customers

It will probably not win the yearly CSS contest, but this will serve as a good starting point.

The next step will be to build the elements necessary to insert a new Customer.

We need a screen with the necessary fields to add a new Customer and an action that returns that view. Open CustomerController.cs and add the following method:

        [HttpGet]
        public ActionResult Create()
        {
            return View();
        }

This will listen to HTTP Get requests and return a View which we don’t have yet. Right click ‘Create’ and select Add view from the context menu. The View name can be Create. Check the Create a strongly-typed view checkbox and select the Customer class as the underlying model. This will insert the View file Create.cshtml in the Views/Customer folder. At first it’s quite empty:

@model MassAssignment.Models.Customer

@{
    ViewBag.Title = "Create";
}

<h2>Create</h2>

Add the following HTML/Razor code underneath the h2 tag:

@using (Html.BeginForm())
{
    <fieldset>
        <legend>Customer</legend>
        <div class="editor-label">
            @Html.LabelFor(model => model.Name)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Name)
        </div>
        <div class="editor-label">
            @Html.LabelFor(model => model.Address)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Address)
        </div>
        <div class="editor-label">
            @Html.LabelFor(model => model.Age)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Age)
        </div>
        <p>
            <input type="submit" value="Create" />
        </p>
    </fieldset>
}

We’re only adding the bare minimum that’s necessary for creating a new customer: labels and fields for all Customer properties bar one. Remember, that the business rules say that every new customer must start with an AccountBalance of 0, this property should not be set to any other value at creation time. Also this form lacks all types of data validation, but that’s not important for our purposes.

We need a link that leads to this View. Open Index.cshtml within the Customer folder and add the following Razor code below the unordered list:

@Html.ActionLink("Create new", "Create", "Customer")

Test the application now to see if you can navigate to the Create view. We need to add the code that will receive the user inputs. Go back to the Customer controller where we just added the Create method. The inputs on Create.cshtml will be posted against the Create view so we need a Create method that will listen to POST requests and accepts a Customer object as parameter.

Insert a method into CustomerController.cs that simulates saving the new Customer object in the database:

        private List<Customer> AddCustomerToDb(Customer customer)
        {
            List<Customer> existingCustomers = GetCustomersFromDb();
            existingCustomers.Add(customer);
            return existingCustomers;
        }

We can now add the overloaded Create method listening to POST requests:

        [HttpPost]
        public ActionResult Create(Customer customer)
        {
            List<Customer> updatedCustomers = AddCustomerToDb(customer);
            Session["Customers"] = updatedCustomers;
            return RedirectToAction("Index");
        }

The incoming Customer object will be populated using the values from the form we created on Create.cshtml. We add the customer to our ‘database’, save the updated list in a Session and redirect to the Index action to see the new list of Customers. Update the Index action to inspect the Session before checking the ‘database’:

public ActionResult Index()
        {
            List<Customer> customers = null;
            if (Session["Customers"] != null)
            {
                customers = Session["Customers"] as List<Customer>;
            }
            else
            {
                customers = GetCustomersFromDb();
            }
            return View(customers);
        }

Run the application and navigate to the Create customer view. Try to add a new Customer by filling out the Name, Address and Age fields. Remember that there is no validation so make sure you insert acceptable values, such as an integer in the Age field. Upon pressing the Create button you will be redirected to the Index view of the Customer controller and the new Customer will be added to the unordered list as follows:

Added a new customer

Note that the AccountBalance property was assigned the default value of 0 according to our business rules. So, we are perfectly safe, right? No-one ever could insert a customer with a positive or negative account balance. Well, it’s not so simple…

When the Create(Customer customer) action receives the POST request then the MVC binding mechanism tries its best to find all constituents of the Customer object. Run the application and navigate to the Create New Customer view. Check the generated HTML code; the form should look similar to this:

<form action="/Customer/Create" method="post">    <fieldset>
        <legend>Customer</legend>
        <div class="editor-label">
            <label for="Name">Name</label>
        </div>
        <div class="editor-field">
            <input class="text-box single-line" id="Name" name="Name" type="text" value="" />
        </div>
        <div class="editor-label">
            <label for="Address">Address</label>
        </div>
        <div class="editor-field">
            <input class="text-box single-line" id="Address" name="Address" type="text" value="" />
        </div>
        <div class="editor-label">
            <label for="Age">Age</label>
        </div>
        <div class="editor-field">
            <input class="text-box single-line" data-val="true" data-val-number="The field Age must be a number." data-val-required="The Age field is required." id="Age" name="Age" type="number" value="" />
        </div>
        <p>
            <input type="submit" value="Create" />
        </p>
    </fieldset>
</form>

Check the ‘name’ attribute of each input tag. They match the names of the properties of the Customer object. These values will be read by the MVC model binding mechanism to build the Customer object in the Create method. An attacker can easily send a HTTP POST request to our action which includes the property value for AccountBalance. You could achieve this with C# as follows:

Uri uri = new Uri("http://validuri/Customer/Create");
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Post, uri);
List<KeyValuePair<String, String>> postValues = new List<KeyValuePair<string, string>>();
postValues.Add(new KeyValuePair<string, string>("name", "hacker"));
postValues.Add(new KeyValuePair<string, string>("address", "neverland"));
postValues.Add(new KeyValuePair<string, string>("age", "1000"));
postValues.Add(new KeyValuePair<string, string>("accountbalance", "5000000"));
message.Content = new FormUrlEncodedContent(postValues);
HttpClient httpClient = new HttpClient();
Task<HttpResponseMessage> responseTask = httpClient.SendAsync(message);
HttpResponseMessage response = responseTask.Result;

Not too difficult, right? There are all sorts of tools out there which can generate POST requests for you, such as Curl, SoapUI and many more, so you don’t even have to write any real code to circumvent the business rule.

This is a common technique to send all sorts of values that you would not expect in your application logic. Attackers will try to populate your models testing for combinations of property names. In a banking application it is quite feasible that a Customer object will have a property name called exactly AccountBalance. After all the developers and domain owners will most likely not give the AccountBalance property some completely unrelated name, such as “Giraffe” and then create a translation table where it says that Giraffe really means AccountBalance.

What can you do to protect your application against overposting?

Property blacklist

You can use the Bind attribute to build a black list of property names i.e. the names of properties that should be excluded from the model binding process:

public ActionResult Create([Bind(Exclude="AccountBalance")] Customer customer)

You can specify a comma-separated list of strings as the Exclude parameter. These properties will be excluded from the model binding process. Even if the key-value pair from the form includes the AccountBalance property it will be ignored.

Property whitelist

You can use the Bind attribute to build a white-list of property names as follows:

public ActionResult Create([Bind(Include="Age,Name,Address")] Customer customer)

This white-list will include all property names that you want to be bound.

Use the UpdateModel/TryUpdateModel methods

You can use the overloads of UpdateModel or TryUpdateModel methods within the Create method as follows:

UpdateModel(customer, "Customer", new string{"Age","Name","Address"});

You can include the names of the properties in the string array that you want to use in the binding process.

Create a ViewModel

Probably the most elegant solution is to create an interim object – a viewmodel – that will hold the properties you want to bind upon a HTTP POST request. Insert a class called CustomerViewModel in the Models folder:

public class CustomerViewModel
    {
        public String Name { get; set; }
        public int Age { get; set; }
        public String Address { get; set; }
    }

Modify the Create method as follows:

        [HttpPost]
        public ActionResult Create(CustomerViewModel customer)
        {
            Customer newCustomer = new Customer()
            {
                AccountBalance = 0
                ,Address = customer.Address
                ,Age = customer.Age
                ,Name = customer.Name
            };
            List<Customer> updatedCustomers = AddCustomerToDb(newCustomer);
            Session["Customers"] = updatedCustomers;
            return RedirectToAction("Index");
        }

This way you can be sure that you only receive the properties that you expect and nothing else.

You can view the list of posts on Security and Cryptography here.

Filed under .NET 4.5, Security Tagged with , , , , ,

Web security in .NET4.5 MVC4 with C#: Cross site request forgery

January 28, 2013 4 Comments

In this post we’ll discuss what a cross-site request forgery means and how you can protect your MVC4 web application against such an attack. The term is abbreviated as CSRF – pronounced ‘see-surf’.

A CSRF is an attack a malicious user can execute using the identity of an unsuspecting user who is logged onto the web application. The effects of the attack can range from annoyances, such as logging out the user from the site, to outright dangerous actions such as stealing data.

Imagine the following: a user with no administrative rights wants to do something on your site that only admins are allowed to do, such as create a new customer in the database. Let’s say that your website is called http://www.mysite.com. This malicious user will then build a site which a user who is logged on to http://www.mysite.com will be tricked into using. Let’s say that the malicious user builds a simple site called http://www.fakesite.com. The attacker will have to know at least a little bit about the HTML of the target site, but that’s easy. He can simply select View Source in IE and similar commands in FF and Chrome and there it is.

Within http://www.mysite.com there will be a page where the user can enter the details of the new customer. The attacker will have to inspect the form element on that site, copy its raw HTML source and paste it to http://www.fakesite.com. He won’t need to build a fully functioning web page: a single-page app will suffice. Let’s say that this single page is called nicecars.html. Let’s imagine that the attacker will actually place some pictures of good cars on this webpage so that it looks real.

Imagine that the original source HTML of the rendered page on http://www.mysite.com looks as follows:

<form action="/Customers/Create" method="post">
    <input id="Name" name="Name" type="text" value="" />
    <input id="City" name="City" type="text" value="" />
    <input id="Country" name="Country" type="text" value="" />
</form>

The attacker will now take this HTML, paste it in http://www.fakesite.com/nicecars.html and insert his own values as follows:

<form action="/Customers/Create" method="post">
    <input id="Name" name="Name" type="text" value="You have been tricked" />
    <input id="City" name="City" type="text" value="Hello" />
    <input id="Country" name="Country" type="text" value="Neverland" />
</form>

The attacker is aiming to insert these values into the database through an authenticated administrator on http://www.mysite.com. In reality these values may be more dangerous such as transferring money from one account to another.

The attacker will also dress up the form a little bit:

<form id="fakeForm" style="display: none;" action="http://www.mysite.com/Customers/Create" method="post">
    <input id="Name" name="Name" type="text" value="You have been tricked" />
    <input id="City" name="City" type="text" value="Hello" />
    <input id="Country" name="Country" type="text" value="Neverland" />
</form>

Note the inline style element: the idea is not to show this form to the administrator. The form must stay invisible on nicecars.html. It’s enough to build a form that has the same structure that http://www.mysite.com expects. Also, as http://www.fakesite.com has nothing to do with Customers, the attacker will not want to post back to his own fake website. Instead he will want to post the data against http://www.mysite.com, hence the updated ‘action’ attribute of the form tag.

You may be wondering how this form will be submitted as there is no submit button. The good news for the attacker is that there’s no need for a button. The form can be submitted using a little JavaScript right underneath the form:

<script>
    setTimeout(function () { window.fakeForm.submit(); }, 2000);
</script>

This code will submit the form after 2 seconds.

If the attacker will try to run his own malicious form then he will be redirected to the Login page of http://www.mysite.com as only admins are allowed to add customers. However, he has no admin rights, so what can he do? He can copy the link to the webpage with the malicious form, i.e. http://www.fakesite.com/nicecars.html, and send it to some user with admin rights on http://www.mysite.com. Examples: insert a link in an email, on Twitter, on Facebook, etc., and then hope that the user will click the link thinking they will see some very nice cars. If the admin actually clicks the link then the damage is done: the values in the malicious form of http://www.fakesite.com/nicecars.html will be posted to http://www.mysite.com/Customers/Create and as the user has admin rights, the new Customer object will be inserted into the database.

The attacker can make it even less obvious that http://www.mysite.com is involved in any way: the form can be submitted with Ajax. The attacker uses the administrative user as a proxy to submit his own malicious data.

This will work seamlessly if the administrative user is logged onto http://www.mysite.com when they click the link to http://www.fakesite.com. Why? When the admin is logged on then the browser will send the authentication cookie .ASPXAUTH along with every subsequent request – even to http://www.fakesite.com. So http://www.fakesite.com will have access to the cookie and send it back to http://www.mysite.com when the malicious form is submitted.

It is clear that the usual authentication and authorisation techniques will not be sufficient against such an attack. You will also want to make sure that the form data that is sent to your application originated from your application and not from an external one. Fortunately this is easy to achieve in MVC4 through the AntiForgeryToken object.

You will need to place this token in two places.

1. On the Controller action that accepts the form data. Example:

[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult Create(CustomerViewModel customerViewModel)

If you now try to enter a new customer by posting against the Create method you’ll receive the following exception:

“The required anti-forgery token from field “_RequestVerificationToken” is not present.”

The reason is that the form that posts against the Create method must have an anti-forgery token which can be verified. You can add this token using a simple Html helper in the .cshtml Razor view. This token will hold a cryptographically significant value. This value will be added to a cookie which the browser will carry along when the form is posted. The value in this cookie will be verified before the request is allowed to go through.

Even if a malicious attacker manages to have an admin click the link to http://www.fakesite.com/nicecars.html they will not be able to set the right cookie value. Even if the attacker knew the exact value of the verification token websites don’t allow setting cookies for another website. This is how an anti-forgery token will prevent a CSRF attack.

2. We set up the verification token on the form as well.

This is easy to do using a Html helper as follows:

@using (Html.BeginForm())
{
    @Html.AntiForgeryToken()
    <fieldset>

    </fieldset>
}

With very little effort you can protect your website against a cross-site request forgery.

You can view the list of posts on Security and Cryptography here.

Filed under .NET 4.5, Security Tagged with , , ,

How to bundle and minify CoffeeScript files in .NET4.5 MVC4 with C#

January 17, 2013 1 Comment

In the previous blog post I showed you how the bundling and minification mechanism in MVC4 works with CSS and JavaScript files. Some of you may have CoffeeScript files in their projects and you may be wondering if they can be bundled as well. The answer is yes, and it’s not complicated at all.

Add a new folder to your MVC4 application called CoffeeScript. Add a file called customers.coffee to it and insert the following CoffeeScript code:

class Customer
	constructor: (@name) ->

	complain: ->
		alert @name + " says: WHERE IS MY ORDER???!!!!"

cust = new Customer("Angry customer")
cust.complain()

This should not be too complicated I believe: we construct a Customer object and call its complain() method.

The next step is to import a package called CoffeeBundler from NuGet: PM> install-package CoffeeBundler. This will add the necessary DLLs to your projects. Now we have access to another type of bundler which will translate CoffeeScript into JavaScript and also perform the bundling and minification processes.

Next go to the BundleConfig.RegisterBundles method and add the following bit of code:

Bundle coffeeBundler = new ScriptBundle("~/bundles/coffee")
                .Include("~/CoffeeScript/customers.coffee");
            coffeeBundler.Transforms.Insert(0, new CoffeeBundler.CoffeeBundler());
            bundles.Add(coffeeBundler);

As CoffeeScript is nothing but JavaScript we still need to create a ScriptBundle object and include our CoffeeScript file in it. We also need to add a transform which will translate the CoffeeScript files into JavaScript, compile them and bundle/minify them. We also want to make sure that this is the first transformation that runs, hence the 0 in the Insert method. By default the JS bundler is the first in the transformation process.

Now let’s open _Layout.cshtml and insert the following bit of code right before the closing /body tag:

<script src="@BundleTable.Bundles.ResolveBundleUrl("~/bundles/coffee")"></script>

In case you do a build now you may receive some errors that complain about a file called AppStart_RegisterCoffeeBundler.cs. This was added automatically when we inserted the CoffeeBundler package from NuGet. You can safely delete this file; its automatically generated code had not quite caught up with .NET4.5 when writing this blog post.

The reason we use the longer script tag to call the ResolveBundleUrl instead of the shorter Scripts.Render is the same: the CoffeeBundler is a bit behind the developments in .NET4.5 and it’s safer to use the longer version.

Now run the application. If all went well then you should see a standard JavaScript alert stating that there’s an angry customer. Open the source view to check the generated HTML. You should see that the CoffeeScript bundle has been created:

<script src="/bundles/coffee?v=_Jtpxzv6QVFgvnfRHCOt5_bHmKJW0D27L4nwCa0u1gU1"></script>

Call that bundle from the browser, the address should be http://localhost:xxxx/bundles/coffee and you should see the JavaScript that the CoffeeScript bundler has generated:

(function(){var n,t;n=function(){function n(n){this.name=n}return n.prototype.complain=function(){return alert(this.name+" says: WHERE IS MY ORDER???!!!!")},n}(),t=new n("Angry customer"),t.complain()}).call(this)

So, simple as that!

View the list of MVC and Web API related posts here.

Filed under .NET 4.5, Web optimisation Tagged with , , , ,

Web optimisation: resource bundling and minification in .NET4.5 MVC4 with C#

January 14, 2013 2 Comments

We will look at the bundling and minification techniques available in a .NET4.5 MVC4 project. Bundling and minification help optimise a web page by reducing the number of requests sent to the server and the size of the downloads. I assume most readers will know what the terms mean, but here comes a short explanation just in case.

Bundling: if your websites requires resource files, typically CSS and JavaScript files then the browser will have to request them from the server. Ideally these resources should be bundled so that the browser can receive the external files in a lower number of requests: all CSS content will be copied to one single CSS file – a CSS bundle – and the same can be made to the JS files creating a JS bundle.

Minification: it is obvious that larger external files take longer to download. CSS and JS files can usually be made smaller by removing comments, white space, making variable names shorter etc.

The two techniques applied together will decrease the page load time making your readers happy.

These two web optimisation techniques are well-known in the web developing world but they are generally cumbersome to carry out manually. Many programmers simply omit them due to time constraints. However, MVC4 in .NET4.5 has made it extremely straightforward to perform bundling and minification so there should be no more excuses.

For the demo project start Visual Studio 2012 and create an MVC4 internet application with .NET4.5 as the underlying framework. Navigate to Index.cshtml of the Home view and reduce its contents to the following:

@{
    ViewBag.Title = "Home Page";
}
<h2>Bundling and minification</h2>
<div>Welcome to bundling and minification.</div>

If you go to the Contents folder you’ll see a default CSS file called Site.css. Add two more CSS files to the folder to simulate tweaking styles: mystyle.css and evenmorestyle.css. For the sake of simplicity I simply copied the contents of Site.css over to the two new files but feel free to fill them with any other css content. Remember, we only pretend that we need 3 style sheets to render our website, we will not really need them.

If you take a look at the Scripts folder it includes a lot of default scripts: jQuery, knockout, modernizr. We will pretend that our site needs most of them.

Locate _Layout.cshtml in the Views/Shared folder. Note that the _Layout view will already include some default bundling and minification features but we’ll start from scratch to make the transition from no optimisation to more optimisation more obvious. So update the contents of _Layout.cshtml to the following – note that you can drag and drop the CSS and JS files onto the editor, VS will create the ‘link’ and ‘script’ tags for you:

<!DOCTYPE html>
<html lang="en">
    <head>        
        <title>@ViewBag.Title - My ASP.NET MVC Application</title>
        <link href="~/Content/Site.css" rel="stylesheet" />
        <link href="~/Content/mystyle.css" rel="stylesheet" />
        <link href="~/Content/evenmorestyle.css" rel="stylesheet" />
        <script src="~/Scripts/modernizr-2.5.3.js"></script>
    </head>
    <body>
        <div>
            @RenderBody()
        </div>
        <script src="~/Scripts/jquery-1.8.3.js"></script>
        <script src="~/Scripts/jquery-ui-1.9.2.js"></script>
        <script src="~/Scripts/jquery.unobtrusive-ajax.js"></script>
        <script src="~/Scripts/jquery.validate.js"></script>
        <script src="~/Scripts/jquery.validate.unobtrusive.js"></script>
        <script src="~/Scripts/knockout-2.1.0.js"></script>
    </body>
</html>

As you see our page needs a lot of external files. You may be wondering why modernizr.js is included separately from the other JS files. The reason is because modernizr.js makes non-HTML5 compatible browsers “understand” the new tags available in HTML5 such as “section” or “aside”. Therefore it needs to be loaded before the HTML appears otherwise its effects will not be seen on the rendered HTML.

You will probably know why JS files are included at the bottom of HTML code: if they come first then the browser will not start rendering the HTML until the JS files have been downloaded. By downloading them as late as possible we can increase the perceived response time. The version numbers of the jQuery files may not be the same as in your solution; I ran an update-package command in the package manager to have the latest version of each. It does not really matter for our discussion though so it’s OK to include the files that Visual Studio inserted by default.

Run the application in Internet Explorer and check the HTML source:

Page source with no optimisation

No surprises here: there is no bundling or minification going on here.

While IE is still open press F12 to turn on the developer tools. Go to the Network tab:

Select Network in developer tools

Clear the browser cache in Internet Options to simulate a user who comes to our site for the first time, press “Start capturing” in web developer tools – marked with an underline above – and refresh the page. The developer tool will capture the rendering time of each component of the home page. You may see something like the following:

Developer tools showing uncached response times without optimisation

The response times on your screen may of course differ but the point is that each resource stands on its own, they are not bundled or minified in any way. Of course some will be downloaded in parallel by the browser – the exact number of parallel connections will depend on the browser type. This number is 6 in IE8 and higher. However, we can do a lot better. Click ‘Go to detailed view’ and select the Timings tab to see the total rendering time. It may look something like this:

Rendering time with no optimisation

The total rendering time in my case was 0.82 sec.

If you go back to the summary view of the developer tools you’ll see in the bottom of the page that the total size of the downloaded content was about 0.9MB.

Our goal is to reduce both the rendering time and the size of the resources to be downloaded. Due to the limited number of parallel requests if you have a lot of resources on your page then some of them will need to wait for other resources to be downloaded. This further increases the response time. If you highlight knockout.js in the web developer tools output and then go to the detailed view, select the Timings tab you’ll see a column called ‘offset’. These values will tell you how much time has passed since the original request:

Start time offset without optimisation

In my case knockout.js started to download 31ms after the original request.

The tools needed to optimise our page are included in the References folder: System.Web.Optimization and WebGrease. WebGrease is a low level command line utility that the framework uses under the hood to carry out the minification and bundling of our external resources.

Locate Global.asax.cs and check the code that’s found there by default. You will see a call to BundleConfig.RegisterBundles. Select ‘RegisterBundles’ and press F12 to go to the source. This is the place where the BundleCollection object is filled with our bundles. As you can see the MVC4 internet template builds some of the bundling for us.

Each type of external resource will have its own BundleType: CSS -> StyleBundle, JS -> ScriptBundle. Both objects will take a string parameter in their constructor: a virtual path to the bundled resource. The constructor is followed by a call to the Include method where we pass in the real paths to the external files to be included in that bundle. Example:

bundles.Add(new StyleBundle("~/Content/themes/base/css").Include(
                        "~/Content/themes/base/jquery.ui.core.css",
                        "~/Content/themes/base/jquery.ui.resizable.css")

The bundle represented by the virtual path ‘/Content/themes/base/css’ will include 2 jQueryUi files. When the browser requests a resource with that path MVC will intercept the request, collect the contents of the files into one consolidated resource and sends that consolidated resource as response to the browser.

The Include() method is quite clever:
– it understands the * wildcard. Example: “~/Scripts/jquery.validate*” will include all resources within the Scrips folder that start with ‘jquery.validate’.
– it will not include redundant files. Example: you may have 3 files that start with jQuery. These are jQuery.js, jQuery.min.js and jQuery.vsdoc for VS intellisense. Include() will apply conventions to exclude the ‘min’ and ‘vsdoc’ files even if you specified with a ‘*’ charater that you want all files whose name starts with ‘jQuery’. Include() will only select the ‘normal’ jQuery.js file. This way we will avoid duplicated downloads.

So let’s bundle our CSS files first. You can erase the default code in the RegisterBundles call to start from scratch:

public static void RegisterBundles(BundleCollection bundles)
        {
            
        }

Knowing what we know now let’s try to construct a bundle of our 3 CSS files:

public static void RegisterBundles(BundleCollection bundles)
        {
            bundles.Add(new StyleBundle("~/bundles/css")
                .Include("~/Content/Site.css", "~/Content/mystyle.css", "~/Content/evenmorestyle.css"));
        }

Next up is our JS files. You may recall that modernizr.js stands on its own before any HTML is rendered so we will build a bundle for that resource as well. You may be wondering why we want to have a bundle with one file and the answer is that bundling also performs minification at the same time. We’ll see how that works in practice later in this post.

Extend the RegisterBundles call as follows:

public static void RegisterBundles(BundleCollection bundles)
        {
            bundles.Add(new StyleBundle("~/bundles/css")
                .Include("~/Content/Site.css", "~/Content/mystyle.css", "~/Content/evenmorestyle.css"));

            bundles.Add(new ScriptBundle("~/bundles/modernizr")
                .Include("~/Scripts/modernizr-*"));
        }

The ‘*’ wildcard will make sure that if we update this file to a newer version in the future the bundle will include that automatically. We don’t need to come back to this code and update the version number.

The last bundle of files we want to create is for the JS files in the bottom of the Index page. Extend the bundle registration as follows:

public static void RegisterBundles(BundleCollection bundles)
        {
            bundles.Add(new StyleBundle("~/bundles/css")
                .Include("~/Content/Site.css", "~/Content/mystyle.css", "~/Content/evenmorestyle.css"));

            bundles.Add(new ScriptBundle("~/bundles/modernizr")
                .Include("~/Scripts/modernizr-2.5.3.js"));

            bundles.Add(new ScriptBundle("~/bundles/js")
                .Include("~/Scripts/jquery*", "~/Scripts/knockout-*"));
        }

-> We want all files starting with ‘jquery’ and the knockout.js file irrespective of its version number. The Include method will usually be intelligent enough to figure out dependencies: jquery.js must come before jquery.ui is loaded and Include() will ‘know’ that. However, it’s not always perfect. You may run into situations where you must spell out each file in the bundle so that they are loaded in the correct order. So in case your js code does not work as expected this just might be the reason.

A note on virtual paths and relative references:

Generally you can use any path as the virtual path in the ScriptBundle/StyleBundle constructor, they don’t need to ist as real physical paths. However, be careful with stylesheets. As you know you can refer to images from within a stylesheet with relative URLs, e.g.:

background-image: url("images/mybackground.png");

The browser will request the image RELATIVE to where it found the stylesheet, i.e. at /images/mybackground.png. Now check the virtual path we provided in the StyleBundle constructor: “~/bundles/css”. This will translate to the following HTML link tag:

<link rel="stylesheet" href="~/bundles/css" />

When the browser sees that it needs to locate images/…png from this particular stylesheet it will think that the stylesheet came from a file called “css” in the directory “bundles”. Therefore it will make a request for “/bundles/images/…png” and that file will probably not exist. That particular request will not be intercepted and the server returns a 404. So to be on the safe side it’s a good idea to specify a virtual path that better corresponds to the real phsyical path to the css files. Update the following:

bundles.Add(new StyleBundle("~/bundles/css")
                .Include("~/Content/Site.css", "~/Content/mystyle.css", "~/Content/evenmorestyle.css"));

to

bundles.Add(new StyleBundle("~/content/css")
                .Include("~/Content/Site.css", "~/Content/mystyle.css", "~/Content/evenmorestyle.css"));

Now the browser will think the css file came from a file called “css” in the directory “content” and sends a request for “/content/images/…png” which will be the correct solution. You can still have e.g. ~/content/styles or ~/content/ohmygod as the virtual path but the folder name, i.e. ‘content’ in this case should correspond to where the css files are located.

Following the same reasoning if you know that any of your JS files makes a reference to other files using relative paths then you will need to update the virtual paths of the ScriptBundle object as well: “~/scripts/js”.

Let’s carry on: how do we render the script tags of those bundles?

This is very easy as Razor has built-in methods to achieve this:

@Styles.Render("~/content/css")
@Styles.Render("~/bundles/js")

MVC will take care of rendering the script tags. Another, more fine-grained way of doing this is to use the following in the HTML code:

<link href="@BundleTable.Bundles.ResolveBundleUrl("~/bundles/css")" rel="stylesheet" type="text/css" />

The optimisation framework will also add more information to the URL: a query string value that will avoid caching old versions of the bundle. If you update one of the files in the bundle then a new query string will be generated and the browser will have to fetch the new updated version instead of using the cached one.

Before we update our code we can test the following: start the application and extend the URL to send a request for one of the bundled resources using its virtual path, e.g. http://localhost:xxxx/bundles/js. You should see something like this:

Request for a bundled resource

It’s looking very much like minified JavaScript. I cannot be sure just by looking at the contents that it includes everything from the bundled resources but believe me it does.

So now we’re ready to update _Layout.cshtml:

<!DOCTYPE html>
<html lang="en">
    <head>        
        <title>@ViewBag.Title - My ASP.NET MVC Application</title>
        @Styles.Render("~/content/css")
        @Scripts.Render("~/bundles/modernizr")
    </head>
    <body>
        <div>
            @RenderBody()
        </div>
        @Scripts.Render("~/bundles/js")
    </body>
</html>

Run the application again, check the generated HTML code and you’ll see…:

HTML source with debug mode

…not quite what you expected, right? The bundled files still appear as individual files, so what’s going on? It turns out that the debug/release settings of your application will make bundling/minification behave differently. In debug mode bundling is ignored for easier debugging and we ran the application exactly in that mode. How do we know that? Check web.config and you’ll see the following tag under system.web:

<compilation debug="true" targetFramework="4.5" />

Change debug to false and re-run the application. The source code should look similar to this:

Html source without debug

That looks better. The files have been bundled and the correct link and script tags have been created. Also note the query string mentioned before attached to the href and src values.

As a result we have 1 link tag instead of 3 and 2 script tags instead of 7 we started off with.

There is an alternative way to force bundling even if debug = true in web config. Go to BundleConfig.RegisterBundles and add the following code after the bundles.Add calls:

BundleTable.EnableOptimizations = true;

This will override debug = true in the web.config file and emit single script and link tags.

As mentioned before bundling also performs minification: you can verify this by requesting the bundled resource in the URL, i.e. by requesting http://localhost:xxxx/bundles/modernizr?v=jmdBhqkI3eMaPZJduAyIYBj7MpXrGd2ZqmHAOSNeYcg1
You should see a jumbled version of the original modernizr.js file.

Have we achieved any improvement in the response time?

Run the same test with the web developer tool as we performed in the beginning. You should definitely get a lower response time. Also check the number of resources and the total size of the resources that the browser needs to download. The number of requests is reduced as we have fewer resources to download and the total size of the resources is greatly reduced due to the minification process.

We achieved all this by very little work – at least compared to how you would have done all this manually.

Some of you may have CoffeeScript files in your web project. You probably would like to bundle and minify them as well. I’ll demonstrate in the next blog post exactly how to do that.

View the list of MVC and Web API related posts here.

Filed under .NET 4.5, Web optimisation Tagged with , , ,

How to post a Multipart http message to a web service in C# and handle it with Java

January 10, 2013 39 Comments

This post is going to handle the following scenario:

  • Upload a file to a server by sending an HTTP POST to a web service in a multipart form data content type with C#
  • Accept and handle the message in the web service side using Java

I admit this may not be the typical scenario you encounter in your job as a .NET developer; however, as I need to switch between the .NET and the Java world relatively frequently in my job this just happened to be a problem I had to solve recently.

Let’s start with the .NET side of the problem: upload the byte array contents of a file to a server using a web service. We’ll take the following steps:

  • Read in the byte array contents of the file
  • Construct the HttpRequestMessage object
  • Set up the request message headers
  • Set the Multipart content of the request
  • Send the request to the web service
  • Await the response

Start Visual Studio 2012 – the below code samples should work in VS2010 as well – and create a new Console application. We will only work within Program.cs for simplicity.

Step 1: read the file contents, this should be straightforward

private static void SendFileToServer(string fileFullPath)
        {
            FileInfo fi = new FileInfo(fileFullPath);
            string fileName = fi.Name;
            byte[] fileContents = File.ReadAllBytes(fi.FullName);
        }

Step2: Construct the HttpRequestMessage object

The HttpRequestMessage within the System.Net.Http namespace represents exactly what it says: a HTTP request. It is a very flexible object that allows you to specify the web method, the contents, the headers and much more properties of the HTTP message. Add the following code to SendFileToServer(string fileFullPath):

Uri webService = new Uri(@"http://avalidwebservice.com");
HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Post, webService);
requestMessage.Headers.ExpectContinue = false;

The last piece of code, i.e. the one that sets ExpectContinue to false means that the Expect header of the message will not contain Continue. This property is set to true by default. However, a number of servers don’t know how to handle the ‘Continue’ value and they will throw an exception. I ran into this problem when I was working on this scenario so I’ll set it to false. This does not mean that you have to turn off this property every time you call a web service with HttpRequestMessage, but in my case it solved an apparently inexplicable problem.

You’ll obviously need to replace the fictional web service address with a real one.

Step 3: set the multipart content of the http request

You should specify the boundary string of the multipart message in the constructor of the MultipartFormDataContent object. This will set the boundary of the individual parts within the multipart message. We’ll then add a byte array content to the message passing in the bytes of the file to be uploaded. Note that we can add the following parameters to the to individual multipart messages:

  • The content itself, e.g. the byte array content
  • A name for that content: this is ideal if the receiving party needs to search for a specific name
  • A filename that will be added to the content-disposition header of the message: this is a name by which the web service can save the file contents

We also specify that the content type header should be of application/octet-stream for obvious reasons.

Add the following code to SendFileToServer(string fileFullPath):

MultipartFormDataContent multiPartContent = new MultipartFormDataContent("----MyGreatBoundary");
ByteArrayContent byteArrayContent = new ByteArrayContent(fileContents);
byteArrayContent.Headers.Add("Content-Type", "application/octet-stream");
multiPartContent.Add(byteArrayContent, "this is the name of the content", fileName);
requestMessage.Content = multiPartContent;

Step 4: send the message to the web service and get the response

We’re now ready to send the message to the server by using the HttpClient object in the System.Net.Http namespace. We’ll also get the response from the server.

HttpClient httpClient = new HttpClient();
Task<HttpResponseMessage> httpRequest = httpClient.SendAsync(requestMessage,    HttpCompletionOption.ResponseContentRead, CancellationToken.None);
HttpResponseMessage httpResponse = httpRequest.Result;

We can send the message using the SendAsync method of the HttpClient object. It returns a Task of type HttpResponseMessage which represents a Task that will be carried out in the future. Note that this call will NOT actually send the message to the service, this is only a preparatory phase. If you are familiar with the Task Parallel Library then this should be no surprise to you – the call to the service will be made upon calling the Result property of the Task object.

This post is not about the TPL so I will not go into any details here – if you are not familiar with the TPL but would like to learn about multipart messaging then read on and please just accept the provided code sample ‘as is’. Otherwise there are a great number of sites on the net discussing the Task object and its workings.


Step 5: read the response from the server

Using the HttpResponseMessage object we can analyse the service response in great detail: status code, response content, headers etc. The response content can be of different types: byte array, form data, string, multipart, stream. In this example we will read the string contents of the message, again using the TPL. Add the following code to SendFileToServer(string fileFullPath):

HttpStatusCode statusCode = httpResponse.StatusCode;
                HttpContent responseContent = httpResponse.Content;

                if (responseContent != null)
                {
                    Task<String> stringContentsTask = responseContent.ReadAsStringAsync();
                    String stringContents = stringContentsTask.Result;                    
                }

It is up to you of course what you do with the string contents.

Ideally we should include the web service call in a try-catch as service calls can throw all sorts of exceptions. Here the final version of the method:

private static void SendFileToServer(string fileFullPath)
        {
            FileInfo fi = new FileInfo(fileFullPath);
            string fileName = fi.Name;
            byte[] fileContents = File.ReadAllBytes(fi.FullName);
            Uri webService = new Uri(@"http://avalidwebservice.com");
            HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Post, webService);
            requestMessage.Headers.ExpectContinue = false;

            MultipartFormDataContent multiPartContent = new MultipartFormDataContent("----MyGreatBoundary");
            ByteArrayContent byteArrayContent = new ByteArrayContent(fileContents);
            byteArrayContent.Headers.Add("Content-Type", "application/octet-stream");
            multiPartContent.Add(byteArrayContent, "this is the name of the content", fileName);
            requestMessage.Content = multiPartContent;

            HttpClient httpClient = new HttpClient();
            try
            {
                Task<HttpResponseMessage> httpRequest = httpClient.SendAsync(requestMessage, HttpCompletionOption.ResponseContentRead, CancellationToken.None);
                HttpResponseMessage httpResponse = httpRequest.Result;
                HttpStatusCode statusCode = httpResponse.StatusCode;
                HttpContent responseContent = httpResponse.Content;

                if (responseContent != null)
                {
                    Task<String> stringContentsTask = responseContent.ReadAsStringAsync();
                    String stringContents = stringContentsTask.Result;                    
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
        }

This concludes the .NET portion of our problem. Let’s now see how the incoming message can be handled in a Java web service.

So you have a Java web service which received the above multipart message. The solution presented below is based on a Servlet with the standard doPost method.

The HttpServletRequest in the signature of the doPost method can be used to inspect the individual parts of the incoming message. This yields a collection which we can iterate through:

@Override
    public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
    {
        Collection<Part> requestParts = request.getParts();
        Iterator<Part> partIterator = requestParts.iterator();
        while (partIterator.hasNext())
        {
        }
    }

If the message is not of type MultipartFormData then the collection of messages will be zero length.

The Part object in the java.servlet.http namespace represents a section in the multipart message delimited by some string token, which we provided in the MultipartFormDataContent constructor. Now our goal is to specifically find the byte array message we named “this is the name of the content” in the .NET code. This name can be extracted using the getName() getter of the Part object. Add the following code to the while loop:

Part actualPart = partIterator.next();
if (actualPart.getName().equals("this is the name of the content"))
{
}

The Part object also offers a getInputStream() method that can be used later to save the byte array in a file. The file name we provided in the C# code will be added to the content-disposition header of the multipart message – or to be exact to the header of the PART of the message. Keep in mind that each individual message within the multipart message can have its own headers. We will need to iterate through the headers of the byte array message to locate the content-disposition header. Add the following to the if clause:

InputStream is = actualPart.getInputStream();
String fileName = "";
Collection<String> headerNames = actualPart.getHeaderNames();
Iterator<String> headerNamesIterator = headerNames.iterator();
while (headerNamesIterator.hasNext())
{
    String headerName = headerNamesIterator.next();
    String headerValue = actualPart.getHeader(headerName);
    if (headerName.equals("content-disposition"))
    {
    }
}

The last step of the problem is to find the file name within the header. The value of the content-disposition header is a collection of comma separated key-value pairs. Within it you will find “filename=myfile.txt” or whatever file name was provided in the C# code. I have not actually found any ready-to-use method to extract exactly the filename so my solution is very a very basic one based on searching the full string. Add the below code within “if (headerName.equals(“content-disposition”))”:

String searchTerm = "filename=";
int startIndex = headerValue.indexOf(searchTerm);
int endIndex = headerValue.indexOf(";", startIndex);
fileName = headerValue.substring(startIndex + searchTerm.length(), endIndex);

So now you have access to all three ingredients of the message:

  • The byte array in form of an InputStream object
  • The name of the byte array contents
  • The file name

The next step would be to save the message in the file system, but that should be straightforward using the ‘read’ method if the InputStream:

OutputStream out = new FileOutputStream(f);
byte buf[] = new byte[1024];
int len;
while ((len = is.read(buf)) > 0)
{
    out.write(buf, 0, len);
}

…where ‘is’ is the InputStream presented above and ‘f’ is a File object where the bytes will be saved.

View the list of posts on Messaging here.

Filed under .NET, Java Tagged with , , , , ,

Exception handling in async methods in .NET4.5 MVC4 with C#

January 7, 2013 1 Comment

In this post we’ll take a look at how to handle exceptions that are thrown by actions that are awaited. My previous post already included some exception handling techniques in MVC4 but here we will concentrate on exceptions thrown by await actions. Check my previous 3 posts for the full story behind the code examples shown here.

We will simulate some problems by intentionally throwing an exception in GetResultAsync and GetDataAsync:

public async Task<String> GetDataAsync(CancellationToken ctk)
        {
            StringBuilder dataBuilder = new StringBuilder();
            dataBuilder.Append("Starting GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            ctk.ThrowIfCancellationRequested();
            await Task.Delay(2000);
            throw new Exception("Something terrible has happened!");
            dataBuilder.Append("Results from the database. ").Append(Environment.NewLine);
            dataBuilder.Append("Finishing GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return dataBuilder.ToString();
        }

public async Task<String> GetResultAsync(CancellationToken ctk)
        {
            StringBuilder resultBuilder = new StringBuilder();
            resultBuilder.Append("Starting GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            ctk.ThrowIfCancellationRequested();
            await Task.Delay(2000);
            throw new Exception("The service is down!");
            resultBuilder.Append("This is the result of a long running calculation. ");
            resultBuilder.Append("Finishing GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return resultBuilder.ToString();
        }

Also, increase the timeout value of the Index action so that we do not get a Timeout Exception:

[AsyncTimeout(4000)]
[HandleError(ExceptionType = typeof(TimeoutException), View = "Timeout")]
public async Task<ActionResult> Index(CancellationToken ctk)

Before you run the application change the ‘mode’ attribute of the customErrors element in the web.config to “Off” as we want to see the debug data.

It does not come as a surprise that we run into an exception:

Intentional exception YSOD

If you had worked with the TPL library before then you may have expected an AggregateException that wraps all exceptions encountered during the parallel calls. However, TPL behaves slightly differently in conjunction with the await keyword. It is still an AggregateException that is instantiated behind the scenes but the .NET runtime will only throw the first exception that was encountered during the method execution.

This is good news: we can set up our try-catch structures as usual; we don’t need to worry about inspecting an AggregateException anymore.

View the list of MVC and Web API related posts here.

Filed under .NET 4.5, Async/Parallelism Tagged with , , , , , , , ,

Timeout exceptions with async/await in .NET4.5 MVC4 with C#

January 4, 2013 1 Comment

This post will discuss timeouts that occur with await and async in .NET4.5. For clarity on async and await in MVC4 check out my previous two blog posts: Await and async in .NET4.5 and Async controllers and actions in .NET4.5 MVC4

As await operations may involve some seriously long running actions, such as calling a slow web service, it can be a good idea to specify a timeout. We may not want to make the visitor wait 60 seconds just to see an error message afterwards. If your experience tells you that a web service normally responds within 5 seconds at most then it may be pointless waiting 50-60 seconds as you can be sure something has gone wrong. ASP.NET has a default request timeout of 90 seconds – correct me here if I’m wrong – but we can specify other values directly in code with an attribute: AsyncTimeout that takes the timeout value in milliseconds as parameter.

In addition to the AsyncTimeout attribute you’ll also need to supply an additional parameter of type CancellationToken to the async action. This parameter can be used by the long running services to check if the user has requested a cancellation. The CancellationToken has an IsCancellationRequested property which provides exactly this type of information. In our example we’ll pass this token to the service calls and use it to throw an exception if the request has been cancelled. As our services are not real service calls, there is no clean-up work to do but imagine that if an IO operation is interrupted by a user then the cancellation token can throw an exception and you can clean up all open resources or roll back the database operations in a catch clause.

You can read more about cancellation tokens on MSDN: Cancellation token on MSDN

Update service methods:

public async Task<String> GetDataAsync(CancellationToken ctk)
        {
            StringBuilder dataBuilder = new StringBuilder();
            dataBuilder.Append("Starting GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            ctk.ThrowIfCancellationRequested();
            await Task.Delay(2000);
            dataBuilder.Append("Results from the database. ").Append(Environment.NewLine);
            dataBuilder.Append("Finishing GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return dataBuilder.ToString();
        }

public async Task<String> GetResultAsync(CancellationToken ctk)
        {
            StringBuilder resultBuilder = new StringBuilder();
            resultBuilder.Append("Starting GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            ctk.ThrowIfCancellationRequested();
            await Task.Delay(2000);
            resultBuilder.Append("This is the result of a long running calculation. ");
            resultBuilder.Append("Finishing GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return resultBuilder.ToString();
        }

We know that the Index() action needs about 2 seconds to complete so let’s try something more aggressive to see what happens:

[AsyncTimeout(1000)]
        public async Task<ActionResult> Index(CancellationToken ctk)
        {
            DateTime startDate = DateTime.UtcNow;

            HomePageViewModel viewModel = new HomePageViewModel();
            viewModel.AddMessage(string.Concat("Starting Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            CalculationService calcService = new CalculationService();
            DatabaseService dataService = new DatabaseService();

            Task<String> calculationResultTask = calcService.GetResultAsync(ctk);
            Task<String> databaseResultTask = dataService.GetDataAsync(ctk);

            await Task.WhenAll(calculationResultTask, databaseResultTask);

            viewModel.AddMessage(calculationResultTask.Result);
            viewModel.AddMessage(databaseResultTask.Result);

            DateTime endDate = DateTime.UtcNow;
            TimeSpan diff = endDate - startDate;

            viewModel.AddMessage(string.Concat("Finishing Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            viewModel.AddMessage(string.Concat("Action processing time: ", diff.TotalSeconds));
            return View(viewModel);
        }

It is no surprise that we get a timout exception upon running the application:

Timeout YSOD

The yellow screen of death is great for debugging but not so nice in a production environment. To turn on custom error messages you must change web.config: locate the customErrors tag under system.web and change the mode attribute to “On” for the production environment. If your web.config does not have this tag then add it:

<system.web>
    <customErrors mode="On"></customErrors>

There is a default view in the Shared folder within Views called Error.cshtml. After modifying the web.config file the user will be redirected to that view upon an unhandled exception:

Error.cshtml screen

You can of course create custom views for errors and then specify which error view to show using attributes. Example:

[AsyncTimeout(1000)]
        [HandleError(ExceptionType = typeof(TimeoutException), View = "Timeout")]
        public async Task<ActionResult> Index(CancellationToken ctk)

This way you can specify error views for specific types of unhandled exceptions.

The next post will look at exception handling in async methods.

View the list of MVC and Web API related posts here.

Filed under .NET 4.5, Async/Parallelism, Timeout Tagged with , , , , , , , , , ,

Async controllers and actions in .NET4.5 MVC4 with C#

January 3, 2013 16 Comments

In the previous post we looked at the basics of async and await in a Console app. Now we’re ready to implement these new features in MVC4 to create async controllers.

The way to build asynchronous controllers has been completely changed compared to how it was done in MVC3 with the AsyncController superclass. This class and the inherent complexity in using it are gone in MVC4.

Start Visual Studio 2012 and create an MVC4 Web Application and choose the Internet template. Create a folder called “Services” in the solution. Insert two classes in this folder: DatabaseService and CalculationService. They represent long running calls to external services and have the following content:

DatabaseService.cs:

public class DatabaseService
    {
        public string GetData()
        {
            StringBuilder dataBuilder = new StringBuilder();
            dataBuilder.Append("Starting GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            Thread.Sleep(2000);
            dataBuilder.Append("Results from the database. ").Append(Environment.NewLine);
            dataBuilder.Append("Finishing GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return dataBuilder.ToString();
        }
    }

CalculationService.cs:

public class CalculationService
    {
        public string GetResult()
        {
            StringBuilder resultBuilder = new StringBuilder();
            resultBuilder.Append("Starting GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            Thread.Sleep(2000);
            resultBuilder.Append("This is the result of a long running calculation. ");
            resultBuilder.Append("Finishing GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return resultBuilder.ToString();
        }
    }

There should be nothing complicated in either class implementation.

Create another folder in the solution called ViewModels. Add a class called HomePageViewModel in that folder:

public class HomePageViewModel
    {
        public List<String> Messages { get; set; }

        public void AddMessage(string message)
        {
            if (Messages == null)
            {
                Messages = new List<string>();
            }
            Messages.Add(message);
        }
    }

Navigate to the Index action of the Home controller and modify it as follows:

public ActionResult Index()
        {
            DateTime startDate = DateTime.UtcNow;

            HomePageViewModel viewModel = new HomePageViewModel();
            viewModel.AddMessage(string.Concat("Starting Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            CalculationService calcService = new CalculationService();
            DatabaseService dataService = new DatabaseService();

            viewModel.AddMessage(calcService.GetResult());
            viewModel.AddMessage(dataService.GetData());

            DateTime endDate = DateTime.UtcNow;
            TimeSpan diff = endDate - startDate;

            viewModel.AddMessage(string.Concat("Finishing Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            viewModel.AddMessage(string.Concat("Action processing time: ", diff.TotalSeconds));
            return View(viewModel);
        }

Nothing complicated here either: we’re just adding messages to the view model, show the thread ids and measure the time it takes to complete the action.

Modify Index.cshtml of the Home view as follows:

@model Mvc4.ViewModels.HomePageViewModel
@{
    ViewBag.Title = "Home Page";
}

<ul>
    @foreach (String message in Model.Messages)
    {
        <li>@message</li>
    }
</ul>

So when you run the web page you should see an output similar to the following:

Screen with no async

It’s easy to see the following:

  • The Index() action blocks the thread when it calls CalculationService and DatabaseService
  • The total processing time took about 4 seconds in total
  • All involved methods executed on the same thread

Now our goal is to make this process more efficient: as it stands now the main thread is only sitting idle for most of the processing time. This can be a serious problem if we’re intending to build a scalable and responsive application.

We need to make a couple of changes to our code:

  • The Index() action needs to return a Task of type ActionResult and turned to an async method: we do not directly return an ActionResult but a Task that represents an ActionResult
  • Remember that if an action is of type async then it needs to have its pair ‘await’ somewhere in the method body
  • We have two long running method calls within the Index() action, so we’ll instruct MVC4 to await them
  • Since we’ll await those two method calls we need to insert async versions of DatabaseService.GetData() and CalculationService.GetResult() as well: they in turn must also return Tasks of type string instead of plain string

Using our experience from the Console app in the previous post we’ll include an async version of GetResult() in CalculationService.cs:

public async Task<String> GetResultAsync()
        {
            StringBuilder resultBuilder = new StringBuilder();
            resultBuilder.Append("Starting GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            await Task.Delay(2000);
            resultBuilder.Append("This is the result of a long running calculation. ");
            resultBuilder.Append("Finishing GetResult on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return resultBuilder.ToString();
        }

We’ll also insert a GetDataAsync() in DatabaseService.cs:

public async Task<String> GetDataAsync()
        {
            StringBuilder dataBuilder = new StringBuilder();
            dataBuilder.Append("Starting GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(". ");
            await Task.Delay(2000);
            dataBuilder.Append("Results from the database. ").Append(Environment.NewLine);
            dataBuilder.Append("Finishing GetData on thread id ").Append(Thread.CurrentThread.ManagedThreadId)
                .Append(".");
            return dataBuilder.ToString();
        }

Update the Index action of the Home controller to call the async method versions of the services:

public async Task Index()
        {
            DateTime startDate = DateTime.UtcNow;

            HomePageViewModel viewModel = new HomePageViewModel();
            viewModel.AddMessage(string.Concat("Starting Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            CalculationService calcService = new CalculationService();
            DatabaseService dataService = new DatabaseService();

            string calculationResult = await calcService.GetResultAsync();
            string databaseResult = await dataService.GetDataAsync();

            viewModel.AddMessage(calculationResult);
            viewModel.AddMessage(databaseResult);

            DateTime endDate = DateTime.UtcNow;
            TimeSpan diff = endDate - startDate;

            viewModel.AddMessage(string.Concat("Finishing Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            viewModel.AddMessage(string.Concat("Action processing time: ", diff.TotalSeconds));
            return View(viewModel);
        }

Run the web app now and you may see an output similar to the following:

MVC screen with async in place

Note the following:

  • The Index action started on thread id 10
  • The same thread enters GetResultAsync
  • The main thread exits GetResultAsync at the await keyword and thread 9 takes over
  • Thread 9 enters GetDataAsync and exits at the await keyword and thread 5 takes over
  • Thread 5 finished Index()
  • The total processing time is still about 4 seconds, but remember from the previous post: if you want to couple asynchronous methods with concurrency you need to include the TPL as well. This will be resolved later in this post.

The output on your screen may be different when you run the sample. It is not guaranteed that 3 different threads will be involved throughout the lifetime of the Index() action. It depends on the availability of threads, may only be 2 in total.

If you are working with WCF services and add a service reference to your project then you’ll have the option to generate the Async() versions of the service calls automatically. Make sure to select the ‘Allow generation of asynchronous operations’ checkbox and the ‘Generate task-based operations’ radiobutton in the Service Reference Settings window.

.NET4.5 is now interspersed with built-in async versions of long running and/or remote methods. Typical examples include: HttpClient.SendAsync that returns a Task of type HttpResponseMessage, or HttpContent.ReadAsStringAsync that returns a Task of type String.

We can now introduce TPL to make the service calls run in parallel. As it stands now the Index action first waits for GetResultAsync to finish before it goes on with GetDataAsync. Ideally Index should wait for both actions to complete in parallel and not one after the other. We will basically hold a reference to the Task values returned by the services and await them both together.

Update the Index action as follows:

public async Task<ActionResult> Index()
        {
            DateTime startDate = DateTime.UtcNow;

            HomePageViewModel viewModel = new HomePageViewModel();
            viewModel.AddMessage(string.Concat("Starting Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            CalculationService calcService = new CalculationService();
            DatabaseService dataService = new DatabaseService();

            Task<String> calculationResultTask = calcService.GetResultAsync();
            Task<String> databaseResultTask = dataService.GetDataAsync();

            await Task.WhenAll(calculationResultTask, databaseResultTask);

            viewModel.AddMessage(calculationResultTask.Result);
            viewModel.AddMessage(databaseResultTask.Result);

            DateTime endDate = DateTime.UtcNow;
            TimeSpan diff = endDate - startDate;

            viewModel.AddMessage(string.Concat("Finishing Action on thread id ", Thread.CurrentThread.ManagedThreadId));
            viewModel.AddMessage(string.Concat("Action processing time: ", diff.TotalSeconds));
            return View(viewModel);
        }

Note the following:

  • We do not await the two service calls one by one
  • Both of them will be awaited using Task.WhenAll
  • Task.WhenAll accepts an array of Task objects that should run in parallel
  • Task.WhenAll will block until all tasks in the array have finished
  • The await keyword will make sure that Index will wait upon all tasks to complete in the array
  • To retrieve the returned value from the service calls just use the Result property of the Task object: this will be populated if the Task has a return value i.e. it is a Task of some type

When you run the updated Index page you may see something like this:

Screen with async and TPL

Again, your results will almost certainly differ. Which thread is allocated to which task is up to the thread scheduler. Refresh the page a couple of times to see some different results.

In the next post we’ll wrap up the discussion with some miscellaneous issues such as timeouts or exception handling in conjunction with asynchronous controllers.

View the list of MVC and Web API related posts here.

Filed under .NET 4.5, Async/Parallelism Tagged with , , , , , , ,

← Older posts

Newer posts →

Elliot Balynn's Blog

A directory of wonderful thoughts

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Once Upon a Camayoc

ARCHIVED: Bite-size insight on Cyber Security for the not too technical.