JSON deserialization with JSON.net: caching results

On January 22 I promised this to be a three-part series. I’ve been kinda busy with upgrading apps, Windows 8 experiments and trivial ;-) stuff like code camps, an MVP summit, preparing my first and second talk about Windows Phone and whatnot and made you wait for the final part for exactly three months – but those who know me, know I stick my promises, so here’s the third and final part of my JSON for Windows Phone series.

In part 1 of this series I described the basics of creating classes from a JSON string and then simply deserializing the string into a (list of) classes. In part 2 I showed how to use JSONConverter subclasses to handle complex stuff the deserializer cannot handle out of the box, like class hierarchies. Part 3, as promised, shows a way to cache results - which makes your application faster, more responsive and more battery/data plan friendly.

Using the demo solution of part 2 as a starting point, I first brought in my wp7nl library on codeplex using NuGet. I am lazy just like any programmer (should be) and I like to defer as much heavy lifting to already existing code as I can ;-).

When dealing with data downloaded from the web everything becomes asynchronous by nature (at least on Windows Phone), and since we don’t have the await and async keyboards aboard our platform yet, I tend to use Observables from Microsoft.Phone.Reactive. To let that work with events, I need an EventArgs child class for my “loading completed” delegate, which I have defined in the following trivial way:

using System;
using System.Collections.Generic;

namespace Wp7nl.Utilities
{
  public class DataLoadCompletedArgs<T> : EventArgs
  {
    public IList<T> Result { get; set; }

    public Exception Error { get; set; }
  }
}

It’s a generic class because I am basically too lazy to write casting statements everywhere. The basic setup of the helper class that does both loading itself is like this:

using System.Linq;
using Microsoft.Phone.Reactive;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Net;
using Newtonsoft.Json;

namespace Wp7nl.Utilities
{
  public class CachedServiceDataLoader<T> where T : class
  {
    private readonly IsolatedStorageHelper<List<T>> storageHelper;

    public CachedServiceDataLoader()
    {
      storageHelper = new IsolatedStorageHelper<List<T>>();
    }

    private void FireDataLoadCompleted(IList<T> result, Exception error)
    {
      if (DataLoadCompleted != null)
      {
        DataLoadCompleted(this,
                          new DataLoadCompletedArgs<T> 
                          {Result = result, Error = error});
      }
    }

    public delegate void DataLoadCompletedHandler(object sender, 
      DataLoadCompletedArgs<T> args);

    public event DataLoadCompletedHandler DataLoadCompleted;
  }
}

So what have we here? A constructor that creates an IsolatedStorageHelper – that’s a class from the latest version from Wp7nl, where it sits in the Wp7nl.Utilities namespace. It’s basically the internal logic of the extension methods I described in my article about tombstoning MVVMLight viewmodels using SilverlightSerializer cut loose from those extension methods – so they can be used for caching all kinds of classes, and not only viewmodels on deactivation or closing of the app. The bottom part of the class is just the event indicating a data load action has been completed, and a convenience method to easily fire that event (I mentioned I was lazy, didn’t I? ;-) ) .

Next up is another little convenience methods for loading stuff from the cache or returning a default (empty) list if the results are not available:

private List<T> LoadFromStorage()
{
  return storageHelper.ExistsInStorage() ? 
     storageHelper.RetrieveFromStorage() : new List<T>();
}

It’s only used once, so I could just a easy have omitted it, but it makes the rest of the code a bit more readable and that’s important too. When you state your intentions in code, that saves on comment.

The method to read stuff from cache is a bit more complex than strictly necessary, but since data coming from the web is coming in asynchronously, why would I want my app to wait on my cache coughing up its results? So I made the cache retrieval asynchronously as well, using some old skool BackgroundWorker hoopla:

public bool StartLoadFromCache()
{
  if (storageHelper.ExistsInStorage())
  {
    var w = new BackgroundWorker();
    w.DoWork += (s, e) =>
      {
        e.Result = LoadFromStorage();
      };

    w.RunWorkerCompleted += (s, e) =>
      FireDataLoadCompleted(e.Result as List<T>, e.Error);

    w.RunWorkerAsync();
    return true;
  }
  return false;
}

I could have used an observable here as well I guess, but I still have not adapted my code snippet. So anyway, I cheat a little by checking first if there’s any cache data at all – if there is not, the method immediately returns false, informing the calling method there’s no cached data and to go get the stuff on the web. Which it can, incidentally, by calling the following method:

public void StartDownloadCacheData(IList<T> currentObjects, Uri serviceUri,
                                   params JsonConverter[] converters)
{
  var w = new SharpGIS.GZipWebClient();
  Observable.FromEvent<DownloadStringCompletedEventArgs>(
    w, "DownloadStringCompleted")
    .Subscribe(r =>
    {
      if (DataLoadCompleted != null)
      {
        if (r.EventArgs.Error == null)
        {
          var deserialized = 
               JsonConvert.DeserializeObject<List<T>>(r.EventArgs.Result,
                                                      converters);
          var result = new List<T>(currentObjects);
          result.AddRange(deserialized.Where(p => !currentObjects.Contains(p)));
          FireDataLoadCompleted(result, r.EventArgs.Error);
          storageHelper.SaveToStorage(result);
        }
        else
        {
          FireDataLoadCompleted(null, r.EventArgs.Error);
        }
      }
    });

  w.DownloadStringAsync(serviceUri);
}

So what this method does is pretty simple:

• It makes a GZipWebClient (that allows data to be loaded zipped, thus saving on amount of bytes actually transmitted)
• It makes an Observable from the event “DownloadStringCompleted” and subscribes an anonymous method
• It fires the DownloadStringAsync method on the uri

Now the anonymous method that is fired when the data arrives

• Checks for errors
• Tries to deserialize the incoming data using the provides converters (if any)
• Adds the existing object list to the result list
• Add the new objects to the list after weeding out duplicates that were already in the list
• Fires the complete event
• Stores the now merged data set in isolated storage.

So the idea is that you can do consecutive downloads, but that resulting data never contains any duplicates. This is why you must provide the list of current objects. This may seem like a little odd, but this is exactly the thing you want to do with working with geographical data – which I do almost all the time for a living. As an inspector you want to download stuff from the areas you want to inspect, not the whole municipality. You move your map to a location, hit “download” and presto, you have the stuff you want to use cached on you device before you go on the road. Repeat until all areas you want to visit today have been processed. Kinda like Nokia Drive does – there is never enough room aboard your phone to store all the possible maps of the world, but when you to Seattle, you download the maps for Washington State, not the whole of North America, and you’re good to go.

But what if you want to start over – clearing the cache in stead of adding it? That’s the last method of this class:

public void StartClearStorage()
{
  var w = new BackgroundWorker();
  w.DoWork += (s, e) => storageHelper.DeletedFromStorage();

  w.RunWorkerCompleted += (s, e) =>
  {
    if (DataLoadCompleted != null)
    {
      FireDataLoadCompleted(new List<T>(), e.Error);
    }
  };

  w.RunWorkerAsync();
}

Also asynchronously, also not strictly necessary, but Microsoft tend to move toward doing everything asynchronously – if you have done any Windows 8 development you know performance requirements are pretty high and strict, so better get used to it already.

I updated the demo solution to show off the workings of this class, that presents itself like showed to the right. If you click on “Load devices”, it will show four devices, as displayed on the image, and the message “Loading from web”. If you hit the “Load devices” button again, it will show the same four devices, but show the message “Loading from cache” and it will show them just a wee bit faster.

If you hit “Load more devices” you will see six devices, – with the Lumia 800 mentioned two times. But wait, what about weeding out the duplicates in StartDownloadCacheData – doesn’t that work? It sure does, but I already mentioned I was lazy, so I did not implement any “Equals” logic on  the Device class – that’s left as exercise for the reader ;-). I assure you it will work properly then.

“Clear list” just clears the list, and if you hit “Load devices” it will load the devices from cache again – very fast.

“Clear cache and list” will actually wipe the cache, so if you hit “Load devices” again it will show “Loading from web”.

I won’t bother you with the XAML – the code in MainPage.xaml.cs is pretty straightforward and shows off all the features of the CachedServiceDataLoader:

using System;
using System.Collections.Generic;
using System.Windows;
using Microsoft.Phone.Controls;
using Microsoft.Phone.Reactive;
using Wp7nl.Utilities;

namespace JsonDemo
{
  public partial class MainPage : PhoneApplicationPage
  {
    private CachedServiceDataLoader<Device> cachedLoader;
    private IList<Device> currentData;
    public MainPage()
    {
      InitializeComponent();
      Loaded += MainPage_Loaded;
    }

    void MainPage_Loaded(object sender, RoutedEventArgs e)
    {
      cachedLoader = new CachedServiceDataLoader<Device>();
      currentData = new List<Device>();
      Observable.FromEvent<DataLoadCompletedArgs<Device>>(
        cachedLoader, "DataLoadCompleted")
       .Subscribe(r =>
                    {
                      currentData = r.EventArgs.Result;
                      PhoneList.ItemsSource = r.EventArgs.Result;
                    });
    }

    private void Load_Click(object sender, RoutedEventArgs e)
    {
      if (!cachedLoader.StartLoadFromCache())
      {
        Message.Text = "Loading from web";
        cachedLoader.StartDownloadCacheData(currentData,
          new Uri("http://www.schaikweb.net/dotnetbyexample/JSONPhones2.txt"),
          new JsonDeviceConverter(), new JsonSpecsConverter());
      }
      else
      {
        Message.Text = "Loading from cache";        
      }
    }

    private void Load2_Click(object sender, RoutedEventArgs e)
    {
      Message.Text = "Loading from web";
      cachedLoader.StartDownloadCacheData(currentData,
        new Uri("http://www.schaikweb.net/dotnetbyexample/JSONPhones3.txt"),
        new JsonDeviceConverter(), new JsonSpecsConverter());
    }      

    private void Clear_Click(object sender, RoutedEventArgs e)
    {
      PhoneList.ItemsSource = null;
      Message.Text = "cleared list (not cache)";
    }

    private void ClearCache_Click(object sender, RoutedEventArgs e)
    {
      cachedLoader.StartClearStorage();
      Message.Text = "cleared list and cache";
    }
  }
}

There isn’t even any of my trademark MVVM code in here. In the MainPage_Loaded – not surprisingly - all the stuff is initialized. The CachedServiceDataLoader is created using a type T, and spits out a IList of T. I use an Observable to keep track of things here as well, but of course you are free to subscribe to events in the ‘old fashioned’ way.

Load_Click shows the usage of StartLoadFromCache and StartDownloadCacheData, the latter one using the JSONconverter child classes I showed in part 2. The rest I assume to be pretty straightforward.

As you can see, working with JSON on Windows Phone is dead easy and my little helper class makes it even more easy. I am still pondering if I should include this in the wp7nl library, as this creates yet two more dependencies (SharpGIS.GZipWebClient and Newtonsoft.Json) to keep in sync. If you have any feedback on this, I’d be happy to hear it. But anyway, this concludes my JSON for Windows Phone series, I hope it will prove to be a useful mini-tutorial for the Windows Phone developer community. And as usual, you can download the full solution from my website.

JSON deserialization with JSON.net: class hierarchies

In part 1 of this series I described the basics of creating classes from a JSON string and then simply deserializing the string into a (list of) classes. That way, you don’t have all the hooplah of SOAP, but still have strongly-typed classes in your client app. But beware, there is no formal contract either, so on a beautiful morning you might start to think that either you had too much of a drink yesterday evening, or that the company providing the data feed for your app indeed has started to sell Windows Phone 7 devices made by Sony, with a 65” screen.

Looking at the JSON string you now see something like this:

[
  {
    "Brand": "Nokia","Type" : "Lumia 800", "Device" : "Phone",
    "Specs":{"Storage" : "16GB", "Memory": "512MB","Screensize" : "3.7"}
  },
  {
    "Brand": "Sony", "Type" : "KDL-65HX920","Device" : "TV",
    "Specs":{"Screensize" : "65", "FullHD" : "Yes", "ThreeD" : "Yes" }
  },  
  { "Brand": "Nokia","Type" : "Lumia 900","Device" : "Phone",
    "Specs":{"Storage" : "8GB", "Memory": "512MB","Screensize" : "4.3" }
  },
  {
    "Brand": "Samsung", "Type" : "UE55C9000","Device" : "TV",
    "Specs":{"Screensize" : "55", "FullHD" : "Yes", "ThreeD" : "Yes" }
  },  
]

None of the two options mentioned before appear to be true: apparently the company has diversified. They are now selling TV's as well. Of course you could run this trough json2csharp, which will give you this:

public class Specs
{
    public string Storage { get; set; }
    public string Memory { get; set; }
    public string Screensize { get; set; }
    public string FullHD { get; set; }
    public string ThreeD { get; set; }
}

public class RootObject
{
    public string Brand { get; set; }
    public string Type { get; set; }
    public string Device { get; set; }
    public Specs Specs { get; set; }
}

This will work, but not for the purpose of what I’d like to show. We refactor the whole stuff into an object structure like this:

Or, in code (put into a single file for the sake of brevity)

namespace JsonDemo
{
  public abstract class Device
  {
    public string Brand { get; set; }
    public string Type { get; set; }
  }
  
  public class Phone : Device
  {
    public PhoneSpecs Specs { get; set; }
  }
  
  public class Tv : Device
  {
    public TvSpecs Specs { get; set; }
  }
  
  public abstract class Specs
  {
    public string Screensize { get; set; }
  }
  
  public class PhoneSpecs : Specs
  {
    public string Storage { get; set; }
    public string Memory { get; set; }
  }
  
    public class TvSpecs: Specs
  {
    public string FullHd { get; set; }
    public string ThreeD { get; set; }
  }
}

If you think this is a ludicrous complicated way to store such a simple data structure I think you are quite right, but a) they don’t call me a Senior Software Architect for nothing, making things complicated is what I am told Architects do for a living, so I try to be a Good Boy ;-) and b) this is just for the purpose of the sample, so bear with me, right?

If you remember the crux of part 1: it all came down to one line of code, namely:

JsonConvert.DeserializeObject<List<Phone>>(r.EventArgs.Result);

This method actually has a second parameter: params Newtonsoft.Json.JsonConverter[] converters, which allows you to provide your own custom converters. Making those is pretty easy, and it becomes even more easy when you use the JsonCreationConverter<T> class that’s floating around the internet in various permutations. I nicked it from StackOverflow here.

using System;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace JsonDemo
{
  public abstract class JsonCreationConverter<T> : JsonConverter
  {
    protected abstract T Create(Type objectType, JObject jsonObject);

    public override bool CanConvert(Type objectType)
    {
      return typeof(T).IsAssignableFrom(objectType);
    }

    public override object ReadJson(JsonReader reader, Type objectType, 
      object existingValue, JsonSerializer serializer)
    {
      var jsonObject = JObject.Load(reader);
      var target = Create(objectType, jsonObject);
      serializer.Populate(jsonObject.CreateReader(), target);
      return target;
    }

    public override void WriteJson(JsonWriter writer, object value, 
   JsonSerializer serializer)
    {
      throw new NotImplementedException();
    }
  }
}

To make a converter, you subclass this object into a templated converter for the base class, override the “Create” method and off you go. Writing a converter for the Device/Phone/Tv hierarchy is pretty easy: you just have to check the value of the “Device” property, which you do like this:

using System;
using Newtonsoft.Json.Linq;

namespace JsonDemo
{
  public class JsonDeviceConverter : JsonCreationConverter<Device>
  {
    protected override Device Create(Type objectType, JObject jsonObject)
    {
      var typeName = jsonObject["Device"].ToString();
      switch(typeName)
      {
        case "TV":
          return new Tv();
        case "Phone":
          return new Phone();
        default: return null;
      }
    }
  }
}

For the Specs/PhoneSpecs/TvSpecs you do more or less the same, only now you have to check for the existence of certain properties, not the value. I decided that if an object has a “Storage” property it’s a PhoneSpecs, and if it has “FullHD” it’s a TVSpecs.

using System;
using Newtonsoft.Json.Linq;

namespace JsonDemo
{
  public class JsonSpecsConverter : JsonCreationConverter<Specs>
  {
    protected override Specs Create(Type objectType, JObject jsonObject)
    {
      if(jsonObject["Storage"] != null)
      {
        return new PhoneSpecs();
      }

      if (jsonObject["FullHD"] != null)
      {
        return new TvSpecs();
      }

      return null;
    }
  }
}

Finally, to get these converters being used by the deserializer you have to slightly modify the deserialization line:

var deserialized = JsonConvert.DeserializeObject<List<Device>>(r.EventArgs.Result, 
  new JsonDeviceConverter(), new JsonSpecsConverter());

And sure enough, if you put a breakpoint behind this line, you can place a watch on “deserialized” and see that the data structure has been deserialized in our artfully crafted class structure.

And that’s all there is to it. Demo solution with full code can be found here.

The third and final part of this series shows how to cache results and can be found here

JSON deserialization with JSON.net: basics

I’ve been contemplating an article about handling JSON for some time now, but it turned out to be a rather long article. So I’d thought to try something new, and write a short series in three parts.

• Part 1 handles the basics
• Part 2 handles advanced deserialization with class hierarchies
• Part 3 handles a caching-and-updating scenarios.

And this is part 1 ;-)

This whole article actually boils down to one line of code, but I need to go trough some hooplah to show you how to use it. It all begins with the data. Consider this piece of quite readable JSON, describing a few recent Windows Phone models.

[
  {
    "Brand": "Nokia","Type" : "Lumia 800",
    "Specs":{"Storage" : "16GB", "Memory": "512MB","Screensize" : "3.7"}
  },
  {
    "Brand": "Nokia", "Type" : "Lumia 710",
    "Specs":{"Storage" : "8GB","Memory": "512MB","Screensize" : "3.7"}
  },  
  { "Brand": "Nokia","Type" : "Lumia 900",
    "Specs":{"Storage" : "8GB", "Memory": "512MB","Screensize" : "4.3" }
  },
  { "Brand": "HTC ","Type" : "Titan II",
    "Specs":{"Storage" : "16GB", "Memory": "512MB","Screensize" : "4.7" }
  },
  { "Brand": "HTC ","Type" : "Radar",
    "Specs":{"Storage" : "8GB", "Memory": "512MB","Screensize" : "3.8" }
  }
]

JSON is rather compact, which is a great feature when you are developing for mobile devices. It has also a few downsides as far as client programming is concerned:

• generating client code for it that does all the parsing and calling, as for SOAP, is not a standard feature of Visual Studio,
• it’s almost impossible to read for an ordinary human being,
• deciphering it into classes is a lot of work,
• hand coding a parser for it is not fun.

Which is why you don’t. There are several ways of generating classes from JSON, the simplest way is this website: json2csharp by Jonathan Keith. You copy a JSON result into the upper textbox, hit the “Generate” button and out come your classes:

There are more sites that do the same, by the way, but this is what I use. Next steps:

• Fire up Visual Studio
• Create a new Windows Phone project (for instance JsonDemo)
• Plonk the classes generated above in the project. Bonus cookies if you split them in separate files and add namespaces to them. Bonus donut if you, like me, think “RootObject” is actually a pretty ugly name for an object - so change it to "Phone".
• Click Tools/Library Package Manager/Manage NuGet Packages for Solution (you do have the NuGet Package Manager installed, don’t you? If not, stop whatever you are doing now and get it right this instance, you hear me ;)? )
• Search for JSON.Net
• Click install. This will add a reference to NewtonSoft.Json.dll to your product.
• Add references to Microsoft.Phone.Reactive and System.Observable because they are going to be needed in the next step.

To make the result visible, add some XAML to the default content panel in Mainpage.Xaml – just a button and a templated ListBox, no rocket science here:

<StackPanel>
  <Button Name="Load"
      VerticalAlignment="Top"
      Content="Load phones" Click="Load_Click" />
  <ListBox x:Name="PhoneList" Height="532">
    <ListBox.ItemTemplate>
      <DataTemplate>
        <StackPanel Orientation="Horizontal">
          <TextBlock Text="{Binding Brand}" 
                 Margin="0,0,12,0" />
          <TextBlock Text="{Binding Type}"/>
        </StackPanel>
      </DataTemplate>
    </ListBox.ItemTemplate>
  </ListBox>
</StackPanel>

Finally, open MainPage.Xaml.cs and add the method Load_Click as displayed below.

using System;
using System.Collections.Generic;
using System.Net;
using System.Windows;
using Microsoft.Phone.Controls;
using Microsoft.Phone.Reactive;
using Newtonsoft.Json;

namespace JsonDemo
{
  public partial class MainPage : PhoneApplicationPage
  {
    // Constructor
    public MainPage()
    {
      InitializeComponent();
    }

    private void Load_Click(object sender, RoutedEventArgs e)
    {
      var w = new WebClient();
      Observable
        .FromEvent<DownloadStringCompletedEventArgs>(w, "DownloadStringCompleted")
        .Subscribe(r =>
        {
          var deserialized = 
            JsonConvert.DeserializeObject<List<Phone>>(r.EventArgs.Result);
          PhoneList.ItemsSource = deserialized;
        });
      w.DownloadStringAsync(
        new Uri("http://www.schaikweb.net/dotnetbyexample/JSONPhones1.txt"));
    }
  }
}

And there it is, the one line of code that this is all about. Call the DeserializeObject method, template it with the return type you want, and stuff the JSON string in it. Result: a list of objects with their properties filled, even if there are things like nested objects (specs in these case) and arrays in there.

If you run the demo solution you get the result displayed in the image on the right. Keep in mind this code is by no means Windows Phone specific. There are JSON.Net implementations for virtually all frameworks available. So should you feel the need to use this from Silverlight or full .NET: it’s there.

You should, by the way, pay attention to the structure of the JSON. The code I show works for a list. A list in JSON starts with a square bracket: [. If your JSON starts with a curly brace: { then you get returned a single object - a so called root object. In that case, your deserialization should code return a single object in stead of a list as well, i.e. something like

var deserialized = 
     JsonConvert.DeserializeObject<Phone>(r.EventArgs.Result);

Finally, a ninja tip:

• Click Tools/Library Package Manager/Manage NuGet Packages for Solution  again
• Search for SharpGIS.GZipWebClient
• Click install
• Change “WebClient” in the Load_Click method to SharpGIS.GZipWebClient

This plug-in replacement for WebClient by Morten Nielsen adds support for GZIP compressed web requests – this reduces network traffic even further, apparently boosting load performance significantly. You won’t really notice the difference on such a small data files as used in this sample, but as your JSON return values get larger, so will be the impact of using this library.

For the record: I am not ill nor do I have forsaken MVVM, but I tried to make the example as simple as possible so yes, I used a little code behind, as to not to cloud the solution in architectural frills. ;-)

Thanks to Matthijs Hoekstra for putting me on track to this, and to fellow #wp7nl developer Leon Zandman for correcting some annoying typos.

Using the Microsoft Ajax library to keep the javascript "this" context intact

This is not strictly .NET but more of a Javascript trick, but nevertheless, one that you might need. It is a simple trick, and actually requires much more text to decribe the problem than the actual solution ;-) Javascript supports the notion of object oriented programming, to an extent. There's plenty to find about that on the web and I don't care to repeat that. One of the more annoying things of Javascript is that it does support callbacks, but is quite careless about object context. This happens for instance when you use callbacks to process data coming back from calling WCF services. Suppose, for instance, I take my project in which I explain how to get data from WCF services in Javascript and rewrite the Javascript code as an object:

function CallBackProcessor()
{
  this.CallSingle = function()
  {
      dotnetbyexample.JSONDemoService.IJSONService.GetSingleObject(
      document.getElementById("tbId").value,
      document.getElementById("tbName").value,
      this.CallBackSingle);
  }
  this.CallBackSingle = function(WebServiceResult)
  {
      resultDiv = document.getElementById("result1");
      resultDiv.innerHTML = this.CreateMessage(WebServiceResult);
  }
  this.CreateMessage = function(WebServiceResult)
  {
    return( "You entered: id = " +
      WebServiceResult.Id + " name = " +
      WebServiceResult.Name);
  }            
}
var processor = new CallBackProcessor();

and then change the code of the button calling the service to call the processor object

<asp:Button ID="btnSelect" runat="server" Text="SingleSelect" 
   UseSubmitBehavior="False"
   OnClientClick="processor.CallSingle(); return false;" />

I will get the notorious "Object doesn't support this property or method" error. Why? Because I try to call the method "CreateMessage" from the "CallBackSingle": the context of "this" is no longer that of "processor", but of whatever is calling the callback. You can find this out by placing a breakpoint before the call to CreateMessage and then view the properties of "this". The logic behind this behaviour defies my imagination, but the solution is actually pretty simple: use Function.createDelegate to set up the callback in stead of calling it directly

this.CallSingle = function()
{
    dotnetbyexample.JSONDemoService.IJSONService.GetSingleObject(
    document.getElementById("tbId").value,
    document.getElementById("tbName").value,
    Function.createDelegate(this, this.CallBackSingle));
}

And that's all there is to it. The "this" context stays intact, thanks to the Microsoft Ajax library, that you already included automatically by placing a ScriptManager on your page.

JSON services revisited: using a Dictionary as a generic parameter

Somebody at Microsoft deserves a Nobel Prize. Well, that is maybe a bit strong, but I ran into the problem that I wanted to create JSON scriptserver that would accept a 'parameter bag' of unknown size and type. I expected problems in the serialization, since Javascript typing is at best a bit lax, but it turns out that when you define a web service with a parameter of type Dictionary<string, object> dict, the JSON serializer automatically derives the object types from the javascript data! It works like this: suppose we define the following method:

[WebMethod]
public string DictMethod(Dictionary<string, object> dict)
{
  StringBuilder b = new StringBuilder("C# reports<BR/>");
  foreach( string s in dict.Keys )
  {
    b.AppendFormat(string.Format("{0}:{1}={2}<BR/>", s, dict[s].GetType(), 
                   dict[s]));
  }
  return b.ToString();
}

We can call this method from javascript like this

<script type="text/javascript">
  function LoadDict()
  {
    dict = new Object();
    dict["test1"] = "Joost";
    dict["test2"] = 21;
    dict["test3"] = true;
    dict["test4"] = 9999999999;
    dict["test5"] = new Date(2008,09,12,13,14,00);
    DictService.DictMethod( dict, ProcessResult );
  }

  function ProcessResult( WebServiceResult )
  {
    document.write( WebServiceResult );
  }
</script>

If you attach the LoadDict javascript function to some client side event (The OnClientClick of a button for example) and launch it, you will see the following output: C# reports test1:System.String=Joost test2:System.Int32=21 test3:System.Boolean=True test4:System.Int64=9999999999 test5:System.DateTime=12-10-2008 11:14:00 So you can see, the javascript string converted into a true .NET string, the value 21 into an Int32 - but the value 9999999999 is too long for an int and is converted into a Int64, a.k.a. long - I really like that one, for it shows true craftmanship of the programmer ;-). The "true" javascript value is converted into a Boolean finally, the javascript Date is converted into a neat DateTime. And presto, all your stuff is converted into real .NET objects. Of course, you should always used typed parameters when possible, but for some generic solutions this may come in very handy. Complete code downloadable here.

Calling a WCF service from Javascript

Note - I wrote this a long time ago. While the information in this article is still valid, nowadays I would use ASP.NET MVC 4 WebAPI on the server, and jQuery on the client to do something like this. I will blog about this soon.

In a previous post I pointed out how to call an ASP.NET ASMX web service from javascript in the browser. Although this still works fine, the world has moved on to .NET 3.5 and WCF services have now replaced ASMX services. Luckily, these are also callable from javascript although the procedure is now a little bit different. In this sample I presume you can use Visual Studio 2008 and know a little about WCF.

1. Create an empty solution, and add a WCF service library to it

We'll call the solution JSONWCFTest, and the library JSONServices.

2. Add a data contract

using System.Collections.Generic;
using System.Runtime.Serialization;

namespace JSONServices
{
 [DataContract]
 public class SampleDataObject
 {
   public SampleDataObject(int NewId, string NewName)
   {
     Id = NewId;
     Name = NewName;
   }
   [DataMember]
   public int Id { get; set; }

   [DataMember]
   public string Name { get; set; }
 }
}

3. Add service contract

Add an interface IJSONService with the following methods using System.Collections.Generic; using System.ServiceModel;

namespace JSONServices
{
 [ServiceContract(Namespace="http://DotNetByExample/JSONDemoService")]
 public interface IJSONService
 {
   [OperationContract]
   SampleDataObject GetSingleObject(int id, string name);

   [OperationContract]
   ICollection GetMultipleObjects(int number);
 }
}

4. Add an implementing class

Implement the service contract like this

System.Collections.Generic;

namespace JSONServices
{
 public class JSONService : IJSONService
 {
   public SampleDataObject GetSingleObject(int id, string name)
   {
     return new SampleDataObject(id, name);
   }

   public ICollection<SampleDataObject> GetMultipleObjects(int number)
   {
     List<SampleDataObject> list = new List<SampleDataObject>();
     for (int i = 0; i < number; i++)
     {
       list.Add(new SampleDataObject(i, "Name" + i));
     }
     return list;
   }
 }
}

So far, this is very basic WCF stuff. Nothing special here.

5. Add new web application

Add a new project of type "ASP.NET Web Application" to the solution. Call it JSONTestApp. Add a reference to the JSONWCFTest project.

6. Add a SVC file to the web application

This part is a bit awkward. Choose "Text File", and name it "AjaxService.svc". Then open the file, and enter the following code

<%@ServiceHost language="C#" debug="true"
Service="JSONServices.JSONService"
Factory="System.ServiceModel.Activation.WebScriptServiceHostFactory"%>

It may be tempting to add an "Ajax-enabled WCF service" to the project but this creates its own operation contract, and in this example I want to simulate the situation in which I re-use an already existing operation contract implemented in a separate service library.

7. Check the javascript interface

Basically, you are done now. Your service is hosted callable from javascript. You can check right-clicking on AjaxService.svc, and you will now see a web page that will say something like "This is a Windows© Communication Foundation service. Metadata publishing for this service is currently disabled." and a lot things more. You can check the javascript interface by adding "/jsdebug" to the url displayed in your browser, in my case this is "http://localhost:1781/AjaxService.svc/jsdebug". This shows the dynamically generated javascript proxy that you can call in your client code:

Type.registerNamespace('dotnetbyexample.JSONDemoService');
dotnetbyexample.JSONDemoService.IJSONService=function() {
dotnetbyexample.JSONDemoService.IJSONService.initializeBase(this);
this._timeout = 0;
this._userContext = null;
this._succeeded = null;
this._failed = null;
}
dotnetbyexample.JSONDemoService.IJSONService.prototype={
_get_path:function() {
var p = this.get_path();
if (p) return p;
else return dotnetbyexample.JSONDemoService.IJSONService._staticInstance.get_path();},
GetSingleObject:function(id,name,succeededCallback, failedCallback, userContext) {
/// <param name="id" type="Number">System.Int32</param>
/// <param name="name" type="String">System.String</param>
/// <param name="succeededCallback" type="Function" optional="true" mayBeNull="true"></param>
/// <param name="failedCallback" type="Function" optional="true" mayBeNull="true"></param>
/// <param name="userContext" optional="true" mayBeNull="true"></param>
return this._invoke(this._get_path(), 'GetSingleObject',false,{id:id,name:name},succeededCallback,failedCallback,userContext); },
GetMultipleObjects:function(number,succeededCallback, failedCallback, userContext) {
/// <param name="number" type="Number">System.Int32</param>
/// <param name="succeededCallback" type="Function" optional="true" mayBeNull="true"></param>
/// <param name="failedCallback" type="Function" optional="true" TomayBeNull="true"></param>
/// <param name="userContext" optional="true" mayBeNull="true"></param>

Note: FireFox displays the javascript directly in the browser; for security reasons, Internet Explorer wants to download it to a file rather than display it, but after saving it and opening it in a text editor you will see the text above. As you can see this contains a lot of plumbing - and this is only the top part - but I emphasized the parts that are interesting right now: the name of the object that you need to call from javascript (dotnetbyexample.JSONDemoService.IJSONService) and the methods, which have - surprise, suprise - exactly the same name as their C# server-side counterparts. From here on, the procedure is pretty much the same as we were used to when calling asmx webservices from javascript:

8. Add an Web Form to the Web Application

Call it TestForm1.aspx

9. Add a script manager and a reference

Add a script manager to the form, right below the form tag, and make a reference to the AjaxService.svc file:

<form id="form1" runat="server">
<asp:scriptmanager runat="server">
<services>
<asp:ServiceReference Path="AjaxService.svc" />
</services>
</asp:scriptmanager>

10. Add some user interface elements to call the service

Right below the script reference, add the following code:

<div>
ID<asp:TextBox ID="tbId" runat="server">21</asp:TextBox>
Name<asp:TextBox ID="tbName" runat="server">John Doe</asp:TextBox><br />
<asp:Button ID="btnSelect" runat="server" Text="SingleSelect"
   UseSubmitBehavior="False"  OnClientClick="CallSingle(); return false;"/>
</div>
<div id="result1"></div>

11. Add Javascript code

Add some javascript code to call the WCF service and process the results. Place this inside the head-tag, below the title tag:

<script type ="text/javascript">
function CallSingle()
{
dotnetbyexample.JSONDemoService.IJSONService.GetSingleObject(
    document.getElementById("tbId").value,
    document.getElementById("tbName").value,
    CallBackSingle );
}
function CallBackSingle( WebServiceResult )
{
resultDiv = document.getElementById("result1");
resultDiv.innerHTML = "You entered: id = " +
 WebServiceResult.Id + " name = " +
 WebServiceResult.Name;
}
</script>

Notice again that calling a WCF method from javascript works the same as when using asmx services: one extra parameter at the end of the list is required, which is the callback. This is a function that is called upon receiving the result from the server - the server method result is put into the single parameter of the callback.

12. Run the test page

Run Test1Form.aspx. You should be able to input and numerical id and a alphanumeric name, which are sent to the server, and returned back to you as and SampleObject.

13. Deploying WCF services in IIS

This all works fine when running it in Cassini, but when you want to host it in IIS, you will need to take care two things. First, you will need to ensure that IIS and WCF are correctly installed and registered. If IIS does not seem to want to cough up your SVC files (you can check that by trying http://<your host here>/<your sitename here>/AjaxService.svc), try running the command ServiceModelReg.exe /i /x from %WINDIR%\Microsoft.NET\Framework\v3.0\Windows Communication Foundation. Restart IIS after running this command. Second, if you created a default ASP.NET 2.0 website, you may run into this error message:

IIS specified authentication schemes 'IntegratedWindowsAuthentication, Anonymous', but the binding only supports specification of exactly one authentication scheme. Valid authentication schemes are Digest, Negotiate, NTLM, Basic, or Anonymous. Change the IIS settings so that only a single authentication scheme is used.

To fix this, right-click the website in your IIS manager and choose "properties". Then click the "Directory" tab, click the "Edit" button in the "Anonymous access and authentication control" area. You will see that both Anonymous access and Integrated Windows Authentication are selected. Unselect one of them, then restart IIS. After that, the application should work smoothly.

Concluding remarks

Calling WCF services from javascript is even simpler than calling asmx services - you don't even have to mark a service as [ScriptService] anymore, you just put an .svc file into your web site with four lines of code, make a reference to an existing service library and you're done. The sample described here does not use the IJSONService.GetMultipleObjects method, but you can download a the complete source code which includes a second test page that does. Notice that although GetMultipleObjects returns a ICollection, in javascript this is translated into a simple array. Thus, you can also use data and operation contract that use generic types, as long as the can simply be translated into an array. Oh - and don't forget to remove "debug=true" from "AjaxService.svc" before publishing it on production. Complete code downloadable here. Author's note: this article is my most viewed posting by far: 20% the traffic of my blog goes to this page. Apparently it meets a growing need in knowlegde. Please spread the word, (and the link) and if you find any errors of have any additions, let me know. Compliments are appreciated as well.

Expanding the length of JSON data returned from ASP.NET web services

A very small example this time, but one that can be very time consuming to find out. The basic example I described earlier works fine, but just try to send back a larger string. Suppose, you change the Hello World Method into this:

[WebMethod]
public string HelloWorld( string ToSomeone)
{
    StringBuilder b = new StringBuilder();
    while (b.Length < 50000)
    {
       b.AppendFormat("Hello World {0}. ", ToSomeone);
    }

    return b.ToString();
}

You get al LOT of "Hello Worlds". Now change 50000 into 200000... you get no error message, but in fact, you get no response at all. To solve this, you need to delve into your web.config. Make sure the JSON configuration declaration at the top looks like this. I highlighted the important part:

<configSections>
  <sectionGroup name="system.web.extensions" type="System.Web.Configuration.SystemWebExtensionsSectionGroup, System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
    <sectionGroup name="scripting" type="System.Web.Configuration.ScriptingSectionGroup, System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
        <section name="scriptResourceHandler" type="System.Web.Configuration.ScriptingScriptResourceHandlerSection, System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false" allowDefinition="MachineToApplication"/>
      <sectionGroup name="webServices" type="System.Web.Configuration.ScriptingWebServicesSectionGroup, System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
        <section name="jsonSerialization" type="System.Web.Configuration.ScriptingJsonSerializationSection, System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false" allowDefinition="Everywhere" />
        <section name="profileService" type="System.Web.Configuration.ScriptingProfileServiceSection, System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false" allowDefinition="MachineToApplication" />
        <section name="authenticationService" type="System.Web.Configuration.ScriptingAuthenticationServiceSection, System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false" allowDefinition="MachineToApplication" />
      </sectionGroup>
    </sectionGroup>
  </sectionGroup>
</configSections>

This may look a bit intimidating, but just copy it. In the system.web part you can define the following tag:

<system.web.extensions>
  <scripting>
    <webServices>
      <jsonSerialization maxJsonLength="999999999"/>
    </webServices>
  </scripting>
</system.web.extensions>

And there you go. The number 999999999 is the absolute max value - so if you send stuff to your client longer than that, you have to think of something else. But for the mapping applications I made on Google Maps this works fine. Bear in mind that sending VERY large portions of data can degrade your user experience considerably, although the asynchronous nature of the ASP.NET JSON service might diminish that problem. By the way, in the latest templates for ASP.NET Ajax enabled web sites the whole configuration declaration is already included, as well as the declaration for the json serialization - but it is commented out. It looks something like this.

<!-- Uncomment this line to customize maxJsonLength and add a custom converter -->
<!--
<jsonSerialization maxJsonLength="500">
  <converters>
    <add name="ConvertMe" type="Acme.SubAcme.ConvertMeTypeConverter"/>
  </converters>
</jsonSerialization>
-->

You have to delete the part that I printed in italics, because that is just an example that does not point to an actual class

Calling ASMX web services directly from javascript

The AJAX hype is all around us, and Microsoft provided us with the very neat ASP.NET AJAX extensions and the control toolkit to make things easier. The UpdatePanel and its nephews enjoy a lot of time in the spotlight. The workhorse in the background that makes this all possible can also be used 'raw' - that is, you can call web services directly from javascript utilizing the standard structure of ASP.NET Ajax. It is a very efficient way to transfer data to and from the browser because the server transforms it into JSON - so no large chunks of verbose XML are send through the wire and you don't have to write elaborate pieces of XML-parsing javascript.
Follow the steps below to get things up and running.

This example assumes you have ASP.NET Ajax and de Web Applications already installed.

1. Create a new project of type "ASP.NET AJAX-Enabled Web Application"
This will create a new Web Application with a Default.aspx that already has a script manager on in, and a preconfigured web.config. Call this project WebServiceDemo

2. Add a web service
Right-click on your web application, click Add/New Item and then Web Service. Call this web service "DemoService".

3. Make a web service callable from script
Open code file "Default.aspx.cs". Notice your class "DemoService" sits in a namespace "WebServiceDemo". You will need this knowlegde later.
Add to the top:

using System.Web.Script.Services;

decorate the class with the attribute [ScriptService] and modify the standard hello world method so it looks like this:

[WebService(Namespace = "http://tempuri.org/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[ToolboxItem(false)]
[ScriptService]
public class DemoService : System.Web.Services.WebService
{
 [WebMethod]
 public string HelloWorld( string ToSomeone )
 {
   return "Hello World" + ToSomeone;
 }
}

4. Register the service on the page where you want to call it from
Open Default.aspx in design view and select the ScriptManager1
Select the "Services" property, and click the button that appears
Click "Add", and enter "DemoService.asmx" for the path property
Click OK. The result should look like this:

<asp:ScriptManager ID="ScriptManager1" runat="server" >
  <Services>
    <asp:ServiceReference Path="DemoService.asmx" />
  </Services>
</asp:ScriptManager>

5. Create a client side script to perform the call
Open Default.aspx in source view and enter just before the <head> tag the following code:

<script type="text/javascript">
 function CallService()
 {
   WebServiceDemo.DemoService.HelloWorld( "Yourself",
   Callback );
 }

 function Callback( result )
 {
   var outDiv = document.getElementById("outputDiv");
   outDiv.innerText = result;
 }
</script>

6. Create a button to start the web service calling function
Drag a button onto the form
Set the property "OnClientClick" to "CallService();return false;"

7. Create div for the output data
Drag a div (from the HTML tab) onto the form.
Set its id to "outputDiv";

If you did everything correctly, the text "Hello World Yourself" should appear beneath your button, without a visible postback.

Notice the following things:

• You always need the fully qualified name of the class to call the webservice: WebServiceDemo.DemoService.HelloWorld
• Calling a webservice from javascript always needs two methods: one to make the actual call, and one to receive the results
• The callback method is added as an extra parameter to the web service method parameters. In C#, the method has one parameter, so in javascript two.

This sample gives a nice under-the-hood sample of how ASP.NET Ajax transfers data to and from the browser.

Complete code downloadable here.