JQuery, JSON, and the REST Starter Kit

7 05 2009

The new version of Zleek will be retrieving all of its data from a RESTful WCF Service that serves up JSON data. It also makes heavy use of the JQuery 1.3 libraries with the JQuery JSON 1.3 plugin. Are they interoperable?

If you’ve worked with the REST Starter Kit on the server and JQuery on the client, you probably already know the answer: almost.

The problem lies in date serialization. Say you have the date 5/6/2009 and you want to pass it to your RESTful WCF service as JSON. Here is how JQuery serializes it:


And here is how the MicrosoftAjax.js library serializes it:


Your WCF service will only be able to deserialize dates in the Microsoft format and will throw an exception otherwise. You will also have the same problem going the other way. If your WCF service returns a DateTime object, it will be deserialized by JQuery as a string. Luckily, due to the nature of JSON (de)serialization, both of these issues are easy problems to fix.

I only wish I had seen Rick Strahl’s excellent posts from a year ago on this issue prior to tackling it. Apparently I’m Google-inept…

Situation 1: Transmitting dates from a JQuery client to a WCF service

For those of you new to JavaScript development, JavaScript is a dynamically typed language. Your prototype object can have a date property that you can change to a string or a number at any time without any issues. This property of JavaScript can be leveraged to fix our problem.

Prior to calling the JQuery library and issuing the request to the WCF service, you can change it to a string that will be serialized like the way MicrosoftAjax serializes dates. To get an output of:


We need an input of:


Here is a function you can use to do just that. It takes a JavaScript Date object and turns it into a string that JQuery will then serialize into the MicrosoftAjax date serialization format.

   1: Agnition.Serialization.SerializeDate = function(date) {

   2:     /// <summary>

   3:     ///    JQuery cannot serialize a Date object so that .NET can read it, this fixes its attempt by calling

   4:     ///    the MS AJAX serializer. JQuery will then serialize a string correctly.

   5:     /// </summary>

   6:     /// <param name="date">The date to serialize</param>

   7:     /// <returns>A JSON serialized date formatted for .NET</returns>

   8:     var dateString = Sys.Serialization.JavaScriptSerializer.serialize(date);

   9:     dateString = dateString.substring(2);

  10:     dateString = dateString.substring(0, dateString.length - 3) + "\/";

  11:     return dateString;

  12: };

Here is an example of it in use:

   1: this.CreateBook = function(successCallback, errorCallback) {

   2:     /// <summary>Attempts to register the user</summary>

   3:     /// <param name="successCallback">The callback fired if the request is successful</param>

   4:     /// <param name="errorCallback">The callback fired if the request failed</param>


   6:     var albumData = this.AlbumData;

   7:     var svcAddress = this.ServiceAddress;


   9:     albumData.LastUpdated = Agnition.Serialization.SerializeDate(albumData.LastUpdated);

  10:     var jsonRequest = new Agnition.Net.WebRequest();

  11:     var url = svcAddress + "/Album/new?includeMedia=false";

  12:     jsonRequest.PutJson(url, albumData, successCallback, errorCallback, null);

  13: };

Situation 2: Receiving dates from a WCF service in a JQuery client

This issue is basically the reverse of Situation 1. Your WCF service will serialize a DateTime object in the MicrosoftAjax format. JQuery will see this and deserialize it as a string. All we have to do is modify this string to put it back into the MicrosoftAjax format, and then use the MicrosoftAjax library to deserialize it. Here is what JQuery will deserialize the response as (notice it’s a string):


Here is a function that will perform the translation for you:

   1: Agnition.Serialization.FixJQueryDate = function(dateString) {

   2:     /// <summary>

   3:     ///    JQuery cannot deserialize the .NET dateTime object, so this fixes its attempt by reconstructing

   4:     ///    the string and calling the MS AJAX deserializer.

   5:     /// </summary>

   6:     /// <param name="dateString">The JQuery deserialized date string</param>

   7:     /// <returns>A JavaScript date object</returns>

   8:     dateString = dateString.substring(1);

   9:     dateString = dateString.substring(0, dateString.indexOf("\/"));

  10:     dateString = "\"\\\/" + dateString + "\\\/\"";

  11:     var date = Sys.Serialization.JavaScriptSerializer.deserialize(dateString);

  12:     return date;

  13: };

And here is an example of it in use:

   1: $(document).ready(function() {

   2:     var request = new Agnition.Zleek.Web.Service.PagedDataRequest(bookUrl, BOOK_PAGE_SIZE, BOOK_MAX_RECORDS);

   3:     request.GetPage(BOOK_CURRENT_PAGE,

   4:             function(data, statusText) {

   5:             /* snip */

   6:                 for (var i = 0; i < numAlbums; i++) {

   7:                     var album = data.Albums[i];

   8:                     album.LastUpdated = Agnition.Serialization.FixJQueryDate(album.LastUpdated);

   9:             /* snip */

  10: };

Now, assuming all of the dates in our service are UTC dates, this function will perform the conversion from UTC to the browser’s local time, and then we can create a new method which fixes both the date serialization and performs the time conversion:

   1: Agnition.ConvertUTCToLocalTime = function(date) {

   2:     /// <summary>Converts a UTC date to local time</summary>

   3:     /// <param name="date">The date object to convert</param>

   4:     /// <returns>The resulting local time</return>

   5:     var utcTime = date.getTime();

   6:     var localOffset = date.getTimezoneOffset() * 60000; // Get offset in milliseconds

   7:     var localTime = utcTime - localOffset;

   8:     return new Date(localTime);

   9: };


  11: Agnition.Serialization.GetJQueryDateAsLocalTime = function(dateString) {

  12:     /// <summary>

  13:     ///    JQuery cannot deserialize the .NET dateTime object, so this fixes its attempt by reconstructing

  14:     ///    the string and calling the MS AJAX deserializer. The resulting date is then converted to local

  15:     ///    time from UTC time.

  16:     /// </summary>

  17:     /// <param name="dateString">The JQuery deserialized date string</param>

  18:     /// <returns>A JavaScript date object, converted from UTC to local time</returns>

  19:     return Agnition.ConvertUTCToLocalTime(Agnition.Serialization.FixJQueryDate(dateString));

  20: };

Caveat Emptor

Because there is no standard date literal in JavaScript, Microsoft AJAX and JQuery may change their respective JSON date serialization formats in the future. So if you’re using this solution (and hopefully created a single function to perform the translations), you may have to modify your translation code when using future releases of JQuery or MS AJAX. In fact, Microsoft has already changed their format once before, and may do so again in the future.


MIX 09 Show Off Entry

10 03 2009

I put together a short video of the recent work I’ve done on Zleek for the MIX 09 Show Off competition.

Here is the video

Hopefully that will whet your appetite for the new changes coming to Zleek

Also, thank you to everyone who voted in the MIX 09 Smart Coding Challenge. I didn’t win a prize, but it was still a lot of fun.

Using Silverlight and WCF to create a RESTful File Upload Utility

12 02 2009


During the development of Zleek, I ran into some problems uploading files from Silverlight. I wanted a file upload utility that provided progress notification to the user. However, there is no built-in progress notification for the WebClient or HttpWebRequest classes! More specifically, the event exists but it currently does not provide enough resolution to be meaningful. It fires at only 100%, so it can only be used as a completion event. This is not good news if you want to provide file upload progress to users.

The second problem is that if you wanted to upload large files, you had to set your WCF Service‘s MaxMessageSize to whatever the maximum file size you are expecting — and if you’re expecting video files, it would have to be absurdly large.

The solution to both of these problems is to have the client break the files up into smaller chunks before sending them off to the server, which would then reassemble them in the correct order. This scenario is not overly complicated, but since it requires a good deal of busy work, I decided to put together a Silverlight library for the client and a WCF library for the service that allows you to quickly and easily build file upload functionality using Silverlight and WCF. This post explores how the libraries were built, and how to use them.

REST Refresher

First off, you should familiarize yourself with REST, WCF, and consuming WCF services from Silverlight by taking a look at Rob Bagby‘s REST in WCF series and my earlier post on Silverlight/WCF communication.

Here is a quick refresher:

REST is part of a service architecture (ROA) that attempts to simplify services by limiting communication to URIs, HTTP Verbs, and HTTP Status Codes. There are no complicated SOAP envelopes that a client needs to create. It just needs to know the URI it wishes to connect to.

URIs should be designed to be hackable, e.g. If sending a GET to http://myhost/myservice.svc/Media/1 retrieves the media item with ID 1, the consumer of the service would be able to guess that a DELETE would delete the media with that ID, while a PUT would insert or update the media with that ID.

Status Codes should be used effectively to report error conditions. Some common ones are:

  • 404 (Not Found), example: Returned for a GET/DELETE attempt if the specified ID does not exist
  • 400 (Bad Request), example: One of the parameters is not valid
  • 403 (Forbidden), example: Client is not authorized to perform the operation

eTag and Last Modified headers should be used effectively to permit caching of GET requests.

Use the WCF REST Starter Kit to create the service.

Client Library

Let’s start with the client. The client is responsible for breaking the files to be uploaded into smaller chunks and sending them off to the server. The main client class is UploadUtility (located in Agnition.Silverlight.Utils/Net), which uses RestfulServiceWrapper (located in Agnition.Silverlight.Utils/ServiceModel) to make calls into a RESTful WCF service.

UploadUtility.PerformUpload() takes a URI, URI format string, and a list of FileInfo objects. It creates a FileWrapper for each of these files, and splits them up into smaller “chunks” (FileChunk). These files are then queued up for upload.

After queueing up the files, the client starts a background process which continually checks to see how many “chunks” are currently being uploaded, and if it is under the MaxUploadCount, pulls the next file out of the queue. When the file is pulled out of the queue, the first “chunk” for the file is uploaded. When the upload completes, progress is reporting and if there is a next “chunk”, it starts the upload for that chunk.

Once the queue is empty, the entire upload process reports as complete.

The URI format string is responsible for telling the client how to perform variable substitution. This format string should at a minimum contain the following placeholders:

  • BaseUri – the service URI.
  • FileName – the original file name of the file being uploaded.
  • FileKey – a unique identifier for the file being uploaded.
  • ChunkNumber – the sequence number for the current chunk being uploaded.
  • TotalChunks – the total number of chunks that make up the file.

The default format string is “{BaseUri}/{FileKey}/{TotalChunks}/{ChunkNumber}?Name={FileName}“, which is used to create the target URI when upload requests are issued. You can also specify custom parameters by inheriting from FileWrapper and overriding UploadUtility.CreateWrapper(), more on this later. FileName and all custom parameters will automatically be URL Encoded.

Important:If a parameter (such as FileName) contains values that might include characters that must be URL encoded, they should be formatted as a query string parameter rather than as part of the main URI. Otherwise, the RESTful WCF service will not correctly map the request to the appropriate method.

The upload utility provides the following settings which can be used to tune the upload process:

  • MaxUploadCount controls the number of concurrent uploads that the client issues. Defaults to 1.
  • MaxUploadSize controls the “chunk” size of the files. If a file size is greater than this setting, it will be split into two or more “chunks”. Defaults to 200KB.
  • MaxFileSize is the maximum file size for any file being uploaded. If a file size is greater than this, it will not be uploaded. Defaults to 30MB.
  • IsApproximatingProgress will increase the resolution of progress notification by using the detected transfer rate to report a calculated progress using the specified reporting interval. When a “chunk” completes upload, the calculated progress will be rest to the actual progress. A “speed test” will be performed prior to uploading the first “chunk” in order to seed the transfer rate.
  • ProgressReportingFrequency sets the progress reporting interval when IsApproximatingProgress is true. Defaults to 20 msec.
  • ProgressReportingMultiplier is a “fudge factor” multiplier to apply to any calculated progress reporting when IsApproximatingProgress is true. This should be between 0 and 1. Defaults to 1.
  • SpeedTestMinimumSize sets the minimum size of the “speed test” to perform when IsApproximatingProgress is true. Defaults to 32KB
  • SpeedTestMaximumSize sets the maximum size of the “speed test” to perform when IsApproximatingProgress is true. Defaults to 200KB.
  • SpeedTestSizePercentage sets the size as a percentage of the total upload size of the “speed test” to perform when IsApproximatingProgress is true. Defaults to 5%.

Server Library

The server logic is very straightforward. Its only job is to reassemble temporary “chunks” uploaded by a client into a complete file, and delete those chunks when the completed file is assembled or the upload is cancelled. It is the job of UploadHandler.cs (located in Agnition.Utils/ServiceModel) to perform these tasks. UploadServiceHostFactory is a subclass of RestServiceHostFactory that sets the MaxMessageSize. Important: This should match the MaxUploadSize of the UploadUtility client!

Consuming services should create a singleton instance of UploadHandler and call UploadHandler.ProcessUpload() and UploadHandler.CancelUpload() to perform these processes.

Putting It All Together

We can use the client and server libraries to put together a file uploader for media files. Let’s start off by building the client.
The UI is pretty straightforward. It contains an upload/cancel button, an overall progress display, and a file progress display. The individual file progress displays are seperated out into another user control that takes an instance of FileWrapper in order to display a preview.

To extend the libraries for media support, I created a few additional classes: MediaFileWrapper, which inherits from FileWrapper by providing a FileType custom parameter, and MediaUploadUtility, which inherits from UploadUtility. The source code for these are as follows:


public class MediaFileWrapper : FileWrapper


    public const string IMAGE_TYPES = "bmp;jpg;jpeg;gif;png;tif;tiff";

    public const string VIDEO_TYPES = "avi;mpg;mpeg;wmv;asf;dvr-ms;m2v;ts;m2t;vob;mod;avs;mov;m4v;mp4;3gp;3g2;dv;mp2";

    public const string SILVERLIGHT_IMAGE_TYPES = "jpg;jpeg;png";

    public const string SILVERLIGHT_VIDEO_TYPES = "wmv";


    public FileType Type {

        get { return _type; }



    public bool CanPreview {

        get { return _canPreview; }



    public MediaFileWrapper(FileInfo file) : base(file) {

        _type = DetermineType(file);

        this.Parameters.Add(new KeyValuePair<string, string>("FileType", Type.ToString()));



    private FileType DetermineType(FileInfo file) {

        string extension = file.Name.Substring(File.Name.LastIndexOf('.') + 1).ToLower();

        List<string> imageExtensions = new List<string>(IMAGE_TYPES.Split(';'));

        List<string> silverlightImageExtensions = new List<string>(SILVERLIGHT_IMAGE_TYPES.Split(';'));

        if (imageExtensions.Contains(extension)) {

            _canPreview = silverlightImageExtensions.Contains(extension);

            return FileType.Image;



        List<string> videoExtensions = new List<string>(VIDEO_TYPES.Split(';'));

        List<string> silverlightVideoExtensions = new List<string>(SILVERLIGHT_VIDEO_TYPES.Split(';'));

        if (videoExtensions.Contains(extension)) {

            _canPreview = silverlightVideoExtensions.Contains(extension);

            return FileType.Video;



        return FileType.Other;



    private FileType _type;

    private bool _canPreview;



public class MediaUploadUtility : UploadUtility


     public List<MediaFileWrapper> MediaFiles {

         get {

             return this.Files.Cast<MediaFileWrapper>().ToList();




     public MediaUploadUtility() : base() { }


     protected override FileWrapper CreateWrapper(FileInfo file) {

         return new MediaFileWrapper(file);



     protected override void ValidateFile(FileWrapper file) {

         ValidateFileType((MediaFileWrapper) file);




     protected void ValidateFileType(MediaFileWrapper file) {

         if (file.Type == FileType.Other) {

             throw new Exception(string.Format("File '{0}' is an unsupported file format.", file.File.Name));




In the actual UI, the pertinent code is the UploadFiles() method, which calls the MediaUploadUtility to initiate the upload process, and the progress notification event handler Uploader_UploadProgressChanged(), which updates the progress notification for the overall upload progress and the progress for each individual file.


public partial class FileUpload : UserControl


    protected MediaUploadUtility Uploader {

        get { return _uploader; }



    public FileUpload() {



        _uploader = new MediaUploadUtility();

        _uploader.UploadProgressChanged += new UploadProgressChangedHandler(Uploader_UploadProgressChanged);

        _uploader.UploadCompleted += new UploadCompletedHandler(Uploader_UploadCompleted);

        _uploader.UploadCancelled += new UploadCancelledHandler(Uploader_UploadCancelled);


        _uploader.IsApproximatingProgress = true;

        _uploader.ProgressReportingMultiplier = 0.8;

        _uploader.MaxUploadCount = BrowserUtil.Name == BrowserName.Firefox ? 6 : 2;



    private void UploadFiles(IEnumerable<FileInfo> files) {

        // Start upload

        Progress.Value = 0;

        ProgressText.Text = "0.00%";

        Uploader.PerformUpload(files, new Uri(Configuration.UploadServiceAddress + "/Media/Upload", UriKind.Absolute),



        // Add files to preview window


        foreach (MediaFileWrapper file in Uploader.MediaFiles) {

            FileProgressContainer.Children.Add(new FileProgressDisplay(file));




    private void CancelUpload() {

        if (Uploader.IsWorking) {

            UploadButton.Content = "Cancelling...";

            UploadButton.IsEnabled = false;

            Uploader.Cancel(new Uri(Configuration.UploadServiceAddress + "/Media/CancelUpload", UriKind.Absolute),





    private static string CreateFileFilter() {

        // Get image extensions

        string imageExtensions = string.Empty;

        foreach (string ext in MediaFileWrapper.IMAGE_TYPES.Split(';')) {

            if (imageExtensions.Length > 0) {

                imageExtensions += ";";


            imageExtensions += "*." + ext;



        // Get video extensions

        string videoExtensions = string.Empty;

        foreach (string ext in MediaFileWrapper.VIDEO_TYPES.Split(';')) {

            if (videoExtensions.Length > 0) {

                videoExtensions += ";";


            videoExtensions += "*." + ext;



        // Create filter string

        return string.Format("ImageFiles|{0}|Movie Files|{1}|All Supported Media|{0};{1}",

          imageExtensions, videoExtensions);



    private void UploadButton_Click(object sender, RoutedEventArgs e) {

        switch (UploadButton.Content.ToString()) {

            case "Upload!":

                OpenFileDialog uploadDialog = new OpenFileDialog();

                uploadDialog.Filter = CreateFileFilter();

                uploadDialog.FilterIndex = 3;

                uploadDialog.Multiselect = true;

                if (uploadDialog.ShowDialog() == true) {

                    UploadButton.Content = "Cancel";





            case "Cancel":






    private void Uploader_UploadCancelled(object sender, EventArgs e) {

        Dispatcher.BeginInvoke(delegate() {

            ProgressText.Text = "Cancelled";

            UploadButton.Content = "Upload!";

            UploadButton.IsEnabled = true;





    private void Uploader_UploadCompleted(object sender, EventArgs e) {

        Dispatcher.BeginInvoke(delegate() {

            ProgressText.Text = "Completed";

            UploadButton.Content = "Upload!";

            UploadButton.IsEnabled = true;





    private void Uploader_UploadProgressChanged(object sender, UploadProgressEventArgs e) {

        Dispatcher.BeginInvoke(delegate() {

            Progress.Value = e.TotalPercentComplete;

            ProgressText.Text = string.Format("{0:n2}%", e.TotalPercentComplete);

            foreach (FileProgress fileProgress in e.FileProgress) {

                foreach (FileProgressDisplay fileDisplay in FileProgressContainer.Children) {

                    if (fileProgress.File.File == fileDisplay.UploadedFile.File) {

                        fileDisplay.Progress = fileProgress.Progress;

                        fileDisplay.ProgressText = string.Format("{0:n0}%", fileProgress.Progress);








    private readonly MediaUploadUtility _uploader;


Easy, right? UploadUtility abstracted away all of the mess!

Now let’s build the service. I created the MediaUploadHandler class, which inherits from UploadHandler, to provide the correct upload paths and provide additional processing dependent on the file type.


public class MediaUploadHandler : UploadHandler


    public override string UploadPath {

        get { return AppDomain.CurrentDomain.BaseDirectory + "/Uploads/"; }



    public override string TemporaryUploadPath {

        get { return UploadPath; }



    protected override void ProcessCompletedFile(string fileKey, string fileName, string outputFilePath, object[] additionalParameters) {

        MediaType mediaType = (MediaType) additionalParameters[0];


        // TODO: Process images/videos differently based on media type



Next, add a new service to your WCF project. Edit the XAML to specify the Agnition.Utils.ServiceModel.UploadServiceHostFactory, and then create three service methods: UploadChunk, CancelChunk, and SpeedTest as follows. Be sure to pay attention to the URI format strings!


<%@ ServiceHost








[ServiceContract(Namespace = "Agnition.Test.Web.Service.Upload")]

[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]

public class Upload


    public static MediaUploadHandler MediaUploadHandler {

        get {

            return _uploadHandler;





    [WebInvoke(Method = HttpVerbs.PUT,

    UriTemplate = "Media/Upload",

    ResponseFormat = WebMessageFormat.Json,

    RequestFormat = WebMessageFormat.Json)]

    public string SpeedTestResponse(Stream data) {

        byte[] binaryData = UploadHandler.ReadInputStream(data);

        return "Speed Test Complete. " + binaryData.Length + " bytes received.";




    [WebInvoke(Method = HttpVerbs.DELETE,

    UriTemplate = "Media/CancelUpload/{fileKey}",

    ResponseFormat = WebMessageFormat.Json,

    RequestFormat = WebMessageFormat.Json)]

    public void CancelUpload(string fileKey) {

        fileKey = HttpUtility.UrlDecode(fileKey);


        // Check input variables

        if (string.IsNullOrEmpty(fileKey)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid FileKey");








    [WebInvoke(Method = HttpVerbs.PUT,

    UriTemplate = "Media/Upload/{fileType}/{fileKey}/{chunkNumber}/{totalChunks}?Name={fileName}",

    ResponseFormat = WebMessageFormat.Json,

    RequestFormat = WebMessageFormat.Json)]

    public void UploadChunk(string fileKey, string fileName, string fileType, string chunkNumber, string totalChunks, Stream data) {

        // Check input variables

        if (string.IsNullOrEmpty(fileKey)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid FileKey");



        if (string.IsNullOrEmpty(fileName)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid FileName");



        if (string.IsNullOrEmpty(fileType)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid FileType");



        if (string.IsNullOrEmpty(chunkNumber)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid ChunkNumber");



        if (string.IsNullOrEmpty(totalChunks)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid TotalChunks");



        if ((data == null) || (!data.CanRead)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid Data");




        // Parse input

        int iChunkNumber = 0;

        int iTotalChunks = 0;

        FileType eFileType = FileType.Other;

        if (!int.TryParse(chunkNumber, out iChunkNumber)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid ChunkNumber");



        if (!int.TryParse(totalChunks, out iTotalChunks)) {

            RestfulServiceHelper.ReturnBadRequest("Invalid TotalChunks");



        fileName = HttpUtility.UrlDecode(fileName).Trim();

        if (fileName.Length == 0) {

            RestfulServiceHelper.ReturnBadRequest("Invalid FileName");



        try {

            fileType = HttpUtility.UrlDecode(fileType);

            eFileType = (FileType) Enum.Parse(typeof(FileType), fileType, true);


        catch {

            RestfulServiceHelper.ReturnBadRequest("Invalid FileType");




        try {

            // Process the uplaoad

            MediaUploadHandler.ProcessUpload(fileKey, fileName, iChunkNumber, iTotalChunks, data, eFileType);


        catch (Exception ex) {





    private enum FileType


        Other = 0,

        Image = 1,

        Video = 2,



    private readonly static MediaUploadHandler _uploadHandler = new MediaUploadHandler();


That’s it! All you had to do was take care of parsing the input, and UploadHandler takes care of the rest.

Source Code

Get the Source Code for this article here. It also includes the Microsoft.ServiceModel.Web library from the WCF Rest Starter Kit.