Designing useful JSON feeds

Technical background: JSON, short for the javascript object notation is, much like XML, a way of encoding data, though in a less spacious fashion, and one which is also compatible with javascript syntax. A chunk of very terse XML to describe this blog might look like this:

<blog>
  <u>http://ecmanaut.blogspot.com/</u>
  <n>Johan Sundström's technical blog</n>
  <d>ecmanaut</d>
</blog>

The top element, as an aside, is required in XML, even if we don't assign any special data to it. In JSON, the above entry would look like this instead:

{"u":"http://ecmanaut.blogspot.com/","n":"Johan Sundström's technical blog","d":"ecmanaut"}

This makes for a smashing combination of light-weight data transport and easy data access for javascript applications. With very little extra weight added to a chunk of JSON, it also enables javascript applications to query cooperative remote servers for data, read and act on the results, which is something the javascript security model explicitly forbids javascript to do with XML, or HTML and plain text. (As a way of protecting you from cross site scripting security holes, which is a good thing to avoid.)

Anyway, with JSON, and a bit of additional javascript to wrap it, we can include data from some other domain given that wants to share data with us, using a script tag we point there, the data will be loaded and available to other scripts further down on the page in a common variable. This is a great thing, whose use is slowly spreading, with pioneering sites such as the cooperative bookmarking and tagging service Del.icio.us, that offers not only RSS feeds of the bookmarks people make, but also JSON feeds.

This article will show what Del.icio.us does, why they do, and suggest best practices for making this kind of JSON feeds even more useful to applications developers. This is what Del.icio.us adds to the bit of JSON above:

if(typeof(Delicious) == 'undefined')
  Delicious = {};
Delicious.posts = [{"u":"http://ecmanaut.blogspot.com/","n":"Johan Sundström's technical blog","d":"ecmanaut"}]

This does two things. First, it peeks to see if there is already a Delicious object defined. If it isn't, it makes a new one, a plain empty container. This is a good way of making sure that other data sources in your API can be included later on in the page, coexisting peacefully without overwriting one another, or adding variable clutter in the global scope. (It is also good for branding! ;-) Then, it assigns our bit of JSON into the first element of an array -- or, in case we asked for lots of entities, fills up an entire arrayful of them.

Now, a later script on the page can do whatever it wants with the data in Delicious.posts, and all is fine and dandy. This is how the category menu on the right of this blog is built, by the way, asking Del.icio.us for posts on my blog carrying the appropriate tag. (In case this sounded very interesting, read more in this article.)

This is all you need for static content pages, being loaded with the page, parsed and executed once only. AJAX applications are typically more long lived, and often do what they do without the luxury of reloading themselves as soon as they want more data to crunch on. Adding support for this isn't hard either: this is what we add at the end of our JSON feed:

if(Delicious.callbacks && Delicious.callbacks.posts)
  Delicious.callbacks.posts( Delicious.posts );

What happens here? Well, first we peek to see if there is a Delicious.callbacks object defined. If there is, we peek inside it, to see whether there is a callback Delicious.callbacks.posts installed for the page that sends the Delicious.posts data. If there was, we call it, passing the data, and let the callback do whatever its application requires of it.

A very rudimentary example script using the API could now look like this:

<script type="text/javascript">
Delicious = { callbacks:{ posts:got_posts, tags:got_tags }};
function got_posts( posts )
{
  alert( posts.length + ' posts matched.' );
}

function got_tags( tagnames, usernames )
{
  alert( tagnames.length + ' tags shared by ' +
         usernames.length + ' users.' );
}</script>
<script type="text/javascript" src="http://api.url/">
</script>

The tags member of the callbacks object is for another (fictious) API, which in this example is called with two arrays of data, one carrying a number of tag names, the other an array of user names who use all of these tags.

With a callback approach like this, an AJAX application can create new <script> tags to its page to perform successive requests to a data source and get alerted of the new data when it arrives. As neither HTML nor javascript provide any means of annotation for when a <script> tag has completed loading, we would otherwise have to do lots of housekeeping and polling to find out when the new content arrives, or to perform advanced feats of magic most javascript programmers are not familiar or comfortable with. Callbacks solve that.

The approach outlined above does not solve another problem, though: handling multiple simultaneous requests. To do that, we also need to be sure each request gets handled by its own callback. This is best solved as a separate mode of invoking your API, by providing an additional URL parameter stating the name of the callback to run, or, more precisely, what javascript expression that should be invoked to call the right callback with your data. With this direct approach, we are best off not assigning any variable at all, but rather just pass the callback the data we feed it, right away.

Assuming our query parameter be named cb, we get invoked like this:

<script type="text/javascript" src="http://api.url/?cb=callbacks[17]">
</script>

and we return the code (as this is a call for our fictious tags call):

if(typeof callbacks[17] == 'function')
  callbacks[17]( ["Web2.0","Del.icio.us"],
                 ["John","Eric","Bruce"] );

-- copying the parameter verbatim.

And the web is suddenly a slightly better place than it was, thanks to your application.

Categories:

• JSON
• javascript
• API
• design
• Del.icio.us