2006-04-15

The stigmergic user script pattern

In a recent (half an hour long) podcast, Jon Udell and Steve Burbeck explore multi-cellular behaviour in a biological and a computer science setting, and what this field could emerge into in a world of computing and computer networks. A fresh perspective on message passing, emergent behaviour and stigmergy structures, concepts not all familiar to people from typical engineering backgrounds, but which have been flourishing in biology for some time already.

Nevertheless, it's growing on us, too, like mold. We are just not very far into putting it in words and common familiarity and frames of reference. The user script is slowly shifting and reshaping our understanding of the web, into something it was not designed to be, but, as many of us have noticed, into a very powerful and shape shifting capabile organism. It's a paradigm shift I believe will evolve the web in a profound way, following along in the trail of previous paradigm shifts:

  • The transition from the web as a set of loosely coupled static hypertext documents into server side dynamically generated pages

  • The large-scale client adoption of XMLHttpRequest, shifting pre-AJAX-ian days to a part client side driven data flow

  • The client driven orchestration or remixing of what the web page does, and how, by way of the user script. This is one of the presently evolving next steps, and useful patterns and practices are slowly forming around it.
While standardization is only just now slowly reaching XMLHttpRequest, and has not moved towards the user script dimension yet, we can in a way be thankful for only having seen a manageable user interface to the user script capability in one of the mainstream browsers yet, making Greasemonkey a de facto standard in the absence of organized work in the field.

But I'm losing track, here; let us get back to the podcast topics, again. A stigmergy structure is the result of emergent behaviour in a system, examples from the podcast being Wikipedia or the Linux source code, evolving over time from contributions by a large number of people, building something fit for a purpose that evolves around some small set of basic principles governing its growth. Or search engines, harnessing the web into a very large data mine, growing larger and more resourceful by the second.

Stigmergy is the method of communication in these systems: how one part sends a message to another part by modifying their common environment. Editing a wikipedia page, submitting a linux patch or writing a web page, registering it with the search engine or being picked up by it from external linkage. And it's a coming, very useful practice in user scripts too.

I recently dropped a reminder on my idea blog repository about finding or devising a gallery browser user script (and Instant Gallery is what came out of it), to transform any page with image links into a common gallery browsing application customized to my own liking, giving me the benefit of a user interface I like and could improve on incrementally and perfect, share with friends and the rest of the web, and use that, pan web, without any prior cooperation from web masters everywhere. This is the raw power of the user script.

And it is also where stigmergy comes into play. Because not all web pages are made to my own and the W3C:s standards and expose image links as hypertext links, handled the same way whether your browser was written this millennium or the former, is capable of javascript or not, and so on, and some employ the most hideous of ways to render click-to-zoom images. The traditional programming approach would be to teach the gallery program how to handle this disorderly input, rendering new versions any time we encounter a new format in need of conquering. But we don't have to.

Recognizing that we have a script that takes image links and makes a gallery from them, we can make a separate script that modifies a standards defiant web site into a standards compliant site with proper image links, prior to running the gallery script -- harvest the page for its image links, add them as proper links, and the gallery is set to go. No added to the gallery tool, easy plug-in fixes from other people who use other sites than I do, and want to use them with the script without to understand and modify my script or have my help doing it.

The web page, in a user script setting, is a stigmergic structure, where an ecology of user scripts can work together to build new and interesting applications, without the blessings of or participation from the webside page host.

And plain image links in a page, is a de facto micro format, specifying a gallery of images. The input fixup scripts can work their magic on a page for the benefit of any other script besides my (and Jos van den Oever's) gallery hack which takes that micro format and renders a gallery of it. We could just as well plug in another script that uploads a copy of those images to a Flickr account, stores them in Google Base or on some social bookmarks service, or anything else we might want to do with a bunch of images.

Making small scripts to bring structure to pages of some common traits and making other scripts to pick up on this structure and perform on it is a very useful pattern from biology that has lots merits in this computer science setting, too. Specialize, do your thing well, and share the result with other specialized entities who do theirs.

Next I would want to stigmergize-enable GM_xmlhttpRequest calls too, to provide the same ecology of specialized helpers to process page-external resources to the scripts I write that pull together data from many locations into tools like Book Burro. But let's play this one step at a time.

4 comments:

  1. Whoa, as usual you have said a mouthful....give me a while to digest, I'm going to need to hire a translator cause I can only understand a bit of this but I get where you're coming from..,or maybe you could develop a userscript for laymans' translation ? B.

    ReplyDelete
  2. "Stigma + emergent patterns?" was my first thought. "When does he get to the stigma?" was my second one.

    The mental article I anticipated was different, and had more to do with the mechanisms for self-regulating the landscape of open services (and the developer crowd that's attracted or repelled by this relative openness). Here are the imagined subsections:

    * How (foul) User scripts stigmatize web services a.k.a. The stigmatizing effects on User Scripts by Script Kiddies
    * Emergence of a Noveau Elite, savvy to the new web, casting a zone of stigma around them, closing the Openness of the Protocols and Standards, socially

    Note: I have to actually read the references to Udell and company, to see if I missed any scattered Greek references.

    PS: My third thought was "Who is Stig?".

    ReplyDelete
  3. Stigmergy is a very powerful idea, definitely. One aspect I haven't seen much research on is the dangers of degenerate signal patterns. Eric Bonabeau talks about how ant colonies moving from one place to another can cross their own trail & end up following their own stigmergic tracks in a circular path that never ends, ultimately dooming the whole colony. It looks to me like at least some parts of Wikipedia are falling into a degenerate pattern, with many pages losing coherence over time. How can we detect & fix, or better yet prevent these degenerate signal patterns from forming?

    ReplyDelete
  4. Now there was an interesting thought. In the basic case of cascading through userscripts by the default invocation mechanism, at least in the case of greasemonkey, just relying on script registration order for deciding the order of invocation, we are actually protected by the fire-once guarantee; no script will come across its own path.

    I would believe all user script environments implement something to the same end, one way or the other, if perhaps not identical to the detail with how Greasemonkey tackles the situation. Script invocation precedence rules are, I believe, something implementation defined, or undefined, at worst.

    In the event of forthcoming, more worked-through / matured user script systems, I would address this problem by book-keeping some more local short-term state, user scripts being able to log script local data, and act on whether or how many times one script has been invoked on the same page earlier already with this page-load. For now though, we would probably not run into the problem at all.

    ReplyDelete

Limited HTML (such as <b>, <i>, <a>) is supported. (All comments are moderated by me amd rel=nofollow gets added to links -- to deter and weed out monetized spam.)

I would prefer not to have to do this as much as you do. Comments straying too far off the post topic often lost due to attention dilution.

Note: Only a member of this blog may post a comment.