This post is triggered by Fridays posting on RWW by Bernarnd Lunn ‘Sorry Google, You Missed the Real-Time Web!’. In it Lunn rightly describes how keeping track of things that are happening right now is a new area of innovation, where big incumbents like Google don’t have much to bring to the table. (Consistent with innovation theory, of course).
As an example he holds up the US Air emergency landing on the river Hudson this week, and I think the terrorist attacks on Mumbai would be another recent one. Information about both events was available earlier on Twitter, Flickr, Qik etc, than in main stream media, as well as quicker than Google could index. Lunn also holds the Dutch/Belgian tool Storytlr up as an example of a tool that shows us the ‘real time web’. I think that reasoning is flawed, and that Storytlr is a very interesting tool, just not for showing us the ‘real time web’ (as witnessed by the amount of effort it took to get the result below).

Example of combining microblogging, photo and video streams into Storytlr
The reasoning is flawed I think because of the fact that we only conclude ‘Twitter was faster’ with hindsight, and that the actual number of people alerted to something happening on the Hudson through their Twitter or other accounts was small in those 7 minutes where main stream media were not yet broadcasting their news alert. I am much more interested in those first 7 minutes while we’re in them.
What is the ‘real time web’?
Real time is when you get your data stream through the web the instant the data is generated. Examples would be microblogging tools like Jaiku and Twitter, photo upload sites (Flickr, 23, Twitpic), live video (Qik, Seesmic), but also things like SMS ( Another example would be the number of mobile phones and their speed of movement on highways, such as TomTom uses to show you traffic congestion in their route navigation software.
When is ‘real time’ useful?
I see three general areas where real time is useful.

  • If it helps me decide to change my course of action right this instant. Traffic information for instance may make me choose a different high way exit than originally planned, or adapt my route while I’m already traveling it. Crisis management would be another example.
  • If it helps me leverage an information advantage immediately. Day trading stocks comes to mind, or having the very latest developments available on your subject of expertise.
  • If speed is more essential than accuracy or completeness. It takes time to filter and to make sense of information, to create a coherent storyline. But sometimes it’s more important to be fast, than to have a clear grasp of events. We accept increased noise to be quicker. When a part of our town was devastated by a large explosion in 2000, we immediately, after feeling the shockwave and hearing the blast, called our family to let them know we were ok, even though we did not know at all what had happened. We just knew that it was very bad, and that telecommunications might collapse soon. We called and said “Something bad has happened, but whatever it is you will see in the news soon, know that we are ok”. For less urgent things we can permit ourselves time to make sense of it first.
  • Faster as it gets closer, as it has bigger consequences
    I want my information faster the closer it is to me. Both geographically and socially.
    If something happened to my wife I want to know this instant, wherever she is, wherever I am. If something is happening in my street or even in my house I want to know this instant. If something is happening in the place I will be at in 5 minutes, I want to know this instant.
    So, anything happening to people close to me, regardless their location or anything serious happening in their location, is important to me. Anything happening in my direct physical environment, or in my near future physical environment, is important to me.
    I want my information also faster as it has bigger consequences for me. Consequences for my life in general, or in my professional fields of expertise. If some tipping point is reached (like the Saudi Ghawar oil field peaking, as it is the singular signal oil production in general is peaking. The Saudi are the only flexibility in oil supply we have, all others are producing at peak capacity), I want to know fast. Because it’s a tell tale sign for different scenarios becoming important. If you need to switch to plan B, or C, you want to know fast when the time to switch has come.
    Early Warning, Weak Signals, and Predictability
    As I said above, the number of people that were aware of US Air having crashed in the Hudson during those first few minutes was limited. Only after the fact did more people conclude it was an early warning. Not that those first Twitter messages and photos were not important, because they were. It’s just that probably not all people for whom it would have been important were privy to those first messages. If you knew what to look for you could set alerts on real time information streams. But of course most of the interesting things are rather unpredictable, and not just the Black Swans. Being unpredictable is what makes them newsworthy and interesting almost by definition. At the same time, the really important bits might go unnoticed. Weak signals are very important in complex situations, but are by definition hard to lift from the noise.
    So what would you look for in real time streams? The number of times a location is being mentioned, perhaps in combination with other key words? You will detect the Black Swans and other ‘big’ events, but you will overlook the Weak Signals that may signal an important trend or a tipping point in developments. How will you go about setting up your Early Warnings?
    Tracking the Real Time Web?
    What does it take to track the Real Time Web, both for detecting the eye-catching unpredictable around locations and people important to you personally, as well as the tell-tale weak signals that will have consequences on your life and work?
    Are there reasonable ways to e.g. determine what a tipping point would look like? Are there detectable characteristics for weak signals that will be important to you? How do you weed out false positives (i.e. getting an alert for heightened activity mentioning your town, to only have it turn out to be about a rock concert taking place?)
    Is there a way of filtering for a Black Swan, when you know that these are events with very low probability at any given time, though bound to happen at some point? Is there a way to create your filter or antenna against reasonable effort/cost in such a situation?
    Technorati Tags: , , , , , ,

    2 reactions on “On the Real Time Web

    1. What is interesting about the crash in New York City is that there was national coverage in the US within the 7 minutes BECAUSE of its location. As many of the national media in the US is headquartered in NYC, there was instant information.
      However, in other parts of the state (not to mention the country or world), this type of access to information is not found. When we had major power outages in December, I had not access to information even though I could access the internet through a cell phone.
      The week before the Hudson River crash, I came into what I thought was an accident and was stuck in traffic for about 5 minutes. I could see the road closed about a kilometer ahead of me on either side of the highway, but nothing else. Suddenly, cars ahead started turning and people started fleeing on foot. Most people were on cell phones as they started leaving the area. We found out hours later that there was a man shooting from a car. Neither the media, facebook, nor twitter had information. Most received the information through word of mouth and cell phones.
      I agree with you that what interests us is immediate information about things that will affect us immediately. On the university campus in the US, this is usually via text messaging and facebook. In fact, I am seeing the growth of facebook as a means for immediate information of all generations (i.e. family events). Most of the students at Virginia Tech learned of an armed gunman on campus through facebook as residence halls were locked down.

    2. Thanks Virginia, for your comment and thoughts. The actual location indeed influences how and if information can be spread. It is the one key factor that sets the starting conditions for the possible paths any piece of information can travel.
      As per your example it of course depended on the people that were in the traffic jam, and the kind of channels and tools they used. At least here in Europe cell phones are used more for non voice communication (internet etc., 85% of usage time) than for voice (only 15% of phone usage time), internet and cell phone have become more or less synonymous. The uptake of services (Twitter is I think still in the early adopter stage, Facebook in Europe still is as well, while having moved more into mainstream in the US, based on its campus/student population kick start) is then the limiting factor. How many Twitter users or Jaiku users are there in an average NYC traffic jam, how many are there in an average traffic jam near a small village somewhere in less populated areas?

    Comments are closed.