Harold Jarche looked at his most visited blog postings over the years, and concludes his blog conforms to Sturgeon’s Revelation that 90% of everything is crap.

I recognise much of what Harold writes. I suspect this is also what feeds impostor syndrome. You see the very mixed bag of results from your own efforts, and how most of it is ‘crap’. The few ‘hits’ for which you get positive feedback are then either ‘luck’ or should be normal, not sparse. Others of course forget most if not all of your less stellar products and remember mostly the ones that stood out. Only you are in a position to compare what others respond to with your internal perspective.

At the same time, like Harold, I’ve realised that it is important to do things, to keep blogging and writing in this space. Not because of its sheer brilliance, but because most of it will be crap, and brilliance will only occur once in a while. You need to produce lots of stuff to increase the likelihood of hitting on something worthwile. Of course that very much feeds the imposter cycle, but it’s the only way. Getting back into a more intensive blogging habit 18 months ago, has helped me explore more and better. Because most of what I blog here isn’t very meaningful, but needs to be gotten out of the way, or helps build towards, scaffolding towards something with more meaning.

It’s why I always love to see (photographs of) artist’s studio’s. The huge mess and mountains of crap. The two hundred attempts at getting a single thing to feel right for once. Often we see master pieces only nicely presented and lighted on a gallery wall. But the artist never saw it like that, s/he inhabits that studio where whatever ends up on a museum wall someday is just one thing in a mountain of other things, between aborted efforts, multiple works in progress, random objects and yesterday’s newspaper.

Shakingtree Award

Today I attended the presentation of this year’s Shaking Tree Award. This annual award started in 2016, and is named after my friend Niels who received the first award during his ‘last lecture‘. Niels died a year ago. The Ministry of Healthcare has pledged to keep going with the award, in the spirit of Niels’ efforts: shake up the system, fighting unneeded and kafkaesque bureaucracy, have hands-on experience with the system at ‘the receiving end’ so you know what you’re talking about, have a sense of humor to go with it, and be able to ‘dance with the system’.

The meeting was attended by a diverse range of people, from the healthcare domain, Niels’ family, and of course a smattering of Niels’ friends.

Before presenting this year’s nominees and the award, time was given to remembering Niels and the reason for this award. This was followed by two conversations between a previous winner and nominee and a representative of an institution they struggled with. First were Annette Stekelenburg and Ria Dijkstra, manager operations at a health care insurer. Annette has a son that needs tube feeding to survive. This situation will not change. Yet every year they need to apply for approval to continue receiving the materials needed. Annette and Ria had a frank conversation about what happened when Annette publicly announced she was fed up with this yearly bureaucracy that should be unneeded. Dijkstra explained how they thought that they had already changed the rules, making the renewal once every 5 years, but that the suppliers never knew, and that forms are being sent out in the insurers name that don’t actually exist anymore.

The second conversation was between Kathi Künnen, a previous nominee, and Betsie Gerrits, department head at UWV, the government agency in charge of employee insurance. Kathi is 29 and has incurable cancer. Because of that she has been determined to be 100% incapable of working, yet there are lots of phases where she actually does want to work. 25% of young professionals with cancer have an incurable form, and most want to remain active as long as possible. Yet the system tells them their ‘earning capacity is 0’ and with a stamp like that there’s no way to find paid activity. Here too, the conversation first of all made the two parties at the table see each other as individual human beings. And from it energy and potential solutions follow. Kathi said she needs reassurance that there can be administrative certainty (other than being tossed out as worthless), as her own life is fluid enough as it is and changing all the time.

I thought both conversations were impressive, and the type of thing we need much more of. Once you get past the frustration, anger and disbelief that often plays a role too, you can see the actual human being at the other side of the table. Dancing with the system is, in part, being able to have these conversations.

The award was presented by the previous winner, Tim Kroesbergen, and the secretary general of the Ministry Erik Gerritsen was host to the event, with Maarten den Braber as MC. The jury, consisting of Sanne (Niels’ wife) and the previous two winners, Annette Stekelenburg and Tim Kroesbergen, made their choice known from amongst the three nominees: Eva Westerhoff, Elianne Speksnijder and Geert-Jan den Hengst. All three nominees were presented by a video, as well as a conversation about their experiences.

Eva Westerhoff is a disability rights advocate & accessibility consultant who happens to be deaf. Next to her job at a bank, she does lots of volunteer work on diversity, inclusion & accessibility in information, communication & tech. She’s been knocking on doors in the Healthcare Ministry for over 20 years. Today she said that because of the political cycle, it seems you need to do everything again every four years or so, to keep awareness high enough.

Elianne Speksnijder is a professional fashion model, photographer and story teller. Lyme disease and epilepsy caused her to land in a wheelchair when she was 15. As she said today, an age which brings enough difficulties as it is. It took her a decade to accept that her wheels were a permanent part of her life. She’s 28 now, a woman with ambitions ‘on wheels’. When she was a teenager she sorely missed a role model (or rolling model, as the Dutch word ‘rolmodel’ can mean both). Now she is setting out to be that role model herself. She hopes for much more inclusivity in media, and challenges companies about it.

Geert-Jan den Hengst, is a 48 year old father of two adult children. He has MS and has been living the last decade or so in an environment that provides 24/7 care. His laptop is his core conduit to the rest of the world. Writing is a need for him. He blogs on his own blog, and writes for the local football team’s website, various media in his hometown and more. At the heart of his writing are everyday observations. He says he is “not a political animal, so I need to stay close to my everyday life in what I do”. Often those observations are examples of how life can be made impractical for someone in his position. He mentioned an early example that got him started: for the local football stadium all types of tickets could be bought online, except for …. tickets for wheel chair access. People with wheel chairs needed to come buy the tickets in person. The group least likely to be able to do that easily.

From all three nominees, I think the main takeaway is taking the time to share and listen to the actual stories of people. Especially when things get complicated or complex. Not news, there’s a reason I’ve been active in participatory narrative inquiry and sense making for a long time, but it bears repeating. Stories are our main way of ‘measurement’ in complex situations, to catch what’s going on for real, to spot the actual (not just the intended) consequences of our actions, structures and regulations, to see the edge cases, and to find the knobs to turn towards getting better results (and know what better actually is).

Jury chairman Tim Kroesbergen after reading the jury motivations for all three nominees, announced Eva Westerhoff as the new Shaking Tree Award winner.

'Last Lecture' Deluxe @shakingtree #fakkeldragers
The Shaking Tree Award statuette (photo by Henk-Jan Winkeldermaat, CC by-nc-sa)

Inside the Ministry a poem by Merel Morre is painted on the wall, that she wrote in honor of Niels ‘Shakingtree’.
A rough translation reads (anything unpoetic is all my doing)

outside

shake goals awake
jump past rules
dance joints wider
dream chances free

out of bounds
outside limitation
it grows
as it grows

tree high
dream high
where it lighter
but never stops

In the ministy’s central hall all the pillars show a face of someone with the words “I care”. That and the poem are promising signs of commitment to the actual stories of people. The Ministry still has 24 statuettes in stock for the Shaking Tree Award, so there’s a likelihood they will keep the annual award up as well. But as this year’s winner Eva Westhoff warned, every 4 years the politics changes, so it’s better to make sure.

20181112_173353
The faces in the Ministry with the text ‘I care’

This Tuesday 2 October sees the annual event of the Dutch Coalition for Humanitarian Innovation. The coalition consists of government entities, knowledge institutions, academia, businesses, and humanitarian organisations in the Netherlands. Together they aim to develop and scale new solutions to increase impact and reduce costs of humanitarian action.

I was asked to join this year’s jury for DCHI’s innovation award. There is a jury award and a public award. For the jury award 8 projects were shortlisted, from which the jury has now selected 3 finalists that were announced last Friday. The public award winner will be selected from the same short list.
At the annual event of DCHI this Tuesday the public award winner will be announced, followed by closing remarks by the Minister of development cooperation mrs Sigrid Kaag, who is very well experienced when it comes to international development. The jury award will be presented to the winner on October 11th at the Partos innovation festival.

The three finalists my colleagues and I in the jury selected are all very interesting, so I briefly want to list them here.

Optimus by the UN’s World Food Program and Tilburg University
Data analysis and mathematical modeling optimises supply and distribution also by taking into account locally available food and conditions. Optimisation means delivering the same nutritional value against lower efforts. It has been successfully used in Syria, Iraq, Yemen and Ethiopia. In Iraq it helped save a million USD per month, allowing the program to provide an additional 100.000 people in need with food packages. (link in Dutch)

Quotidian early warning solutions by Oxfam India
Flood prediction models in India are accurate, but still flooding causes many fatalities. The cause is often not being able to timely reach and warn everyone. Oxfam India came up with ways to integrate early warning systems with existing local infrastructure, and so create a low cost option for real time distribution of flood warnings.

Words of Relief / Translators without borders
Being able to provide key information to people in need depends on having that information in the right language. Information only saves lives if those who need it understand it. Translators without Borders creates glossaries which can be used for humanitarian response. Their Gamayun initiative wants to bring 20 underserved languages online by creating such glossaries and providing that as open data to all who can use it. They see it as a key tool for equality as well. In a slightly different setting I saw this work in practice, during the Syrian refugee wave in Germany, at a hackathon I attended such glossaries were used to build apps to help refugees navigate German bureaucracy and find the help they needed.

These three projects are very different, in terms of technology used, in the issues they address, and the way they involve the communities concerned, and all three highly fascinating.

For the UNDP in Serbia, I made an overview of existing studies into the impact of open data. I’ve done something similar for the Flemish government a few years ago, so I had a good list of studies to start from. I updated that first list with more recent publications, resulting in a list of 45 studies from the past 10 years. The UNDP also asked me to suggest a measurement framework. Here’s a summary overview of some of the things I formulated in the report. I’ll start with 10 things that make measuring impact hard, and in a later post zoom in on what makes measuring impact doable.

While it is tempting to ask for a ‘killer app’ or ‘the next tech giant’ as proof of impact of open data, establishing the socio-economic impact of open data cannot depend on that. Both because answering such a question is only possible with long term hindsight which doesn’t help make decisions in the here and now, as well as because it would ignore the diversity of types of impacts of varying sizes known to be possible with open data. Judging by the available studies and cases there are several issues that make any easy answers to the question of open data impact impossible.

1 Dealing with variety and aggregating small increments

There are different varieties of impact, in all shapes and sizes. If an individual stakeholder, such as a citizen, does a very small thing based on open data, like making a different decision on some day, how do we express that value? Can it be expressed at all? E.g. in the Netherlands the open data based rain radar is used daily by most cyclists, to see if they can get to the rail way station dry, better wait ten minutes, or rather take the car. The impact of a decision to cycle can mean lower individual costs (no car usage), personal health benefits, economic benefits (lower traffic congestion) environmental benefits (lower emissions) etc., but is nearly impossible to quantify meaningfully in itself as a single act. Only where such decisions are stimulated, e.g. by providing open data that allows much smarter, multi-modal, route planning, aggregate effects may become visible, such as reduction of traffic congestion hours in a year, general health benefits of the population, reduction of traffic fatalities, which can be much better expressed in a monetary value to the economy.

2 Spotting new entrants, and tracking SME’s

The existing research shows that previously inactive stakeholders, and small to medium sized enterprises are better positioned to create benefits with open data. Smaller absolute improvements are of bigger value to them relatively, compared to e.g. larger corporations. Such large corporations usually overcome data access barriers with their size and capital. To them open data may even mean creating new competitive vulnerabilities at the lower end of their markets. (As a result larger corporations are more likely to say they have no problem with paying for data, as that protects market incumbents with the price of data as a barrier to entry.) This also means that establishing impacts requires simultaneously mapping new emerging stakeholders and aggregating that range of smaller impacts, which both can be hard to do (see point 1).

3 Network effects are costly to track

The research shows the presence of network effects, meaning that the impact of open data is not contained or even mostly specific to the first order of re-use of that data. Causal effects as well as second and higher order forms of re-use regularly occur and quickly become, certainly in aggregate, much higher than the value of the original form of re-use. For instance the European Space Agency (ESA) commissioned my company for a study into the impact of open satellite data for ice breakers in the Gulf of Bothnia. The direct impact for ice breakers is saving costs on helicopters and fuel, as the satellite data makes determining where the ice is thinnest much easier. But the aggregate value of the consequences of that is much higher: it creates a much higher predictability of ships and the (food)products they carry arriving in Finnish harbours, which means lower stocks are needed to ensure supply of these goods. This reverberates across the entire supply chain, saving costs in logistics and allowing lower retail prices across Finland. When 
mapping such higher order and network effects, every step further down the chain of causality shows that while the bandwidth of value created increases, at the same time the certainty that open data is the primary contributing factor decreases. Such studies also are time consuming and costly. It is often unlikely and unrealistic to expect data holders to go through such lengths to establish impact. The mentioned ESA example, is part of a series of over 20 such case studies ESA commissioned over the course of 5 years, at considerable cost for instance.

4 Comparison needs context

Without context, of a specific domain or a specific issue, it is hard to asses benefits, and compare their associated costs, which is often the underlying question concerning the impact of open data: does it weigh up against the costs of open data efforts? Even though in general open data efforts shouldn’t be costly, how does some type of open data benefit compare to the costs and benefits of other actions? Such comparisons can be made in a specific context (e.g. comparing the cost and benefit of open data for route planning with other measures to fight traffic congestion, such as increasing the number of lanes on a motor way, or increasing the availability of public transport).

5 Open data maturity determines impact and type of measurement possible

Because open data provisioning is a prerequisite for it having any impact, the availability of data and the maturity of open data efforts determine not only how much impact can be expected, but also determine what can be measured (mature impact might be measured as impact on e.g. traffic congestion hours in a year, but early impact might be measured in how the number of re-users of a data set is still steadily growing year over year)

6 Demand side maturity determines impact and type of measurement possible

Whether open data creates much impact is not only dependent on the availability of open data and the maturity of the supply-side, even if it is as mentioned a prerequisite. Impact, judging by the existing research, is certain to emerge, but the size and timing of such impact depends on a wide range of other factors on the demand-side as well, including things as the skills and capabilities of stakeholders, time to market, location and timing. An idea for open data re-use that may find no traction in France because the initiators can’t bring it to fruition, or because the potential French demand is too low, may well find its way to success in Bulgaria or Spain, because local circumstances and markets differ. In the Serbian national open data readiness assessment performed by me for the World Bank and the UNDP in 2015 this is reflected in the various dimensions assessed, that cover both supply and demand, as well as general aspects of Serbian infrastructure and society.

7 We don’t understand how infrastructure creates impact

The notion of broad open data provision as public infrastructure (such as the UK, Netherlands, Denmark and Belgium are already doing, and Switzerland is starting to do) further underlines the difficulty of establishing the general impact of open data on e.g. growth. The point that infrastructure (such as roads, telecoms, electricity) is important to growth is broadly acknowledged, with the corresponding acceptance of that within policy making. This acceptance of quantity and quality of infrastructure increasing human and physical capital however does not mean that it is clear how much what type of infrastructure contributes at what time to economic production and growth. Public capital is often used as a proxy to ascertain the impact of infrastructure on growth. Consensus is that there is a positive elasticity, meaning that an increase in public capital results in an increase in GDP, averaging at around 0.08, but varying across studies and types of infrastructure. Assuming such positive elasticity extends to open data provision as infrastructure (and we have very good reasons to do so), it will result in GDP growth, but without a clear view overall as to how much.

8 E pur si muove

Most measurements concerning open data impact need to be understood as proxies. They are not measuring how open data is creating impact directly, but from measuring a certain movement it can be surmised that something is doing the moving. Where opening data can be assumed to be doing the moving, and where opening data was a deliberate effort to create such movement, impact can then be assessed. We may not be able to easily see it, but still it moves.

9 Motives often shape measurements

Apart from the difficulty of measuring impact and the effort involved in doing so, there is also the question of why such impact assessments are needed. Is an impact assessment needed to create support for ongoing open data efforts, or to make existing efforts sustainable? Is an impact measurement needed for comparison with specific costs for a specific data holder? Is it to be used for evaluation of open data policies in general? In other words, in whose perception should an impact measurement be meaningful?
The purpose of impact assessments for open data further determines and/or limits the way such assessments can be shaped.

10 Measurements get gamed, become targets

Finally, with any type of measurement, there needs to be awareness that those with a stake of interest into a measurement are likely to try and game the system. Especially so where measurements determine funding for further projects, or the continuation of an effort. This must lead to caution when determining indicators. Measurements easily become a target in themselves. For instance in the early days of national open data portals being launched worldwide, a simple metric often reported was the number of datasets a portal contained. This is an example of a ‘point’ measurement that can be easily gamed for instance by subdividing a dataset into several subsets. The first version of the national portal of a major EU member did precisely that and boasted several hundred thousand data sets at launch, which were mostly small subsets of a bigger whole. It briefly made for good headlines, but did not make for impact.

In a second part I will take a closer look at what these 10 points mean for designing a measurement framework to track open data impact.

The US government is looking at whether to start asking money again for providing satellite imagery and data from Landsat satellites, according to an article in Nature.

Officials at the Department of the Interior, which oversees the USGS, have asked a federal advisory committee to explore how putting a price on Landsat data might affect scientists and other users; the panel’s analysis is due later this year. And the USDA is contemplating a plan to institute fees for its data as early as 2019.

To “explore how putting a price on Landsat data might affect” the users of the data, will result in predictable answers, I feel.

  • Public digital government held data, such as Landsat imagery, is both non-rivalrous and non-exclusionary.
  • The initial production costs of such data may be very high, and surely is in the case of satellite data as it involves space launches. Yet these costs are made in the execution of a public and mandated task, and as such are sunk costs. These costs are not made so others can re-use the data, but made anyway for an internal task (such as national security in this case).
  • The copying costs and distribution costs of additional copies of such digital data is marginal, tending to zero
  • Government held data usually, and certainly in the case of satellite data, constitute a (near) monopoly, with no easily available alternatives. As a consequence price elasticity is above 1: when the price of such data is reduced, the demand for it will rise non-lineary. The inverse is also true: setting a price for government data that currently is free will not mean all current users will pay, it will mean a disproportionate part of current usage will simply evaporate, and the usage will be much less both in terms of numbers of users as well as of volume of usage per user.
  • Data sales from one public entity to another publicly funded one, such as in this case academic institutions, are always a net loss to the public sector, due to administration costs, transaction costs and enforcement costs. It moves money from one pocket to another of the same outfit, but that transfer costs money itself.
  • The (socio-economic) value of re-use of such data is always higher than the possible revenue of selling that data. That value will also accrue to the public sector in the form of additional tax revenue. Loss of revenue from data sales will always over time become smaller than that. Free provision or at most at marginal costs (the true incremental cost of providing the data to one single additional user) is economically the only logical path.
  • Additionally the value of data re-use is not limited to the first order of re-use (in this case e.g. academic research it enables), but knows “downstream” higher order and network effects. E.g. the value that such academic research results create in society, in this case for instance in agriculture, public health and climatic impact mitigation. Also “upstream” value is derived from re-use, e.g. in the form of data quality improvement.

This precisely was why the data was made free in 2008 in the first place:

Since the USGS made the data freely available, the rate at which users download it has jumped 100-fold. The images have enabled groundbreaking studies of changes in forests, surface water, and cities, among other topics. Searching Google Scholar for “Landsat” turns up nearly 100,000 papers published since 2008.

That 100-fold jump in usage? That’s the price elasticity being higher than 1, I mentioned. It is a regularly occurring pattern where fees for data are dropped, whether it concerns statistics, meteo, hydrological, cadastral, business register or indeed satellite data.

The economic benefit of the free Landsat data was estimated by the USGS in 2013 at $2 billion per year, while the programme costs about $80 million per year. That’s an ROI factor for US Government of 25. If the total combined tax burden (payroll, sales/VAT, income, profit, dividend etc) on that economic benefit would only be as low as 4% it still means it’s no loss to the US government.

It’s not surprising then, when previously in 2012 a committee was asked to look into reinstating fees for Landsat data, it concluded

“Landsat benefits far outweigh the cost”. Charging money for the satellite data would waste money, stifle science and innovation, and hamper the government’s ability to monitor national security, the panel added. “It is in the U.S. national interest to fund and distribute Landsat data to the public without cost now and in the future,”

European satellite data open by design

In contrast the European Space Agency’s Copernicus program which is a multiyear effort to launch a range of Sentinel satellites for earth observation, is designed to provide free and open data. In fact my company, together with EARSC, in the past 2 years and in the coming 3 years will document over 25 cases establishing the socio-economic impact of the usage of this data, to show both primary and network effects, such as for instance for ice breakers in Finnish waters, Swedish forestry management, Danish precision farming and Dutch gas mains preventative maintenance and infrastructure subsidence.

(Nature article found via Tuula Packalen)