Category Archives: Open Data

Financial Transparency in the Netherlands, an overview

Recently I participated in a session of the Dutch permanent parliamentary commission on national spending, discussing open financial data. A good reason to give a quick update on open government spending data in the Netherlands.

Current status of open spending
Let’s give you a general overview of open spending in the Netherlands first. As you can see in the Open Data Census, open spending data is the single biggest missing chunk of data in the Netherlands. The national budget is available as open data, since 2012, thanks to the work of the Dutch national audit office, but only on an aggregated level. The Ministry for Foreign Affairs is publishing transaction level data on international aid since 2012 as part of IATI, and is the only Dutch public sector body doing this. On a local level some aggregated spending data is available through the Open State Foundation‘s project openspending.nl. In the past months I have gathered local spending data from 25 local councils, and provided it to this project to make comparisons across local governments possible. In a current project with the Province of North-Holland, we are, in collaboration with 10 local governments, aiming to open up the spending data of 50+ local councils. There is no requirement, unlike in the UK, for government bodies to publish open spending data.

Old Parliament
The session took place in the old plenary meeting room of the Parliament

National Audit Authority: Forwards with open spending!
President of the National Audit Authority Saskia Stuiveling had the clearest message during the parliamentary committee meeting, in terms of general outlook as well as leading by example. Even for the audit authority it is often hard to get the right data to properly audit government spending. Opening up spending data by default will help them to concentrate on those parts of public policy where it matters most, e.g. health care spending. To lead by example the audit authority has opened up their own spending data this spring. They also published a ‘Trend Report Open Data‘ tracking the open data efforts of all Ministries, and urging them to do more. Opening up data is becoming a standard advice given in all their audit reports. In other words they are building up pressure for Ministries to do more. (disclosure: I worked with the audit authority on the trend report open data)

Foreign Affairs: Open spending is useful instrument
The Ministry for Foreign Affairs presented itself as a proponent of more financial transparency. Having started publishing open spending data on international development in 2012, they will be launching a (Tableau) based viewer for that data on June 11th, which includes the possibility to drill down to project level information and can link to external sources such as project descriptions published by NGO’s. A viewer like this serves as a replacement for yearly paper based reporting, makes a step towards visualizing impact and not just spending, as well as is a means to motivate more NGO’s towards bigger spending transparency.

Finance Ministry: following Audit Authority’s lead
The Finance Ministry until now has done little towards open spending, but during the session in the Parliament they showed how the work done by the audit authority mentioned above has prodded them into action as well. Triggered by the open data trend report last March, they have now opened up aggregated spending for the first time (update from Rense Posthumus in the comments: data is located at opendata.rijksbegroting.nl). Also the Finance Ministry announced that subsidies data and basic financial data of independent government agencies is available in a viewer in sneak preview, though no URL was given yet. It wasn’t indicated when this would be made publicly available. (UPDATE: see comment by Rense Posthumus) The plan to publish departmental spending for all ministries by 2016 was announced, but made dependent on ‘creating a standard reporting method’ first. That met with resistance in the audience: if the data is good enough for the Finance Ministry to work with, why isn’t it good enough to publish? That argument did seem to resonate with the Ministry director present.

Interior Affairs: very disappointing
A very disappointing contribution was made by the Ministry for the Interior’s deputy director-general. This Ministry is nominally responsible for the open government and open data efforts of the government, as well as in the lead to reform the Freedom of Information Act in light of the new EU Directive on the re-use of public sector information, but in this session showed a shocking lack of vision and no will to act. In 20 minutes nothing was said about open government at all, leaving the attending Members of Parliament confused. Even the actions the Ministry hás taken, such as the launch of the national data portal in 2011, and joining the Open Government Partnership (albeit with an Action Plan that adroitly avoids formulating action), weren’t mentioned. From this presentation one can only conclude that nothing much can be expected from this Ministry in the near future. This means other public sector bodies are left largely to their own devices, which is a shame as it means lots of time will be lost clearing up confusion and raising the general level of knowledge on how to do open government data well. The Ministry for the Interior, being in charge of the open government dossier, is the only one inside government who could claim a much needed role of ‘lighthouse’ and beacon for established good practice, but they’re not on the ball, nor seem to aim to be.

OECD Regional Well-Being Index

At Re:Publica in a session on data visualization to make sense of globalization, the release of a very cool dataviz project was announced for next week: The OECD Regional Well-Being Index. ‘Truth and beauty operator’ Moritz Stefaner, who contributed to the visual aspects, made this announcement during the session and gave a sneak preview.

It is a follow-up of the OECD Better Life Index (also very cool), and a new incarnation of the statistical regional explorer.

What it allows you to do is explore regional data, on the basis of what you deem relevant, and then find out which regions in other OECD countries have similar profiles. This is important, as until now OECD data was mostly presented on national level, but the more profound differences are usually found within a country, or when comparing regions, not countries.

If you do such a comparison for Berlin, as shown in the pictures, you find out why Peter Rukavina likes Berlin so much: it is statistically similar to his home Prince Edward Island, just more urban and with a wider variety of things on offer.

Re Publica 2014 Berlin
Berlin, with Prince Edward Island mentioned as similar region

Re Publica 2014 Berlin
PEI, statistically similar to Berlin

The existing OECD Regional Well-Being Index is already a great and beautiful project. It moves away from ranking countries, as that has no real meaning (in the sense of scope of interventions or policy consequences). You can create your own set of important indicators, and your choice as well as those of other visitors is used again as data to improve the visualization of the project itself. The top layer of the index is playful, and doesn’t throw all of the statistics in your face at the beginning. If you want you can dig much deeper and get much richer detailed numbers.

For more OECD data visualizatons see their Data Lab. Also check out the dataviz portfolio of Moritz Stefaner, who created the key elements of the OECD visualizations.

Update on Local Spending Data FOIA Requests

Four weeks ago I asked all 25 municipalities in my Province for their spending data, as reported in so called IV3 files to the Dutch national statistics office. As all municipalities use the same format, this makes it possible to compare spending and budgets across communities, for instance as is done at openspending.nl

Because I asked 25 government bodies the same question at the same time, it also makes for interesting comparisons on how each of them deals with requests for information, and how that compares to the legal obligations in place in the Freedom of Information Act (WOB, FOIA).

Today is day 28, and that is the end of the initial period, stated in the law, government bodies have to respond to requests. So how did the 25 municipalities do?

As of today I have received 15 out of 25 requested data sets (60%). The shortest response time was 4 days, and the last week, as the deadline was approaching, saw most activity.
Just over half (14 out of 25, 56%) turn out to only accept FOIA requests on paper, and not through e-mail. This is an mostly unnecessary obstructive effort to reduce the number of citizen requests received, and especially to prevent overlooking requests and thus penalties.

Five municipalities have announced postponing their answer with (the legally defined) additional 4 weeks. Four have a few days of the first 4 weeks remaining (the days used for me responding on paper where the original e-mail wasn’t accepted). One municipality is now officially late.

All in all a pretty good result thusfar in my opinion.

An Exercise In Freedom of Information: Local Spending Data

I have approached all 25 municipalities in my province with a freedom of information (foia) request for local spending data. This is a little side project that serves two purposes:

  • Bringing together spending data for the entire region
  • Establishing the FOIA readiness and processes of municipalities


  • Where does my money go
    Where does my money go? The first financial transparency open data project.

    OpenSpending: getting local spending data
    The main trigger for this is the OpenSpending project which exists as a global project, but also has a separate national Dutch clone at openspending.nl by the Open State Foundation. All Dutch municipalities report their spending and revenue in a fixed format, called IV3, to the Dutch Statistics Office CBS on a quarterly basis. If this data would be available for all municipalities, it would enable great comparison opportunities. Right now, only the data for the city of Amsterdam is available.

    So last October I did a FOIA request in my home town Enschede, to get the spending data, and promptly received it within a week. That data is now findable through the Enschede city data portal. Now that openspending.nl announced it is ready for more data, I decided to try and get some for my entire region. Last Monday I sent out 24 FOIA requests to municipalities in my province for their IV3 files.

    FOIA readiness and process assessment
    Now that I have send out 24 identical FOIA requests for spending data, and have the original one as benchmark, this provides good opportunity to compare the way municipalities deal with FOIA requests. So that provides the second purpose of this exercise.

    I will track the progress of my 24 FOIA requests, and document the results. Thusfar 5 out of 24 have let me know their digital communication path is closed for FOIA, so I have posted letters to those. One (1) municipality quickly confirmed my request, properly recognizing it as a FOIA request and stating it had been forwarded to the right person internally, a handful of others automatically confirmed reception of my e-mail.

    Going To 3D Camp Limerick: Connecting Open Data and Making

    Schermafbeelding 2014-03-11 om 19.00.36

    Elmine and I will be speaking at 3DCamp in Limerick, Ireland in May. It’s the 7th annual barcamp dedicated to a “broad range of technologies that change the way we interact with computers”. At the invitation of Gabriela Avram who is with the Limerick University’s Department of Computer Science and Information Systems, and the Interaction Design Centre, we’ll be visiting the event, and might also give a workshop the day before at the University of Limerick.

    I’ve set myself the task to bridge open data and making in a meaningful way.

    I pitched it like this:
    What we make should matter, help solve and mean something for you or for others, provide a perspective for action and new affordances. In this talk, building on my experiences in both worlds, I will explore how Open Data and Making can mutually reinforce each other.

  • What data do you need, in order to decide what to make to solve a local issue?
  • How do you get that data open?
  • What can you make to gather data yourself?
  • What can you make that let’s you act on what data tells you?
  • Can we make tangible things that let us understand data better?
  • Taking practical examples from across Europe as inspiration, let’s see what we can come up with!

    I also proposed a hands on workshop on this topic. There we could hands on explore the entire process of:

  • pick a local issue,
  • map out stakeholders and data sets,
  • design actions to get or gather data,
  • ideate what type of things would provide stakeholders with perspectives for action / affordances needed,
  • decided on what to make from ideas generated, and how
  • I think this is a great way to also prepare for our own MidSummer Unconference “Making Stuff That Matters”, when Elmine and I will welcome a wide range of peers to our home to explore Making.

    You can follow 3D Camp on Twitter, for the latest on other sessions.

    Open Data Program With 10 Local Govs

    Today, in the presence of some 60 local government representatives, saw the kick-off of a project I’m proud of to see launched. Nine municipalities in the Province North-Holland are embarking with us on a 9 month program to connect locally relevant policy themes with new stakeholders and publishing open data. Open data as a policy intervention to provide citizens and organisations with new affordances and new pathways for action.

    PNH Slimmer has started
    The beautiful ‘open data manifesto’ the municipalities and province signed today. Made by my colleague Frank (laser cutter to the rescue!) and our artist in residence Ate Hes (hand painted data visualizations, and the overall concept)

    The connection to local policy themes is intended as a source of intrinsic motivation for data holders. Economic potential or transparency impacts often see their benefits impact elsewhere, not with the data holder, and may not suffice to get the open data ball rolling pro-actively.
    The Province acts as an umbrella for the effort, providing a broader context and allowing local governments to build on each other.

    In the past 2 months we talked to 16 local governments to find local themes, and see whether the time needed can be allocated internally. Nine municipalities signed up today, a few more may follow in the coming days.

    PNH Slimmer has started
    A quick and stimulating game to come up with applications for a combination of 3 random data sets

    We will be working with these municipalities both as a group, as well as ‘on the ground’ to help make it happen in practice. Working on raising awareness, spotting opportunities. Working on creating a steady process for regular data publishing. Finding and mobilizing people to start using the data.

    Because just publishing data is not a result to us. Getting to the point where the data is being used to strengthen local communities is the result we are after.

    It is an expedition, not a prescribed route. So also to me and my The Green Land colleagues it is an adventure with unknown outcomes. But it is an adventure we are looking forward to take on, as we’re confident there are plenty of building blocks and experiences already available to make this happen.

    It takes a village…..to see open data impact

    An interesting paper by Ben Worthy has been published, looking at the impact of open spending data in the UK in terms of transparency and accountability. The abstract says the impact is currently very limited “as it lacks the narrative or accountability instruments to fully bring such effects. Nor has it created a new stream of information to underpin citizen choice, though new innovations offer this possibility. The evidence points to third party innovations as the key. They can contextualise and ‘localise’ information as a first step in more effective accountability.

    The superficially simple and neutral reforms conceal complex political dynamics. The very design lends itself to certain framing effects, further compounded by assumptions and blurred concepts and a lack of accountability instruments to resolve problems raised by the data.

    Definitely makes sense to me. Context, narratives, and additional tools with which new stakeholders can be reached and brought into the discussion, are needed. Just as much as we constantly need to try and avoid making assumptions when publishing data, as it will create a bias towards what type of usage will be likely to occur, framing it, or setting boundaries on the evolutionary space available as it were. Whether it is open data or big data, the effects are similar, though some such as context are even more pronounced in big data.

    It is why, when working on open data with local governments, and positioning it as a policy instrument, I spend energy on providing context, seeking out new or unusual stakeholders to stimulate them to take a look at the data, and to create lots of new conversations between them and the data holders. Aiming to create an ecosystem. In a sense, “it takes a village” to create the impact we are aiming for.

    International Open Data Panel at Swedish Internet Days

    On November 25th an international Open Data panel took place at the Swedish Internet Days. On the kind invitation of Vinnova, Sweden’s innovation agency, I took part in the panel, to present on the current status and coming steps in Open Data in the Netherlands.

    I was lucky to join Malte Beyer-Katzenberger (EC), Richard Stirling (ODI), Cathrine Lippert (Digitisation Agency of Denmark), and Daniel Dietrich (OKFN Germany), who also gave their views on Open Data in their respective countries, or the EC’s plans in the case of Malte.
    It was great to share a panel with them, as well as to have the opportunity to talk to each other and share our insights, new experiences and examples. Face to face time is scarce, and I thoroughly enjoyed spending time in Stockholm.

    Below you will find the videos of the various panel contributions, as well as my own slides.

    Malte Beyer-Katzenberger on the open data efforts by the EC

    Cathrine Lippert on open data and basic registries in Denmark

    Richard Stirling on the ODI’s work in the UK

    Ton Zijlstra on open data in the Netherlands

    Daniel Dietrich on open data in Germany

    After the panel I had the opportunity to catch up with Peter Krantz over lunch. Peter’s a long time advocate for open data in Sweden, and although we had interacted often on-line in the past years, this was the first time we met.

    Digital Disruption Key-Note

    Yesterday and today I am at the ARAS Community Event Europe in Frankfurt, Germany. The conference brings together people around PLM (product lifecycle management). I was asked to provide the opening keynote. Digital disruption and the new (or rather often not so new) methods we have available to deal with that were the topic of my talk. Starting with Steve Denning’s recent observation that we’re in the Golden Age of Management now as we are setting the scene for what management looks like in the networked age, I talked about the unintended impact of internet and mobile communications, that make a range of existing management methods obsolete if not dangerous. Simultaneously I went into the different emerging instruments and methods that are emerging in response.

    Find the slides below.

    Theoretically Open Post Codes Not So Much In Practice

    Currently the 2013 Open Data Census of the OKFN, for which I am the lead editor, is taking place. It tracks 10 data sets in each country. For the Netherlands questions were raised by a.o. Andrew Stott on the Dutch postcodes which are shown as fully open. Specifically it seemed there was no download link. So, as an experiment, I aimed to get my hands on the Dutch postcodes. Follow me down the rabbit hole.

    Dutch postcodes, after a court verdict, became open in February 2012, something celebrated by the responsible Ministry for Infrastructure and Environment (the link to the press release in that article is dead). So, where’s that data?

    The postcodes, first of all, are not a separate data set but part of the much larger BAG data set which holds all addresses and all buildings for the Netherlands, with their geospatial references.

    National Data Portal
    First port of call is the Dutch National Data Portal, which indeed holds a page for the BAG data set. It points to three data files, and lists the license as Public Domain. Nice, we’re done! Except that none of the links to the data files work. Further down the hole.

    Cadastral Office, the data holder
    Next stop, is the Cadastral Office, as they are the data holder, and the links in the national data portal all point there. The Cadastre has a page about the BAG listing a number of ‘products’. No links to the actual data, but the descriptions make clear that to get access to the data you need to register first, then apply for a subscription on the data, which is not free if you are not in the public sector. No indication whatsoever that any data from the BAG is available as Open Data. However they do make mention of PDOK (acronym for “public service on the map”) which offers a WFS and WMS service as well as a viewer for the data. Further down the hole.

    PDOK
    PDOK is the next incarnation of the INSPIRE based national geodata portal. Looking for the BAG there does not provide any useful links: It does expose a ‘temporary’ WMS and WFS service. WMS provides map images, WFS provides data, but its access is restricted and fees apply the XML feature description says (none of that in human readable text on the site though). It probably would not give us the desired postcodes anyway. The links it provides are to nationaalgeoregister.nl, PDOK’s predecessor. Further down the hole.

    National Geo Register
    At the National Geo Register, I try another search for BAG, which yields nothing new. It appears it conflates my BAG search with ‘baggeren’ (dredging in English) as it starts the same. Then I try ‘adressen’ (addresses) with more success, and one of the results reads “INSPIRE Download Service Addresses” (incidently a stripped down version of that search result also comes up at PDOK under addresses.) Following that link it finally gets interesting, as the next page provides both a download link, an e-mail address at the Cadastral Office for enquiries, and some information on re-use rights. It says the last metadata edit was last August.

    The usage conditions are listed in the National Geo Register as unrestricted, but access is listed as restricted, referencing a license called ‘GEO Gedeeld’. The National Data Portal (all the way up the rabbit hole) promised us Public Domain, so what is this GEO Gedeeld license? The link to the license is dead however. Looking for the same license title at the Cadastral Office site, yields a PDF that says attribution is mandatory, and resharing the data set is forbidden. This turns out to be a ‘roll-your-own’ license by the geodata sector, adding restrictions on top of the Creative Commons framework, including confusing pseudo-CC icons. It is however entirely unclear if this is the license that is attached to the data mentioned in the National Geo Register.

    Getting to the data, sort of
    As said the National Geo Register lists a download link, and if you open that XML file, in the ‘subtitle’ field it says this is the actual full BAG data. The same XML file also provides a link to a description in XML, that points to a Public Domain license again for the acccess restrictions on the file. So the National Geo Register page says a restricted license applies, but the XML metadata that comes with it specifies Public Domain as does the national data protal. Finally it also provides a link to the actual data dump.

    The data dump is a 1.4 GB zip file, that contains 7 other zip files, yielding over 30GB of data when unpacked, split up in 21MB xml files. The files contain however no reference to a license. This is the full BAG data set, containing all addresses and buildings for all of the Netherlands. From this dataset you need to combine several subsets to get to a postcode list: The ‘NUM’ files give you address index numbers, its georeferences, the house number and the postcode, and a number that corresponds with a street. The ‘OPR’ files give you the corresponding street name, and the number of the place it is in. The ‘WPL’ files give you the place name, and through a separate table also the municipality it is in. (Do note that is does not include postcodes that are not connected to geolocations, such as PO Boxes. The primary purpose of this database is not postcodes but addresses and buildings.)

    Open or not?
    So how open is this much welcomed open post code data? It seems the data holder, the Cadastral Office, purposefully obfuscates the existence of the data dump, making no direct reference to it at all, and offering their paid for services instead. There is however a full machine readable data dump that is also directly provided by the Cadastral Office. Whether it is openly licensed is not entirely clear, as the National Geo Register states a restrictive license on the webpage, but provides a Public Domain dedication in the XML metadata, and there is the Public Domain dedication in the National Data Portal.

    Make it Open for real!
    To make this data Open in practice, and not just in theory the National Data Portal as well as the Cadastral Office should reference the existing data dump directly, as well as provide the Public Domain license info with the data dump. If no one can find your data, then it is not open. Next to technical openness (machine readable, online) and legal openness (public domain, or attribution) also social openness is needed (easily findable, contact info readily available).