Last year I spent a large amount of time participating in the study that provided advice on which government data sets to include in the mandatory list that is part of the Open Data Directive.

The Open Data Directive, which should have been transposed by EU Member States into national law by last July, but still mostly isn’t, provides a role to the EC to maintain a list of ‘high value data sets’ that all EU countries must make freely available for re-use through APIs and bulk download. This is the first time that it becomes mandatory to pro-actively publish certain data as open government data. Until now, there were mandatory ways to provide open data upon request, but the pro-active publication of such open data has always been voluntary (with various countries making a wide variety of voluntary efforts btw). Also the availability of government data builds on the national freedom of information framework, so the actual availability of a certain data set depends on different legal considerations in different places. The high value data list is the first pan-EU legal requirement that is equal in all EU Member States.

I was part of a team that provided a study into which data sets should appear on that high value data list. The first iteration of this list (to be extended and amended periodically) by the EC covers six thematic areas: geographic data, statistics, mobility data, company information, earth observation and environment, and meteorology. I was responsible for the sections on earth observation and environment, and meteorology, and I’m eager to see how it has been translated into the implementation act as for both those thematic areas it would mean a very significant jump in open data availability if the study results get adopted. We submitted our final report by September 2020, and in the year since then we’re all waiting to see how the implementation act for the high value data will turn out. Our study only is a part in that, as it is itself an input for the EC’s impact assessment for different choices and options, which in turn forms the basis for a negotiation process that includes all Member States.

Originally the implementation act was expected to be published together with other EC proposals, such as the Data Governance Act last December. This as the EU High Value Data list is part of a wider newly emerging EU legal framework on digitisation and data. But nothing much happened until now. First the expectation was Q1, then by the summer, then shortly after the summer, and now the latest I hear from within the EC is ‘hopefully by the end of the year’.

It all depends on poltical will at this stage it seems, to move the dossier forward. The obstacle to getting the implementing act done apparently is what to do with company data (and ultimate beneficial ownership data). Opening up company registers has clear socio-economic benefits, outweighing the costs of opening them up. There are privacy aspects to consider, which can be dealt with well enough I think, but were not part of our study as it only considered socio-economic impacts and expected transition costs, and the demarcation between the Open Data Directive and the GDPR was placed outside our scope.

There apparently is significant political pressure to limit the openness of such company registers. There must be similarly significant political pressure to move to more openness, or the discussion would already have been resolved. It sounds to me that the Netherlands is one of those politically blocking progress towards more openness. Even before our study commenced I heard rumours that certain wealthy families had the ear of the Dutch prime minister to prevent general free access to this data, and that the pm seemed to agree up to the point of being willing to risk infringement proceedings for not transposing the Open Data Directive completely. As it stands the transposition into national law of the Open Data Directive hasn’t happened mostly, and the implementing act for high value data hasn’t been proposed at all yet.

Access Info, the Madrid based European NGO promoting the right to access to information, has in June requested documents concerning the high value data list from the EC, including the study report. The report hasn’t been published by the EC themselves yet because it needs to be published together with the EC’s impact assessment for which the study is an input, and alongside the implementation act itself.
Access Info has received documents, and has published the 400+ page study report (PDF) that was submitted a year ago, alongside a critical take on the company register data issue.

I am pleased the study is now out there informally at least, so I can point people to sections of it in my current work discussions on how to accomodate the new EU legal framework w.r.t. data, of which the high value open data is a part. Previously there were only publicly available slides from the last workshop that was part of the study, which by neccessity held only general information about the study results. Next to company registers, which I am assuming is the roadblock, there is much in the study that also is of importance and now equally suffering under the delays. I hope the formal publication of the report will follow soon. The publication of the impementing act is a key step for European open data given its first ever EU-wide mandates.

Virk Data Dag
A 2014 workshop, Virk Data Dag, at the Danish Business Authority discussing use cases for the open Danish company register, where I presented and participated.

Finally, a declaration of interests is in order for this posting I think:

  • My company was part of the consortium that did the mentioned study. I led the efforts on earth observation and environmental data, and on meteorological data.
  • In current work for my company, the implementation act for high value data, and other recent EC legal proposals are of importance, as I am helping translate their impact and potential to Dutch national (open) data infrastructure and facilitating data re-use for public issues.
  • I am a voluntary board member of the NGO Open State Foundation. OSF advocates full openness of company registers, and co-signed the critical take Access Info published. The board has no influence on day to day actions, which are the responsibility of the NGO’s director and team.
  • I am personally in favor of opening up company registers as open data.
    • I think that privacy issues can be readily addressed (something that is directly relevant to me as a sole trader business, as co-owner of an incorporated business, and as a board member of an association for which my home address is currently visible in the company register)
    • I think that being visible as a business owner or decision maker is part of the social contract I entered into in exchange for being personally shielded from the business risks I am exposed to. Society is partially shielding me from those risks, as it allows social benefits to emerge (such as creating an income for our team), but in turn people need to be able to see who they’re dealing with.
    • I think there is never a business case where charging fees for access to a monopolistic government database such as company registries makes sense. Such fees merely limit access to those able to afford it, causing unequality of access, and power and information assymmetries. Data collection for public tasks is a sunk cost by definition, access fees are always less over time than the additional tax revenue and social value resulting from re-use of freely available data. The only relevant financial aspect to address is that provision costs accrue with the dataholder and benefits with the treasury, which general budget financing is the remedy for
    • I think that already open company registers in Europe and elsewhere provide ample evidence that many formulated fears w.r.t. such openness don’t become reality.

Since the start of this year I am actively tracking the suite of new European laws being proposed on digitisation and data. Together they are the expression into law of the geopolitical position the EU is taking on everything digital and data, and all the proposed laws follow the same logic and reasoning. Taken together they shape how Europe wants to use the potential and benefits of digitisation and data use, including specifically for a range of societal challenges, while defending and strengthening citizen rights. Of course other EU legal initiatives in parallel sometimes point in different directions (e.g. EU copyright regulations leading to upload filters, and the attempts at backdooring end-to-end encryption in messaging apps for mass surveillance), but that is precisely why to me this suite of regulations stands out. Where other legal initiatives often seem to stand on their own, and bear the marks of lobbying and singular industry interests, this group of measures all build on the same logic and read internally consistent as well as an expression of an actual vision.

My work is to help translate the proposed legal framework to how it will impact and provide opportunity to large Dutch government data holders and policy departments, and to build connections and networks between all kinds of stakeholders around relevant societal issues and related use cases. This to shape the transition from the data provision oriented INSPIRE program (sharing and harmonising geo-data across the EU), to a use needs and benefits oriented approach (reasoning from a societal issue to solve towards with a network of relevant parties towards the data that can provide agency for reaching a solution). My work follows directly from the research I did last year to establish a list of EU wide high value data sets to be opened, where I dived deeply into all government data and its governance concerning earth observation, environment and meteorology, while other team members did the same for geo-data, statistics, company registers, and mobility.

All the elements in the proposed legal framework will be decided upon in the coming year or so, and enter into force probably after a 2 year grace period. So by 2025 this should be in place. In the meantime many organisations, as well as public funding, will focus on already implementing elements of it even while nothing is mandatory yet. As with the GDPR, the legal framework once in place will also be an export mechanism of the notions and values expressed in it to the rest of the world. This as compliance is tied to EU market access and having EU citizens as clients wherever they are.

One element of the framework is already in place, the GDPR. The newly proposed elements mimic the fine structures of the GDPR for non-compliance.
The new elements take the EU Digital Compass and EU Digital Rights and Principles for which a public consultation is now open until 2 September as a starting point.

The new proposed laws are:

Digital Markets Act (download), which applies to all dominant market parties, in terms of platform providers as well as physical network providers, that de facto are gatekeepers to access by both citizens and market entities. It aims for a digital unified market, and sets requirements for interoperability, ‘service neutrality’ of platforms, and to prevent lock-in. Proposed in November 2020.

Digital Services Act (download), applies to both gatekeepers (see previous point) and other digital service providers that act as intermediaries. Aims for a level playing field and diversity of service providers, protection of citizen rights, and requires transparency and accountability mechanisms. Proposed in November 2020.

AI Regulatory Proposal (download), does not regulate AI technology, but the EU market access of AI applications and usage. Market access is based on an assessment of risk to citizen rights and to safety (think of use in vehicles etc). It’s a CE mark for AI. It periodically updates a list of technologies considered within scope, and a list of areas that count as high risk. With increasing risk more stringent requirements on transparency, accountability and explainability are set. Creates GDPR style national and European authorities for complaints and enforcement. Responsibilities are given to the producer of an application, distributors as well as users of such an application. It’s the world’s first attempt of regulating AI and I think it is rather elegant in tying market access to citizen rights. Proposed in April 2021.

Data Governance Act (download), makes government held data that isn’t available under open data regulations available for use (but not for sharing), introduces the European dataspace (created from multiple sectoral data spaces), mandates EU wide interoperable infrastructure around which data governance and standardisation practices are positioned, and coins the concept of data altruism (meaning you can securely share your personal data or company confidential data for specific temporary use cases). This law aims at making more data available for usage, if not for (public) sharing. Proposed November 2020.

Data Act, currently open for public consultation until 2 September 2021. Will introduce rules around the possibilities the Data Governance Act creates, will set conditions and requirements for B2B cross-border and cross-sectoral data sharing, for B2G data sharing in the context of societal challenges, and will set transparency and accountability requirements for them. To be proposed towards the end of 2021.

Open Data Directive, which sets the conditions and requirements for open government data (which build on the national access to information regulations in the member states, hence the Data Governance Act as well which does not build on national access regimes). The Open Data Directive was proposed in 2018 and decided in 2019, as the new iteration of the preceding Public Sector Information directives. It should have been transposed into national law by 1 July 2021, but not all MS have done so (in fact the Netherlands has just recently started the work). An important element in this Directive is EU High Value Data list, which will make publication of open data through APIs and machine readable bulk download mandatory for all EU member states for the data listed. As mentioned above, last year I was part of the research team that did the impact assessments and proposed the policy options for that list (I led the research for earth observation, environment and meteorology). The implementation act for the EU High Value Data list will be published in September, and I expect it to e.g. add an open data requirement to most of the INSPIRE themes.

Most of the elements in this list are proposed as Acts, meaning they will have power of law across the EU as soon as they are agreed between the European Parliament, the EU council of heads of government and the European Commission and don’t require transposition into national law first. Also of note is that currently ongoing revisions and evaluations of connected EU directives (INSPIRE, ITS etc.) are being shaped along the lines of the Acts mentioned above. This means that more specific data oriented regulations closer to specific policy domains are already being changed in this direction. Similarly policy proposals such as the European Green Deal are very clearly building on the EU digital and data strategies to achieving and monitoring those policy ambitions. All in all it will be a very interesting few years in which this legal framework develops and gets applied, as it is a new fundamental wave of changes after the role the initial PSI Directive and INSPIRE directive had 15 to 20 years ago, with a much wider scope and much more at stake.


The geopolitics of digitisation and data. Image ‘Risk Board Game’ by Rob Bertholf, license CC BY

De Gemeente Amsterdam wil een meldingsplicht voor sensoren in de publieke ruimte. Iedere organisatie die sensoren in de buitenruimte plaatst zou vanaf het najaar moeten melden waar die sensoren staan. Dit is nuttig om meerdere redenen. Allereerst omwille van transparantie en om de discussie over nut en noodzaak van al die sensoren om ons heen diepgaand te kunnen voeren. Of om te zien welke gegevens die nu door private organisaties worden verzameld, eventueel ook voor een gedeeld publiek belang kunnen worden gebruikt.

Amsterdam gaf eerder al een voorbeeld dat navolging verdient met de start van een algoritme-register, en dit sensorenregister lijkt me een uitstekende aanvulling.

(Defect) reclamebord op Utrecht Centraal Station dat ik in 2018 fotografeerde, met een ondoordachte camera in de publieke ruimte om aandacht voor de advertentie te meten. Burgerlijk verzet plakte de camera af.

Today we had national parliamentary elections. Just before lunch E and I went to vote. As usual voting was uneventful, this time with added face masks and hand sanitizer.


Our neighbourhood polling station

This time around, to enable people to avoid lines, our city published live data on how busy polling stations are. Allowing you to choose one without too many people (you can pick any polling station in the municipality you reside in). Based on public data streams, and on Open Street Map. Well done.


amersfoort.stemdrukte.nl showing current activity at polling stations

The Open State Foundation (I’m a board member) published their current map of polling stations again at ‘Where’s my polling station‘, as they did for the national elections 4 years ago and the local elections since then. Their map is also Open Street Map based. OSF gets their data from each single municipality and republishes it as a national dataset.


Waarismijnstemlokaal.nl, where is my polling station, with info on each polling station in the Netherlands

Good to see how open data is used to strengthen the democratic process in its most direct form.

This morning I gave a guest lecture at the Amsterdam University College as part of a course on Big Data mostly (using Rob Kitchin’s book The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences). I talked about open data, open government data more particularly. How it creates impact, the challenges for government in publishing it, and also quite a bit on the pitfalls connected to using open data for some sort of application. I ended on the note that the ethical issues tied to open data usage are also connected to the notion that data is now a prime geopolitical factor. Any choices you make w.r.t. re-using open data will therefore tell the world a lot about who you are, and to which of the geopolitical data propositions you adhere (e.g. surveillance capitalism, data driven statism, data driven enlightenment)

The lecturer I’ve known for a decade or so from open data efforts, and he invited me as well as TU Delft’s Frederika Welle Donker. She and I have been speaking together in various settings before. A combination that worked well again this time I think, my own practice based perspectives in combination with the insights that research provides from approaching in a more rigorous manner the same questions I deal with.

I published the slides and transcript in my new set-up running ‘my own Slideshare‘, and shared the URL at the start of the talk.
This came in handy as this of course was an online event, and convenient and immediate sharing of content makes more sense in such a setting than when doing a talk in the same room as the students.

It has been a while I did such a general introduction about open data. So I spent time yesterday evening and early this morning first rewatching a general talk I gave 8 years ago, and one two years ago, thinking about what are the current developments that are relevant, and current things we are actually working on (e.g. data governance and ethical issues).

16 years ago today I blogged about this new thing Jeremy Ruston had built and released, TiddlyWiki, a wiki that you could save as a single file. Using javascript and css within the file allowed for expanding the view on items, and added the wiki elements like WikiWords to build links. It was of interest because it was local first software and extremely easy to start using (even if you couldn’t yet save your Tiddlywiki easily then). Jeremy saw it as a tool for local first note taking, and as it was all contained in a single file it was easy to publish, or even send the whole thing as an email attachment. At the time I was using a wiki for notetaking, and so this tool was of interest to me.

I am glad that TiddlyWiki is still around all these years later. In the current attention for Digital Gardens, TiddlyWiki is mentioned and used frequently. A big thank you to Jeremy Ruston for bringing this tool into the world. He’s currently running Federatial which is “exploring a new wave of collaboration tools that eschew the old, crude hierarchical models of organization”. I read that as he is building tools that fit well with networked agency.