Tag: PSIdirective

Posted on
by

Juni is een goede maand voor open data dit jaar.

Ten eerste keurde vorige week dinsdag 4 juni de Eerste Kamer de wet goed die de Europese open data richtlijn implementeert in de Nederlandse Wet Hergebruik Overheidsinformatie. Al is de wet nog niet gepubliceerd en dus nog niet van kracht komt daarmee een einde aan drie jaar vertraging. De wet had al per juli 2021 in moeten gaan. De Europese richtlijn ging namelijk in juli 2019 in en gaf Lidstaten twee jaar de tijd voor omzetting in nationale wetgeving.

Ten tweede ging afgelopen zondag 9 juni de verplichting voor het actief publiceren door overheden via API’s van belangrijke data op zes thema’s in. Die Europese verordening werd eind 2022 aanvaard, werd begin februari 2023 van kracht, en gaf overheden 16 maanden d.w.z. tot zondag om er aan te voldoen. De eerste rapportage over de implementatie moeten Lidstaten in februari 2025 doen, dus ik neem aan dat veel landen die periode nog gebruiken om aan de verplichtingen te voldoen. Maar het begin is er. In Nederland is de impact van deze High Value Data verordening relatief gering, want het merendeel van de data die er onder valt was hier al open. Tegelijkertijd was dat in andere EU landen niet altijd het geval. Nu kun je dus Europees dekkende datasets samenstellen.

Posted on
by

This week the draft implementation act (PDF) and annex listing the first batch of European High Value Data sets (PDF) has finally been published. In the first half of 2020 I was involved in preparatory research to advise on what data, spread across six predetermined themes, should be put on this mandatory list. It’s the first time open data policy makes the publication of certain data mandatory through an API. Until now European open data policy built upon the freedom of information measures of each EU Member State (MS), and added mandatory conditions to what MS published voluntarily, and to how to respond to public data re-use requests. This new law arranges for the pro-active publication of certain data sets.

In the 2020 research I was responsible for the sections about earth observation, environmental, and meteorological data. We submitted our final report in September 2020, and since then there had been total silence w.r.t. the progress in negotiating the list with the MS, and putting together the implementation act. I knew that at least the earth observation and environmental data would largely be included the way I suggested, when last summer I got a sneak preview of the adaptation of the INSPIRE portal where such data is made available.

The Implementation Act

In the Open Data Directive there’s a provision that the European Commission can, through a separate implementation act, set mandatory open data requirements for data belonging to themes listed in the Directive’s Annex. At launch in 2019, 6 such themes were listed: Geo-information, statistics, mobility, company information, earth observation / environment, and meteorology.
The list of themes can also be amended, through another separate implementation act, and a process to determine the second set of themes is currently underway.

The draft implementation law (PDF) states that government-held datasets mentioned in its Annex must be published through APIs, under an open license such as Creative Commons Zero, By Attribution or equivalent / less restrictive. Governments must publish the terms of use for such APIs and these terms may not be used to discourage re-use. APIs must also be fully publicly documented, and a point of contact must be provided.

MS can temporarily exempt some of the high value datasets, a decision that must be made public, but limited to two years after entry into force of this implementation act. Additional usage restrictions are allowed for personal data within the data sets concerned, but only to the extent needed to protect personal data of individuals (so not as an excuse to disallow re-use and access to the data as a whole).

MS must report on their implementation actions every two years, in which they need to list the actual data sets opened, the links to licenses, API and documentation, and exemptions still in place. The implementation is immediately binding for all MS (no need to first transpose into national law to be enforcable), will apply 20 days after publication in the EU Journal, and MS have 6 months to comply.

The Data Sets Per Theme

In this first batch of mandatory open data, 6 themes are covered (PDF). Some brief remarks on all of them.

Mobility

This is, contrary to what you’d expect, the smallest theme of the six covered. Because everything that is already covered in the Intelligent Transport (ITS) Directive is out of scope, which is most of everything concerning land based mobility. What remains for the High Value Data list is data on transport networks contained in the INSPIRE Annex I theme Transport Networks, and static and dynamic data about inland waterways, as well as the electronic navigational charts (ENC) for inland waterways. This is much in line with the 2020 study report. There was some concern with national hydrographical services about ENCs for seas being included (making it harder to force sea going vessels to use the latest version), but my reassurances that it would be unlikely held true.

Geospatial data

Geospatial data is I would say the ‘original’ high value government data, and has been for centuries. The data sets from the four INSPIRE Annex I themes Administrative Units, Geographical Names, Addresses, Buildings and Cadastral Parcels are within scope. Additionally reference parcels and agricultural parcels as described in the 1306/2013 and 1307/2013 Regulations on the Common Agricultural Policy (CAP) are on the list.

Earth Observation and Environment

This was a theme I was responsible for in the 2020 study. It is an extremely broad category, covering a very wide spectrum of types of data. It was basically impossible to choose something from this list, not in the least because re-use value usually comes from combinations of data, not from any single source used. Therefore my proposed solution was to not choose, and advise to treat it as a coherent whole needed in addressing the EU goals concerning environment/nature, climate adaptation, and pollution. The High Value Data list adopts this approach and puts 19 INSPIRE themes within scope. These are:

  • Annex I: Hydrography, and Protected Sites
  • Annex II in full: Elevation, Geology, Land Cover, and Ortho-imagery
  • Annex III: Area management, Bio-geographical regions, Energy resources, Environmental monitoring facilities, Habitats and biotopes, Land use, Mineral resources, Natural risk zones, Oceanographic geographical features, Production and industrial facilities, Sea regions, Soil, and Species distribution

Additionally all environmental information as covered by the 2003/4/EC Directive on public access to such information is added to the list, and all data originating in the context of a wide range of EU Regulations and Directives on air, climate, emissions, nature preservation and biodiversity, noise, waste and water. I miss soil in this environmental list, but perhaps the Annex III INSPIRE theme is seen as sufficiently covering it. I still need to follow up on the precise formulations w.r.t. data in 31 additionally referenced regulations and directives.

What to me is a surprising phrasing is that earth observation is defined here including satellite based data. Not surprising in terms of earth observation itself, but because satellite data was specifically excluded from the scope of our 2020 study. First because the EU level satellite data is already open. Second because this list deals with data from MS, and not many MS have their own satellite data. When they do it is usually the result of public private collaborative investment, and such private investment may dry up if there are no longer temporary exclusive access arrangements possible, which would have resulted in considerable political objections. Perhaps adding space based data collection is currently being well enough watered down by defining the INSPIRE themes as its scope, while at the same time future proofing the definition for when satellite data does become part of INSPIRE themes.

Together these first three, mobility, geospatial, and EO/environment, place a full 24 out of 34 INSPIRE themes on the list for mandatory open data. This basically amounts to adding an open data requirement to INSPIRE. It places MS’ INSPIRE compliance very much in the focus of attention, which now often is limited, and further positions INSPIRE as a key building block in the coming Green Deal dataspace. It will be of high interest to see what the coming new version of the INSPIRE directive, currently under review, makes of all that.

Statistics

This topic is more widely covered in the High Value Data list, than it was in the 2020 study, both in the types of statistics included, and in the demands made of those types of statistics. Still there are lots of statistics that MS hold, that aren’t included here (while some MS do publish most of their statistics already btw): the selection is based on European reporting obligations that follow from a list of various European laws.
Topics for which statistics must be published as open data in a specified way:

  • Industrial production
  • Industrial producer price index, by activity
  • Volume of sales by activity
  • EU international trade in goods
  • Tourism flows in Europe
  • Harmonised consumer prices indices
  • National accounts: GDP, key indicators on corporations and households
  • Government expenditure and revenue, government gross debt
  • Population, fertility, mortality
  • Current healthcare expenditure
  • Poverty
  • Inequality
  • Employment, unemployment, potential labour force

Data for these reporting obligations should be available from the moment the law creating them has been in force. That means for instance that healthcare expenditure should be available from at least 2008, whereas employment statistics must be available from at least 2019, because of the different years in which these laws were enacted.

Company information

Company information from the start has been the most controversial theme of the six covered by this implementation act. I assume this theme has also been the prime political reason for the long delay in the proposal being published. In my perception because this is the only data set that actually might end up challenging the status quo in society (as it involves ownership and power structures, and touches tax evasion). In the 2020 study four aspects were considered, the basic company information, company documents and accounts, ownership information, and insolvency status. Two ended up in the draft law: basic company information and company documents. Opening ownership information, not even the ultimate beneficial ownership (UBO) information, from the start drew vehement objections (including from the Dutch government). Many stakeholders (including the NGO I chair) are disappointed with the current outcome. (Here’s an old blogpost where I explain UBO, and here’s SF writer Brin on what transparent UBO might mean to our societies.) The data that will become open data still may be 2 years in the future: the Open Data Directive allows a 2 year exemption, and this is the data where that exemption will be used I think.
That said, mandatory open company data and documents, even with the delay through exemptions, is already a step forward that puts an end to literally decades of court cases, obstruction, and lobbying for more openness. The very first PSI Directive in 2003 was already an expression of a broad demand for this data, now 20 years on it finally becomes mandatory across the EU. Some people I know have been after this for their entire professional careers and already retired. It’s easy to loose sight of that win when we only focus on not having (ultimate) ownership data included.

Meteorological data

This is the other theme I was responsible for in the 2020 study. Like with company information this is an area where the discussion about making it available for re-use is decades old and precedes digitisation becoming ubiquitous. When I started my open data work in 2008, most of the existing documentation and argumentation for the value of and need for open data concerned meteorological data. A range of EU countries already have this as open data, others not at all. While progress has been made in the past decades, the High Value Data list provides a blanket obligation for all EU MS, a result that would otherwise still be a very long time away if entirely voluntary for the MS involved.
Data included here includes all weather observation data, validated observations / climate data, radar data (useful for things like cloud heights, precipitation and wind), and numerical weather prediction data (these are the outputs of the combined models used for predictions).

The implementation act is up for public feedback until 21 June, but likely will retain its current form. I think it’s a pretty good result, and I am happy that I have been able to contribute to it.

Posted on
by

It’s finally here, published today: the proposal for the EU High Value Data list. The list for the first time makes open data publication mandatory for government concerning (for now) 6 themes (geographic information, meteorology, mobility, statistics, earth observation and environment, and company information). Already in September 2020 an impact assessment and advice on policy options was delivered to the European Commission. I was part of that assessment team, and responsible for the themes Meteorology, Earth Observation and environment. Now we get to see what has been proposed to be implementend in law. I haven’t read it yet, will do that tomorrow first thing, but wanted to share the link here. There’s a window for feedback on the proposal open until 21 June 2022.

Posted on
by

Vandaag sluit de internetconsultatie voor de implementatiewet van de open data richtlijn. Ik schreef er met Kerst al over. Zojuist diende ik mijn zienswijze op het wetsvoorstel in. Die is samen met de andere openbare reacties beschikbaar online.

Als basis voor mijn reactie vergeleek ik de ontwerptekst (pdf), de memorie van toelichting (pdf), de Europese open data richtlijn, en ook de tekst van de huidige hergebruikswet. Want ook van wat er niet verandert kun je iets vinden tenslotte. Mijn/onze ervaringen in het veld sinds de allereerste open data richtlijn van kracht werd, vormen de bril die ik opzette.

Oorspronkelijk zou ik mijn reactie schrijven vanuit het perspectief van een klant. Maar na grondige beschouwing wordt die niet of nauwelijks geraakt door de wijzigingen, en dat maakte een reactie vanuit die organisatie niet heel nuttig. Tegelijkertijd is er wel genoeg te zeggen over het wetsvoortsel. Dus deed ik het vanuit mijn persoonlijke perspectief, en dat van ons bedrijf.

Het is prettig te zien dat ook andere organisaties hebben gereageerd. Dat maakte het mogelijk om zelf nog een paar dingen aan te voeren die anderen juist niet noemden, en er op te vertrouwen dat andere punten al voldoende door andere partijen waren belicht.

Nu is het wachten op het uiteindelijk voorstel, en de goedkeuring daarvan. Nederland is laat met deze wet, hij had vorig jaar juli al van kracht moeten zijn. De invoering van de Open Data richtlijn is één van de bouwstenen van de Europese datastrategie, en er staat veel te gebeuren. Dus hoog tijd dat Nederland hier stappen in zet. Dan kunnen we ook makkelijker met elkaar de kansen die er bij horen verzilveren.

Posted on
by

In juni 2019 nam de Europese Unie een nieuwe Open Data Richtlijn aan. Deze moest vervolgens binnen 2 jaren in alle Lidstaten in nationale wetgeving worden omgezet. Dat is in veel landen niet op tijd gebeurd, waaronder in Nederland. Ondermeer door de pandemie ging de aandacht naar andere dingen. Het Ministerie van Binnenlandse Zaken heeft de draad een tijdje geleden weer opgepakt, en dat resulteert in een Kerstcadeautje: de internetconsultatie voor de nieuwe Wet Hergebruik Overheidsinformatie is op 24 december geopend. Tot 6 februari kan iedereen zijn inzichten delen over de ontwerptekst.

De nieuwe Open Data Richtlijn breidt de scope van organisaties die onder de Richtlijn uit, voegt dynamische data nadrukkelijk toe (vroeger kwamen overheden er mee weg om real time meetgegevens niet te delen omdat op het moment van aanvragen die gegevens nog niet bestonden, alleen een jurist met ‘nee’ als uitgangspunt verzint zoiets) en voegt een lijst verplichte open data publicaties toe (die regels moeten nog in een aparte wet gepubliceerd worden, de Europese Commissie heeft dat nog niet gedaan, al zou dat al een jaar geleden rond zijn).
Verder worden weer stappen gezet in het verder beperken van exclusieve overeenkomsten en het vragen van geld voor gegevensverstrekking, zoals vorige edities van deze Richtlijn ook telkens deden.

Ik ga dit kerstcadeau nog niet openmaken, maar neem in januari uitgebreid de tijd om de tekst te lezen en zonodig van opmerkingen te voorzien.

(Full disclosure: ik werkte in 2017/2018 mee aan de evaluatie en impact assessment van de vorige Hergebruiksrichtlijn. De resultaten daarvan hebben mede de inhoud van de nieuwe Open Data Richtlijn in 2019 bepaald.)

Posted on
by

Today I learned that on 30 September the EC has initiated infringement procedures against 19 EU Member States because of failing to provide complete information on the transposition in national law of the new Open Data Directive.

Just one day before that announcement I already wrote that transposition was far from complete, but I hadn’t noticed that the day after that turned out to apply to 19 countries or about 70% of Member States.

One of those countries, unsurprisingly, is the Netherlands. Unsurprising because the work on the transposition only started in earnest last February. A year delayed because of shifting priorities due to the pandemic, and much too late to ensure timely compliance, for which the deadline was last July.

My contact on this within the responsible ministry however told me that progress has been made. An internet consultation on the new law should open shortly after New Year, meaning the text of the proposed transposition will be publicly available then. I am eager to read it.