I have a Google Alert set up for my name, to find new mentions of it online. Today I received a mail that my name came up in an article in the South China Morning Post (SCMP), as part of a photo credit. This made me curious.

An article on the amount of time elderly US citizens spend behind their computer screens published August 25th, uses a photo I made it turns out.

My mom is in a Hong Kong newspaper

The photo is from 2008, and shows my mom trying her first steps on a laptop, which we gave her for her 71st birthday when she started having mobility problems. E’s hand is pointing out things in the Gmail interface. This image is available in my Flickr account, and that is how the SCMP ended up finding it (it says as much in the photo credit). They likely used the Flickr’s search filter and had it set to ‘any Creative Commons license’. And that’s where it went wrong.

SCMP is a commercial company, and my photo is licensed with Creative Commons Attribution, Non-Commercial, Share Alike. Creative Commons is a way for copyright holders to preemptively state which uses of a work are always permitted. I license all my photos, and using Creative Commons give permission for any use that isn’t commercial, as long as the result is shared the same way, and as long my name is mentioned as the author.

SCMP did mention my name (which is how I found the article), but cannot comply with the non-commercial part of the Creative Commons license, and thus should have asked for my permission before using the image. Now I’ve sent them an e-mail with an invoice, for using my photo, and another 100% added for using it without permission. Payable in 15 business days.

To my pleasant surprise the SCMP’s photo editor (whom I mailed), responded within 20 minutes apologising and promising payment.

(full disclosure: I’m a board member for Open Nederland, the association of Dutch makers that serves as the Dutch chapter for Creative Commons.)

A project I’m involved has won funding from the SIDN Fund. SIDN is the Dutch domain name authority, and they run a fund to promote, innovate, and stimulate internet use, to build a ‘stronger internet for all’.
With the Open Nederland association, the collective of makers behind the Dutch Creative Commons Chapter, of which I’m a board member, we received funding for our new project “Filter me niet!” (Don’t filter me.)

With the new EU Copyright Directive, the position of copyrights holders is in flux the coming two years. Online platforms will be responsible for ensuring copyrights on content you upload. In practice this will mean that YouTube, Facebook, and all those other platforms will filter out content where they have doubts concerning origin, license or metadata. For makers this is a direct threat, as they run the risk of seeing their uploads blocked even while they clearly hold the needed copyright. False positives are already a very common phenomenon, and this will likely get worse.

With Filtermeniet.nl (Don’t filter me) we want to aid makers that want to upload their work, by inserting a bit of advice and assistance right when they want to hit that upload button. We’ll create a tool, guide and information source for Dutch media makers, through which they can declare the license that fits them best, as well as improve metadata. In order to lower the risk of being automatically filtered out for the wrong reasons.

At the end of March the European Commission (EC) has announced it is adopting the Creative Commons By Attribution license as its standard license.

The CC-BY license will be used for videos and photos, studies published in peer-reviewed journals, data and visualisations on the EU open data portal and documents published on EU websites.

Re-use of EC material has been possible since 2006 (and rephrased in 2011), but in practice it wasn’t always clear to potential re-users what was allowed and what wasn’t.
While re-use and attribution is part of the EC’s copyright notice, it is likely re-users are discouraged by the copyright claim above it, and missing the permissions underneath it:


Current default copyright notice on EC websites, to be exchanged for a CC-BY license

In contrast adding the Creative Commons By Attribution license sends a clear message about permissions that are granted up-front without the need for a re-user to seek consent: any re-use is permitted, including commercial re-use, provided the EC is attributed as its source, and provided re-use forms or alterations don’t suggest they are endorsed by or coming from the EC.


The clarity that a Creative Commons license provides

(full disclosure: I am a board member of Open Nederland, the Dutch Creative Commons chapter)

Open Nederland heeft een eerste podcast geproduceerd. Sebastiaan ter Burg is de gastheer en Maarten Brinkerink deed de productie en muziek.

In de Open Nederland podcast komen mensen aan het woord komen die kennis en creativiteit delen om een eerlijke, toegankelijke en innovatieve wereld te bouwen. In deze eerste aflevering gaat het over open in verschillende domeinen, zoals open overheid en open onderwijs, en hoe deze op elkaar aansluiten.

De gasten in deze aflevering zijn:

  • Wilma Haan, algemeen directeur van de Open State Foundation,
  • Jan-Bart de Vreede, domeinmanager leermiddelen en metadata van Kennisnet en
  • Maarten Zeinstra van Vereniging Open Nederland en Chapter Lead van Creative Commons Nederland.

(full disclosure: ik ben zowel bestuurslid van Open Nederland als bestuursvoorzitter van Open State Foundation, waarvan CEO Wilma Haan in deze podcast deelneemt.)

This week NBC published an article exploring the source of training data sets for facial recognition. It makes the claim that we ourselves are providing, without consent, the data that may well be used to put us under surveillance.

In January IBM made a database available for research into facial recognition algorithms. The database contains some 1 million face descriptions that can be used as a training set. Called “Diversity in Faces” the stated aim is to reduce bias in current facial recognition abilities. Such bias is rampant often due to too small and too heterogenous (compared to the global population) data sets used in training. That stated goal is ethically sound it seems, but the means used to get there raises a few questions with me. Specifically if the means live up to the same ethical standards that IBM says it seeks to attain with the result of their work. This and the next post explore the origins of the DiF data, my presence in it, and the questions it raises to me.

What did IBM collect in “Diversity in Faces”?
Let’s look at what the data is first. Flickr is a photo sharing site, launched in 2004, that started supporting publishing photos with a Creative Commons license from early on. In 2014 a team led by Bart Thomee at Yahoo, which then owned Flickr, created a database of 100 million photos and videos with any type of Creative Commons license published in previous years on Flickr. This database is available for research purposes and known as the ‘YFCC-100M’ dataset. It does not contain the actual photos or videos per se, but the static metadata for those photos and videos (urls to the image, user id’s, geo locations, descriptions, tags etc.) and the Creative Commons license it was released under. See the video below published at the time:

YFCC100M: The New Data in Multimedia Research from CACM on Vimeo.

IBM used this YFCC-100M data set as a basis, and selected 1 million of the photos in it to build a large collection of human faces. It does not contain the actual photos, but the metadata of that photo, and a large range of some 200 additional attributes describing the faces in those photos, including measurements and skin tones. Where YFC-100M was meant to train more or less any image recognition algorithm, IBM’s derivative subset focuses on faces. IBM describes the dataset in their Terms of Service as:

a list of links (URLs) of Flickr images that are publicly available under certain Creative Commons Licenses (CCLs) and that are listed on the YFCC100M dataset (List of URLs together with coding schemes aimed to provide objective measures of human faces, such as cranio-facial features, as well as subjective annotations, such as human-labeled annotation predictions of age and gender(“Coding Schemes Annotations”). The Coding Schemes Annotations are attached to each URL entry.

My photos are in IBM’s DiF
NBC, in their above mentioned reporting on IBM’s DiF database, provide a little tool to determine if photos you published on Flickr are in the database. I am an intensive user of Flickr since early 2005, and published over 25.000 photos there. A large number of those carry a Creative Commons license, BY-NC-SA, meaning that as long as you attribute me, don’t use an image commercially and share your result under the same license you’re allowed to use my photos. As the YFCC-100M covers the years 2004-2014 and I published images for most of those years, it was likely my photos are in it, and by extension likely my photos are in IBM’s DiF. Using NBC’s tool, based on my user name, it turns out 68 of my photos are in IBM’s DiF data set.

One set of photos that apparently is in IBM’s DiF cover the BlogTalk Reloaded conference in Vienna in 2006. There I made various photos of participants and speakers. The NBC tool I mentioned provides one photo from that set as an example:

Thomas Burg

My face is likely in IBM’s DiF
Although IBM doesn’t allow a public check who is in their database, it is very likely that my face is in it. There is a half-way functional way to explore the YFCC-100M database, and DiF is derived from the YFCC-100M. It is reasonable to assume that faces that can be found in YFCC-100M are to be found in IBM’s DiF. The German university of Kaiserslautern at the time created a browser for the YFCC-100M database. Judging by some tests it is far from complete in the results it shows (for instance if I search for my Flickr user name it shows results that don’t contain the example image above and the total number of results is lower than the number of my photos in IBM’s DiF) Using that same browser to search for my name, and for Flickr user names that are likely to have taken pictures of me during the mentioned BlogTalk conference and other conferences, show that there is indeed a number of pictures of my face in YFCC-100M. Although the limited search in IBM’s DiF possible with NBC’s tool doesn’t return any telling results for those Flickr user names. it is very likely my face is in IBM’s DiF therefore. I do find a number of pictures of friends and peers in IBM’s DiF that way, taken at the same time as pictures of myself.


Photos of me in YFCC-100M

But IBM won’t tell you
IBM is disingenuous when it comes to being transparent about what is in their DiF data. Their TOS allows anyone whose Flickr images have been incorporated to request to be excluded from now on, but only if you can provide the exact URLs of the images you want excluded. That is only possible if you can verify what is in their data, but there is no public way to do so, and only university affiliated researchers can request access to the data by stating their research interest. Requests can be denied. Their TOS says:

3.2.4. Upon request from IBM or from any person who has rights to or is the subject of certain images, Licensee shall delete and cease use of images specified in such request.

Time to explore the questions this raises
Now that the context of this data set is clear, in a next posting we can take a closer look at the practical, legal and ethical questions this raises.