Bookmarked CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition (
The popularity of ASR (automatic speech recognition) systems, like Google Voice, Cortana, brings in security concerns, as demonstrated by recent attacks. The impacts of such threats, however, are less clear, since they are either less stealthy (producing noise-like voice commands) or requiring the physical presence of an attack device (using ultrasound). In this paper, we demonstrate that not only are more practical and surreptitious attacks feasible but they can even be automatically constructed. Specifically, we find that the voice commands can be stealthily embedded into songs, which, when played, can effectively control the target system through ASR without being noticed. For this purpose, we developed novel techniques that address a key technical challenge: integrating the commands into a song in a way that can be effectively recognized by ASR through the air, in the presence of background noise, while not being detected by a human listener. Our research shows that this can be done automatically against real world ASR applications. We also demonstrate that such CommanderSongs can be spread through Internet (e.g., YouTube) and radio, potentially affecting millions of ASR users. We further present a new mitigation technique that controls this threat.

It seems sharing play-lists is no longer an innocent behaviour, nor is playing YouTube with the sound on in the presence of automated speech recognition like Google’s, Amazon’s and Apple’s cloud connected microphones in your living room. “CommanderSongs can be spread through Internet (e.g., YouTube) and radio.

The easiest mitigation of course is not having such microphones in your home in the first place. Another is running your own ASR with your 95% standard commands localized in the device. Edge first like Candle proposes, not cloud first.

Since shortly after we moved in we have a temperature and humidity sensor in our garden.

This week’s heat wave is breaking records across Europe including here in the Netherlands. So I’ve kept an eye on the temperature in our garden. Our sensor is part of a city wide network of sensors, which includes two sensors nearby. Of the three sensors, ours indicates the lowest temperature at 36.8 (at 16:45), the other two hover just under 40 and at 41.8 respectively. Such differences are caused by the surroundings of the sensor. That ours is the lowest is because it’s placed in a very green garden, while the others are out on the street. In our completely paved and bricked up courtyard the temperature is 42.1 in the shade, due to the radiation heat of sun and stones. Goes to show that greenery in a city is key in lowering temperatures.

Three sensors in our neighbourhood, ours is in the middle, showing the lowest temperature. Note that the color scale is relative, for these 3 sensors running from 36.6 to 41.8.

In the past days since our return from France the temperature has been steadily rising, as per the graph below (which currently ends at the peak of 36.8 at 16:45). Staying inside is the best option, although the also increasingly higher lowest temperatures (from 15 to above 20) mean that the nights are slowly becoming more uncomfortable as the outside temperature will stay above the in house temperature during most or all of the night.

UPDATE as of 26/7 June noon, here you can see how the night minimum jumped 5 degrees in 24 hours, bringing it above the in house temperature for the entire night, except a brief moment around 6 am. At noon the maximum for the day before is already nearly reached.

The way to make this graph yourself is

  • Go to, where you can select various data types and time frames. Our sensor is number 51, and I selected a time frame starting at July 19th at midnight. This allows me to download the data as CSV.
  • The data in that download is Tab separated, not comma,when you select a comma to be used as decimal point.
  • The file contains columns for the sensor number and its latitude and longitude, that are not needed as this is data for just one sensor. Likewise, empty columns for measurement values for which my sensor kit doesn’t contain sensors, such as particulate matter, can be removed. Finally the columns for battery level and humidity are also not needed on this occasion.
  • With the remaining columns, time and temperature it is easy to build the graph. In this case I replaced the timestamps with sequential numbers, as I intend to make a sparkline graph with it later.

Some links I think worth reading today.

My current thinking about what to bring to my open data and data governance work, as well as to technology development, especially in the context of networked agency, can be summarised under the moniker ‘ethics by design’. In a practical sense this means setting non-functional requirements at the start of a design or development process, or when tweaking or altering existing systems and processes. Non-functional requirements that reflect the values you want to safeguard or ensure, or potential negative consequences you want to mitigate. Privacy, power asymmetries, individual autonomy, equality, and democratic control are examples of this.

Today I attended the ‘Big Data Festival’ in The Hague, organised by the Dutch Ministry of Infrastructure and Water Management. Here several government organisations presented themselves and the work they do using data as an intensive resource. Stuff that speaks to the technologist in me. In parallel there were various presentations and workshops, and there I was most interested in what was said about ethical issues around data.

Author and interviewer Bas Heijne set the scene at the start by pointing to the contrast between the technology optimism concerning digitisation of years back and the more dystopian discussion (triggered by things like the Cambridge Analytica scandal and cyberwars), and sought the balance in the middle. I think that contrast is largely due to the difference in assumptions underneath the utopian and dystopian views. The techno-optimist perspective, at least in the webscene I frequented in the late 90’s and early 00’s assumed the tools would be in the hands of individuals, who would independently weave the world wide web, smart at the edges and dumb at the center. The dystopian views, including those of early criticaster like Aron Lanier, assumed, and were proven at least partly right, a centralisation into walled gardens where individuals are mere passive users or an object, and no longer a subject with autonomy. This introduces wildly different development paths concerning power distribution, equality and agency.

In the afternoon a session with professor Jeroen van den Hoven, of Delft University, focused on making the ethical challenges more tangible as well as pointed to the beginnings of practical ways to address them. It was the second time I heard him present in a month. A few weeks ago I attended an Ethics and Internet of Things workshop at University of Twente, organised by UNESCO World Commission on the Ethics of Science and Technology (COMEST). There he gave a very worthwile presentation as well.

Van den Hoven “if we don’t design for our values…”

What I call ethics by design, a term I first heard from prof Valerie Frissen, Van den Hoven calls value sensitive design. That term sounds more pragmatic but I feel conveys the point less strongly. This time he also incorporated the geopolitical aspects of data governance, which echoed what Rob van Kranenburg (IoT Council, Next Generation Internet) presented at that workshop last month (and which I really should write down separately). It was good to hear it reinforced for today’s audience of mainly civil servants, as currently there is a certain level of naivety involved in how (mainly local governments) collaborate with commercial partners around data collection and e.g. sensors in the public space.

(Malfunctioning) billboard at Utrecht Central Station a few days ago, with not thought through camera in a public space (to measure engagement with adverts). Civic resistance taped over the camera.

Value sensitive design, said Van den Hoven, should seek to combine the power of technology with the ethical values, into services and products. Instead of treating it as a dilemma with an either/or choice, which is the usual way it is framed: Social networking OR privacy, security OR privacy, surveillance capitalism OR personal autonomy, smart cities OR human messiness and serendipity. In value sensitive design it is about ensuring the individual is still a subject in the philosophical sense, and not merely the object on which data based services feed. By addressing both values and technological benefits as the same design challenge (security AND privacy, etc.), one creates a path for responsible innovation.

The audience saw both responsibilities for individual citizens as well as governments in building that path, and none thought turning one’s back on technology to fictitious simpler times would work, although some were doubtful if there was still room to stem the tide.

After having the sensor revised as it stopped working having dropped out of the tree during a storm (turned out to be a battery draining firmware issue), I have finally reinstalled the Measure Your City sensor hub. I took the opportunity to spray paint the sensor hut white (preventing the dark grey hut from quickly heating up in sunlight and thus influence the measurements), add the Measure Your City sticker (so passers by can figure out what this thing is), and use a stronger wire to hang it back in the garden.

In the past weeks we used the sensor indoors to measure humidity, as it was very dry during cold spells. From now on you should see the temperature measurements change from about a steady 20 degrees inside, to the day and night rhythm outside.