I very much appreciate how Sven Knebel extensively responded to my previous posting on some Webmention issues I came across. Some of his responses do make me have new questions.

About the wrong URL, i.e. not the source of the webmention, showing up in a Webmention, Sven writes:

…. There’s a href=”https://news.indieweb.org/nl” class=”u-syndication” as the only top-level link inside his post, and no explicit url property set. This causes the microformats parser to assume that this link points to the canonical location of the post, and it is thus used for comment display. This seems like a problem with the microformats specification, and I’ll follow up on it there, but for now the easy fix would be for Frank’s posts to mark up their permalink, e.g. by adding a class=”u-url” to the link on the headline.

To me this reads as a vulnerability. I would expect my site to always take the source from the webmention message as URL. That is the only one that has been checked from my end for the presence of a reference to my site (the target). If the source page is allowed to set a different URL, even by mistake like here, that feels extremely counterintuitive. It opens it up to spam. In this case the faulty link is to a benign site, but it could have been pills or malware. It is also strange to me that my server in the comments table of the database correctly stores the source url, but in the meta data table stores a url at the discretion of the source’s website. (Meanwhile Frank has fixed it for now on his end as demonstrated by his webmention to my previous post, but my point remains)

About no content being shown of the blogpost that links to my blogposts Sven says:

“This is intentional. Frank’s post only mentions your post (=includes a link to it), it is not marked up as an explicit reply. Only replies are shown with content, since for mentions this is often misleading.”

This to me doesn’t make a lot of sense. [update: and for my site at least it isn’t true either, I linked back as an explicit reply to my own posting, but it still shows it as a mention].
There is indeed a difference between a direct reply to something (@Frank….) and mentioning that something as part of something else (As Frank says….). Yet that doesn’t warrant a difference in presentation, where a reply would be shown, yet for a mention just the address of the site. It also gives the source control over how something is shown on my site (by setting a different microformat for a link), while I do not have that control.
From the perspective of the reader of my blog it is not enough to only see that ‘some site links to this blogpost’ to click on that link to find out if it might be of interest, it is tremendously helpful to see a piece of that referring page to determine the context in which it refers to my blogpost.

Most if not all of my mentions of others’ blogposts aren’t meant as a direct response but as building or continuing on a line of reasoning, riffing off other people’s ideas. This is the way distributed conversations take place, how ambient humanity is established. Distributed conversations are a fundamental part of blogging to me. It’s not back and forth replies, it’s a jam session. To enjoy the jam session, you need to see the whole band at a glance, not just a list of the line-up while listening to a sole musician. Discoverability and serendipity flow from it.
It used to be that trackbacks did precisely that, show the context in which someone else referred to my blogposts. It is enriching my own posts to show that context underneath them. See below how that looked a long time ago, in a post on information strategies from 2005.

Three trackbacks on an old post of mine, showing context of the linking blogpost



These three posts are not in response to me, but reflections triggered by my posts and extensions of my contribution

So I’d definitely want to show that context for webmentions. What strikes me as odd now is how little control I have over how the Webmention and Semantic Linkbacks plugins actually deal with webmention data. The stuff I’d like to show is stored in my database, but I can’t through the plugins determine how that is shown.
The same is true on the flipside: my site adds microformats so others can machine read my blog, but apparently it doesn’t do it right. Yet I have no control from the mentioned plugins interfaces over how that is done, nor do I have documentation / insight into how the plugins are designed to comply with microformat specifications. So the next step is: read up on microformat specifications, and dive into the code of the plugins to see where it does what, and whether I can change that in ways that won’t be simply overwritten with the first update of WordPress or the plugins. [UPDATE: I installed a different WordPress Theme, called Sempress, as it should be better at adding the correct microformats for this site]

Webmentions is what makes it possible for me to write here about someone else’s blogpost and have my response show up beneath theirs. And vice versa. Earlier mechanisms such as pingback and trackback did the same thing, but slipped under the radar or succumbed to spam. Webmention is a W3C recommendation.

The webmention itself is simple
The core of webmention is straightforward: if I write something here, my webserver will try to let every site I link to in my text know I link to them. This by checking if the sites I link to have an ‘endpoint’, an antenna basically, for webmentions. If a site does, then it will send a simple message to that antenna stating two web addresses, the source (here my blogpost) and the target (here your blogpost). When your site receives a webmention it will do some checking: does my source blogpost indeed link to your target address?

What happens next is less simple
It can quickly get confusing during what happens next.
When my site receives a webmention (this source x links to your target y), all it knows is just the URL of a page that links to me. What my site displays and how it displays that as a consequence of a webmention message depends on multiple factors:

My server will try to read the source blogpost, and see what machine readable information it contains, and what it can know about the source blogpost. These machine readable parts are in the form of microformats.
My server will store some of the information it finds.
Then my website template will show some information from what the server stored when showing the target blogpost on my site.

How well that works depends on multiple factors therefore:

  1. The available machine readable info in the source blogpost, and whether that info is properly encoded
  2. The settings of my server for what it stores
  3. The settings of my site template for what it shows

When something seems to be going wrong, it could be a problem with your site, my site template or my server settings, and it is never obvious which one it is, or if it is the aggregation of multiple issues. It also depends on how easy it is to alter any settings whether you can repair or change things when webmentions are not properly dealt with. Supposedly the Webmention and Semantic Linkbacks plug-ins I use should take care of those issues but it is not obvious that they indeed do.

An example, me and Frank’s sites webmentioning each other
Frank Meeuwsen and I have been mentioning eachother several times and we’ve seen some strange webmention behaviours. For instance in one case Frank’s blog displayed not just a short part of my posting mentioning him, but my entire page including header, footer and sidebar. Clearly something wrong, likely with some of my machine readable encoding, but maybe also something wrong on his end. I suspect my machine readable encoding is indeed faulty but there’s no clear way in which I can change how my webmention plugins deal with that. And if I alter the code, which I could, it is likely the next software update will simply overwrite it.

Yesterday Frank posted about the puzzle webmention is to him in Dutch. Here are some screenshots on how pieces of that puzzle look on my end of things.

Frank’s posting lives at http://diggingthedigital.com//Waar-te-beginnen-met-Webmentions/ In his posting he refers to a posting on my site. He did not send a webmention. But I can do that myself, using a simple form at the bottom of my posting (visible at the bottom of this page too). In that webform I pasted the mentioned url, and that sends the simple webmention message. That message has been received and stored on my server, with the correct source and target address and a timestamp:

What ended up underneath my posting is:

Or as it looks for me as the site’s owner:

A few things stand out:

  • There’s no link to the actual blogpost by Frank (the source), just to his general domain
  • There’s a link to news.indieweb.org, which is a completely different domain
  • There’s no image of the author or an avatar in absence of an image
  • There isn’t any content from Frank’s post shown as part of the mention

So what’s happening? Is this an issue at Frank’s end, is it an issue with what I store on my server, or what I show in my site template? One, two, all three of them?

Puzzling over the pieces in this example

The missing avatar. My site tries to look for an avatar in the source, and if there isn’t one, it shows a general one. Here neither happens, it’s just a blank space. The HTML source of my page reveals it does try to show an avatar, the one that Frank sets in his own blog page as the one to use. His site says in the source code:

<a href="/" class="site-avatar"><img src="/images/dtd-avatar.png" class="u-photo" /></a>

The micro format u-photo is interpreted correctly by my site, and it tries to show the linked image. When you go to that image in your browser it works, but if you try to embed it in your own page it doesn’t.

Frank’s image should be visible below this line,
Frank's avatar
and above this one, but it isn’t.

Probably Frank’s web server prevents bandwidth theft by sending back a white pixel and not the requested image.
[UPDATE] The issue, as Sven points out in the comments, is that this site is https and Frank’s is http. My browser is set-up to reject http material on an otherwise https site. A case of my browser being my castle.[/UPDATE]
Making the avatar fail because my site doesn’t try to store the avatar locally.

The link to news.indieweb.org and the absence of a link to the actual blog post by Frank. The source (Frank’s blogpost) was sent and received correctly as we saw. In the machine readable part of Frank’s site a value is set as ‘canonical’ address for his blogpost.

There is an extra / in that url, and I’m not sure what that might cause, but on my end the canonical that gets saved is very different, it’s that indieweb address.

The odd bit is that indieweb.org address is not mentioned in the source of Frank’s page. At the same time, it seems it isn’t unique to my server, as underneath a posting about webmentions by Sebastiaan Andeweg you see the same thing happening. Frank’s webmention from May 12th shows the indieweb link (and no avatar). Sebastiaan doesn’t use WordPress or the plugins I use as far as I can tell.

So where’s the actual link to Frank’s blogpost? The canonical URL Frank’s posts provides is stored on my server, in the database table for comments as the URL for the author. The indieweb URL however is stored as canonical URL in the comment metadata table in my database. And that gets used for displaying the webmention underneath my blogposting.

The same is true for the absence of the content of Frank’s mention of me. It is collected and stored in the comment table of my site’s database. Yet what is shown underneath my blogpost as mention is constructed only from the comment meta data table, and not the comment table.


Frank’s mention’s content is in my comment database, yet not shown


The metadata fields stored for Frank’s mention in my database

So what’s happening here is a mix of elements from Frank’s site, my webmention plugins and my site template. But how to influence the behaviour of my plugins without seeing that undone with the next update is not clear to me at this point. Nor is how to alter the plugins so I can improve the machine readable microformats on my site.

For the Province of South-Holland we’re currently helping them to extend their open data provision. Next to looking at data they hold relevant to key policy domains, we also look at what other data is available elsewhere for those domains. For instance nationwide datasets with local granular level of detail. In those cases it can be of interest to take the subset relevant for the Province and republish that through their own channels.

One of the relevant topics is energy transition (to sustainable energy sources). Current and historic household usage is of interest here. The companies that maintain the grid publish yearly data per postcode, or at least some of them do. There are seven of these companies.
Luckily all three companies active in South-Holland do publish that data.


In South-Holland three companies are active (number 3, 5 and 6)
(Source: Energielevernanciers.nl

Having this subset of data is useful for any organisation in the region that wants to limit the amount of data they have to dig through to get what they need, for the provincial organisation itself, and for individual citizens. Households that have digital meters have access to their daily energy usage readings online. This data allows them to easily compare their personal usage with their neighbours and wider surrounding area. For instance I established that our usage is lower for both electricity and gas than average in our street. It is also easier to map, or otherwise visualise, in a meaningful way for the province and relevant regional stakeholders.

Here’s a brief overview of the steps we’re taking to get to a province-wide data set.

  • Download the data for the years available for Westland, Liander and Stedin (Westland goes back to 2010, the others to 2008)
  • Check the data formats: Westland and Stedin provide CSV, Liander XLSX
  • Check data structure: all use the same structure of fields and conventions
  • To get only the data for South-Holland we use the postcode that is mentioned in the data.
  • The Dutch postcode zones do not conform to provincial boundaries however, so we take the list of four position postcodes and determine the ones that fall within South-Holland:
    • 1428-1429
    • 2159-2164
    • 2170-3381
    • 3465-3466
    • 4126-4129
    • 4140-4146
    • 4163-4169
    • 4200-4209
    • 4213
    • 4220-4249
  • The data contains 6 position postcodes of the structure 1234AB. We need to split them into the four digits and the two letters, to be able to match them with the ranges that fall within the province.
  • For personal data protection purposes, in the data, for 6 position postcodes where the number of addresses in that postcode is less than 10, the data is aggregated with a neighbouring postcode, until the number of addresses is higher than 9. It is not certain that those aggregations fall within a single province. The data provides a ‘from’ 6 position postcode and a ‘to’ 6 position postcode. This is the same value where the number of addresses in a postcode is high enough but can be a wider range.
    • We need to test if the entire postcode range in a single data record falls within one of the ranges of postcodes that belong in South-Holland.
    • For the small number of aggregates that fall into two provinces we can adopt the average usage number, but need to mark that the number of households in that area is unknown,
    • or retrieve the actual number of addresses from the national address and building database, and mark that the average energy usage values are from a larger number of addresses.
    • Alternatively we can keep the entire range, including the part outside the province,
    • or we exclude the entire range and leave a ‘hole in the map’.
    • In any case we need to mark in the data what we did, and why.
  • The result is then a data set in CSV that consolidates the three sources for all those records that fall within the province.
  • This dataset can then be mapped, e.g. in Q-GIS or other tools in use within the province South-Holland.
  • We provide a recipe and/or script from the above steps that can take the future yearly data sets from the three sources and turn them into a consolidated subset for South-Holland, so that the province can automate keeping the data up to date.

When I talk about Networked Agency, I talk about reducing the barrier to entry for all kinds of technology as well as working methods, that we know work well in a fully networked situation. Reducing those barriers allows others to adopt these tools more easily and find power in refound ability to act. Networked agency needs tech and methods that can be easily deployed by groups, and that work even better when federated across groups and the globe-spanning digital human network.

The IndieWeb’s principles (own your own data, use tools that work well on their own, and better when federated, avoid silos as the primary place of where you post content) fit well with that notion.

Recently I said that I was coming back to a lot of my material on information strategies and metablogging from 2003-2006, but now with more urgency and a change in scope. Frank asked what I meant, and I answered

“that the principles of the open web (free to use, alter, tinker, control, trust by you/your group) also apply to other techs (for instance energy production, blockchain, biohacking, open source hardware, cheap computing hardware, algorithms, IoT sensors and actuators) and methods (p2p, community building, social media usage/production, group facilitation etc.). Only then are they truly empowering, otherwise you’re just the person it is ‘done to’.”

Blockchain isn’t empowering you to run your own local currency if you can only run it on de-facto centralised infrastructure, where you’re exposed to propagating negative externalities. Whether it is sudden Ethereum forks, or the majority of BTC transactions being run on opaque Chinese computing clusters. It is empowering only if it is yours to deploy for a specific use. Until you can e.g. run a block chain based LETS easily for your neighbourhood or home town on nodes that are Raspberry Pi’s attached to the LETS-members’ routers, there is no reliable agency in blockchain.

IoT is not empowering if it means Amazon is listening into all your conversations, or your fire alarm sensors run through centralised infrastructure run by a telco. It is empowering if you can easily deploy your own sensors and have them communicate to an open infrastructure for which you can run your own gateway or trust your neighbour’s gateway. And on top of which your group does their own data crunching.

Community building methods are not empowering if it is only used to purposefully draw you closer to a clothing brand or football club so they can sell your more of their stuff. Where tribalism is used to drive sales. It is empowering if you can, with your own direct environment, use those methods to strengthen local community relationships, learn how to collectively accommodate differences in opinions, needs, strengths and weaknesses, and timely reorient yourself as a group to keep momentum. Dave Winer spoke about working together at State of the Net, and 3 years ago wrote about working together in the context of the open web. To work together there are all kinds of methods, but like community building, those methods aren’t widely known or adopted.

So, what applies to the open web, IndieWeb, I see applies to any technology and method we think help increase the agency of groups in our networked world. More so as technologies and methods often need to be used in tandem. All these tools need to be ‘smaller’ than us, be ours. This is a key element of Networked Agency, next to seeing the group, you and a set of meaningful relationships, as the unit of agency.

Not just IndieWeb. More IndieTech. More IndieMethods.

How would the ‘Generations‘ model of the IndieWeb look if transposed to IndieTech and IndieMethods? What is Selfdogfooding when it comes to methods?

More on this in the coming months I think, and in the runup to ‘Smart Stuff That Matters‘ late August.

Triggered by some of the previous postings on RSS, I started thinking about what my ideal set-up for RSS reading would be. Because maybe there’s a way to create that for myself.

A description of how I approach my feeds, and what I would ideally like to be able to do, I already penned a decade ago, and it hasn’t really changed much.

The basic outline is:

  • I think of feed subscriptions as subscribing to people. I don’t follow your blog, but I follow and interact with you. I used to have a blogroll that reflected that by showing the faces of people whose writing I read. Basically the web is my social network always, In my feed reader every feed title is the name of the author, not the blog’s title.
    my blogroll in 2005, people’s faces, not site names
  • The feeds I subscribe to, I group in folders by subjective social distance, roughly following Dunbar-style group sizes. The dozen closest to me, the 50, the 150, the 500 beyond that, and above that 999 for people I don’t have a direct connection with at all. So my wife’s blog feed is in folder a12, and if I’ve just come across your blog this week and we never met, your feed will be in e999. The Keep Track folder are my own content feeds from various platforms.
    the folders in my current feedreader by social distance
  • There are three reading styles I’d like my reader to support, of which it only does one.
    • I read to see what is going on with people I know, by browsing through their writing from closer to further away, so from the a12 folder towards the e999 folder. This my reader supports, by way of allowing a folder structure for it
    • I read outside-in, looking at the general patterns in all the new postings of a day: what topics come up, what are people working on, what do they care about. This is not supported yet, other than scrolling through the whole thing. A quick overview of topics versus social distance would be useful here.
    • I read inside-out, where I have specific questions, ideas or topics on my mind and want to see if some of the people in my reader have been wrting about it recently. This is not supported yet. A good way to search my feeds would be needed.
  • I would like to be able to tag feeds. So I can contextualise the author (coder, lives in Portugal, interested in privacy by design, works independently). This allows me to look at different groups of people across the social distance related folders. E.g. “what are the people I follow in Berlin up to this week, as I will be visiting in a few days?” “What are the current concerns in the IndieWeb community?” Ten years ago I visualised that as below

    Social distances with community and multi-faceted contexts plotted on them

  • I would like to be able to pull in tags of postings and have full content search functionality. This would support my inside-out reading. “What is being said today in my feeds about that conference I didn’t go to?” “Any postings today on privacy by design?”
  • I think I’d like visual representations of which communities are currently most active, and for topics, like heat maps. Alerts on when the level of activity for a feed or a community or subsets of people changes would be nice too.
  • From the reader follow actions, such as saving an article, creating a todo from it, bookmarking it, or sharing it in some channel. An ideal reader should support all those actions, or let me configure actions
  • [UPDATED OCTOBER 2018] From reading a posting by Peter Rukavina I realised I’d also like to have built-in machine translation.

From the whole IndieWeb exploration of late, I realized that while no feedreader does all the above, it might be possible to build something myself. TinyTiny RSS seems a good starting point. It’s an open source tool you can run as your own instance. It comes with features such as filtering and auto-tagging that might fit my needs. It can be hosted on my own domain, and it has a database I then have back-end access to, to build features it doesn’t have itself (such as visualisations and specific sharing actions). It can also produce RSS feeds. It seems with TinyTiny RSS I could do all kinds of things to the RSS feeds I pull in on my server, and push the results out again as RSS feeds themselves. Those I could load into my regular reader, or republish etc.

Now need to find a bit of time to set it up and to play with it.

Came across this post by Ruben Verborgh from last December, “Paradigm Shifts for the Decentralised Web“.

I find it helpful because of how it puts different aspects of wanting to decentralise the web into words. Ruben Verborgh mentions 3 simultaneous shifts:

1) End-users own their data, which is the one mostly highlighted in light of things like the Cambridge Analytica / Facebook scandal.

2) Apps become views, when they are disconnected from the data, as they are no longer the single way to see that data

3) Interfaces become queries, when data is spread out over many sources.

Those last two specifically help me think of decentralisation in different ways. Do read the whole thing.