Webmentions is what makes it possible for me to write here about someone else’s blogpost and have my response show up beneath theirs. And vice versa. Earlier mechanisms such as pingback and trackback did the same thing, but slipped under the radar or succumbed to spam. Webmention is a W3C recommendation.

The webmention itself is simple
The core of webmention is straightforward: if I write something here, my webserver will try to let every site I link to in my text know I link to them. This by checking if the sites I link to have an ‘endpoint’, an antenna basically, for webmentions. If a site does, then it will send a simple message to that antenna stating two web addresses, the source (here my blogpost) and the target (here your blogpost). When your site receives a webmention it will do some checking: does my source blogpost indeed link to your target address?

What happens next is less simple
It can quickly get confusing during what happens next.
When my site receives a webmention (this source x links to your target y), all it knows is just the URL of a page that links to me. What my site displays and how it displays that as a consequence of a webmention message depends on multiple factors:

My server will try to read the source blogpost, and see what machine readable information it contains, and what it can know about the source blogpost. These machine readable parts are in the form of microformats.
My server will store some of the information it finds.
Then my website template will show some information from what the server stored when showing the target blogpost on my site.

How well that works depends on multiple factors therefore:

  1. The available machine readable info in the source blogpost, and whether that info is properly encoded
  2. The settings of my server for what it stores
  3. The settings of my site template for what it shows

When something seems to be going wrong, it could be a problem with your site, my site template or my server settings, and it is never obvious which one it is, or if it is the aggregation of multiple issues. It also depends on how easy it is to alter any settings whether you can repair or change things when webmentions are not properly dealt with. Supposedly the Webmention and Semantic Linkbacks plug-ins I use should take care of those issues but it is not obvious that they indeed do.

An example, me and Frank’s sites webmentioning each other
Frank Meeuwsen and I have been mentioning eachother several times and we’ve seen some strange webmention behaviours. For instance in one case Frank’s blog displayed not just a short part of my posting mentioning him, but my entire page including header, footer and sidebar. Clearly something wrong, likely with some of my machine readable encoding, but maybe also something wrong on his end. I suspect my machine readable encoding is indeed faulty but there’s no clear way in which I can change how my webmention plugins deal with that. And if I alter the code, which I could, it is likely the next software update will simply overwrite it.

Yesterday Frank posted about the puzzle webmention is to him in Dutch. Here are some screenshots on how pieces of that puzzle look on my end of things.

Frank’s posting lives at http://diggingthedigital.com//Waar-te-beginnen-met-Webmentions/ In his posting he refers to a posting on my site. He did not send a webmention. But I can do that myself, using a simple form at the bottom of my posting (visible at the bottom of this page too). In that webform I pasted the mentioned url, and that sends the simple webmention message. That message has been received and stored on my server, with the correct source and target address and a timestamp:

What ended up underneath my posting is:

Or as it looks for me as the site’s owner:

A few things stand out:

  • There’s no link to the actual blogpost by Frank (the source), just to his general domain
  • There’s a link to news.indieweb.org, which is a completely different domain
  • There’s no image of the author or an avatar in absence of an image
  • There isn’t any content from Frank’s post shown as part of the mention

So what’s happening? Is this an issue at Frank’s end, is it an issue with what I store on my server, or what I show in my site template? One, two, all three of them?

Puzzling over the pieces in this example

The missing avatar. My site tries to look for an avatar in the source, and if there isn’t one, it shows a general one. Here neither happens, it’s just a blank space. The HTML source of my page reveals it does try to show an avatar, the one that Frank sets in his own blog page as the one to use. His site says in the source code:

<a href="/" class="site-avatar"><img src="/images/dtd-avatar.png" class="u-photo" /></a>

The micro format u-photo is interpreted correctly by my site, and it tries to show the linked image. When you go to that image in your browser it works, but if you try to embed it in your own page it doesn’t.

Frank’s image should be visible below this line,
Frank's avatar
and above this one, but it isn’t.

Probably Frank’s web server prevents bandwidth theft by sending back a white pixel and not the requested image.
[UPDATE] The issue, as Sven points out in the comments, is that this site is https and Frank’s is http. My browser is set-up to reject http material on an otherwise https site. A case of my browser being my castle.[/UPDATE]
Making the avatar fail because my site doesn’t try to store the avatar locally.

The link to news.indieweb.org and the absence of a link to the actual blog post by Frank. The source (Frank’s blogpost) was sent and received correctly as we saw. In the machine readable part of Frank’s site a value is set as ‘canonical’ address for his blogpost.

There is an extra / in that url, and I’m not sure what that might cause, but on my end the canonical that gets saved is very different, it’s that indieweb address.

The odd bit is that indieweb.org address is not mentioned in the source of Frank’s page. At the same time, it seems it isn’t unique to my server, as underneath a posting about webmentions by Sebastiaan Andeweg you see the same thing happening. Frank’s webmention from May 12th shows the indieweb link (and no avatar). Sebastiaan doesn’t use WordPress or the plugins I use as far as I can tell.

So where’s the actual link to Frank’s blogpost? The canonical URL Frank’s posts provides is stored on my server, in the database table for comments as the URL for the author. The indieweb URL however is stored as canonical URL in the comment metadata table in my database. And that gets used for displaying the webmention underneath my blogposting.

The same is true for the absence of the content of Frank’s mention of me. It is collected and stored in the comment table of my site’s database. Yet what is shown underneath my blogpost as mention is constructed only from the comment meta data table, and not the comment table.


Frank’s mention’s content is in my comment database, yet not shown


The metadata fields stored for Frank’s mention in my database

So what’s happening here is a mix of elements from Frank’s site, my webmention plugins and my site template. But how to influence the behaviour of my plugins without seeing that undone with the next update is not clear to me at this point. Nor is how to alter the plugins so I can improve the machine readable microformats on my site.

I’ve finished building an AppleScript for automatically creating a Suggested Reading blogpost from my Evernote bookmarks quicker than I thought.

Mostly because in my previous posting on this I, in an example of blogging as thinking out loud, had already created a list of steps I wanted to take. That made it easier to build the step by step solution in AppleScript and find online examples where needed.

Other key ingredients were the AppleScript Language Guide, the Evernote dictionary for AppleScript (which contains the objects from Evernote available to AppleScript), the Evernote query language (for retrieving material from Evernote), and the Postie plugin documentation (which I use to mail to WordPress).

In the end I spent most time on getting the syntax right of talking to the WordPress plugin Postie. By testing it multiple times I ultimately got the sequence of elements right.

The resulting script is on duty from now on. I automatically call the script every Monday afternoon. The result is automatically mailed to my WordPress installation which saves it as a posting with a publication date set for Tuesday afternoon. This allows me time to review or edit the posting if I want, but if I don’t WordPress will go ahead and post it.

There is still some room for improvement. First, I currently use Apple Mail to send the posting to WordPress. My default mail tool is Thunderbird, so I had to configure Mail for this, which I had rather not. Second, the tags from Evernote that I use in the title of the posting aren’t capitalised yet, which I would prefer. Good enough for now though.

I’ve posted the code to my GitHub account, where it is available under an open license. For my own reference I also posted it in the wiki pages of this blog.

The bookmarks to use as listed in Evernote..


…and the resulting posting scheduled in WordPress

This is an AppleScript I wrote to automatically post weekly overviews of bookmarks I made in Evernote, as a Suggested Reading post on my WordPress blog.

set singletaglist to {}
set multitaglist to {}
set alltaglist to {}
set allbulletlist to {}

--get the notes from Evernote
tell application "Evernote"
set query_string to "notebook:\"linklog\" created:day-7" --get notes since last week
set notities to find notes query_string --gives you list of relevant notes
set notenum to count of notities
repeat with notitie in notities
--for every note fetch title, tags and source
set titel to title of notitie
set bron to source URL of notitie
set taglist to tags of notitie
set numtags to number of items in taglist
--the taglist needs to be processed
repeat with counter from 1 to numtags
set tagtest to name of item counter of taglist
--if tag already encountered it goes to multitaglist
if (number of items in alltaglist is 0) then
copy tagtest to end of alltaglist
else
if (alltaglist contains tagtest) then
copy tagtest to end of multitaglist
--if not encountered it goes to all tags
else
copy tagtest to end of alltaglist
end if
end if
end repeat
--fetch content and cut down to first line
set inhoud to ENML content of notitie --gets you en xml, we want text between and


set AppleScript's text item delimiters to "" --text to the right of
set tinhoud to text item 2 of sinhoud
set AppleScript's text item delimiters to ""
set inhoud to ""
set sinhoud to ""
set thisbullet to {titel, bron, tinhoud} --list of this particual ever note
set end of allbulletlist to thisbullet
end repeat
end tell
-- we got what we need from Evernote
-- we now have a list of all items, but need to look at tags to create the title of post
-- I want a title like Suggested reading: tag1, tag2, tag3 and more
-- where two tags are used more than once and one just once
-- alltaglist contains all tags used, multitaglist contains all tags occurring multiple times
-- now create a list of tags that are used once (basically alltags minus multitags)
set allnum to number of items in alltaglist
set doubles to number of items in multitaglist
repeat with counter from 1 to allnum
if (multitaglist does not contain item counter of alltaglist) then copy item counter of alltaglist to end of singletaglist
end repeat
set singles to number of items in singletaglist
--I need 3 tags for the title, two used more than once, and one single. Unless there's not enough.
set posttitle to "Suggested Reading: "
if (allnum < 4) then -- too few tags, use them all repeat with counter from 1 to allnum - 1 set posttitle to posttitle & item counter of alltaglist & ", " end repeat set posttitle to posttitle & item allnum of alltaglist & " and more" else -- at least four tags available set takedouble to 2 --default value if (doubles < 2) then --take from doubles what possible set takedouble to doubles --0 or 1 end if if (singles = 0) then -- take all from doubles set takedouble to 3 end if --in all other cases default value works -- with value of takedouble now build posttitle if (takedouble = 3) then -- take first & last, and in the middle from doubles set middlelist to round doubles / 2 rounding up set posttitle to posttitle & item 1 of multitaglist & ", " & item middlelist of multitaglist & ", " & item doubles of multitaglist & " and more" end if if (takedouble = 2) then -- take first & last from doubles, first from singles set posttitle to posttitle & item 1 of multitaglist & ", " & item 1 of singletaglist & ", " & item doubles of multitaglist & " and more" end if if (takedouble = 1) then -- take first from doubles, first and last from singles set posttitle to posttitle & item 1 of multitaglist & ", " & item 1 of singletaglist & ", " & item singles of singletaglist & " and more" end if if (takedouble = 0) then -- take first middle & last,from singles set middlelist to round singles / 2 rounding up set posttitle to posttitle & item 1 of singletaglist & ", " & item middlelist of singletaglist & ", " & item singles of singletaglist & " and more" end if end if -- we now can start building the blog posting -- title is available posttitle -- now let's build the html for the posting --first opening line and start of UL set blogpostext to "

Some links I thought worth reading the past few days

    "
    --for each bullet in bulletlist a new LI
    --with the blurb, and the title as href to the link
    repeat with listitem in allbulletlist
    set thislisitem to "

  • " & item 3 of listitem & ": " & item 1 of listitem & "
  • "
    set blogpostext to blogpostext & thislisitem
    end repeat
    --end the ul started at the top
    set blogpostext to blogpostext & "

" --we got it all now
--mail it to wordpress cat linklog, tags all tags, title and body.
--the content of the mail is determined by the Postie plugin settings
--the plugin used in my WordPress
-- cat and title in subject, rest in body: delay to post must come first, then body, then tags after one empty line
set mailsubject to "standard//[Linklog] " & posttitle
set tagtext to ""
repeat with counter from 1 to allnum - 1
set tagtext to tagtext & item counter of alltaglist & ", "
end repeat
set tagtext to tagtext & item allnum of alltaglist
set mailbody to "delay: 1d
"
set mailbody to mailbody & ":start " & blogpostext & "

tags: " & tagtext --don't use :end delimiter

--now send it as email
tell application "Mail"
set myMessage to make new outgoing message with properties {sender:"your@mail.com", subject:mailsubject, content:mailbody}
tell myMessage
make new to recipient with properties {address:"yoursecret@blogmail.com"}
end tell
send myMessage
delay 5 --wait for the mail to be processed
quit --quit Mail as I don't use it for anything else
end tell

I added a page-based section to this blog, to serve as a wiki-like extension. Where blogs are a stream of content, I find I have need of a more static part of the site, with content that can serve as reference, as a jump-off page to blog content, or to document things.

In the 00’s I used to have a wiki living alongside this blog, and think of ways of connecting my blog to a wiki (in 2004 I wrote a WordPress and a Movable Type plugin to let blogposts and wiki-pages synchronise). The wiki I ran was wikkawiki, which based on functionality would still be my goto choice for an open source self hosted wiki.
The issue with running a wiki exposed to the public was that it attracted loads of spam attacks, something that in practice never was outweighed by the use bona fide visitors made of the wiki to alter or add content.

In short to add wiki-style functionality to my blog, the only functionality that is really needed is that 1) I myself have a edit button on static items, 2) the ability to categorise and tag those items, and 3) keep those items outside of the blog posting stream on the front page, and outside of the RSS feed. WordPress pages fit that description, when I’m logged in, and after adding a plugin to allow categories and tags on pages. So a page based section it is, or rather, will be over time.

Following up on yesterday’s posting on blogging more, I looked at using the WordPress desktop and Android apps. This to see if using those apps makes it easier to blog something on the go (triggered by Peter’s comment that enabling mailing entries to his blog helped his workflow.)

It turns out that I can’t connect to this blog from those apps. WordPress is designed to build the connection using its own plugin Jetpack. I’ve been using Jetpack for visitor statistics already, and previously noticed how the statistics function was the only bit that ever worked. Jetpack needs the xmlrpc file that allows remote access to work. While that file exists in my install and responds as if its active when accessed (“XML-RPC server accepts POST requests only”), in practice it does not seem to be functional. Running the Jetpack debugger to test xmlrpc returns an error message.

None of the suggested fixes by Jetpack help, like disabling all plugins to see if there’s a conflict with Jetpack, and I already am running a default theme. As a final measure they suggest to disconnect Jetpack from the Jetpack settings, but that does not work…..as Jetpack says it can’t save settings. Which was the issue I started with, so I’ve come full circle without a solution.

[UPDATE: my hoster confirms they’ve disabled external calls to xmlrpc as they see it as an attack-vector]