After creating a Micropub client, so that I can post automatically to this site from a script, I decided to make the next step: creating my own Microsub client. What that is, and why I want it I’ll discusse some other time. But quickly after beginning that I realised a key first step needed to be solved first: getting the right authorisation to access my site.

My micropub client also requires that authorisation, but I circumvented doing that myself by using Jamie Tanna’s online tool to create the right access token to gain entry to my site. However, Jamie’s tool sets the scope of the permissions to ‘draft’, meaning you can save new posts as draft. That was fine for micropub, but isn’t useful for microsub.

For microsub I need a different scope, read and follow at first. So instead of starting to build a Microsub client, I first needed to create something to get the right access tokens. Luckily somewhere at the end of the Microsub documentation there’s a summary of the steps you need to take, and once I read that I could imagine a solution, and could recognise the different steps involved. I sketched them out for myself:

I then started to code each of those steps in PHP. In the end I needed about 30 lines of code.
It’s not a fancy script, just one for personal use, so it needs a bit of cleaning before I can share it.

The 4 steps I coded were:

  • Create a URL of the IndieAuth endpoint listed in the site to visit, that contains various pieces of information
    https://domain.com/indieauthendpoint?me=me&responsetype=code&client_id=mydomain&state=a_number_you_make_up&scope=the_access_you_ask&redirect_uri=where_to_send_answer_on_client_id
  • Make the script able to receive the answer to that at the given redirect_uri:
    http://redirect_uri?code=the_secret_code_you_asked&status=the_made_up_number_you_sent
    The returned status ensures that the response belongs to the request you sent. The code is what you need to use for the next step.
  • For the next step you send a POST request (basically it’s sending in a webform) to the Token endpoint provided by the site, which contains the code you received, grant_type with the value authorization_code, the same client_id you sent earlier, and again a redirect_uri where to receive the answer. That last one can be different form before but must be on the same domain.
  • What you get back is some JSON data that contains the access token. For each interaction you have with the site after this step you use that access token to get access to the site.

Where the notion of authorising myself with some website doing some digital variation of a multistage secret handshake for a long time has seemed daunting to me to create from scratch, once I was able to see the different building blocks it became simpler: each of those building blocks, visiting a URL, reading a response, sending in a webform, and reading JSON data I had written before, so I had some PHP code to re-use.

Two years ago I wrote about the features I’d like to see in an ideal feed reader. Today I’ve tried to sketch out the various components, based on the various IndieWeb protocols, and then match the features I listed previously to the component that I think should provide it.

Separating feed reading into IndieWeb components

Feeds from blogs, to the left in the sketch below, end up in a microsub server, whose function is to fetch and store feeds. A microsub client is what presents those fetched feeds to me. A regular feed reader usually does both these things, but splitting them like the IndieWeb does allows multiple clients to use the same collection of feeds and feed items. It allows a wider diversity of readers, if the fetching is dealt with separately.
The microsub client also contains a micropub client. This allows having action buttons underneath each item presented in your microsub client. Hitting a button posts your reply or remarks, or shares something to your own website, and through your site to e.g. Twitter or Mastodon. Ideally it would be possible to have different websites to share to, next to storing locally in a specified format. The website that receives such an action has a micropub endpoint, and may need authentication through IndieAuth.

Such a reader set-up should be able to run locally on my laptop, as well as on a basic hosting package, so likely php / mysql based. Locally because I want to run my stuff local first, but the same thing online, even if a home server, because I’m not always working on my laptop and would like access from my tablet too, and point my Android Indigenous app to my subscriptions. Locally I would not need authentication from the microsub client to the microsub server, but in all other situations I would.

The sketch above is completely congruent with how I sketched my information gathering and filtering process in 2005:

filter1.jpg

Matching the ideal functionality to IndieWeb components

When I match the features of my ideal feed reader to those various IndieWeb components I think this is what results:

Microsub server needs to be able to :
* use various kinds of feeds (rss, atom, json, h-feed)
* allow folders (so I can arrange feeds on social distance)
* recognize and store tags if feed items have them
* allow me to tag _feeds_, really meaning tagging authors
* keep track of posting frequency, last post seen of feeds
* keep track of tags or predefined topics mentions/frequency
* pull in machine translations by default for certain feeds and store them with the orginal item

Microsub client needs to be able to:
* present the feed items in the server’s folder structure as a long list (the classic feed reader view)
* present views based on patterns in current feed items: what’s hot, what’s unique? Also set against social distance. (Topics discussed in my communities today)
* present views based on feed tags (show me all Germany based blog items of this morning, show me every feed of from EU based coders)
* present views based on feed tags and item tags: show me Germany based blog items talking about topic X.
* show full text search results of all items mentioning a certain topic.
* store full text search queries
* present visually which topics seem to be hot in which community, or where the frequency of mentioning a topic has changed
* provide a search of feeds (not feeds content): do I already have this feed in my list, where’s the feed of author Y?
* pull in a machine translation on request

Micropub client needs to be embedded in the microsub client and should support:
* saving an article as markdown or as html, to disk, to Zotero
* creating a todo from it by amending a textfile,
* bookmarking it, either to my blog or some other target
* sharing something about it to my blog, to Mastodon through my blog
* replying to it, through my blog to Mastodon, Twitter, other blogs
* allow configuring new actions.
* choosing from multiple micropub endpoints

What do you think, should some of these features be provided by other components than they’re currently listed? Are features missing that you’d like to see in your ideal feed reader?

Boris wrote that iA Writer has support for Micropub now. That sounds interesting, as it would allow me to write locally, and publish to several of my sites, both online and on my local system.

Somehow I can’t get the iA app to talk to my IndieAuth endpoint yet. Not sure what the issue is, but duckduckgo-ing the error message seems to indicate it has to do with some system settings on my laptop?

The resource could not be loaded because the App Transport Security policy requires the use of a secure connection

Bookmarked Private posts: the move of the checkins (by Sebastiaan Andeweg)

Private posts is something I’d like to have too. In WP it is possible, by having posts you need a login for. Finding a way to smooth that, which doesn’t require me to have other people having an account here, would be great. Automating IndieAuth access looks like a viable path.

However, private posts is just a first step in my mind. On my wish list is a deeper form of allowing selective publishing: private elements in otherwise public postings. Where one site visitor might read ‘my daughter’, friend might read her name. Where other read ‘a client’, colleagues would read the organisation’s name. Building a smooth spectrum from fully public to fully private. Along the lines of how we in conversations also continuously switch between different degrees of disclosure, and not just between conversations.

Recently, the call for private posts became louder again. Aaron Parecki is trying to get a group of people together to exchange private posts between Readers. I would like to be one of them…

Sebastiaan Andeweg

A great effect of spending a day in the same room with 20 or so more geeking inclined others, is you get a lot of examples, tools and services mentioned. And geek is as geek does, I try them out on the spot. Today this helped me become aware that something is wrong on my server with the OAuth authentication I run. I thought that it was working fine, as it is no problem to actually use it, for instance to log in with my own domain name at the IndieWeb wiki. But when interacting with my micropublishing endpoint not all goes well.

Today I noticed that:

  • When I try to post from Micropublish.net, I can log in at micropublish.net, but when I try to post I get an ‘unauthorized’ error
  • When I try to use the Omnibear Firefox add-on it authorises ok, but then endlessly tries to load the list of syndication targets
  • When I use Quill to post, it posts fine, but does not load the list of syndication targets

Those missing syndication targets (now that I understand what they are from todays sessions) was what first caught my eye. Testing the micropublish endpoint on my server myself I got the correct response, but Quill turned out to get ‘unauthorized’ as response for that request, just like micropublish.net got for posting.

The endpoint gives a correct response

In WordPress my IndieAuth plugin has a diagnostic tool, and running that, it turns out an authorisation header is not send out.

Which seems to be causing the problems. Reading in the links provided it seems like with XML-RPC, my hoster is actively blocking that header. [UPDATE: It is not, it’s just not available in the way the server currently runs PHP] Resulting in exactly the same experience as I had with XML-RPC, that it seems to be only half working (namely the ‘safe’ uses work, while the rest fails). There’s a work around, renaming the headers that get send out, and implementing that work-around is a thing for me to do tomorrow. To see if I can get around being unauthorised. [UPDATE: That workaround did not work until now]

I plan to dedicate some learning time in the coming 12 weeks to better understand the protocols that drive the independent web, or IndieWeb. During our STM18 birthday unconference Frank Meeuwsen presented his experiences on the IndieWeb. Frank, Peter and I have formed an impromptu triade to explore the IndieWeb in the past months. In one of his slides Frank conveniently listed the relevant protocols. I’ll plan for 24 hours to explore 6 protocols. Some of them I already understand better than others, so I’ll start with the ones I feel less knowledgeable about.

The ones I want to explore in more detail, in planned order, are:

  • ActivityPub / OStatus, a decentralized networking protocol (as this ties into my Mastodon experiments as well, this comes first)
  • Micropub, publish on your own domain with 3rd party tools
  • Microsub, own your feed-subscriptions (although I already run my own TinyTinyRss instance)
  • Microformats, markup for data, text, people, events (already used on my blog, but curious to see how I can extend that to more types of data)
  • Indieauth, federated login protocol to sign in with your own domain on other services (already active on my blog, but interested in where else I could use it)
  • Webmentions, respond to a blogpost through your own site (already active on my site, but strongly wish to better format and style it on my site)