In reply to Social Readers by James G

Over the years I’ve blogged about what would be an ideal feed reader to me, and also mapped it to how IndieWeb standards might help realise it. In the end it all goes back to how I in 2005 described using feed reading as information filtering, and the inputs, reading and resulting actions it is built out of. That is still my approach, and it is as high friction as it was back then in terms of how well existing readers and tools cater to those needs. Plenty of space for feed reader evolution as you mention!

I would particularly love to hear parts of web readers you like and dislike. If you could build a social reader tomorrow, what features would it have?

James G

Houtje-touwtje, literally ‘small stick-short rope’, or cord loops fasteners in English, is the more down to earth and old Dutch phrase for putting stuff together with duct tape (known as MacGyver tape to many here, after the 80’s tv series). After messing around in the FreshRSS back-end last weekend, yesterday I made a houtje-touwtje working version to let me post more easily about the things I find in my feed reader.

In my previous post I mentioned I want to support three different reactions:

  • favourites, something I liked, or want to let the author know I appreciated
  • bookmarks, something I (want to) read, and keep reference to for my notes
  • replies, something where I comment on or reply to a posting and its author directly

Four labeled posts in my feed reader

The way I mark postings in my feed reader is attaching a label to them (favourite, bookmark, reply)
Then with the SQL statement from the last posting I pull out those labeled postings (label, url, title, author, and content).

Yesterday evening I attempted to bring it a step or two further, but ended up with something that works.
I created a php html page that shows me, using the SQL query, every labeled posting and for each posting shows a webform. That webform is needed because I want to be able to add two things to every labeled posting (though they’re not mandatory)

  • My motivation for favouriting, or bookmarking, or my reply
  • A quote of the bit in the original posting I am picking up on

The form for me to add motivation and selected quote

Next to that the form is needed to forward the other information (link, title, URL) to the next stage, along with my response and quote. This simultaneously allows me to clean up URLs, shorten the title or provide the actual author name if the post only contains a shortened username e.g.

The webform also contains a field for the total number of labeled items (hidden), and a field for the label itself. Again needed to forward them to the next stage of the process.

The second stage is that submitting the form, calls the same php page again, but this time to process the submitted form information (no longer the database query)

Based on the label (fav, bookmark or reply) it selects a template for how I want to show those in my blog (each have their own svg-icon and first sentence), uses the right microformat, and adds in the url, author, title and my reaction and selected quote in the right spots. The microformat is relevant for when my posting sends a webmention to the website I’m reacting to, and lets that site know if it’s a reply (to show underneath in their comments), or a bookmark or favourite. Assuming that the other website can receive webmentions that is.

Resulting HTML

The result of that is then shown to me in HTML. I paste that HTML into my blog editor, add things like posting categories and tags, and hit publish. In essence I’ve replaced what the Post Kinds plugin, that I stopped using, in my WordPress blog did before: automating fetching some info, and adding it into my posting. It is akin to what I had previously, where I weekly posted links to my blog from my Evernote bookmarks using e-mail.

The posting on the blog

This ‘houtje-touwtje‘ set-up doesn’t do the fancy things yet, like posting itself automatically to my blog through micropub (UPDATE January 2022 it now does!), or providing me this functionality within my FreshRSS reader’s interface. Also, after posting things, I need to manually delete the labels, so they don’t resurface the next time I run the php script.

Another path to explore is using the same or similar workflow to put labeled posts into my personal notes.

This morning I tested out the workflow, and I was pleased with the result. I loaded up the feed reader, and did my usual reading. I marked things for follow up. I ran the script, added my motivation and selected quotes. Pasted the resulting HTML into my blog, added categories and hit publish. This definitely feels smoother and less time consuming. It’s been a while I posted more than 1 thing based on my feedreading that day, let alone four (1, 2, 3, 4) within minutes of reading it. The ‘normal’ routine was ending up with a bunch of open tabs in my browser, and at the end of the day giving up and hitting close-all for those I didn’t get around to during the day.

[Update: the php script is available in my GitHub account]

I am working on following somewhat in Peter’s footsteps. He has his FreshRSS reader connected to his weblog, for everything he ‘stars’ in his feed reader. Those get posted to his ‘favourites’ stream, and send webmentions so they end up as ‘likes’ underneath the original posts. See Peter’s description on his blog.

My plans are a bit more ambitious, an ambition that may yet well bump into the limtis of my php / mysql capabilities.

I want to be able to mark articles in the feed reader for three things: bookmarking, favouriting, and replies. Then I want to add two pieces of content, and post that to my site.

When I favourite something on my blog, I basically always add 2 things when posting it: a rationale for why I like something, and a quote from the original article that meant something to me. See the image below, a screenshot of a ‘favourite’ I marked earlier this month.

That post is made of a short template that I now activate when in my blog’s editor with a hotkey (using AlfredApp) that creates the star, and has a place for the URL, title and author, plus my own motivation for posting, and a quote with the author’s name repeated.
Favourites, bookmarks and replies follow the same template. The only difference is the symbol shown, and the microformat used to signal to the original article what sort of webmention I’m sending their way (so they know how to treat it, e.g. as a like or as a comment/reply)

Peter’s example of favouriting by starring leads to setting a boolean field in the content table itself of the database.
FreshRSS lacks three distinct ‘starring’ buttons, but I can easily add labels to a posting. Those labels are stored in a separate table from the feed articles, unlike when starring.

The first step to take then is to gain access to the postings I mark for follow-up during feed reading. I’ve used the following SQL query:

SELECT ton_entry.link, ton_entry.author, ton_entry.title, ton_entrytag.id_tag, uncompress(ton_entry.content_bin) as content FROM ton_entry JOIN ton_entrytag ON ton_entry.id=ton_entrytag.id_entry

The two key differences with Peter’s SQL statement are, the JOIN part, and the uncompress part. The JOIN statement combines the table that knows what labels I applied with the table that contains all articles, and selects only those where an article ID is in both tables, resulting in a list of the articles I applied labels to. Because in a later step I want to select a quote from the source article the SQL statement also grabs the content of an article. That content is stored as a compressed binary blob (yes, blob is its official name) in the database. Using ‘uncompress’ makes the content blob human readable again, and the ‘as content’ bit puts it in a variable called content.

The next step is allowing me to provide my remarks and select a quote through a form, and the third step to add all that in to the right template based on the label (favourite, bookmark, reply), after which I need to put it into my WordPress install in the right categories and publish it.

Drummers by Alper Çuğun, license CC BY

Drummer

Drummer is a new outliner tool for blogging launched last month, created by Dave Winer. As it is popping up in various places, connections are built to other tools (like Microblog), I found myself rolling the topic of outlines around in my head.

I have an somewhat ambivalent attitude towards outlining. Actually I need to split that up in being ambivalent on outlining, on outliners, and on OPML, the standard for exchanging outline files. Some remarks on all three things.

Outlines

Outlines are very useful. I use them for braindumps, idea generation, project plannning and design, and when making the storyline for my presentations. They’re great because you can quickly write out many points and then start shifting things around, changing the order, placing items in branches (or chapters) etc.

Outlines are limited as well, because they are hierarchical and linear in nature. This is similar for me with mind maps. They require a beginning and an end, main or central points with sub points etc. Even though you could link or include other outlines on a branch it still is just another part of a tree structure. A large amount of the information I work with is not like that, and a lot of my work processes (especially those where structure needs to emerge from the information, not to be applied to information) are not like that. Stuff is always linked, but those links can loop back, unlike in outlines. In my description of my personal knowledge management system last year I wrote about the distinction between the parts of it that are organised as networks, and the parts that are hierarchical. This is the same thing, at a lower level of aggregation.

Fast Drummer by Hsing Wei, license CC BY

Outliners

Outliners are great tools, and I use them regularly. When done well you can seamlessly move items around, changing the order, nesting them, or moving them several levels up. Dave Winer talks about moving things around ‘on rails’, and indeed that is what it feels like in a well working outliner, an almost frictionless rearranging of things.

I have used a variety of outliner tools over the years. E.g. I used to make my presentation outlines in Cloudliner, which could sync with Evernote. Once the outline was there, I’d move to Evernote to flesh out the story in more detail.
Currently I mostly work in Obsidian, which isn’t a full outliner in the sense that it lacks the ‘on rails’ features, although by now it is way better at outlining than earlier through the use of hotkeys. (See this video by Nick Milo on outlining with Obsidian)
Tinderbox is also an outliner I use, which is also able to work non-hierarchically: I can start there with adding notes visually, and then switching to an outline view of the same information to do the ordering and branching I want.

Where outliners are less great in my eyes is how they generally imply that all outlines are glorified shopping lists. When writing anything it is about creating prose. Sentences need to flow into each other. Outliners in their interface however suggest that every item in an outline is short. A brief statement at most, definitely not something like a full sentence or even a paragraph. This is where for me friction originates, outliner UIs imply bullet lists. The mental model of a bullet list clashes with that of a text, even if well structured. I don’t think outlines need to be bullet lists, but outliner tools apparently do. Obsidian is the only exception, as it works fully in notes, and you can mix up longer pieces of prose with lists of short items, and have bullets as long as a novel if you want. (This is what I gain for Obsidian not having the ‘on rails’ experience)

Even the original demonstration of an outliner, in Doug Engelbart’s famous 1968 demo, shows that clash to me. It demonstrates the power of outliners: rearranging items at will, moving an item to become a sub-item or a sub-item to become a main item, stacking lists multiple levels deep, as well as adding the link to another outline as item in an outline and it being navigable. But the content of the outline demonstrated is short, a shopping list and a task list.

Drummers by George N, license CC BY

OPML

Dave Winer has been developing outliners for a long time, and we also have to thank him for the OPML standard, meant to exchange outlines in XML. Most people that use RSS readers are familiar with OPML because it is the common format in which you can export and import lists of RSS subscriptions.

Outliners are usually capable of exporting OPML. Most outliner tools however only export to OPML, and don’t store them in files that way or in some other text based format (for the ‘on rails’ effect they regularly keep outlines not as files but in an internal database. Obsidian works on files, and thus doesn’t have the ‘on rails’ style). This means there’s no seamless transition from an outliner tool to another tool, or vice versa, nor a good way to switch between different text based formats such as markdown or OPML. Drummer is an exception, it natively stores everything in OPML files.

Another issue I have with outliners is in how they deal with importing OPML. OPML is an extensible format, which allows adding data attributes to the text information in the outline. This allows me to attach meaning to content that is machine readable. It’s what I did for my book lists for instance. Outliners however in general never attempt to check if such data attributes are present. OPML is short for Outline Processor Markup Language, yet outliners never do some actual processing upon import.

I don’t expect general outliners to be able to do something with such attributes, but I would expect general outliners to at least alert me to their presence in an import, and if possible ask me if I’d like to explore them or do something with them. That general outliners only look at the mandatory parts of the OPML file means they never even look if there’s other semantic information present in the OPML file, though the standard supports it.

Yes, you could create your own outliner tool that reads specific additional data attributes but no regular user would be able to. Tinderbox that I mentioned, allows me to set a wide variety of attributes to notes, and I can create an OPML template in Tinderbox that includes (parts of) them in OPML exports. As far as I can tell though it doesn’t support templating OPML imports. Without this there’s no chance of an OPML using ecosystem evolving, and there hasn’t. The lack of interoperability means novel use cases for OPML always need to come with their own bespoke outliner. This is why I add XSLT to my OPML files for RSS subscriptions and for books, basically packaging a reader right inside the file: it makes them human readable in their entirety and independent from outliners.

Hear The Drummer Get Wicked

I think this is how Dave blogs: He opens up an outline at the start of the day and adds items (thoughts, annotated links, bookmarks, comments) to it during the day which get pushed to his blog. Every line has its own permalink, but it’s a single post for the day, evolving during the day and fixed thereafter. I like the ease of use I can imagine that brings to him.

This is much like how I create my day logs in my personal notes. I open it up first thing behind my laptop in the morning, and during the day it’s my jumping off point as well as where I write things down first. What gets bigger or has more permanent value is then split off in its own note, with the note linked in the day log it sprouted from. (I create my weekly notes on this blog from collating the day logs and then picking the things I want to mention from). I wouldn’t post my day logs though, they’re not fit for publication.

Does Dave also have a personal day log, or is that another branch on his daily outline that doesn’t get published?

The reduction of friction between note making and blogging Dave Winer’s workflow suggests sounds valuable though, and Drummer therefore a tool to watch.


Drummer by Jonas Bengtsson, license CC BY

I have been using FreshRSS for a while as reader. I am interested in adding additional functionality to it. E.g. pushing entries from my reader to my clippings in Obsidian, or like Peter does, being able to push the things I favourite to my blog. Basically I am after the ‘social reader‘, where I respond to things and collect things directly from my reader. However I’m nowhere near capable enough to roll my own stuff building on existing IndieWeb tools. For now I’m therefore trying to add a few simple things on top of the reader I already use.

However it turned out, my FreshRSS set-up on a domain I have didn’t even have all functionality working properly. For instance starring an item results in a call to a URL that runs afoul of the filtering rules of my hoster. It gets blocked.
My feed reading is always ever on my main device, meaning I don’t really need to have it on an external webserver. Today I created a local install on my laptop, using the laptop’s (MAMP) webserver. Now everything works, and as a bonus I have direct access to the database behind FreshRSS which means I can do things as Peter describes in his posting linked above, run local queries on the database.

I should also try again to get Aperture running locally once more. That somehow never really worked out yet. The upside of Aperture is that it can parse not just existing RSS feeds but also other things as feeds.


My FreshRSS instance on localhost