Having created a working flow to generate OPML booklists directly from the individual book notes in my PKM system, I did the first actual run in production of those scripts today.

It took a few steps to get to using the scripts in production.

  • I have over 300 book note files in my Obsidian vault.
  • Of course most lacked the templated inline data fields that allow me to create lists. For the 67 fiction books I read in 2021 I already had a manual list with links to the individual files. Where needed I added the templated data fields.
  • Having added those inline fields where they were missing I can easily build lists in Obsidian with the Dataview plugin. Using this code


    results in

  • The same inline data fields are used by my scripts to read the individual files and build the same list in OPML
  • That gets automatically posted to my website where the file is both machine and human readable.

Doing this in production made me discover a small typo in the script that builds the OPML, now fixed (also in the GitHub repository). It also made me realise I want to add a way of ordering the OPML outline entries by month read.

Lists to take into production next are those for currently reading (done), non-fiction 2021, and the anti-library. That last one will be the most work, I have a very long list of books to potentially read. I will approach that not as a task of building the list, but as an ongoing effort of evaluating books I have and why they are potentially of interest to me. A way, in short, to extend my learning, with the list as a useful side effect. The one for currently reading is the least work, and from it the lists for fiction 2022 and non-fiction 2022 will automatically follow. The work is in the backlog, getting history to conform to the convention I came up with, not in moving forward from this point.

In parallel it is great to see that Tom Critchlow is also looking at creating such book lists, in JSON, and at digesting such lists from others. The latter would implement the ‘federated’ part of federated bookshelves. Right now I just point to other people’s list and rss feeds in my ‘list of lists‘. To me getting to federation doesn’t require a ‘standard’. Because JSON, OPML and e.g. schema.org have enough specificity and overlap between them to allow both publishers of lists and parsers or such lists enough freedom to use or discard data fields as they see fit. But there is definitely a discussion to be had on identifying that overlap and how to use it best. Chris Aldrich is planning an IndieWeb event on this and other personal libraries related topics next month. I look forward to participating in that, quite a number of interesting people have expressed interest, and I hope we’ll get to not just talk but also experiment with book lists.

As a form of WAB* I’ve made it easier for myself to update my OPML book lists. I created those lists earlier this year as a proof of concept of publishing federated bookshelves. Updating OPML files residing on my hosted webserver is not a fun manual task. Ultimately I want to automate pushing lists from my personal working environment (notes in Obsidian) to my site. Given my limited coding skills I made a first easier step and created a webform in a php script that allows me to add a book to an opml list. It has a drop-down menu for the various OPML lists I keep (e.g. fiction2021, non-fiction2021, currently reading, anti-library), provides the right fields to add the right OPML data attributes, and then writes them to the correct list (each list is a separate file).

That now works well. Having a way to post to my book lists by submitting a form now, I can take the next step of generating such form submissions to replace manually filling out the form.

* Work Avoiding Behaviour, a continuation of the SAB, Study Avoiding Behaviour that I excelled in at university. WAB seems to fit very well with the current locked down last days until the end of year. The Dutch terms ‘studie/werk ontwijkend gedrag’ SOG/WOG lend themselves to the verb to ‘sog’ and to ‘wog’. Yesterday when Y asked E what she had been doing today, E said ‘I’ve been wogging’, and I realised I had been too.

TIL: In an xml style sheet, an xslt file, testing for the existence of an attribute is not the same as testing whether an attribute has content.

I have a machine readable file (OPML) with lines such as

<outline type="book" text="title by author" name="book title" author="author name" url="" comment="some remarks" authorurl="" inLanguage="en" />

I want to show that file in a browser for people to see, and I want to test for the attribute url in that line so that I can mark up the text with a web link in the human readable version.

What I first used to test for the presence of a link was:

<xsl:when test="@url">

I assumed this would test for the actual presence of a value for attribute ‘url’. But that is not correct. The statement actually tests whether the attribute ‘url’ is present in the line of machine readable code.

To test if the url attribute also has content in it, e.g. url=”https://weblink.tld/page” and not url=”” I need to check that the field is not empty:

<xsl:when test="@url!=''">

I experimentally create OPML lists of books I read (see why and how). For instance I have a ‘fiction 2021’ list. OPML files are meant to be read by machines, but I use the same file to make a human readable version. In the human readable version I want to link to a book and to an author if I have a direct link to a page about the book (by the publisher or author) and a link to the author’s website. I test for non-empty url attributes so I know when to construct a link in the human readable output. You can download the XSLT I use to do that and of course the OPML file that lists the fiction books for 2021 (also see right hand side column, in the feeds section).

When I wrote about outlining last weekend, I mentioned Dave Winer’s blog being an outline document. Yesterday, in the context of Drummer he referred to a 2013 posting “Two ways of looking at an outliner“. In it he goes into detail how outliners aren’t only creating files (a single outline, saved in a file), but can be viewed as file systems as well. At the end of that posting he talks about how his entire blog is an outline, all stored in a single opml file. When I mentioned how Dave Winer seems to blog by starting an outline each day, I was partly right. He’s starting a new branch (i.e. a day file) in a month branch (i.e. a folder), in a year branch (i.e. a folder), in the entirety of his blog that is a single OPML file.

Drummers by Alper Çuğun, license CC BY

Drummer

Drummer is a new outliner tool for blogging launched last month, created by Dave Winer. As it is popping up in various places, connections are built to other tools (like Microblog), I found myself rolling the topic of outlines around in my head.

I have an somewhat ambivalent attitude towards outlining. Actually I need to split that up in being ambivalent on outlining, on outliners, and on OPML, the standard for exchanging outline files. Some remarks on all three things.

Outlines

Outlines are very useful. I use them for braindumps, idea generation, project plannning and design, and when making the storyline for my presentations. They’re great because you can quickly write out many points and then start shifting things around, changing the order, placing items in branches (or chapters) etc.

Outlines are limited as well, because they are hierarchical and linear in nature. This is similar for me with mind maps. They require a beginning and an end, main or central points with sub points etc. Even though you could link or include other outlines on a branch it still is just another part of a tree structure. A large amount of the information I work with is not like that, and a lot of my work processes (especially those where structure needs to emerge from the information, not to be applied to information) are not like that. Stuff is always linked, but those links can loop back, unlike in outlines. In my description of my personal knowledge management system last year I wrote about the distinction between the parts of it that are organised as networks, and the parts that are hierarchical. This is the same thing, at a lower level of aggregation.

Fast Drummer by Hsing Wei, license CC BY

Outliners

Outliners are great tools, and I use them regularly. When done well you can seamlessly move items around, changing the order, nesting them, or moving them several levels up. Dave Winer talks about moving things around ‘on rails’, and indeed that is what it feels like in a well working outliner, an almost frictionless rearranging of things.

I have used a variety of outliner tools over the years. E.g. I used to make my presentation outlines in Cloudliner, which could sync with Evernote. Once the outline was there, I’d move to Evernote to flesh out the story in more detail.
Currently I mostly work in Obsidian, which isn’t a full outliner in the sense that it lacks the ‘on rails’ features, although by now it is way better at outlining than earlier through the use of hotkeys. (See this video by Nick Milo on outlining with Obsidian)
Tinderbox is also an outliner I use, which is also able to work non-hierarchically: I can start there with adding notes visually, and then switching to an outline view of the same information to do the ordering and branching I want.

Where outliners are less great in my eyes is how they generally imply that all outlines are glorified shopping lists. When writing anything it is about creating prose. Sentences need to flow into each other. Outliners in their interface however suggest that every item in an outline is short. A brief statement at most, definitely not something like a full sentence or even a paragraph. This is where for me friction originates, outliner UIs imply bullet lists. The mental model of a bullet list clashes with that of a text, even if well structured. I don’t think outlines need to be bullet lists, but outliner tools apparently do. Obsidian is the only exception, as it works fully in notes, and you can mix up longer pieces of prose with lists of short items, and have bullets as long as a novel if you want. (This is what I gain for Obsidian not having the ‘on rails’ experience)

Even the original demonstration of an outliner, in Doug Engelbart’s famous 1968 demo, shows that clash to me. It demonstrates the power of outliners: rearranging items at will, moving an item to become a sub-item or a sub-item to become a main item, stacking lists multiple levels deep, as well as adding the link to another outline as item in an outline and it being navigable. But the content of the outline demonstrated is short, a shopping list and a task list.

Drummers by George N, license CC BY

OPML

Dave Winer has been developing outliners for a long time, and we also have to thank him for the OPML standard, meant to exchange outlines in XML. Most people that use RSS readers are familiar with OPML because it is the common format in which you can export and import lists of RSS subscriptions.

Outliners are usually capable of exporting OPML. Most outliner tools however only export to OPML, and don’t store them in files that way or in some other text based format (for the ‘on rails’ effect they regularly keep outlines not as files but in an internal database. Obsidian works on files, and thus doesn’t have the ‘on rails’ style). This means there’s no seamless transition from an outliner tool to another tool, or vice versa, nor a good way to switch between different text based formats such as markdown or OPML. Drummer is an exception, it natively stores everything in OPML files.

Another issue I have with outliners is in how they deal with importing OPML. OPML is an extensible format, which allows adding data attributes to the text information in the outline. This allows me to attach meaning to content that is machine readable. It’s what I did for my book lists for instance. Outliners however in general never attempt to check if such data attributes are present. OPML is short for Outline Processor Markup Language, yet outliners never do some actual processing upon import.

I don’t expect general outliners to be able to do something with such attributes, but I would expect general outliners to at least alert me to their presence in an import, and if possible ask me if I’d like to explore them or do something with them. That general outliners only look at the mandatory parts of the OPML file means they never even look if there’s other semantic information present in the OPML file, though the standard supports it.

Yes, you could create your own outliner tool that reads specific additional data attributes but no regular user would be able to. Tinderbox that I mentioned, allows me to set a wide variety of attributes to notes, and I can create an OPML template in Tinderbox that includes (parts of) them in OPML exports. As far as I can tell though it doesn’t support templating OPML imports. Without this there’s no chance of an OPML using ecosystem evolving, and there hasn’t. The lack of interoperability means novel use cases for OPML always need to come with their own bespoke outliner. This is why I add XSLT to my OPML files for RSS subscriptions and for books, basically packaging a reader right inside the file: it makes them human readable in their entirety and independent from outliners.

Hear The Drummer Get Wicked

I think this is how Dave blogs: He opens up an outline at the start of the day and adds items (thoughts, annotated links, bookmarks, comments) to it during the day which get pushed to his blog. Every line has its own permalink, but it’s a single post for the day, evolving during the day and fixed thereafter. I like the ease of use I can imagine that brings to him.

This is much like how I create my day logs in my personal notes. I open it up first thing behind my laptop in the morning, and during the day it’s my jumping off point as well as where I write things down first. What gets bigger or has more permanent value is then split off in its own note, with the note linked in the day log it sprouted from. (I create my weekly notes on this blog from collating the day logs and then picking the things I want to mention from). I wouldn’t post my day logs though, they’re not fit for publication.

Does Dave also have a personal day log, or is that another branch on his daily outline that doesn’t get published?

The reduction of friction between note making and blogging Dave Winer’s workflow suggests sounds valuable though, and Drummer therefore a tool to watch.


Drummer by Jonas Bengtsson, license CC BY

I assume that in its most basic form I could redo Dopplr of sorts by announcing travel plans in an OPML file, much like book lists or my rss subscriptions. Then it comes down to how to share such travel plans with a known and limited network only. (You don’t want to announce to just everyone when you won’t be home.)

The IndieWeb efforts concerning travel seem to focus on posting actual travel movements, like planned flights. A sort-of check-in style post. The socially shared Dopplr info was much simpler: a city and a set of dates. Because its purpose was aiding serendipitous meet-ups. Exact travel plans or exact location aren’t needed for it, just a way to flag paths more or less crossing to those involved.

Of course making such an OPML file currently is as easy as posting an empty file, as there’s no significant travel during the pandemic.

Theoretically I could use such an OPML file to announce several things:

  • The various cities I consider as home turf, as they’re within easy reach in an hour.
  • Selected cities I’m willing to travel to at short notice outside that hour travel time if there’s a good reason to.
    From where I am a visit to Antwerp, Brussels, Eindhoven would count in that category, or maybe on specific occasions Düsseldorf or Cologne.
  • Upcoming travel plans, things like ‘Copenhagen, Denmark, 4th-7th September’ (actually a 2019 example)

Such a list would allow comparison with your list to see whether any of your travel plans match with my ‘home turf’ and destinations I’m willing to consider outside of it, whether any of your travel plans match with my travel plans, or whether any of my travel plans line up with your home turf and other relatively nearby destinations you’re willing to consider. Cities and countries are part of schema.org vocabularies and as such usable in OPML as data attributes.

I think there’s a space for location based services, such as Dopplr was, that don’t depend on or use maps, but provide location contextualized information that influences my actions, choices and my relationships to my networks (a quote from a 2012 blogpost on moving beyond the map).

Or this is just me applying my current opml hammer to anything that might be a nail 😀


I couldn’t resist making this mock-up mimicking the colorful Dopplr