I want to make it easy to publish lists of books I am reading and have read, or any other list. And do so without using centralised platforms like e.g. Goodreads (Amazon). A book list is a small library.
The route I am currently on, is publishing a machine readable list others can easily incorporate. These lists are in OPML, an exchange format for outlines. It’s the same format generally used to share lists of RSS feed subscriptions.
Current situation and usage: automated lists
Currently I am able to directly automatically create the lists in OPML from my individual book notes in Obsidian.md (which I use for PKM).
In Q1 2022 I experienced that creating lists and posting them works nicely and smoothly, with no friction. I do currently only create a few lists (fiction and non-fiction in the running year, antilibrary). I’m also working through the books I’ve read in the last decade or so, and gradually creating those lists. I’m not generating those as OPML however, they currently are just a list in my own notes.
Next steps: consuming other lists
Next steps will look at how to do the federating itself: how can I ‘consume’, or even include in my own lists, the OPML, ActivityPub or JSON lists of others in a meaningful way? I think a first step is consuming one list published by someone else, treating it as a recommendation list perhaps or some other form of input, much like I’m reading feeds. It might be useful to be able to pick out mentions about books I’ve already read, are in my anti-library, match an author I like, or match my interests while being unknown to me. I suspect a slightly tweaked parser for every new list might be needed, as using a list depends both on format and on content fields.
Ealier steps: proof of concept and data model
In 2020 I came across a posting by Tom Critchlow on this topic, and a year later I started looking into using OPML to create the lists.
I created a proof of concept, with a data format.
Using that I created a webform to update a book list by hand with a new entry.
Then I automated generating the lists (code on GitHub).
All as proofs of concept.
This is the frontpage of my emerging wiki-like collection of semi-permanent content. Where blogposts form a ‘river’ of items, for reference it is useful to have a range of more static ‘pools’ of content. Both to provide additional context and background to blogposts, as well as a useful documentation in itself. Documentation of ongoing work, reading, research, or experiments. (April 2018).
Topics
Networked Agency
Ethics by Design
Indieweb
Information strategies and PKM
Site tweaks
Linqurator bookmarking tool
Bringing Slides home / self-hosting my presentation slides
Federated bookshelves
I’ve read something, so then what?
How and when do I turn what I read into notes, for learning, for retention, for use elsewhere? Having at least some system for this I found is crucial to have somewhere to go with the ideas, associations and thoughts that reading non-fiction creates. I know this because how the absence of a system to take note of my reading feels. For an extended time at some point I did not read any non-fiction simply because I had nowhere to go with the material I learned about, no application, no outlet, and the ideas plus the urge to do something with them kept swirling around in my head creating noisy chaos and frustration.
Having some system is key therefore. What I describe below is my current incarnation of a system, it shifts over time, tools come and go, and therefore it is a snapshot. At the same time my general information strategies and personal knowledge management haven’t fundamentally changed in 20 years. The source of any continuity or consistency is me, not my tools. That said, none of what follows is very strict, it is meant to be forgiving. The source of inconsistency is also me, not my tooling. Not much of it is blind routine. Yet it is way more than doing nothing. My notes create a ratcheting effect, not allowing movement backwards, which is very valuable to me.
Inputs to outputs and the role of friction
Previously I described the flows of various reading inputs to my main tool in working with notes, Obsidian (my entire Obsidian set-up I described in Oct 2000), and the few outputs that flow from them. The image below gives an overview (click for larger version).
The specific steps and tools mentioned in the image are of less importance here. Key is the locus and role of friction in the depicted flows. There is friction in getting material, both source texts and my annotations for them, into my note making tool. This friction I seek to reduce, making it easier to get material to the place where I can work with it. Similarly I’ve reduced friction in getting outputs into the world (my blog, client websites, book lists for instance). Where I don’t seek to reduce friction is in working with notes (expect perhaps for functionality like search), because there friction is the actual work, where the thinking, rewriting, rearranging etc is happening. There is no way around friction there, because it is how I add value to my notes, how I learn and remember things.
The rest of this posting focuses mostly on what happens in the middle section, where the work is, between the inputs and outputs: what do I do there with annotations from what I read?
Web articles via browser
I use a markdown clipper in my browser to save web articles directly as markdown in a folder NewClippings that is part of my notes vault in Obsidian. Usually these come from my feedreading (over 400 individual blogs) or clicking links in those blogs. Saving is not a postponement and promise to self to ‘read later’ which never happens. It is the result of a curation decision, an intentional step, after skimming the web article. I skim looking for suprisal, and I jot down specific reasons for saving it with the web article. This way my future self will know why I was interested in the article originally. Those reasons may be a novel insight in the article, associations I make with other notes I have etc.
Two examples:
I currently have some 825 clippings, of which about 150 are my own old blogposts recycling my earlier writings that way. I do not intend to always process them into other more permanent notes, I don’t treat it as an inbox. It is a repository in its own right, that I can search for additional material. I do pick out clipped webpages for processing, if that is logical from what I’m doing at some point in time.
Whenever I do process an article, it means deleting everything that I don’t think is interesting from my point of view, and summarising and paraphrasing the points I do find interesting. At the top, where the original reason for saving an article is, I keep track of the status of that process. The reworked article remains in the same folder as the other saved articles during this process. If I lift specific notes or notions from the article that can stand on their own, those will link to the original web article by url as source, and end up in one of three notes folders (one for conceptual notes, one for actionable ideas, one for more factual notes and examples) where they are woven into the wider collection by linking. Where that lifting is done I delete the original article from the clippings folder.
Scientific Articles and PDFs via Zotero
I use Zotero to keep a library of scientific articles and other PDFs (e.g. European Commission legal documentation). My motivation for saving them is stored as remark with the material in Zotero. When I save something I usually mention it in my Day logs in my notes, so that I can stumble across it again for processing.
I read those PDFs and highlight and comment in the PDF itself.
Until last week’s Zotero update I could easily grab those highlights and notes using a plugin and save them in markdown into my Obsidian notes vault. The update broke the plugin, so that flow is temporarily out of order. Any highlights and annotations in such a markdown file have a link to the corresponding location in the PDF in Zotero, meaning I can directly jump to the source.
Then, like with web articles I summarise, paraphrase and connect to other notes in steps. Notes/notions that stand on their own, contain the links to Zotero sources. I write, link and fill those stand alone notes from within the annotations first. Once I’ve taken out all I want from an article I delete the annotations note, because all created individual notes link to their source in Zotero. Currently some 10% of learning related notes link to a source in Zotero, after two years of using Zotero.
Books via Kindle
I don’t nearly read the amount of non-fiction I’d like. So none of this is ‘routine’ but it is what has emerged so far as workflow.
When I start reading a non-fiction book, I create a note that serves as the place for thinking and processing what I read. I prefill that note with a template that contains some datafields for my book lists and provide a structure of questions to explore the book before reading. What do I think it is about, what seems the author’s purpose with this book, why am I interested in reading it, what kind of surprisal am I after? What do the different chapters discuss, which ones seem most interesting to me and why? This all before I start reading parts of the book.
While I’m reading I follow two ways of making notes:
I make notes alongside reading the book, and put them in the note I created for the book. These are descriptions and summaries of key ideas, but also associations and links to other things, questions that arise while reading. Basically my half of being in dialogue with the book. This is what I’ve done exlusively until recently.
Recently I started using the Kindle sync plugin for Obsidian, which grabs all my Kindle highlights and annotations and puts them in a markdown note in my Obsidian vault, including links to the right paragraph in the book. I link that note with annotations to the note I made about the book. I never was a big highlighter / annotator in my e-books because of the difficulty of doing anything with it, but thanks to this plugin removing the barriers I started annotating and highlighting much more intensively.
In the book note that I created before reading I then work through the things I highlighted and annotated. Thinking about connections, contrasts etc, and linking to existing notes accordingly. It is also where I start paraphrasing and writing snippets that can become notes in their own right. The book note is the jumping off point for it. When I’m done with a book, the book note will have the links to the notes it brought forth, and the material that I didn’t in the end use for new notes. I keep the book note, and I keep the note with the highlights and annotations (it would resync from Kindle anyway), so I can always trace a note back to the book note and the location in the book itself.
Handwritten notes
I write some reading notes by hand on my BOOX Nova 2 e-ink device, as well as hand written marginalia, but I don’t have an easy flow bringing those to my notes in Obsidian yet. They are stored as PDFs on my e-ink device, and I need to bring them manually into Zotero, from which there then is a working flow to my notes.
This friction on the input side hinders regular use.
Over the years I’ve filled many note books by hand, not just with reading notes, but also annotations of talks, conversations and any other things I jotted down. Recent notes from the past day(s) I usually go through as needed and transcribe into my digital notes. I add them to my Day Log notes, which then can be a jumping off point to create additional individual notes.
Older notebooks, I have in the past scanned a lot of the key material, and have more recently scanned a few note books entirely. In my digital notes I have created an index file for each scanned notebook, where next to a link to each scanned page there’s a brief description of the content, and perhaps a link to relevant other material. That makes it possible to stumble across those notes in search.
Invitation to share
This post describes how I currently make notes from things that I read or wrote. It is a transcription and adaptation of a presentation I gave on April 3rd to the Micro.blog Readers Republic (video), an informal group of book readers around the world meeting every month for conversation. The question of note taking and learning came up as ‘So you’ve read a book, and then what?’ and a few of us volunteered to show what we do. I’m always interested in how other people organise their work, and I think that requires I also share how I work.
Essentially, this description of how I digest my reading is an invitation to you to write up your modes of working too.
Yesterday, musing about traversing my social graph through blogrolls, I suggested using OPML’s include attribute as a way of adding the blogrolls of the blogs I follow in my own blogroll. Ideally using a spec compliant OPML reader, you’d be able to seamlessly navigate from my blogroll, through the blogroll of one of the blogs I follow, to the blogroll of someone they follow, and presumably back to me at some point.
It does require having an OPML version of such blogrolls available. Peter publishes his blogroll as OPML as I do, allowing a first simple experiment: do includes get correctly parsed in some of the Outliner tools I have?
Adding an include into my OPML file
This little experiment starts with adding to my list of RSS feeds I follow a reference to Peter’s own OPML file of feeds he follows. I already follow two of Peter’s RSS feeds (blogposts and favourites) which I now placed in their own subfolder and to which I added an outline node of the include type, with the URL of Peter’s OPML file.
Screenshot of my OPML file listing the RSS feeds I follow. Click to enlarge. On line 22 you see the line that includes Peter’s OPML file by mentioning its URL.
Trying three outliners
Cloud Outliner (which I in the past used to first create outlines that could then be sent to Evernote) does not parse OPML includes correctly upon import. It also doesn’t maintain any additional attributes from OPML outline nodes, just the text attribute.
Screenshot of Cloud Outliner showing incorrect import of my OPML file. Click to enlarge.
Tinderbox like Cloud Outliner fails to load OPML includes as per spec. It does load some of the attributes (web url, and description, next to the standard text attribute), but not any others (such as the feed url for instance, the crucial element in a list of RSS feeds). It looks like it only picks up on attributes that are directly mappable on pre-existing default attributes within Tinderbox itself.
Screenshot of how Tinderbox imports my OPML file. It keeps some attributes but ignores most, and for includes just mentions the URL
Electric Drummer does correctly import the entire OPML outline. As Dave Winer is both the original creator of the OPML specification and more recently of the Electric Drummer app, this is consistent. Electric Drummer picks up on all attributes in an imported OPML file. Upon import it also fetches the external OPML files listed as includes from their URLs, and fully incorporates them into the imported outline.
Screenshot of Drummer, which incorporates the content of Peter’s OPML file I linked to in my OPML file. Click to enlarge.
Opening up options for tinkering
So at least there is 1 general outliner tool that can work with includes. It probably also means that Dave’s OPML package can do the same, which allows me to tinker at script level with this. One candidate for tinkering is, where a blogger has a blogroll, just not in OPML, to use the OPML package to convert scraped HTML to OPML, and include it locally. That allows me to traverse sets of blogrolls and see the overlap, closed triangles, feedback loops etc. I could also extend my own published blogroll by referencing all the published blogrolls of the bloggers I follow. For you my blogroll would then support exploration and discovery one step further outwards in the network. In parallel I can do something similar for federated bookshelves (both in terms of books as in terms of lists of people who’s booklists and their lists of people you follow)