Bookmarked Interoperable Personal Libraries and Ad Hoc Reading Groups by Maggie Appleton

I somehow missed Maggie Appleton’s blogpost (bookmarked above) about the IndieWeb pop-up session on personal libraries of a few weeks ago. During the session I found her suggestion for ad-hoc reading clubs very interesting, as an application of having book lists on your site. I first and foremost think about discovery in the context of publishing book lists: if I enjoy your blog, or know you and you share book lists those may contain good suggestions to read. Discovery is also why in my ‘data format‘ for such lists I allow for sharing the URLs of lists of others, as well as share the URL of where I found the recommendation for a specific book. What Maggie Appleton suggests is something else and interesting: what if you could see when several others in your network are also currently reading the same book you are reading, allowing an ad-hoc book reading club for that book. It would require a way to compare lists you follow. My sharing of lists I follow is a useful start for it I think, but you’d add a match detection layer on top of it. Whether that matching needs to take place in my site, I don’t know. To me it feels like a personal tool perhaps, alerting me to other readers, and allowing me to privately think about whether I’d want to form an ad-hoc book club with them at that time.

An idea I brought to the event and ended up hosting a session on was ad hoc reading groups – discussions and meetups facilitated by our public bookshelves.

Maggie Appleton

Came across this in a The Hague bookshop. Books wrapped in brown paper with a few key words written on them. Customers are invited to buy them wihthout judging a book by its cover. I wonder how often people do that. There were a few who were trying to guess the book based on the key words provided.

I find that I feel writing a non-fiction subject oriented book is nonsense for non-academics. I feel a strong aversion to the idea of writing a non-fiction book, as people have suggested to me occasionally since university.

Different elements are part of that aversion:

  • There’s a plethora of non-fiction books that to me seem 300 to 400 pages of anecdotal padding around a core idea that would fit on the backflap. Many such books lack tables of content and indexes, seemingly to better hide that one or few core ideas, so you need to go through all pages to find them.
  • The motivation for non-fiction writers to write a book I often find suspect. Aimed at marketing and PR, in support of selling themselves as consultant for instance. Written not to serve an audience, or even find one, but as a branding prop. That makes the actual content often even thinner. Such as taking something anecdotal like “I had this great project I enormously enjoyed doing” and anointing it as the new truth, “Organise all your projects like this, it’s a universal method!”
  • I equally find my own favourite topics suspect as material for writing a book. I don’t think any of the topics I work on, and have been working on, are deep enough or have enough solid foundation to stand on their own as a book. It could only become a range of anecdotes around ideas that themselves fit in a sentence or two. In my activities context and environment are key in working out how an idea can be made to work for a client, and that’s the work. That’s a good source of anecdotes, but not more. See the first bullet. A book about it would be a collection of opinions, and in my eyes would take a rather large amount of work to give those ideas a more solid footing.

In a conversation with E about this a few months ago, she said that’s a very arrogant stance towards authors (they have nothing to say), as well as belittling myself (I have nothing to say). I think those are both the same things, that most people, including me, don’t have enough to say to fill a book, to spend tens of thousands of words on. Many have enough to say on enough moments to at that time fill a great blogpost, article, a pamphlet (like the one about birthday unconferences shown in the right hand column), or an essay. But not a book, an artefact that seems such a heavyweight creation and production process in comparison. There are those who write a book by collating material that was previously written as blogposts, or as internal notes, and then somewhat rearranged. I see that as case in point more than counter argument.

As stated at the top, I make exceptions for academic books, explaining or introducing a field or actual research and their popular science counterparts, and for non-subject non-fiction, that e.g. describes a journey (geographically, or through life for instance, ‘true stories’, the history of a topic and how we ended up in the current situation, that sort of thing).
I also don’t mean fiction. Fiction’s role is very different, and any story that makes you read the next sentence and the next and the next is not what I mean here.
In that sense I very much appreciate the work of Cory Doctorow, who writes articles, essays, columns and blogposts about the topics he cares about, and writes fiction books to explore those same topics along different and novel routes.

Yet, our house holds many non-fiction books. A stack of books that keeps ever growing. So, why is that? Is it that there is more value in the whole, the collection of books read, and those unread, as opposed to the lack of value I perceive in any singular book in itself? Or maybe I don’t understand what writing a non-fiction book is, and what it is for. There are people reading my blog who have written non-fiction books. What were your motivations and aims? Why a book?

Having created a working flow to generate OPML booklists directly from the individual book notes in my PKM system, I did the first actual run in production of those scripts today.

It took a few steps to get to using the scripts in production.

  • I have over 300 book note files in my Obsidian vault.
  • Of course most lacked the templated inline data fields that allow me to create lists. For the 67 fiction books I read in 2021 I already had a manual list with links to the individual files. Where needed I added the templated data fields.
  • Having added those inline fields where they were missing I can easily build lists in Obsidian with the Dataview plugin. Using this code


    results in

  • The same inline data fields are used by my scripts to read the individual files and build the same list in OPML
  • That gets automatically posted to my website where the file is both machine and human readable.

Doing this in production made me discover a small typo in the script that builds the OPML, now fixed (also in the GitHub repository). It also made me realise I want to add a way of ordering the OPML outline entries by month read.

Lists to take into production next are those for currently reading (done), non-fiction 2021, and the anti-library. That last one will be the most work, I have a very long list of books to potentially read. I will approach that not as a task of building the list, but as an ongoing effort of evaluating books I have and why they are potentially of interest to me. A way, in short, to extend my learning, with the list as a useful side effect. The one for currently reading is the least work, and from it the lists for fiction 2022 and non-fiction 2022 will automatically follow. The work is in the backlog, getting history to conform to the convention I came up with, not in moving forward from this point.

In parallel it is great to see that Tom Critchlow is also looking at creating such book lists, in JSON, and at digesting such lists from others. The latter would implement the ‘federated’ part of federated bookshelves. Right now I just point to other people’s list and rss feeds in my ‘list of lists‘. To me getting to federation doesn’t require a ‘standard’. Because JSON, OPML and e.g. schema.org have enough specificity and overlap between them to allow both publishers of lists and parsers or such lists enough freedom to use or discard data fields as they see fit. But there is definitely a discussion to be had on identifying that overlap and how to use it best. Chris Aldrich is planning an IndieWeb event on this and other personal libraries related topics next month. I look forward to participating in that, quite a number of interesting people have expressed interest, and I hope we’ll get to not just talk but also experiment with book lists.

I have a working proof of concept to take individual book notes from Obsidian, turn them into an OPML list of books, and publish them on this webserver. As I had time off these past days I’ve allowed myself to do some code tinkering, resulting in the set-up shown in the image below.


A sketch of my set-up, made in Excalidraw within Obsidian. The blue items now exist, the grey items are still to be done.

The workflow is now as follows:

  • Within Obsidian I have made a template for book notes, which has a number of inline data fields (shaped ‘field:: value’). These fields contain the same attributes that I use in my OPML files, using the data structure I made earlier. It also contains one additional field, the booklist it is part of.
  • When I first create a new book note I use the template and fill out the inline data fields. If it changes status (to read, reading, read) I update the attributes if needed. Next to those data fields it can contain anything else (e.g. my Kindle hihglights and remarks end up in those book notes too.)
  • I can create lists of books in Obsidian using the Dataview plugin, which can find and interpret the inline data fields.
  • I can run a PHP script, on my laptop, that iterates through all the files in the folder that contains my book notes. It reads the inline data fields and turns them into OPML lines with the same attributes. It saves it in the correct OPML file using the booklist field. This means that when I move a book in Obsidian from my “anti-library” to the “currently reading” list and then to the “non-fiction 2021” or “fiction 2021” list by changing that single booklist data field, that will get reflected in the OPML as well. The OPML files are saved in a folder, and both human and machine readable.
  • I have a second PHP script that also runs locally on my laptop, that iterates through the files in the folder. For each of the .opml files it finds that have changed in the past week, it will get the filename and the file content. It then sends those two data fields (and an access code) to a script on my web server as POST form data.
  • The script on my web server accepts POST form data and if the access code is ok, will save the submitted file content using the submitted file name. After that the OPML files on my webserver have the same content as my Dataview overviews within Obsidian, and are fully based on the inline data fields in my individual book notes.

I’ve tested this flow and it works correctly. There’s one important improvement to still make. It currently goes through all my book notes and creates all opml files anew. I want to change that to start from the recently changed book notes and then generate the corresponding opml files. For now it is fast enough locally to not be an issue though that it iterates through the entire folder of book notes.
A second step to take is an addition: to render the same information as JSON files. Dave Winer’s OPMLpackage is likely useful here. Early on there was some discussion on which format to use, and I don’t see a need to choose. I’ve created it using my preference, but the same information can be formatted differently in parallel if it aids usage and federation.
To fully automate this, I still need to set a cron job that calls the first and second local script in turn, every other week or so.

Now that it all works, I will need to see how it goes in practice when I pick a new book, or finish one.
I also need to clean up the code (removing the tests I added in various steps) and translate some of the comments in English. Then I can share I’ve shared the scripts on GitHub, so others can use it for inspiration.

Future steps may include generating book postings in my blog here, directly based on Obsidian notes as well.

As a form of WAB* I’ve made it easier for myself to update my OPML book lists. I created those lists earlier this year as a proof of concept of publishing federated bookshelves. Updating OPML files residing on my hosted webserver is not a fun manual task. Ultimately I want to automate pushing lists from my personal working environment (notes in Obsidian) to my site. Given my limited coding skills I made a first easier step and created a webform in a php script that allows me to add a book to an opml list. It has a drop-down menu for the various OPML lists I keep (e.g. fiction2021, non-fiction2021, currently reading, anti-library), provides the right fields to add the right OPML data attributes, and then writes them to the correct list (each list is a separate file).

That now works well. Having a way to post to my book lists by submitting a form now, I can take the next step of generating such form submissions to replace manually filling out the form.

* Work Avoiding Behaviour, a continuation of the SAB, Study Avoiding Behaviour that I excelled in at university. WAB seems to fit very well with the current locked down last days until the end of year. The Dutch terms ‘studie/werk ontwijkend gedrag’ SOG/WOG lend themselves to the verb to ‘sog’ and to ‘wog’. Yesterday when Y asked E what she had been doing today, E said ‘I’ve been wogging’, and I realised I had been too.