A year ago I blogged about federated bookshelves, in response to Tom Critchlow’s posting Library JSON, A Proposal for a Decentralized Goodreads.

As I reread both postings this morning as well as some of the links Tom points, specifically Phil Gyford’s posting as he starts from the reading experience, not from the tech, and Matt Webb’s for suggesting RSS/OPML, I jotted down a few additional notes.

  • Since the previous posting I stopped linking to Amazon and Goodreads, and having a way to point others to books and vice versa, for discovery is of more interest to me now
  • I envisage myself and others having multiple lists (by topic of interest, genre, language, year, author maybe)
  • I’d like to be able to point from one of my lists to another (from an author field in one list to an author centered list e.g.)
  • I care less about ‘factual’ reviews, more about reasons why people chose a book (‘the cover design jumped out at me in the store’ or ‘this book touches upon X connected to the topic Y that I’m currently exploring’, which goes back to my notions of social filtering
  • Similarly I don’t need images of book covers, which also potentially carry copyright issues, but links to author websites or their publisher would be useful, as is a link to a list sharer’s/reader’s blogpost
  • I’d like to be able to see/get/follow other people’s lists
  • I’d like sharing a list of other people’s lists I follow
  • I’d like to be able to adopt entries in other people’s lists into one of my lists (e.g. an authour, a book or thematic list
  • It would be great if such lists could be imported somehow into tools people might use, e.g. Calibre, Delicious Library, Zotero
  • I don’t think you need a unique ID for a book, like Tom originally suggested, if the aim is discovery. It’s enough to be able to build triangles that allow navigation and discovery, from me to a title or author, to another reader or more books by an author, or other books in lists where this one shows up
  • OPML with our without RSS seems the most simple approach here, as the type of info we’re talking about is very well suited to outliners. OPML outlines, and outlines of outlines, can be machine readable and human readable at the same time (case in point, my OPML list of blogs I follow, which is human readable as a blogroll and can also directly be imported into any feedreader
  • The first list I think I should make as an experiment, is the list of things I might read, my current non-fiction Anti-Library

That last point I’ve added to my things to do if I find some spare moments.


The other side of a book case, image by Ton Zijlstra, license CC BY NC SA

7 reactions on “Federated Bookshelves Revisited

  1. After I wrote about federated bookshelves again two weeks ago, I decided to build a proof of concept. A proof of concept for providing an OPML file that contains a list of books, in a way that can be parsed by others. I roughly follow Tom Critchlow’s “spec”.
    Because I am making up my own data attributes (although I follow schema.org where possible), I decided to not just create an OPML list, but also to add an XSL template so that OPML is not just machine readable but also human readable in a browser.
    The general idea is I have a list that contains lists of books. A list of books can contain books directly, or only be a link to that list of books. A list of books can be one of my own lists on my own domain, or it can be a list published by someone else on a different web address. This allows me to point to other people’s lists when it is somehow relevant to me.
    A book in a list I provided with data attributes like title, author and urls for the book and author, and again fields pointing to other people, like the url for the list I may have found the book, or the url for the person / review which was my recommendation.
    This allows discovery for both you and me. It makes it work like social software: in triangles, where you can navigate from a person, to a piece of content (a book or list), and to a piece of metadata which is itself the url of another list, or the url of another person, that then have their own metadata pointing to others etc.
    Because my lists are structured opml, I should be able to automatically create list files from my own book notes.
    Let’s have a look at the proof of concept:
    I have an OPML file, called ‘books.opml

    At the top of that screenshot you can see the opml file calls a XSL stylesheet, named test. I created it by adapting the similar set-up I have for my OPML blogroll. Because of that stylesheet the book list is human readable in a browser and looks like this:

    What you see is first some info about me as the creator of the list. It has a link to this list, which is my main list, and a link to my site.
    Below it is a list of book lists.

    The first of those book lists, called ‘Fiction I read in 2021’ doesn’t have its own url, and the books are shown directly. Those books may have a link to the book, to the author, some notes, or a link to who recommended it to me, or in which list I found it. It also has a short list description at the top.
    Underneath it is another list by me, called ‘Current non-fiction anti-library’, that is just linked.
    Underneath that are two lists, created by Tom Critchlow and Tom Macwright, both of which are just links. The list item in the outline has an author attribute, and if it’s not my name it gets shown as a ‘followed’ book list. Theoretically if an external link is an OPML file itself, I could include it and show it right here.
    Now if you click my other book list, the anti-library list (read here what an anti-library is), that is another opml file.

    It calls the same XSL stylesheet as the other list, and renders in the browser as

    What you can see in this second list is that it starts with the same link to the main list and info about me, and then shows how the list itself has both a URL and a description. It contains books, and see how one of those books has a link to the book itself. (I don’t link to Amazon or Goodreads, so won’t have links for most books, only if there’s a link to the publisher’s or author’s page on that book)
    In the description of the last book you also see how it has a link to the list (Tom Macwright’s) in which I found it, as well as a link to a posting that served as recommendation for me.
    For now what I like is that these lists bring their own viewer with it (your browser using my stylesheet), and can point to other people’s lists both directly as a list in my own collection, or as a reference for one of the books. Without having to make assumptions about other people’s lists or parse them somehow, it still allows connecting to them (federating), and discovery. My own lists use schema.org terms for book lists (collection) and books (book). Three attributes I cannot place within schema.org terms, at least not without adding additional subnodes in the outline: author url, reference list (url of the list I found a book in), and reference url (the person or posting that recommended the book to me).
    UPDATE: I described the data structure for booklists I used.

  2. Writing it down may help in getting out of the loop…
    I’m continuing my tinkering with federated bookshelves, for which I made an OPML based way of publishing both lists of books, as well as point to other people’s lists and to RSS feeds of content about books. I now changed my XSL style sheet to parse my OPML files to be able to also parse mentions of RSS feeds.
    Meanwhile I read Matt Webb’s posting on using RSS (and OPML) a few more times, and I keep thinking, “yes, but where do you leave the actual data?”
    Then I read Stephen Downes’ recent posting on distributing reading material and entire books for courses through RSS, and realised it gave me the same sense of not sounding quite right, like Matt’s posting. That feeling probably means I’m not fully understanding their argument.
    RSS is a by design simple XML format as a way to syndicate web content, including videos and podcasts. Content is an important word here, as is syndication: if you have something where new material gets added regularly, an RSS feed is a good way to push it out to those interested.
    OPML is another by design simple XML format as a way to share outlines. Outlines are content themselves, and outlines can contain links to other content (including further outlines). One of the common uses of OPML is to share a list of RSS feeds through it, ‘these are the blogs I follow’.
    In Matt’s and Stephen’s posts I think there are examples that fail to satisfy either the content part of RSS, or the syndication of new content part. In Matt’s case he talks about feeds of postings about books, like my book category in this site, which is fine, but also in terms of lists of books, which is where I struggle: a list doesn’t necessarily list pieces of content, let alone pieces of web content which RSS seems to require. It more likely is just a list. At the same time he mentions OPML as ‘library’, to use to point to such lists of books. Why would you use OPML for the list of lists, but not for the lists themselves, when those book lists themselves have no content per book, only a number of data attributes which aren’t the content items but only descriptions of items? And when the whole point of OPML outlines is branching lists? When a library isn’t any different from a list, other than maybe in size? Again it is different for actual postings about books, but you can already subscribe to those feeds as existing rivers of content, and point to those feeds (in the same OPML, as I do in my experimental set-up now as well).
    In Stephen’s posting he talks about providing the content of educational resources through RSS. He suggests it for the distribution of complete books, and for course material. I do like the idea of providing the material for a course as a ‘blob’. We’re talking about static material here, a book is a finished artefact. Where then is the point in syndication through RSS (other than maybe if the book is a PDF or EPUB or something that might be an enclosure in a RSS feed)? Why not provide the material from its original web source, with its original (semantic) mark-up? Is it in any way likely that such content is going to be read in the same tool the RSS feed is loaded into? And what is the ‘change’ the RSS feed is supposed to convey here, when it’s a one-off distribution and no further change beyond that moment of distribution is expected?
    OPML outlines can have additions and deletions, though at a slower pace than e.g. blogs. You could have an RSS feed for additions to an OPML outline (although OPML isn’t web content). But you could also monitor OPML outlines themselves for changes (both additions and deletions) over time. Or reload and use the current version as is, without caring about the specific changes in them.
    The plus side of OPML and RSS is that there are many different pieces of code around that can deal with these formats. But most won’t be able to deal as-is with adding data attributes that we need to describe books as data, but aren’t part of the few basic mandatory attributes RSS and OPML are expected to contain. Both RSS and OPML do allow for the extension of attributes, if you follow existing name spaces, such as e.g. schema.org’s for creative works, which seems applicable here (both for collections of books, i.e. a shelf or a library, as well as books themselves). If the use of RSS (and OPML for lists of RSS files) is suggested because there’s an existing eco-system, but we need to change it in a way that ensures the existing ecosystem won’t be able to use it, then where’s the benefit of doing so? To be able to build readers and to build OPML/RSS creators, it is useful to be able to re-use existing bits and pieces of code. But is that really different from creating ones own XML spec? At what point are our adaptations to overcome the purposeful simplicity of OPML and RSS destroying the ease of use we hope to gain from using that simplicity?
    Another thing that I keep thinking about is that book lists (shelves, libraries) and book data, basically anything other than web published reviews of books, don’t necessarily get created or live on the web. I can see how I could easily use my website to create OPML and RSS feeds for a variety book lists. But it would require me to have those books and lists as content in my website first, which isn’t a given. Keeping reading lists, and writing reading notes, are part of my personal knowledge management workflow, and it all lives in markdown textfiles on my local harddrive. I have a database of e-books I own, which is in Calibre. I have an old database of book descriptions/data of physical books I owned and did away with in 2012, which is in Delicious Library. None of that lives on the web, or online in any form. If I am going to consistently share bookshelves/lists, then I probably need to create them from where I use that information already. I think Calibre has the ability to work with OPML, and has an API I could use to create lists.
    Putting that stuff first into my website in order to generate one some or all of XML/OPML/RSS/JSON from it there, is work and friction I don’t want. If it is possible to automatically put it in my website from my own local notes and databases, that is fine, but then it is just as possible to automatically create all the XML/OPML/RSS/JSON stuff directly from those local notes and databases as well. Even if I would use my website to generate sharable bookshelves, I wouldn’t work with other people’s lists there.
    I also think that it is very unlikely that a ‘standard’ emerges. There will always be differences in how people share data about books, because of the different things they care about when it comes to reading and books. Having namespaces like schema.org is useful of course, but I don’t expect everyone will use them. And even if a standard emerges, I am certain there will be many different interpretations thereof in practice. It is key to me that discoverability, of both people sharing book lists and of new to me books, exists regardless. That is why I think, in order to read/consume other people’s lists, other than through the human readable versions in a browser/reader, and to tie them into my information filtering and internal tools/processes, I likely need to have a way to flexibly map someone else’s shared list to what I internally use.
    I’m not sure where that leaves me. I think somewhere along these lines:
    Discovery, of books and people reading them, is my core aim for federation
    OPML seems useful for lists (of lists)
    RSS seems useful for content about books
    Both depend on using specific book related data attributes which will have limited standardisation, even if they follow existing namespaces. It is impossible to depend on or assume standardisation, something more flexible is needed
    My current OPML lists points to other lists by me and others, and to RSS feeds by me and others
    I’m willing to generate OPML, RSS and JSON versions of the same lists and content if useful for others, other than templating there’s no key difference after all
    Probably my website is not the core element in creating or maintaining lists. It is for publishing things about books.
    I’m interested in other people’s RSS feeds about books, and will share my list of feeds I follow as OPML
    I need to figure out ways to create OPM/RSS/JSON etc directly from where that information now lives in my workflow and toolset
    I need to figure out ways to incorporate what others share with me into my workflow and toolset. Whatever is shared through RSS already fits existing information strategies.
    For a limited number of sources shared with me by others, it might make sense to create mappings of their content to my own content structures, so I can import/integrate them more fully.
    Related postings:Federated Bookshelves (April 2020)Federated Bookshelves Revisited (April 2021)Federated Bookshelves Proof of Concept (May 2021)Booklist OPML Data Structure (May 2021)

  3. Some time ago, I read a post from Ton Zijlstra on the concept of federated bookshelves. His original post was riffing on an idea by Tom Critchlow called Library-JSON. Recently, Ton revisited this topic, and in a short period of time created a working prototype OPML file and a XSL stylesheet to view the OPML file.
    I finally had some time this weekend to play around with this. Here is my work so far:
    My first booklist in OPML format
    The transformed file
    I had some problems using Ton’s XSL file. On my server, when I entered the URL of the OPML file, all that was displayed was the OPML file. However, I found a tool to apply the XSL transformation, and the resulting file was a valid HTML page (the “transformed file” above). I will continue to investigate this and also reach out to Ton to see if I did something wrong.

  4. Recently, Ton Zijlstra wrote about the concept of “federated bookshelves”, in which he referenced a post by Tom Critchlow on a similar concept called “Library JSON”. I have played with this idea based on Ton’s proof of concept. I also read through the postings flowing from Tom Crichlow’s post in 2020, and the ones from Ton Zijlstra’s post this year, and thought it would be helpful to provide a chronology of the development of this idea. To this end, I have created a Github repo with a chronology and links to tools and other distributed book info concepts that I have come across in reading on this topic. Pull requests and other comments are welcome! I am planning to spend more time on this in the next few weeks, so hopefully more prototypes and ideas to come….

  5. TIL: In an xml style sheet, an xslt file, testing for the existence of an attribute is not the same as testing whether an attribute has content.
    I have a machine readable file (OPML) with lines such as
    <outline type="book" text="title by author" name="book title" author="author name" url="" comment="some remarks" authorurl="" inLanguage="en" />
    I want to show that file in a browser for people to see, and I want to test for the attribute url in that line so that I can mark up the text with a web link in the human readable version.
    What I first used to test for the presence of a link was:
    <xsl:when test="@url">
    I assumed this would test for the actual presence of a value for attribute ‘url’. But that is not correct. The statement actually tests whether the attribute ‘url’ is present in the line of machine readable code.
    To test if the url attribute also has content in it, e.g. url=”https://weblink.tld/page” and not url=”” I need to check that the field is not empty:
    <xsl:when test="@url!=''">
    I experimentally create OPML lists of books I read (see why and how). For instance I have a ‘fiction 2021’ list. OPML files are meant to be read by machines, but I use the same file to make a human readable version. In the human readable version I want to link to a book and to an author if I have a direct link to a page about the book (by the publisher or author) and a link to the author’s website. I test for non-empty url attributes so I know when to construct a link in the human readable output. You can download the XSLT I use to do that and of course the OPML file that lists the fiction books for 2021 (also see right hand side column, in the feeds section).

  6. I want to make it easy to publish lists of books I am reading and have read, or any other list. And do so without using centralised platforms like e.g. Goodreads (Amazon).
    The route I am currently on, is to publish a machine readable list others can easily incorporate. These lists are in OPML, an exchange format for outlines. It’s the same format generally used to share lists of RSS feed subscriptions.
    In 2020 I came across a posting by Tom Critchlow on this topic, and a year later I started looking into using OPML to create the lists.
    I created a proof of concept, with a data format.
    Using that I created a webform to update a book list by hand with a new entry.
    Then I automated generating the lists (code on GitHub).
    All as proofs of concept.
    Currently I am able to directly automatically create the lists in OPML from my individual book notes in Obsidian.md (which I use for PKM).
    In the coming weeks (Q1 2022) I will work with the scripts I created to see how I like them, and what to tweak to further reduce friction in my workflows.
    Next steps will look at how to do the federating itself: how can I ‘consume’, or even include in my own lists, the OPML, ActivityPub or JSON lists of others in a meaningful way?

Comments are closed.

To respond on your own website, enter the URL of your response which should contain a link to this post's permalink URL. Your response will then appear (possibly after moderation) on this page. Want to update or remove your response? Update or delete your post and re-enter your post's URL again. (Find out more about Webmentions.)

Mentions

Likes