After I wrote about federated bookshelves again two weeks ago, I decided to build a proof of concept. A proof of concept for providing an OPML file that contains a list of books, in a way that can be parsed by others. I roughly follow Tom Critchlow’s “spec”.
Because I am making up my own data attributes (although I follow schema.org where possible), I decided to not just create an OPML list, but also to add an XSL template so that OPML is not just machine readable but also human readable in a browser.
The general idea is I have a list that contains lists of books. A list of books can contain books directly, or only be a link to that list of books. A list of books can be one of my own lists on my own domain, or it can be a list published by someone else on a different web address. This allows me to point to other people’s lists when it is somehow relevant to me.
A book in a list I provided with data attributes like title, author and urls for the book and author, and again fields pointing to other people, like the url for the list I may have found the book, or the url for the person / review which was my recommendation.
This allows discovery for both you and me. It makes it work like social software: in triangles, where you can navigate from a person, to a piece of content (a book or list), and to a piece of metadata which is itself the url of another list, or the url of another person, that then have their own metadata pointing to others etc.
Because my lists are structured opml, I should be able to automatically create list files from my own book notes.
Let’s have a look at the proof of concept:
I have an OPML file, called ‘books.opml‘
At the top of that screenshot you can see the opml file calls a XSL stylesheet, named test. I created it by adapting the similar set-up I have for my OPML blogroll. Because of that stylesheet the book list is human readable in a browser and looks like this:
What you see is first some info about me as the creator of the list. It has a link to this list, which is my main list, and a link to my site.
Below it is a list of book lists.
- The first of those book lists, called ‘Fiction I read in 2021’ doesn’t have its own url, and the books are shown directly. Those books may have a link to the book, to the author, some notes, or a link to who recommended it to me, or in which list I found it. It also has a short list description at the top.
- Underneath it is another list by me, called ‘Current non-fiction anti-library’, that is just linked.
- Underneath that are two lists, created by Tom Critchlow and Tom Macwright, both of which are just links. The list item in the outline has an author attribute, and if it’s not my name it gets shown as a ‘followed’ book list. Theoretically if an external link is an OPML file itself, I could include it and show it right here.
Now if you click my other book list, the anti-library list (read here what an anti-library is), that is another opml file.
It calls the same XSL stylesheet as the other list, and renders in the browser as
What you can see in this second list is that it starts with the same link to the main list and info about me, and then shows how the list itself has both a URL and a description. It contains books, and see how one of those books has a link to the book itself. (I don’t link to Amazon or Goodreads, so won’t have links for most books, only if there’s a link to the publisher’s or author’s page on that book)
In the description of the last book you also see how it has a link to the list (Tom Macwright’s) in which I found it, as well as a link to a posting that served as recommendation for me.
For now what I like is that these lists bring their own viewer with it (your browser using my stylesheet), and can point to other people’s lists both directly as a list in my own collection, or as a reference for one of the books. Without having to make assumptions about other people’s lists or parse them somehow, it still allows connecting to them (federating), and discovery. My own lists use schema.org terms for book lists (collection) and books (book). Three attributes I cannot place within schema.org terms, at least not without adding additional subnodes in the outline: author url, reference list (url of the list I found a book in), and reference url (the person or posting that recommended the book to me).
UPDATE: I described the data structure for booklists I used.
@ton I miss the “text” attribute so my tool failed on your OPML. Isn’t “text” mandatory according to http://www.opml.org/spec2 ?
@x28de yes, it is. Thanks, I realise I had postponed the decision on what to put in ‘text’, title, author or both. In a next step I probably should also move some of the other attributes in underlying outline nodes. And a question is what of the things I’d want to use are and aren’t defined in other w3 namespaces. Now the choices are mine, so not to spec as such.
@x28de Will fix the text attribute though.
@x28de done.
This is great!
Can you subscribe to an OPML list directly in an RSS reader? Feedly doesn’t seem to subscribe to this list…
My limited understanding of OPML was that it contained lists of RSS feeds – but I may well be missing something.
Either way – this is great exploration and thanks for going down this rabbit hole!
OPML is for outlines, basically building tree structures of any type of content and data, viewed/manipulated in an outliner. I also use outlines e.g. to create and structure talks. They could be anything though, in the 1968 mother of all demo’s Douglas Engelbart uses an outliner to create and organise a shopping list. Using it as a list of RSS feeds is one specific use case, which allows import/exporting lists in rss readers. Because an opml file is an outline, you can group elements together e.g. the folders I separate my RSS feeds in.
The OPML spec doesn’t itself describe subscription. It does allow for inclusion: if an outline element points to an opml file, that opml file’s outline should be included from that node. This makes it possible to adopt someone else’s opml outline of books, or another one of my own, as a branch on my own tree structured opml outline. If you do that not just statically, replicating it once, but would do it dynamically, pulling in a branch whenever it is getting viewed, or by comparing your version to the current external one, you are de facto subscribed.
Inoreader some years ago implemented subscribing to opml outlines : https://www.inoreader.com/blog/2014/05/opml-subscriptions.html The functionality for it was created by Marjolein Hoekstra and she blogged a lengthy description of opml ‘subscriptions’ and lots of other opml facets at https://cleverclogs.org/2014/05/rss-reader-inoreader-to-support-dynamic-opml-subscriptions.html
Ooh! Excited to see some exploration with OPML files here cc @genmon
After I built a proof of concept of using OPML to share and federate book lists yesterday (UPDATE: description of the data structure for booklists), Tom Chritchlow asked me about subscribing to OPML lists in the comments. I also reread Matt Webb’s earlier posting about using OPML and RSS for book lists.
That results in a few remarks and questions I’d like to make and ask:
OPML serves 2 purposes
In the words of Dave Winer, opml’s creator, OPML is meant as a “transparently simple, self-documenting, extensible and human readable format that’s capable of representing a wide variety of data that’s easily browsed and edited” to create and manipulate outlines, i.e. content structured hiearchically / tree-like.
the format is a way to exchange such outlines between outliner tools.
In other words OPML is great for making (nested) lists, and for exchanging them. I use outlines to build my talks and presentations. It could be shopping lists like in Doug Engelbart’s 1968 ‘mother of all demos’. And indeed it can be lists of books.
A list I regard as an artefact in itself. A list of something is not just iterating the somethings mentioned, the list itself has a purpose and meaning for its creator. It’s a result of some creative act, e.g. curation, planning, writing, or desk research.
A book list I regard as a library, of any size. The list can be as short as the stack on my night reading table is high, as long as a book shelf in my home is wide, or as enormous as the full catalogue of the Royal Library. Judging by Tom Critchlow’s name for his booklist data ‘library.json‘ he sees that similarly.
A book list, as I wrote in my posting about the proof of concept, can have books in them, and other book lists by myself or others. That is where the potential for federation lies. I can from a book point to Tom’s list as the source of inspiration. I could include one of Tom’s booklists into my own booklists.
A list of books is different from a group of individual postings about books as also e.g. presented on my blog’s reading category page. I blog about books I read, but not always. In fact I haven’t written any postings at all this year, but have read 25 books or so since January 1st. It is easier to keep a list of books, than to write postings about each of the books listed. This distinction is expressed too in Tom Macwright’s set-up. There’s a list of books he’s read, which points to pages with a posting about an entry in that list, but the list is useful without those postings.
The difference between booklists as artefacts and groups of postings about books that may also be listed has impact on what it means to ‘subscribe’ to them.
A book list, though it can change over time, is a steady artefact. Books may get added or removed just like in a library, but those changes are an expression of the will of its maker, not a direct function of time.
My list of blogsposts about books, in contrast is fully determined by time: new entries get added on top, older ones drop off the list because the list has a fixed length.
OPML is very suited for my lists as artefacts
RSS is very suited for lists as expression of time, providing the x most recent posts
Subscribing to RSS feeds is widely available
Subscription is not something that has a definition for OPML (that you can use OPML to list RSS subscriptions may be confusing though)
Inclusion however is a concept in OPML: I can add a list as a new branch in another list. If you do that once you only clone a list, and go your own seperate way again. You could also do it dynamically, where you always re-import the other list into your own. Doing it dynamically is a de-facto subscription. For both however, changes in the imported list are non-obvious.
If you keep a previously seen copy and compare it to the current one, you could monitor for changes over time in an OPML list (Inoreader did that in 2014 so you could see and subscribe to new RSS feeds in other people’s OPML feed lists, also see Marjolein Hoekstra’s posting on the functionality she created.).
I am interested in both book lists, i.e. libraries / bookshelves, the way I am interested in browsing a book case when I visit somebody’s home, and in reading people’s reviews of books in the form of postings. With OPML there is also a middle ground: a book list can for each book include a brief comment, without being a full review or opinion. In the shape of ‘I bought this because….’ this is useful input for social filtering for me.
While interested in both those types, libraries, and reviews, I think we need to treat them as completely different things, and separate them out. It is fine to have an OPML list of RSS feeds of reviews, but it’s not the same as having an OPML book list, I think.
I started at the top with quoting Dave Winer about OPML being a “simple, self-documenting, extensible and human readable format that’s capable of representing a wide variety of data that’s easily browsed and edited“. That is true, but needs some qualification:
While I can indeed add all kinds of data attributes, e.g. using namespaces and standardised vocabularies like schema.org, there’s no guarantee nor expectation that any OPML parser/reader/viewer would do anything with them.
This is the primary reason I used an XSL template for my OPML book lists, as it allows me to provide a working parser right along with the data itself. Next to looking at the raw file content itself, you can easily view in a browser what data is contained in it.
In fact I haven’t seen any regular outliner tool that does anything with imported OPML files beyond looking at the must have ‘text’ attribute for any outline node. Tinderbox, when importing OPML, does look also at URL attributes and a few specific others.
I know of no opml viewer that shows you which attributes are available in an OPML list, let alone one that asks you whether to do something with them or not. Yet exploring the data in an OPML file is a key part of discovery of other people’s lists, of the aim to federate booklists, and for adopting better or more widely shared conventions over time.
Are there generic OPML attribute explorers, which let you then configure what to pay attention to? Could you create something like an airtable on the fly from an OPML list?
Monitoring changes in OPML list you’re interested in is possible as such, but if OPML book lists you follow have different structures it quickly becomes a lot of work. That’s different from the mentioned Inoreader example because OPML lists of RSS feeds have a predefined expected structure and set of attributes right in the OPML specification.
Should it be the default to provide XSL templates with OPML files, so that parsing a list as intended by the creator of the list is built right into the OPML list itself?
Should we ‘dumb down’ lists by moving data attributes of an outline node to a sub-node each? You will reduce machine readability in favor of having basic OPML outliners show all information, because there are no machines reading everything yet anayway.
I think for the coming weeks I’ll be on the lookout for sites that have book lists and book posting feeds, to see what commonalities and differences I find.
After I built a proof of concept of using OPML to share and federate book lists yesterday (UPDATE: description of the data structure for booklists), Tom Chritchlow asked me about subscribing to OPML lists in the comments. I also reread Matt Webb’s earlier posting about using OPML and RSS for book lists.
That results in a few remarks and questions I’d like to make and ask:
OPML serves 2 purposes
In the words of Dave Winer, opml’s creator, OPML is meant as a “transparently simple, self-documenting, extensible and human readable format that’s capable of representing a wide variety of data that’s easily browsed and edited” to create and manipulate outlines, i.e. content structured hiearchically / tree-like.
the format is a way to exchange such outlines between outliner tools.
In other words OPML is great for making (nested) lists, and for exchanging them. I use outlines to build my talks and presentations. It could be shopping lists like in Doug Engelbart’s 1968 ‘mother of all demos’. And indeed it can be lists of books.
A list I regard as an artefact in itself. A list of something is not just iterating the somethings mentioned, the list itself has a purpose and meaning for its creator. It’s a result of some creative act, e.g. curation, planning, writing, or desk research.
A book list I regard as a library, of any size. The list can be as short as the stack on my night reading table is high, as long as a book shelf in my home is wide, or as enormous as the full catalogue of the Royal Library. Judging by Tom Critchlow’s name for his booklist data ‘library.json‘ he sees that similarly.
A book list, as I wrote in my posting about the proof of concept, can have books in them, and other book lists by myself or others. That is where the potential for federation lies. I can from a book point to Tom’s list as the source of inspiration. I could include one of Tom’s booklists into my own booklists.
A list of books is different from a group of individual postings about books as also e.g. presented on my blog’s reading category page. I blog about books I read, but not always. In fact I haven’t written any postings at all this year, but have read 25 books or so since January 1st. It is easier to keep a list of books, than to write postings about each of the books listed. This distinction is expressed too in Tom Macwright’s set-up. There’s a list of books he’s read, which points to pages with a posting about an entry in that list, but the list is useful without those postings.
The difference between booklists as artefacts and groups of postings about books that may also be listed has impact on what it means to ‘subscribe’ to them.
A book list, though it can change over time, is a steady artefact. Books may get added or removed just like in a library, but those changes are an expression of the will of its maker, not a direct function of time.
My list of blogsposts about books, in contrast is fully determined by time: new entries get added on top, older ones drop off the list because the list has a fixed length.
OPML is very suited for my lists as artefacts
RSS is very suited for lists as expression of time, providing the x most recent posts
Subscribing to RSS feeds is widely available
Subscription is not something that has a definition for OPML (that you can use OPML to list RSS subscriptions may be confusing though)
Inclusion however is a concept in OPML: I can add a list as a new branch in another list. If you do that once you only clone a list, and go your own seperate way again. You could also do it dynamically, where you always re-import the other list into your own. Doing it dynamically is a de-facto subscription. For both however, changes in the imported list are non-obvious.
If you keep a previously seen copy and compare it to the current one, you could monitor for changes over time in an OPML list (Inoreader did that in 2014 so you could see and subscribe to new RSS feeds in other people’s OPML feed lists, also see Marjolein Hoekstra’s posting on the functionality she created.).
I am interested in both book lists, i.e. libraries / bookshelves, the way I am interested in browsing a book case when I visit somebody’s home, and in reading people’s reviews of books in the form of postings. With OPML there is also a middle ground: a book list can for each book include a brief comment, without being a full review or opinion. In the shape of ‘I bought this because….’ this is useful input for social filtering for me.
While interested in both those types, libraries, and reviews, I think we need to treat them as completely different things, and separate them out. It is fine to have an OPML list of RSS feeds of reviews, but it’s not the same as having an OPML book list, I think.
I started at the top with quoting Dave Winer about OPML being a “simple, self-documenting, extensible and human readable format that’s capable of representing a wide variety of data that’s easily browsed and edited“. That is true, but needs some qualification:
While I can indeed add all kinds of data attributes, e.g. using namespaces and standardised vocabularies like schema.org, there’s no guarantee nor expectation that any OPML parser/reader/viewer would do anything with them.
This is the primary reason I used an XSL template for my OPML book lists, as it allows me to provide a working parser right along with the data itself. Next to looking at the raw file content itself, you can easily view in a browser what data is contained in it.
In fact I haven’t seen any regular outliner tool that does anything with imported OPML files beyond looking at the must have ‘text’ attribute for any outline node. Tinderbox, when importing OPML, does look also at URL attributes and a few specific others.
I know of no opml viewer that shows you which attributes are available in an OPML list, let alone one that asks you whether to do something with them or not. Yet exploring the data in an OPML file is a key part of discovery of other people’s lists, of the aim to federate booklists, and for adopting better or more widely shared conventions over time.
Are there generic OPML attribute explorers, which let you then configure what to pay attention to? Could you create something like an airtable on the fly from an OPML list?
Monitoring changes in OPML list you’re interested in is possible as such, but if OPML book lists you follow have different structures it quickly becomes a lot of work. That’s different from the mentioned Inoreader example because OPML lists of RSS feeds have a predefined expected structure and set of attributes right in the OPML specification.
Should it be the default to provide XSL templates with OPML files, so that parsing a list as intended by the creator of the list is built right into the OPML list itself?
Should we ‘dumb down’ lists by moving data attributes of an outline node to a sub-node each? You will reduce machine readability in favor of having basic OPML outliners show all information, because there are no machines reading everything yet anayway.
I think for the coming weeks I’ll be on the lookout for sites that have book lists and book posting feeds, to see what commonalities and differences I find.
@ton Hi Tom. This is very cool! I’m curious though about your thoughts on the terms machine readable and federated.I’ve always interpreted machine readable as requiring semantic encoding, but I don’t see that in your data.Is there something I’m missing, or do you have a different view of machine readability? Something else?
This was a messy week, mentally mostly, but also because it was a school holiday week with a national holiday. I didn’t come to much actual work, despite there being two things that really need urgent attention. So tomorrow will be a hectic day to repair some of that.
A clear sign of that messiness was that I even struggled to keep my daily logs. This week I
Had a few conversations with people in my network
Had the weekly client conversations
Had a MT meeting
Started bringing my old photo blog back into this site
Let myself get distracted by tinkering with opml for booklists
Booked a house for a few summer weeks in Copenhagen, even if it felt premature to do so. It has a generous cancellation policy thankfully.
Helped Y plot her mother’s day surprises for E
Did some gardening today, as it was a rare warm and sunny spring day
This Article was mentioned on halfanhour.blogspot.com
Writing it down may help in getting out of the loop…
I’m continuing my tinkering with federated bookshelves, for which I made an OPML based way of publishing both lists of books, as well as point to other people’s lists and to RSS feeds of content about books. I now changed my XSL style sheet to parse my OPML files to be able to also parse mentions of RSS feeds.
Meanwhile I read Matt Webb’s posting on using RSS (and OPML) a few more times, and I keep thinking, “yes, but where do you leave the actual data?”
Then I read Stephen Downes’ recent posting on distributing reading material and entire books for courses through RSS, and realised it gave me the same sense of not sounding quite right, like Matt’s posting. That feeling probably means I’m not fully understanding their argument.
RSS is a by design simple XML format as a way to syndicate web content, including videos and podcasts. Content is an important word here, as is syndication: if you have something where new material gets added regularly, an RSS feed is a good way to push it out to those interested.
OPML is another by design simple XML format as a way to share outlines. Outlines are content themselves, and outlines can contain links to other content (including further outlines). One of the common uses of OPML is to share a list of RSS feeds through it, ‘these are the blogs I follow’.
In Matt’s and Stephen’s posts I think there are examples that fail to satisfy either the content part of RSS, or the syndication of new content part. In Matt’s case he talks about feeds of postings about books, like my book category in this site, which is fine, but also in terms of lists of books, which is where I struggle: a list doesn’t necessarily list pieces of content, let alone pieces of web content which RSS seems to require. It more likely is just a list. At the same time he mentions OPML as ‘library’, to use to point to such lists of books. Why would you use OPML for the list of lists, but not for the lists themselves, when those book lists themselves have no content per book, only a number of data attributes which aren’t the content items but only descriptions of items? And when the whole point of OPML outlines is branching lists? When a library isn’t any different from a list, other than maybe in size? Again it is different for actual postings about books, but you can already subscribe to those feeds as existing rivers of content, and point to those feeds (in the same OPML, as I do in my experimental set-up now as well).
In Stephen’s posting he talks about providing the content of educational resources through RSS. He suggests it for the distribution of complete books, and for course material. I do like the idea of providing the material for a course as a ‘blob’. We’re talking about static material here, a book is a finished artefact. Where then is the point in syndication through RSS (other than maybe if the book is a PDF or EPUB or something that might be an enclosure in a RSS feed)? Why not provide the material from its original web source, with its original (semantic) mark-up? Is it in any way likely that such content is going to be read in the same tool the RSS feed is loaded into? And what is the ‘change’ the RSS feed is supposed to convey here, when it’s a one-off distribution and no further change beyond that moment of distribution is expected?
OPML outlines can have additions and deletions, though at a slower pace than e.g. blogs. You could have an RSS feed for additions to an OPML outline (although OPML isn’t web content). But you could also monitor OPML outlines themselves for changes (both additions and deletions) over time. Or reload and use the current version as is, without caring about the specific changes in them.
The plus side of OPML and RSS is that there are many different pieces of code around that can deal with these formats. But most won’t be able to deal as-is with adding data attributes that we need to describe books as data, but aren’t part of the few basic mandatory attributes RSS and OPML are expected to contain. Both RSS and OPML do allow for the extension of attributes, if you follow existing name spaces, such as e.g. schema.org’s for creative works, which seems applicable here (both for collections of books, i.e. a shelf or a library, as well as books themselves). If the use of RSS (and OPML for lists of RSS files) is suggested because there’s an existing eco-system, but we need to change it in a way that ensures the existing ecosystem won’t be able to use it, then where’s the benefit of doing so? To be able to build readers and to build OPML/RSS creators, it is useful to be able to re-use existing bits and pieces of code. But is that really different from creating ones own XML spec? At what point are our adaptations to overcome the purposeful simplicity of OPML and RSS destroying the ease of use we hope to gain from using that simplicity?
Another thing that I keep thinking about is that book lists (shelves, libraries) and book data, basically anything other than web published reviews of books, don’t necessarily get created or live on the web. I can see how I could easily use my website to create OPML and RSS feeds for a variety book lists. But it would require me to have those books and lists as content in my website first, which isn’t a given. Keeping reading lists, and writing reading notes, are part of my personal knowledge management workflow, and it all lives in markdown textfiles on my local harddrive. I have a database of e-books I own, which is in Calibre. I have an old database of book descriptions/data of physical books I owned and did away with in 2012, which is in Delicious Library. None of that lives on the web, or online in any form. If I am going to consistently share bookshelves/lists, then I probably need to create them from where I use that information already. I think Calibre has the ability to work with OPML, and has an API I could use to create lists.
Putting that stuff first into my website in order to generate one some or all of XML/OPML/RSS/JSON from it there, is work and friction I don’t want. If it is possible to automatically put it in my website from my own local notes and databases, that is fine, but then it is just as possible to automatically create all the XML/OPML/RSS/JSON stuff directly from those local notes and databases as well. Even if I would use my website to generate sharable bookshelves, I wouldn’t work with other people’s lists there.
I also think that it is very unlikely that a ‘standard’ emerges. There will always be differences in how people share data about books, because of the different things they care about when it comes to reading and books. Having namespaces like schema.org is useful of course, but I don’t expect everyone will use them. And even if a standard emerges, I am certain there will be many different interpretations thereof in practice. It is key to me that discoverability, of both people sharing book lists and of new to me books, exists regardless. That is why I think, in order to read/consume other people’s lists, other than through the human readable versions in a browser/reader, and to tie them into my information filtering and internal tools/processes, I likely need to have a way to flexibly map someone else’s shared list to what I internally use.
I’m not sure where that leaves me. I think somewhere along these lines:
Discovery, of books and people reading them, is my core aim for federation
OPML seems useful for lists (of lists)
RSS seems useful for content about books
Both depend on using specific book related data attributes which will have limited standardisation, even if they follow existing namespaces. It is impossible to depend on or assume standardisation, something more flexible is needed
My current OPML lists points to other lists by me and others, and to RSS feeds by me and others
I’m willing to generate OPML, RSS and JSON versions of the same lists and content if useful for others, other than templating there’s no key difference after all
Probably my website is not the core element in creating or maintaining lists. It is for publishing things about books.
I’m interested in other people’s RSS feeds about books, and will share my list of feeds I follow as OPML
I need to figure out ways to create OPM/RSS/JSON etc directly from where that information now lives in my workflow and toolset
I need to figure out ways to incorporate what others share with me into my workflow and toolset. Whatever is shared through RSS already fits existing information strategies.
For a limited number of sources shared with me by others, it might make sense to create mappings of their content to my own content structures, so I can import/integrate them more fully.
Related postings:Federated Bookshelves (April 2020)Federated Bookshelves Revisited (April 2021)Federated Bookshelves Proof of Concept (May 2021)Booklist OPML Data Structure (May 2021)
The proof of concept book list I made in opml (also see these additional remarks) currently has the following structure:
It follows the OPML 2 specification
It uses schema.org specifications w.r.t. ‘thing’, ‘creative work’, ‘collection’ and ‘book’ for outline elements and data attributes within them, with a few exceptions.
The file
A booklist file is in OPML format, and has a .opml file extension.
It opens with declaring it to be XML version 1.0 and utf-8 encoding.
It declares an XSL stylesheet, for which the URL is specified, which allows HTML rendering of the file. I think it’s important to package a opml to html parser with the booklist file, so that regardless of data structure, anyone can see what data is contained within it.
It declares OPML version 2.0
The HEADER section
In the HEADER section of the OPML file the following fields are used:
title: mandatory, the name of this booklist file, or of the owner’s main list of lists if this is a sublist
url: mandatory, the url of the booklist file meant in the title
dateCreated: date created, optional
dateModified: date modified, optional
ownerName: mandatory, name of the list owner
ownerId: the url of the owner, optional
ownerEmail: email address of the owner, optional
the OPML HEADER fields for expansion state, vertical scroll state, and for window location are not used (and ignored by the included XSL parser if present).
The BODY section
The body section contains one or more
outlineelements, with a number of attributes. Each attribute can exist only once within an outline element.type=”collection” : At least one is needed. A collection is a single booklist. With the following data attributes, which are all strings:
text: mandatory, the name of the booklist
author: the name of the creator of the booklist, expected
url: the URL of the collection, if it has its own URL, optional if the current file outlines books within the collection
comment: a brief description of the list, optional
type=”book”: A book is always part of a collection. If a collection has its own URL attribute (different from the url of the current file), it does not need to have any book within the file where the collection is listed. If a collection does not have its own URL attribute (or is the current file’s url), it is expected have at least one book (otherwise it’s simply an empty collection). With the following data attributes:
text: mandatory, a string “[title of book] by [name of author(s)/editor]
name: mandatory, the title of the book
author: mandatory, the name of the author(s) or editor of the book
isbn: the ISBN number of the book, optional
comment: a short comment by the booklist owner about the inclusion of the book in the list, optional
url: an url for the book itself, optional
authorurl: the url to the website of the book’s author. This attribute is not listed as part of schema.org. Optional
referencelisturl: the url of a list by a different owner, where this list’s owner found the book. This attribute is not listed as part of schema.org. Optional.
referenceurl: the url of a posting or a person’s url that served as recommendation or motivation for the inclusion of the book in this list by its owner. This attribute is not listed as part of schema.org. Optional.
inLanguage: the language in which the book is written as ISO-639(-1/2/3) code, optional
category: a list of tags, comma separated, optional
type=”rss”: a booklist opml file can point to one or more RSS feeds, optional. Multiple rss-type nodes can be grouped together nested in a typeless outline node with only a text attribute for the name of the group. Not a node within a ‘collection’, not a sub node of a ‘book’. E.g. the book reviews site and feed of someone. These feeds are not booklists or collections but content streams, to which the booklist file owner may want to point. With the following data attributes:
text: mandatory, the name of the feed
xmlUrl: mandatory, the url of the RSS feed
htmlUrl: the url of the website the RSS feed originates from, optional
author: the author of the RSS feed, optional. I use it mostly to mark my own feeds in the XSL style sheet, so I can display it differently than feed I myself subscribe to
type=”include”: points to an OPML file, preferrably a booklist file, that then should be included at this point in this booklist file. In booklists files only to be used at the top level, not as sub node in a ‘collection’ or ‘book’. Optional, and at this point only foreseen, not implemented. With the following attributes:
text: mandatory, descrption or title of the file to be included. This is what is shown in outliners and html renderings.
url: mandatory, the link to the opml file to be included, the linked file must be an .opml file.
Some time ago, I read a post from Ton Zijlstra on the concept of federated bookshelves. His original post was riffing on an idea by Tom Critchlow called Library-JSON. Recently, Ton revisited this topic, and in a short period of time created a working prototype OPML file and a XSL stylesheet to view the OPML file.
I finally had some time this weekend to play around with this. Here is my work so far:
My first booklist in OPML format
The transformed file
I had some problems using Ton’s XSL file. On my server, when I entered the URL of the OPML file, all that was displayed was the OPML file. However, I found a tool to apply the XSL transformation, and the resulting file was a valid HTML page (the “transformed file” above). I will continue to investigate this and also reach out to Ton to see if I did something wrong.
Recently I created a proof of concept of publishing book lists, what I’ve read, what I plan to read etc., using OPML. Andy Sylvester picked it up and created his own list, as a way of experimenting with federated bookshelves. He used the XSLT style sheet I created to be able to render the OPML file in a human readable way in your browser. It seems to work, although it doesn’t render in the browser yet.
Andy, I think the reason it doesn’t render in the browser is because you are loading the style sheet from my domain. The XSLT file must be hosted on the same domain as the OPML file, otherwise it triggers cross-site scripting protections. It should work properly if you download the XSLT I use and put it on your own host.
Hosting the XSLT style sheet also allows you to adapt one other detail: right now there are some tweaks in my version based on the author name of a collection or feed. If it is my name it renders as ‘my list’ and otherwise as ‘list I follow’. When you self host the style sheet you can change the mentions of my name to yours and it will make the proper distinction between your lists and lists you follow.
I assume that in its most basic form I could redo Dopplr of sorts by announcing travel plans in an OPML file, much like book lists or my rss subscriptions. Then it comes down to how to share such travel plans with a known and limited network only. (You don’t want to announce to just everyone when you won’t be home.)
The IndieWeb efforts concerning travel seem to focus on posting actual travel movements, like planned flights. A sort-of check-in style post. The socially shared Dopplr info was much simpler: a city and a set of dates. Because its purpose was aiding serendipitous meet-ups. Exact travel plans or exact location aren’t needed for it, just a way to flag paths more or less crossing to those involved.
Of course making such an OPML file currently is as easy as posting an empty file, as there’s no travel currently.
Theoretically I could use such an OPML file to announce several things:
The various cities I consider as home turf, as they’re within easy reach in an hour.
Selected cities I’m willing to travel to at short notice outside that hour travel time if there’s a good reason to.
From where I am a visit to Antwerp, Brussels, Eindhoven would count in that category, or maybe on specific occasions Düsseldorf or Cologne.
Upcoming travel plans, things like ‘Copenhagen, Denmark, 4th-7th September’ (actually a 2019 example)
Such a list would allow comparison with your list to see whether any of your travel plans match with my ‘home turf’ and destinations I’m willing to consider outside of it, whether any of your travel plans match with my travel plans, or whether any of my travel plans line up with your home turf and other relatively nearby destinations you’re willing to consider. Cities and countries are part of schema.org vocabularies and as such usable in OPML as data attributes.
I think there’s a space for location based services, such as Dopplr was, that don’t depend on or use maps, but provide location contextualized information that influences my actions, choices and my relationships to my networks (a quote from a 2012 blogpost on moving beyond the map).
Or this is just me applying my current opml hammer to anything that might be a nail
I couldn’t resist making this mock-up mimicking the colorful Dopplr
Dave Winer posted a few days ago that he was looking for examples of OPML in “production”. I assume that means “being used by someone or in an app”. I thought I would contribute a few items…
Zylstra.org – Ton Zijlstra’s OPML blog roll in a human readable form (reference)
Zylstra.org – Ton Zijlstra’s booklist in OPML (reference)
AndySylvester.com – Andy’s beginning booklist in OPML (reference)
AndySylvester.com – Reading lists in OPML for the River5 feed reader/aggregator
Recently, Ton Zijlstra wrote about the concept of “federated bookshelves”, in which he referenced a post by Tom Critchlow on a similar concept called “Library JSON”. I have played with this idea based on Ton’s proof of concept. I also read through the postings flowing from Tom Crichlow’s post in 2020, and the ones from Ton Zijlstra’s post this year, and thought it would be helpful to provide a chronology of the development of this idea. To this end, I have created a Github repo with a chronology and links to tools and other distributed book info concepts that I have come across in reading on this topic. Pull requests and other comments are welcome! I am planning to spend more time on this in the next few weeks, so hopefully more prototypes and ideas to come….
Schomburg Center’s Black Liberation Reading List for Adults
New York Public Library has a list of 94 books! I’ve read a few. I definitely should read many more. I find it very intimidating to go through other folks’ book lists and turn them into want-to-read posts on my site, so I’m going to try something I saw Ton Zijlstra do with his reading lists – simply link to the list here and tag this post `books` so it shows up with my other reading posts!
I look forward to adding a few others to clean up old bookmarks and marked-as-unread newsletters in my inbox.
Some examples for how people create book lists / shelves on their own sites.
Boris Anthony, library, sortable by various aspects into different shelves.
Tom Critchlow, wiki of various book lists in markdown files, and proposal for sharing data with JSON
John Johnston, a list of books read, by year, including remarks and starred rating
Tom Macwright, list of books read since 2007, each mention links to a review
Ton Zijlstra, a blog category ‘Books I read‘ with ocassional mentions and reviews of books read, and an OPML proof of concept for data exchange
Tom Critchlow has a great and long list of other people’s book lists / sites.
Other things of interest:
IndieWeb microformat for ‘read’, which you could aggregate into lists.
Bookwyrm, an ActivityPub based federated book club platform. Can be self hosted. Allows creation and curation of ‘shelves’ and lists. It isn’t primarily meant for cataloguing or as a data-source for books, but it does do both of those things to some degree…The application is set up to share book and author data between instances, and get book data from arbitrary outside sources. Right now, the only connector is to OpenLibrary, but other connectors could be written.
Open Library of the Internet Archive has list making functionality, as well as book records in json and rdf.
I want to make it easy to publish lists of books I am reading and have read, or any other list. And do so without using centralised platforms like e.g. Goodreads (Amazon).
The route I am currently on, is to publish a machine readable list others can easily incorporate. These lists are in OPML, an exchange format for outlines. It’s the same format generally used to share lists of RSS feed subscriptions.
In 2020 I came across a posting by Tom Critchlow on this topic, and a year later I started looking into using OPML to create the lists.
I created a proof of concept, with a data format.
Using that I created a webform to update a book list by hand with a new entry.
Then I automated generating the lists (code on GitHub).
All as proofs of concept.
Currently I am able to directly automatically create the lists in OPML from my individual book notes in Obsidian.md (which I use for PKM).
In the coming weeks (Q1 2022) I will work with the scripts I created to see how I like them, and what to tweak to further reduce friction in my workflows.
Next steps will look at how to do the federating itself: how can I ‘consume’, or even include in my own lists, the OPML, ActivityPub or JSON lists of others in a meaningful way?
This Article was mentioned on adhoc.systems