Category: Wiki

All pages considered part of the wiki section

Current status:

  • Created a working way to submit JSON formatted blogposts to this site, code on GitHub
  • Included that in my earlier scripts to create posts from my feed reading, that I now no longer then have to post by hand.
  • Created a Microsub client to replace my feed reader, in which I can respond directly from within the page I am reading, and do so to different websites I maintain.
  • Combined the same basic script with a local webform so I can very quickly post something. I don’t think I will be using this possibility much but it was a good way to add a front-end to the micropub script first and fast.
  • Can take a local markdown file written in Obsidian and post it as html to my site. This is by far the most useful to me
    • I write my blogposts in Obsidian, drafts live in a specific writing folder. They have two inline data fields, status and tags. While writing a note has status ‘writing’, when it is ready to publish I set the status to ‘draft’.
    • Within Obsidian I use the same status field to create a dynamic overview of posts being written, ready to publish, and previously published (using the DataView plugin).
    • When I’m ready to post, I hit a hotkey which launches my PHP script. It looks at all files in the specific writing folder and checks for files that changed within the last few hours and if those contain a status field ‘draft’. For those that do it transforms the markdown in those files to html, and then posts that to my site, using the tags in the other data field to tag and categorise the post. It also sets the status field in my notes from ‘draft’ to ‘posted’.
    • I could also run the script every hour or so using a cron job, so that anything posts automatically, while I go on with my other work.
  • Use the form in my feedreader to save an article in markdown to my notes in Obsidian with my rationale and some remarks.
  • Use the form in my feedreader to post an annotation to Hypothes.is (similarly formatted to posting it to my notes in Obsidian)

Rationale:

Local first, personal, narrow band
See blogpost.

Narrow band means:
my preferences can be treated as default inputs
my tasks are predictable to me
together they are functions with parameters, aka code.

Sources:

Micropub standard
IndieWeb wiki on Micropub
Tips from Jan Boddez (in Dutch)
Jamie Tanna’s work on his personal micropub client
Jamie Tanna’s tool to get authorisation tokens manually, great for testing/development.
Parsedown, which I use to translate markdown files written in Obsidian, to HTML for my site.

I want to make it easy to publish lists of books I am reading and have read, or any other list. And do so without using centralised platforms like e.g. Goodreads (Amazon). A book list is a small library.
The route I am currently on, is publishing a machine readable list others can easily incorporate. These lists are in OPML, an exchange format for outlines. It’s the same format generally used to share lists of RSS feed subscriptions.

Current situation and usage: automated lists
Currently I am able to directly automatically create the lists in OPML from my individual book notes in Obsidian.md (which I use for PKM).
In Q1 2022 I experienced that creating lists and posting them works nicely and smoothly, with no friction. I do currently only create a few lists (fiction and non-fiction in the running year, antilibrary). I’m also working through the books I’ve read in the last decade or so, and gradually creating those lists. I’m not generating those as OPML however, they currently are just a list in my own notes.

Next steps: consuming other lists
Next steps will look at how to do the federating itself: how can I ‘consume’, or even include in my own lists, the OPML, ActivityPub or JSON lists of others in a meaningful way? I think a first step is consuming one list published by someone else, treating it as a recommendation list perhaps or some other form of input, much like I’m reading feeds. It might be useful to be able to pick out mentions about books I’ve already read, are in my anti-library, match an author I like, or match my interests while being unknown to me. I suspect a slightly tweaked parser for every new list might be needed, as using a list depends both on format and on content fields.

Ealier steps: proof of concept and data model
In 2020 I came across a posting by Tom Critchlow on this topic, and a year later I started looking into using OPML to create the lists.

I created a proof of concept, with a data format.
Using that I created a webform to update a book list by hand with a new entry.
Then I automated generating the lists (code on GitHub).
All as proofs of concept.

The annual Tadaa! list is a posting I write listing the things that gave me a feeling of accomplishment that year. I started it in 2010, and made one every year since. The primary reason for these lists is that I easily forget things I did in a year, because I tend to move on to other things before letting something register or sink in properly. For instance one year I forgot I organised, hosted and moderated what turned out as national level conference with a colleague, because I left for an extended international work trip the day after. That made organising a conference, originally meant as a project retrospective, a thing I had to finish before I could go to the airport. The lists over the years helped me resurface things that are worth looking back on. They are unordered lists, often roughly chronological as I leaf through my calendar and notes to write them.
Here is the full list, most recent at the top.

2023
2022
2021
2020
2019
2018
2017,
2016,
2015,
2014,
2013,
2012,
2011
2010

A (short) list of applications that were very useful to me at one time, but then went away or astray. The question is, could one redo these in a current and useful way?

  • Dopplr: showing simple travel plans (city and dates) to facilitate serendipitous meet-ups outside your regular movements. (went away after being acquired)
  • Delicious: social bookmarking (went astray by dropping/breaking-by-redoing the social functionality, then went away). Have a project on the shelf to redo this for myself, called Linqurator.
  • Skype: p2p voip (went astray by dropping p2p in favor of centralised servers, after acquisition by Microsoft). See this and this posting asking questions about the current p2p voip space.

Some examples for how people create book lists / shelves on their own sites.

Other things of interest:

  • IndieWeb microformat for ‘read’, which you could aggregate into lists.
  • Bookwyrm, an ActivityPub based federated book club platform. Can be self hosted. Allows creation and curation of ‘shelves’ and lists. It isn’t primarily meant for cataloguing or as a data-source for books, but it does do both of those things to some degree…The application is set up to share book and author data between instances, and get book data from arbitrary outside sources. Right now, the only connector is to OpenLibrary, but other connectors could be written.
  • Open Library of the Internet Archive has list making functionality, as well as book records in json and rdf.

The proof of concept book list I made in opml (also see these additional remarks) currently has the following structure:

It follows the OPML 2 specification
It uses schema.org specifications w.r.t. ‘thing’, ‘creative work’, ‘collection’ and ‘book’ for outline elements and data attributes within them, with a few exceptions.

The file

  • A booklist file is in OPML format, and has a .opml file extension.
  • It opens with declaring it to be XML version 1.0 and utf-8 encoding.
  • It declares an XSL stylesheet, for which the URL is specified, which allows HTML rendering of the file. I think it’s important to package a opml to html parser with the booklist file, so that regardless of data structure, anyone can see what data is contained within it.
  • It declares OPML version 2.0

The HEADER section
In the HEADER section of the OPML file the following fields are used:

  • title: mandatory, the name of this booklist file, or of the owner’s main list of lists if this is a sublist
  • url: mandatory, the url of the booklist file meant in the title
  • dateCreated: date created, optional
  • dateModified: date modified, optional
  • ownerName: mandatory, name of the list owner
  • ownerId: the url of the owner, optional
  • ownerEmail: email address of the owner, optional

the OPML HEADER fields for expansion state, vertical scroll state, and for window location are not used (and ignored by the included XSL parser if present).

The BODY section

The body section contains one or more outline elements, with a number of attributes. Each attribute can exist only once within an outline element.

  • type=”collection” : At least one is needed. A collection is a single booklist. With the following data attributes, which are all strings:
    • text: mandatory, the name of the booklist
    • author: the name of the creator of the booklist, expected
    • url: the URL of the collection, if it has its own URL, optional if the current file outlines books within the collection
    • comment: a brief description of the list, optional
  • type=”book”: A book is always part of a collection. If a collection has its own URL attribute (different from the url of the current file), it does not need to have any book within the file where the collection is listed. If a collection does not have its own URL attribute (or is the current file’s url), it is expected have at least one book (otherwise it’s simply an empty collection). With the following data attributes:
    • text: mandatory, a string “[title of book] by [name of author(s)/editor]
    • name: mandatory, the title of the book
    • author: mandatory, the name of the author(s) or editor of the book
    • isbn: the ISBN number of the book, optional
    • comment: a short comment by the booklist owner about the inclusion of the book in the list, optional
    • url: an url for the book itself, optional
    • authorurl: the url to the website of the book’s author. This attribute is not listed as part of schema.org. Optional
    • referencelisturl: the url of a list by a different owner, where this list’s owner found the book. This attribute is not listed as part of schema.org. Optional.
    • referenceurl: the url of a posting or a person’s url that served as recommendation or motivation for the inclusion of the book in this list by its owner. This attribute is not listed as part of schema.org. Optional.
    • inLanguage: the language in which the book is written as ISO-639(-1/2/3) code, optional
    • category: a list of tags, comma separated, optional
  • type=”rss”: a booklist opml file can point to one or more RSS feeds, optional. Multiple rss-type nodes can be grouped together nested in a typeless outline node with only a text attribute for the name of the group. Not a node within a ‘collection’, not a sub node of a ‘book’. E.g. the book reviews site and feed of someone. These feeds are not booklists or collections but content streams, to which the booklist file owner may want to point. With the following data attributes:
    • text: mandatory, the name of the feed
    • xmlUrl: mandatory, the url of the RSS feed
    • htmlUrl: the url of the website the RSS feed originates from, optional
    • author: the author of the RSS feed, optional. I use it mostly to mark my own feeds in the XSL style sheet, so I can display it differently than feed I myself subscribe to
  • type=”include”: points to an OPML file, preferrably a booklist file, that then should be included at this point in this booklist file. In booklists files only to be used at the top level, not as sub node in a ‘collection’ or ‘book’. Optional, and at this point only foreseen, not implemented. With the following attributes:
    • text: mandatory, descrption or title of the file to be included. This is what is shown in outliners and html renderings.
    • url: mandatory, the link to the opml file to be included, the linked file must be an .opml file.