I have a working proof of concept to take individual book notes from Obsidian, turn them into an OPML list of books, and publish them on this webserver. As I had time off these past days I’ve allowed myself to do some code tinkering, resulting in the set-up shown in the image below.
A sketch of my set-up, made in Excalidraw within Obsidian. The blue items now exist, the grey items are still to be done.
The workflow is now as follows:
- Within Obsidian I have made a template for book notes, which has a number of inline data fields (shaped ‘field:: value’). These fields contain the same attributes that I use in my OPML files, using the data structure I made earlier. It also contains one additional field, the booklist it is part of.
- When I first create a new book note I use the template and fill out the inline data fields. If it changes status (to read, reading, read) I update the attributes if needed. Next to those data fields it can contain anything else (e.g. my Kindle hihglights and remarks end up in those book notes too.)
- I can create lists of books in Obsidian using the Dataview plugin, which can find and interpret the inline data fields.
- I can run a PHP script, on my laptop, that iterates through all the files in the folder that contains my book notes. It reads the inline data fields and turns them into OPML lines with the same attributes. It saves it in the correct OPML file using the booklist field. This means that when I move a book in Obsidian from my “anti-library” to the “currently reading” list and then to the “non-fiction 2021” or “fiction 2021” list by changing that single booklist data field, that will get reflected in the OPML as well. The OPML files are saved in a folder, and both human and machine readable.
- I have a second PHP script that also runs locally on my laptop, that iterates through the files in the folder. For each of the .opml files it finds that have changed in the past week, it will get the filename and the file content. It then sends those two data fields (and an access code) to a script on my web server as POST form data.
- The script on my web server accepts POST form data and if the access code is ok, will save the submitted file content using the submitted file name. After that the OPML files on my webserver have the same content as my Dataview overviews within Obsidian, and are fully based on the inline data fields in my individual book notes.
I’ve tested this flow and it works correctly. There’s one important improvement to still make. It currently goes through all my book notes and creates all opml files anew. I want to change that to start from the recently changed book notes and then generate the corresponding opml files. For now it is fast enough locally to not be an issue though that it iterates through the entire folder of book notes.
A second step to take is an addition: to render the same information as JSON files. Dave Winer’s OPMLpackage is likely useful here. Early on there was some discussion on which format to use, and I don’t see a need to choose. I’ve created it using my preference, but the same information can be formatted differently in parallel if it aids usage and federation.
To fully automate this, I still need to set a cron job that calls the first and second local script in turn, every other week or so.
Now that it all works, I will need to see how it goes in practice when I pick a new book, or finish one.
I also need to clean up the code (removing the tests I added in various steps) and translate some of the comments in English. Then I can share I’ve shared the scripts on GitHub, so others can use it for inspiration.
Future steps may include generating book postings in my blog here, directly based on Obsidian notes as well.