Favorited dev Notes for Markdown in RSS by Dave Winer

As part of celebrating twenty years of RSS, Dave Winer adds the ability to incorporate markdown in RSS feeds. Essentially this was always possible, but there was no way to tell a RSS reader that something was to be interpreted not as HTML but as Markdown. Doing this makes it possible to provide both HTML and Markdown in the same feed, if Markdown is e.g. the way you’ve written a posting and want to be able to also edit it again in Markdown, and not in HTML.

After my hiatus I think this is worth an experiment to see if I can generate an RSS feed directly from my markdown notes on my local system. Just like I already can generate OPML feeds and blogposts or website pages from my notes. Chris Aldrich recently asked about using WordPress and Webmention as a way of publishing your own notes with the capability of linking them to other peoples notes. Could RSS play a role there too? Could I provide selected RSS feeds for specific topics directly from my notes? Or for specific people? For them to read along? Is there something here that can play a role in social sharing of annotations, such as Hypothes.is provides? I need to play with this thought. RSS is well understood an broadly used, providing not just HTML but also Markdown through it sounds like a step worth exploring.

Web pages are predictably untrustworthy to remain online as they were when you dropped by to see them. They can change while their address remains the same, can redirect an address to someplace else entirely, and entire websites can disappear meaning there’s no response when attempting to visit an address.

This means that when I link to something here, there’s no guarantee at all that when you click such a link that you actually get to see what I thought I’ve linked to. And no, screenshots don’t help: fake is easy and we need to back up our words with hyperlinks.

The same is true for stuff I don’t link to here, but save to my archive. That’s why I don’t just save URLs but the entire article for future reference to my markdown notes. That still means the actual source might disappear, without me having a way of proving what I saved is what I saw. This not only is relevant to the content itself, but also for instance for licensing information. There are photos in this blog that were openly licensed when I used them, but no longer, leaving it impossible for me to prove I still can use the image because of the license at the time.

This makes an archiving service useful, like Archive.org. I can use that to store URLs I find interesting and I do that with some regularity. It is why I am a monthly donor to the Web Archive, I’d like it to remain a more robust reference point on the web.

Currently I have one way of adding web pages to an archive, using the Wayback Machine add-on in my browser. The same add-on helps me find previous versions of a page already archived, tweets about that page, and annotations made by others. Very useful, during browsing.


The Web Archive browser add-on bookmarklet

Writing blogposts and saving webpages as markdown in my local notes, or starting to annotate a page in Hypothes.is are another matter however. There I’d like to automate getting or creating an archive link.

In all cases it would need to be an archive link next to the original. If I link to something in a blogpost here I want to still send WebMentions to the linked site, and that requires the link to the original to be in my posting. Similarly for my notes, I want to have the original url as well, although it would be reconstructable from the archive link. For online social annotations in Hypothes.is, the original link is needed because that is how you find other people’s annotations alongside your own. The last one is probably easiest, by using the browser add-on manually and adding the result as a first annotation for instance.


An archive link as first Hypothes.is annotation

On the page in the IndieWeb wiki about using the Internet Archive there are some code snippets to be found to use with the Archive’s API, or using the basic string to save something https://web.archive.org/save/urlhere. It also mentions bloggers who either send the URLs they mention, or their own postings, or both (e.g. when sending a WebMention) to the Internet Archive.
When posting to my blog from my local markdown notes I could potentially add a function to the markdown-to-html parser I use where it detects external links, runs them through the Archive and writes the html for both the direct and the archived link.

For saving web articles as local notes in markdown there are several options to explore:

  • When saving from the browser using the Markdownload add-on, first saving and copying the saved url using the Achive add-on, then pasting that archive link in the dialog box.
  • Adding [Opslaan in Internet Archive](https://web.archive.org/save/{baseURI}) to the Markdownload template so I can directly save a URL from within the local note later, if wanted. I added this experimentally, to see if I would actually use it like this.
  • When saving to notes from my microsub feedreader I could add a function to the html-to-markdown parser I use there to run external links through the Archive and write the Archive link in markdown after the original link.

Chris Aldrich’s Hypothes.is feed pointed me to this resource on Personal Knowledge Graphs (which he co-authors/-ed). In it Martynas Jusevičius mentions as an issue with markdown being used for personal knowledge graphs is their incompatibility with the RDF ecosystem as there is no support for typed links in markdown. I disagree with the statement that that would turn markdown knowledge collections into a ‘walled garden’. There’s no garden there. It does hinder interoperability with more complex environments like RDF / semantic web, and potential connections and interactions with other graphs and people.

A question I think is whether the burden of arranging interoperability lies with the least complex or with the most complex part. Probably the latter, which points away from plaintext and markdown, to at least having a parser that adds the additional complexity. Markdown always needs a parser, it’s not intended for other people to see other than the author(s).

On the other hand, being able to characterise links sounds like at least a somewhat doable step in a markdown text environment. In terms of publishing such notes to my blog that should translate into microformats in html, but parsing it to xml is a similar step.

A quick search came up with this post ‘Semantic Markdown‘.
It shows the issue of adding more complexity to markdown well I think. As an authoring tool you don’t want to make writing more complicated, nor make reading back what you’ve written less easy for human eyes. This is why I e.g. generally avoid frontmatter in my markdown notes in Obsidian, as it reduces the ease of reading for myself. Inline data fields used sparingly are less disruptive to me.

The article also provides imo mostly unconvincing examples, like labeling Berlin as a geographic place, or an event name as an event. Especially given the effort involved marking them up as city or event. I’d like to form more meaningful triples around external links (a reply to b, a part of b), also between notes (‘a counterexample of b’) where the information is in the type of relationship between two points. There’s some association here with one OPML file embedding other OPML outlines by pointing to them, and the branching in a classic Zettelkasten or even code repo’s.
Don’t want to just add links to mundane things (this word refers to a city): I e.g. dislike a lot of Wikipedia links inside Wikipedia lemma’s where they link to the lemma of a thing in general, but not to its meaning within the context of the link placement, while the phrasing suggests the link will provide more context. That’s just like adding every single word in a sentence to its dictionary page to me.

In the comments underneath that Semantic Markdown link, similar things are mentioned, alongside links to a semantic markup language bridging xml and markdown, and another one describing the same.

(I’m not sure what this type of post is, I find myself writing several in parallel yesterday and today, triggered by reading streams of public Hypothes.is annotations. I’ve dubbed it Jottings. It’s an attempt at blogging something in the stage between just bookmarking a url with some motivation and something more well formed based on an actual experience or exploration. More grasping at first connections, less formed opinion, but also not just annotation as it is rather somewhat removed from the source text and not anchored to it. More holding questions than providing insights or answers.)

I extended the capabilities of my microsub feed reader with the option to save web articles directly from the reader to my Obsidian notes in markdown format.

Until now if I wanted to save an entire article I found in my feed reader, I would open it in the browser and then use the markdownclipper browser add-on to add some context and then save the article in markdown in my notes. I wanted to cut out that step of opening it in the feed reader, by saving it directly to my markdown notes. In my feedreader I already have a response form to e.g. post a reply to a posting on my own site. Posting it to my notes means adding a path to how I process that form.

I had to find a suitable script for converting HTML to MarkDown first. Which I found in PHP League’s HTML-to-Markdown, as suggested by Jan Boddez. It requires Composer which I already had installed on my laptop.

I tweaked my feed reader’s response form to also (as a hidden field) include the original HTML of a posting (using htmlentities to stuff it into a form field value). The script that processes the form I altered to both have a path for posting to websites (using micropub) and a new path to make a note in Obsidian, which is then saved as a .md file to the folder I store all clipped articles in.
To make a note I shape the available input the same way I template clipping things from the browser. At the top is my rationale for clipping something and reference to the source, followed by the original posting after which I add some keywords as tags and again the reference to the source.

In the images below you see the corresponding elements marked both as they appear in the reader as well as the resulting note.

The article as shown in my feed reader:

1: the original HTML content from a feed
2: title of the article (prefilled by my feed reader)
3: name of the author (prefilled by my feed reader)
4: original article’s URL (prefilled by my feed reader)
5: the reason and context why I am saving this to notes (also used to write a reply to a post, or the reason for bookmarking something if it will be posted on my site)
6: a quote I want to highlight
7: keywords that will become tags or categories on my site, and tags in my notes
8: selector for which site to post to (zyl is my blog), or ‘obs’ for making a note in Obsidian

Except for that last one those numbers are marked on the image of the resulting markdown note.

The resulting note in Obsidian:

1: the original HTML content from a feed shown in Markdown as the main body of the note
2: title of the article, both shown as part of the content of the note, as well as the title of the note (where a timestamp is added)
3: name of the author (mentioned with the source both at the top and bottom)
4: original article’s URL (mentioned with the author both at the top and bottom)
5: the reason and context why I am saving this, always at the top as it helps me process the content better
6: a quote I wanted to highlight
7: keywords that have become hashtags

(This posting was also written in my notes and, except for the images, posted directly from Obsidian to my site. Meaning I can both automatically move material into Obsidian, as well as automatically move material out of Obsidian. I quite enjoy the feeling of using that ‘magic’.)