In reply to Indexing, filing systems, and the art of finding what you havee by Austin Kleon

What I have started doing last April, following a tip by Wouter Groeneveld during the first Dutch language Obsidian meet-up, is scanning notebooks (I use a camera above the notebook activated with a foot button, so you can quickly flip through it). Those scans are available as a folder in my notes app (obsidian.md), where I work through them to create an index first, and to turn them into notes within my digital system later when it becomes useful / of interest.

A writer has to have a system for going back through old work and finding ideas….

But I have a ton of material that never makes it online, and I need to get it out of my notebooks and into an indexed and fully searchable system. I think this will be easiest if I do it as I go, and keep it simple: the minute I finish a notebook, go back and type the whole thing into a .txt file and save it. (And back it up.)

I suspect that rather than being totally dreary, this transcribing step can also be a creative step, and I will see patterns of thought, generate new ideas…

Austin Kleon

Ever since I was a child in primary school, my mental image of a year has been that of a circle, with January and December at the bottom, and July and August at the top (you’ll notice that this means the months aren’t evenly spaced around the circle in my mental image, spring takes up less of the circle than the fall. It’s a mental image, not a precise graph, likely influenced by my childhood sense of the endlessness of summers, and the long period of darkening days of fall and winter).


My mental image of a year ever since childhood

This Monday I completed a full circle around that image: I’ve been reading my own blog posts from previous years on each day, to see which of those I can take an idea or notion from to convert into a note in my digital garden (named ‘Garden of the Forking Paths‘). Peter has been going around the circle with me I read today (which in turn prompted me to write this), starting from my posting about it last year. He’s been reading his old blogposts every day, not just to reread but also to repair links, bring home images to self-host, clean up lay-out etc. I’m sure I am and have been my own blog’s most avid reader ever since I started writing in this space 19 years ago, and like Peter had been using my ‘on this day’ widget to repair old blog posts since I added the widget in early 2019.

Now I’ve come full circle on reading those blogposts for a year to mine them for their ideas and notions. The next cycle until the summer of 2022 I am adding a layer.

I will of course be making another round through my own blogposts like I did before. Because sometimes I missed a day, I haven’t repaired all of them each day, and I may take new meaning from them the next time I read them.
The layer I’m adding is also reviewing the personal notes I made on this day last year. This concerns the daily notes I make (a habit I started in April 2020), the other notes I’ve created on a certain date (work notes, ideas, travel etc), and indeed the blog posts I converted to notes dated on this day last year.

I see Frank has also picked up on Peter’s posting, and is embarking on a year of reading his own daily postings as well. Like Frank, I have never blogged on this day of the year since this blog started. And like for his blog, that has now changed.

Going in circles… I suspect life is circles, not turtles, all the way down. At least when you get a bit older that is.

In this next part looking at my use of Obsidian I want to describe in more detail what notes I take and how I take them.

Taking better notes is the actual reason I started using Obsidian. Using Obsidian for my work, day logs, and task management came later, and that covers the hierarchical part of my PKM system. The note taking part is the networked part of it. The system works for me because it combines those two things and has them interact: My internal dialogue is all about connected ideas and factoids, whereas doing activities and completing projects is more hierarchical in structure.

I make four types of notes: Notions, Notes, Ideas and work notes.

That last type, work notes, are the project and task related notes. Things I write down during meetings, notes from interviews, or ideas on how to move forward in a project. These live in the hierarchical structure I described in Pt 2. They can be linked to Ideas, Notes or Notions, or may give rise to them, but they serve a purpose firmly rooted in ongoing work. They are always placed within the context, and folder, of a specific project. This post isn’t about those notes.

The other notes live in the non-hierarchical, networked part of my system. They are added as I go along based on things I think about and information I come across. They become part of the system and get context not by the folder they are placed in (as is the case with work notes in a project folder), they become part of the system because they get linked to existing notes that I associate them with. They are never not linked to at least one other note. The links over time form patterns, and emergent patterns lead to new insights. Those new insights get expressed in additional links and in new notes.

These networked type notes come in three shapes, Notions, Notes, and Ideas. Each has their own folder to keep them separated from other material.
The folders are named Garden of the Forking Paths (for Notions), Notes (which I may yet rename), and the Ideas-greenhouse. I will discuss them one by one.

Ideas-greenhouse
The ideas-greenhouse holds ideas I have, ideas that seem like something that can be put to action more or less quickly. They may be connected to notes in the other two folders, or to notes in the project folders. An example would be, that I jotted down the idea of making a digital garden for my company two months ago, triggered by a posting on how a community should have its governance documented in combination with having reread the communication handbook of Basecamp while thinking about remote working. It has since morphed into building a collective memory, and turned into a budding internal website documenting the first few things. This is useful when we are onboarding new people, and as a reference for all of us, so colleagues feel better equiped to decide something on their own or ask better questions if they do need someone else’s advice on a decision. So the ideas greenhouse contains ideas that can be acted upon after some tweaks. They may be refined over time, before such action, or connected to and recombined with other ideas in the greenhouse.

Notes
Notes, things I come across that strike me as interesting (filtered by my current favourite topics, but not exclusive to it, to aid serendipity), which I jot down while adding why I find it of interest. They’re more like general resources, in which I can keep/find examples, quotes or pointers, extended with some notes on what I think of them. Notes are things that may result from work notes (something someone said in a meeting, an example held up), or from feed reading (pointers to things, perspectives found in someone else’s blog), regular browsing and reading, or questions that I come up with. It’s a mix of stuff, and that’s why it still is called simply ‘Notes’. I may yet come up with a different name for them. An example would be a note I made last week, called SensRnet. SensRnet is the attempt at creating a national database of sensors placed in publice spaces, by local and national government entities. It came up in a meeting with a client. I jotted it down, adding a few links to its source code on GitHub published by the Dutch Cadastre, and links to articles written by some local governments about it. I also mention the outcome of a project my colleague Marc did a few years ago writing local regulations governing the placement of such sensors in public spaces, and doing an analysis from the legal viewpoint. That’s how a Note starts. I may copy some text into at some point, and summarise it over time, or add other context in which I encountered the same thing again. Notes are ‘factoid’ like, resources written down with the context added of how I found them and why I was interested. That’s different from Notions which are already at creation more about the future use it may have.

Notions and proto-notions
Notions are conceptual notes taken from my own work and experience mostly, and give my own perspective on these concepts. Most of the current well over 500 come directly in my own words from my blogposts of the past 18 years and presentations I gave during that time.

As they are more conceptual than factual I started calling them Notions to distinguish them from the other more general resource Notes. I keep these Notions in a folder called Garden of the Forking Paths (see the name explained)

Each Notion links to at least one other Notion, and while I write them I think about how what I am writing connects to other things already in the Garden of the Forking Paths (GotFP). I may also add additional links or tags, as I come across a Notion while pursuing something else.

Usually while writing a Notion, I show the graph of how it connects to other Notions/Notes alongside it. I set the graph to show not only the 1st level links, as that only shows the links already apparent from the text I have in front of me. I set it to show 3 steps out at the start, and reduce to two steps when there are more links. That way you see the entire vicinity of a Notion, and it may trigger additional perspectives and associations. It’s a way to leverage the ‘weak ties’ between Notions, which is the place where new information generally comes from.

Below you see two graphs for a Notion called ‘3D to navigate information’, gleaned from a 2006 blogpost I wrote. The first image is the graph for direct links, showing two links. The second image is the graph for a distance of 2 (links of links), and it shows a much wider picture. It may well be that seeing that graph being created alongside a Notion while I am writing it, leads to adding in another link.

The Notion ‘3D to navigate information’ is linked to two others, one on how the physical and information landscape overlap and correlate, and one about what I think would be useful functionality for social software tools.

If you look at the same graph with distance 2, the layer of additionally visible nodes show how my new Notion might be connected to things like online identity, using the environment to store memory and layered access to information. This triggers additional thoughts during the writing process.

I spin out notes and potential Notions from my project notes, as I encounter things in my work where some idea or thought jumps out. Those potential Notions I put in a folder called proto notions, inside my GotFP.

Processing notes and proto-notions
Both the notes and proto-notions I touch upon every now and then, further summarising them or adding explanation and perspective, rewording them, linking them to other notes (this is what Tiago Forte calls progressive summarising). Proto-notions may yet become Notes and not evolve into Notions. Some of what starts as a Note may become a Notion in the GotFP, but most will always remain notes. Most ‘factoids’, even if reworded and put into the context of why I find them interesting will always be Notes. Notions usually are about concepts pertaining to vision, values and practices. Linking them is a key part of those concepts, as it binds them into my network of concepts and thoughts, it puts them as atoms into the constellations that make up my perspective of things. Notes can be specific examples of Notions.

I previously described how I use certain tags and referencing and naming conventions for Notes and Notions.

Using Notions and Notes
I use Notions and Notes in my work directly, pulling them into project notes, by transclusion, or e.g. when writing project proposals. I regularly call them up in conversations when something related gets being discussed, so I can re-use parts of them.

I also use Notions to create new blogposts and presentations. Last month I gave two presentations which were entirely created from collating a few Notions and adding a line or two to have them flow over into each other. One was on government core (base) registers, the other on Ethics as a Practice. Two months ago I blogged about how I see the role of cities, and that too was constructed from Notions.

Next to actual output, I pull together Notions, and sometimes Notes in what I call ’emergent outlines’ (Söhnke Ahrens in his book about Zettelkasten calls them speculative outlines, I like emergence better than speculation as a term). These are brief lists to which I add Notions that I think together flow into a story. As I use transclusion I can read them using the underlying Notions. Emergent outlines are a lightweight and bottom-up way to write more, that has a much lower threshold than thinking up a writing project and sitting down trying to write it out.

Feedreading and Notes / Notions
Feedreading is a source for Notes, sporadically for Notions. I notice a rising need with myself for higher quality material as input. Blog reading is conversational to me, and for a long time I’ve been content with that conversation as it is. Now I more often want to look into things more deeply. A blog conversation is no longer mostly the endpoint and more frequently the starting point for an exploration, leading me down a trail of links deeper into a topic. In the past three months I’ve read more scientific articles than in the past 3 years I think. Scientific articles and other documents I keep in Zotero, and from my notes I reference the Zotero entry. This difference in how I perceive my feed reading will likely shift my focus to how to read those feeds much more ‘inside-out’, i.e. starting from a question or topic, and checking what specific people in my blogroll say about them. This is funcionality more or less missing from feed readers, so it may lead me to want to tinker some more.

This concludes the 4th part of describing how I use Obsidian. There’s one more coming, which is all about Obsidian’s functionality as a viewer on my markdown files: the use of workspaces.

I’ve been using Obsidian a little over 100 days now. So, with over three months of daily use it’s good to review the experience. I will do this in some detail, and it will span several blogposts. To explain both the evolution over time, as well as how I currently work with Obsidian in practice in a more detailed way, as Frank (rightly!) requested.

My system leads the use of tools

First off, a key point to make. I am using a system for myself to plan and do my work, maintain lots of things in parallel, and keep notes. That system consists of several interlocking methods, and those methods are supported by various tools. What I describe in my review of 100 days of using Obsidian, is not about Obsidian’s functionality per se, but more about how the functionality and affordances of Obsidian fit with my system and the methods in that system. With a better fit with my system and methods, I can reduce friction in my methods, and reduce the number of tools I need to use in support of those methods. At the same time, the use of a new tool like Obsidian influences the practical application of methods, it creates a different daily practice. Those shifts are of interest as well.

What I started with

The image below shows you how my overall system of work and taking in information looks. It’s a personal knowledge management system, that both takes care of the networked nature of making sense of new information and evolving interests, as well as the more hierarchical nature of working on projects and executing tasks. Both start with my general notion of where I want to be headed (‘goals’).

I used different tools for different parts of that image:

  • Excel (orange) for: listing goals (3-10 yrs out), the 3 month planning cycle I keep (along the lines of ’12 week year’), the habits I want to maintain or introduce, and tracking of those habits and project progress/fulfillment.
  • Things (red) for: areas of my life I’m active in, projects within those areas, and tasks in those projects.
  • WordPress (darkblue) for: daily logs (which I started keeping end of April this year, on an internal WP instance), week logs (internal draft blogposting), and of course for public blogging itself.
  • Evernote (blue) for: a list of all my current interests/favourite topics, all types of note taking, related to my work/projects and my information diet.
  • Other tools (grey) come into play for feedreading (Readkit), blocking time (Nextcloud calendar in Thunderbird), book reading (Kindle, Nova2), keeping references (Zotero since June, Evernote before that)

While evaluating my system, I tried Obsidian

In the spring I had started evaluating my system. I found I was not keeping up several parts of it, had fallen out of practice with a number of elements, and had changed some of my practices without adapting the flow in my tools. It had therefore suffered in its usefulness. Being at home because of the pandemic allowed me to allocate some time to take a better look, and to start testing some changes. On the tool side of that evaluation, I want to get rid of Evernote (as a silo and single point of failure) since some years.

One change in my system I was experimenting with, was keeping better atomic notes about the core concepts and key elements in how I work. Late last year I thought a bit about atomic notes, i.e. cards with individual snippets, and bringing those collections of snippets and the process of curating them and threading them into e.g. a blogpost or a line of argumentation. In January I came across Zettelkasten and took a closer look, in the spring I read a book about Zettelkasten and knew I wanted to adopt parts of it into my system (linking notes first and foremost, and storing references in a better way). That’s when I started using Zotero to keep references, and stopped doing that in Evernote (Zotero can take website snapshots and store them locally, something I used Evernote for a lot. On top of it if you give Zotero a reference it will find and store a PDF of a scientific article, very useful to read more deeply).

I started to keep atomic notes, sometimes called ‘evergreen notes’ which I to myself now call Notions, capturing concepts from my work (so not work related notes, but conceptual notes) first in both WordPress and Evernote simultaneously. WordPress (a local instance on my laptop, not online) because I already used it for day logs since April, and it allows relatively easy linking, and Evernote because it is much easier to keep notes there than WP, but linking in Evernote is much harder. I also played with some note taking tools, and that’s when I came across Obsidian. It immediately felt comfortable to use it.

How after 100 days Obsidian has covered my system

After over 100 days of Obsidian my use of it has expanded to include a much larger part of my system. Along the way it made my use within that system of Things, Evernote and almost Excel obsolete. It also means I sharpened my system and practice of using it again. This is how the tool use within my system, with the use of Obsidian in green, now looks

Obsidian now contains some 1200 mark down files. 500 are Notions, atomic notes almost exclusively about my own concepts and other core concepts in my work, in my own words. Mostly taken from my own blogposts, reports, and presentations over the years. The other 700 are some 115 day log / week log / month maps, about 100 proto-notions and notes that contain conceptual info to keep from other sources, and some 500 work and project related notes from conversations and work in progress. This sounds as a very quantitative take, and it is. I have in the past months definitely focused on the volume of ‘production’, to ensure I could quickly experience whether the tool helped me as intended. I think that monitoring the pace of production, which I’ve done in the past months, will no longer be relevant by the end of this year. I used the quantity as a lead indicator basically, but have been on the lookout for the lag indicators: is building a collection of linked notes leading to new connections, to more easily creating output like blogposts and presentations, having concepts concisely worded at hand in conversations to re-use? And it did. One very important thing, central to the Zettelkasten method, I haven’t really tried yet however, which is to use the current collection as a thinking tool. Because I was more focused on creating notions first.

On Obsidian as a tool

There are four things in Obsidian that are to me key affordances:

  1. it is a viewer/editor, a fancy viewer/editor, on top of plain markdown text files on my laptop. It builds its own local database to keep track of links between notes. Whatever happens to Obsidian, my data is always available.It being ‘just’ a viewer is important because Obsidian is not open source and won’t be. There is a potential open source alternative, Foam, but that tool is not yet developed enough.
  2. being ‘just’ an editor means using regular text files, it feels like coming full circle, as I have for the most part been note taking in simple text files since the late ’80s. Textfiles always had my preference, as they’re fast and easy to create, but it needed a way to connect them, add tags etc., and that was always the sticking point. It means text files are available outside of Obsidian. This allows me to access and manipulate notes from outside Obsidian without issue, and I do (e.g. on mobile, but also with other software on my laptop such as Tinderbox that I used for the images in this post).
  3. it makes linking between notes (or future links) as simple as writing their filenames, which is supported by forward search while you’re typing.
  4. it shows graphs of your note network, which to me is useful especially for 2 steps around a note you’re working on.

I use Obsidian as simple as possible; I do not use plugins that are supposed to help you create notes (e.g. the existing Zettelkasten and Day log plugin), because they make assumptions about how to create notes (how to name them, which links to create in them). I created my own workflow for creating notes to avoid functionality lock-in in Obsidian: day logs are created manually by keyboard shortcuts using Alfred (previously TextExpander), as are the timestamps I use to create unique file names for notes.

Timeline of three months of Obsidian use

Below is a timeline of steps taken in the past months, which gives you an impression of how my use of Obsidian in support of my system has evolved.

November 2019 I discuss the concept of cards (i.e. atomic notes), curation and writing output

January 2020 I first looked at the Zettelkasten method and some tools suggested for it. I mention the value of linking notes (possible in Evernote, but high friction to do)

May 2020, read the book about Zettelkasten by Sönke Ahrens, adopted Zotero as a consequence.

7 July started with deliberately making Zettelkasten style atomic notes in WordPres en Evernote in parallel, to move away from collecting as dumping stuff in your back yard. Atomic notes only concerning my concepts in my work.

8 July started using Obsidian, after having just started creating ‘evergreen’ notes

15 July having made 35 atomic notes, I make a new association between two of them for the first time.

28 July I’m at 140 conceptual notes. I named the collection Garden of the Forking Paths. I switched my digital tickler files (a part of the GTD method) from Evernote to Obsidian. I had stopped using them, but now it felt normal again to use them. The post I wrote about this, was made from atomic notes I already had made beforehand.

5 August I find I haven’t used WordPress anymore for my day logs ever since starting with Obsidian, and that I also added week logs (an automatic collation of day logs), and monthmaps (a mindmap at the start of the month listing key upcoming things and potential barriers). My Evernote use dropped to 4 notes in 4 weeks, whereas it was 47 the 4 weeks before it. After almost a month of Obsidian, I am getting more convinced that I am on a path of ditching Evernote.

12 August I renamed my ‘evergreen’ notes, that contain my concepts mostly, to Notions, as the generic word notes doesn’t make a distinction in the character of some the things I’m putting into notes.

12 August I write a first long form blogpost made from Notions

13 August Added Nextcloud synchronisation of the note files, allowing mobile viewing and editing of notes

31 August I keep track of tasks in Obsidian and drop Things. There was a time I always did such things in straightforward text files. Being able to do so again but now with a much better way of viewing and navigating such text files and the connections between them, makes it easy to ‘revert’ to my old ways so to speak.

13 September I am at 300 Notions. These first 300 notions are mostly my notions, the things that are core to my thinking about my own work, and the things I internalised over the past 25 years or so, of doing that work. I expect that going forward other people’s ideas and notions will become more important in my collection.

13 September I describe how I make notions and notes

September / October I increasingly use my conceptual Notions as reference while in (online) conversations.

5 October I gave a client presentation (about the Dutch system of base registers) pulled together completely from existing Notions.

7 October added a ‘decision log’ to my note keeping.

16 October 100 days in Obsidian, 500 Notions and about 700 other types of notes.

16 October reinstated a thorough Weekly Review (a component of GTD) into my system.

21 October I gave a brief presentation Ethics as a Practice, the second this month pulled together from existing notes.

This all as a first post looking back on 100 days of Obsidian.
Part 2: Hierarchy and Logs
Part 3: Task management
Part 4: Writing connected Notions, Ideas, and Notes
Part 5: Flow using workspaces
Part 6: Obsidian development vs my usage

This morning I wrote my 300th notion, the term I use for my permanent notes (it was on how the societal impacts of novel infrastructures change the scope and paths of your empathy). These notions I started writing from July 8th, so it took about two months. I hadn’t written almost any in the past week or two, and that felt uncomfortable. Thinking about it I realised I wasn’t going through my old presentations as much anymore, and just picking a presentation from the list helped me get back into it. Regularly those presentations contain 1 idea per slide, so they are a rich source. A second much used source are my existing blog posts. Each day I look at which postings I created on this date in the previous 17 years of this blog.

These first 300 notions are mostly my notions, the things that are core to my thinking about my own work, and the things I internalised over the past 25 years or so, of doing that work. Likely there will be a point where I’ve worked through most of my existing material, and new notions will come more from reading novel material, reading other people’s material. This will take more effort I expect, because I’d need to digest my reading and think about it before lifting out the bits I want to keep. That’s different from what is now basically transcribing my slides or my old blogposts. Likely it will need a slightly different process, with more notes in the process of turning into notions in parallel.

I just realised that it’s a month this Friday that I started using markdown textfiles and Obsidian for notes, and that I have not used my local WordPress install at all during that time, nor Evernote much. I made 4 notes in EN in a month: 1 bookmark, 1 shopping list, 2 call logs. Compared to 47 notes the month prior to it.

Day logs and work notes are now in markdown files, internal wikipages are now my Garden of the Forking Path notes in markdown files. Those were previously in my local WP install. Bookmarks aren’t mindlessly send to Evernote at a touch of a button anymore, with the vague intention of reading later and/or having it come up in a search at some point in the future. Reading ‘later’ never really works for me (Instapaper never succeeded in really landing in my workflow). So now it’s either I read it and want to keep it for reference by adding a snapshot to Zotero, or I did not read it and trust that if it’s important it will resurface at some point again. Other elements in my use of Evernote I’ve recreated on the go in text files quite naturally: Folders for each of my areas of activity match up with what I have as Notebooks in EN.

It feels like coming full circle, as I have for the most part been note taking in simple text files since the late ’80s. I started paying for Evernote in 2010, after using the free version for a while, and used wiki in parallel to text files for note taking for a number of years before that (2004-2008 I think). Textfiles always had my preference, as they’re fast and easy to create, but it needed a way to connect them, add tags etc., and that was always the sticking point. Tools like Obsidian, Foam and others like it are mere viewers on top of those text files in my file system. Viewers that add useful things like visualising connections, and showing multiple queries on the underlying files in parallel. It adds what was missing. So after a month, I am getting more convinced that I am on a path ditching Evernote.

Time to start syncing some of my notes folders to my phone (through NextCloud), and choose a good editor for Android, so I can add/use/edit them there too.