To keep the database size down on my personal Mastodon instance I routinely delete everything older than a few days. This includes anything I bookmarked. The same is true for E’s instance. There’s no ready way to get those bookmarks out of Mastodon into something else. Unlike for public things where you can get an RSS feed from instances by adding .rss to a url, for the non-public bookmarks you need to use the API.

With some suggestions by my automaton junior coding assistant I quickly had a working API call to read the bookmarks (the url is yourinstance/api/v1/bookmarks, and you need to create an access token in your Mastodon instance under the developers menu heading).

Outputting those bookmarks as RSS is a straightforward way to make it accessible to various other applications. So I added code to make an RSS feed. And it works. The code is up on Github.
I’ve added the feed both to my regular feed reader (a self-hosted FreshRSS instance), and to the RSS plugin for Obsidian. The latter so that I can easily access the bookmarks in my notes. The former so that I can from within my feedreader send it to various websites I control as well as have a second route to my notes.

A quick and satisfying home cooked coding snack.

On the Obsidian forum I came across an intriguing post by Andy Matuschak. Matthew Siu and Andy have made an Obsidian plugin to help with sensemaking and they are looking for people with use cases to test it out.
I filled out the survey saying I had a large variety of notes about EU data law, digital ethics, and (data) governance, which I need to make sense of to guide public sector entities. They asked about online traces of me as well. Soon Matthew reached out and we decided on a time for a call.

And that is how I ended up working in Obsidian for an hour while Matthew and Andy were watching my shared screen. Sort-of how I once watched Andy work through his notes after reading a book. They’re on the US west coast, so with the nine hour time difference it was 22:00-23:00 hours here, which plus my cold meant I wasn’t as focused as earlier in the day. It also feels slightly odd to me having people watch me while doing actual work.

Because that was what I did, doing some actual work. Using notes from several conversations earlier this week, plus EU legal texts and EU work plans, and notes from workshop output from over a year ago, I was working towards the right scope of a workshop to be held early March.

The plugin I tried out is called the Obsidian reference plugin.
It allows you to select something in one note, and paste it in another. It links back to the source, is uneditable where you paste it, and marked where you copied it. When you hover over it, it can preview the snippet in its original context, when you click it the source opens/focuses in another tab. It seems a simple thing, and similar to block transclusion/references, yet still it had some interesting effects:

  • Initially I saw myself using it to cut and paste some things from different notes together in a new note. This is a bit like canvassing, but then solely in text, and focused on snippets rather than full notes.
  • The snippets you paste aren’t editable, and the idea is you can paraphrase them, rather than use them as-is like in block transclusion. I did a bit of that paraphrasing, but not a lot, it was more like gathering material only. Perhaps as I was bringing together parts of my own (conversation) notes. I can see that when going through the annotations of a source text, this can be a second step, thinking highlights and annotations through, remixing them, and come up with some notions to keep (see second to last bullet).
  • It was easy to bring in material from a range of notes, without it becoming hard to keep an overview of what came from where. This is I think key when comparing different inputs and existing own notes.
  • Once I was interacting with the collected material, my use of additional snippets from other notes shifted: I started to use them inline, as part of a sentence I was writing. This resembles how I currently use titles of my main notions, they’re sentences too that can be used inside another sentence, as a reference inside a flowing text rather than listed at the end. I often do this because it marks the exact spot where I think two notions link. This means using smaller snippets (part of a phrase), and it is possible because the reference to the source is kept, and its context immediately accessible through hovering over it.
  • Discussing this effect with Matthew and Andy I realised another use case for this plug-in is working with the content of the core of my conceptual notes (that I call Notions) inside project or work notes. Now that reference is only to Notions as a whole. Adding a snippet makes a qualitative difference I want to explore.
  • You can collapse the snippets you create, but I didn’t do that during the hour I let Matthew and Andy watch me work. I can imagine doing that if I’m working through a range of snippets to paraphrase or use. I can see this being useful when for instance collating results from in-depth interviews. For my World Bank data readiness assessments the report was based on snippets from some 70 (group) interviews. A lot of material that I would mine along the lines of ‘what was said about X across all these conversations’, or ‘what assumptions are voiced in these interviews regarding Y’.
  • I spent the hour working with notes mainly from conversations, which are often pseudo-verbatim with my associations and questions I had during the conversation mixed in. Reading old notes often allows me to be ‘transported’ back into the conversation, its setting etc in my memory. Being able to glance at a snippet’s context from conversation notes as I work with it, and getting transported back into a conversation, felt like a rich layer of potential meaning being made easily available.
  • What I created in the hour was something I otherwise likely wouldn’t have. I was able to build or at least start a line of detailed argumentation for both the scope of the workshop in March I was working on this hour, as well as a line of argumentation to be used within that workshop to show participants why taking EU developments into account is key when working on a regional or local issue with data. In a more explicit way and I think I might otherwise have come up with a ‘result’ rather than the path to that result. ‘Thinking on paper’ in short. Useful stuff.
  • Reflecting on all this afterwards before falling asleep, I realised that a key way to use this is connected to the video I linked to above in which Andy gathers his thoughts and notes after reading a book: reflecting on an article or book I just read. A key part of the work there is seeking out the friction with previous reading and Notions. Not just to work with the annotations from a book as-is, but also the messy juxtaposition and integration with earlier notes. Then bringing in snippets from here and there, paraphrasing them into some sort of synthesis (at least one in my mind) is valuable. Collapsing of snippets also plays a role here, as you work through multiple annotations and ‘confrontations’ in parallel, to temporarily remove them from consideration, or as a mark of them having been used ‘up’.
  • Once you delete a snippet, the marking at its source is also removed, so if a link to source is important enough to keep you need to do that purposefully, just as before.

Bookmarked Coding on Copilot: 2023 Data Suggests Downward Pressure on Code Quality by William Harding and Matthew Kloster

Gitclear takes a look at how the use of Copilot is impact coding projects on GitHub. They signal several trends that impact the overall code quality negatively. Churn is increasing (though by the looks of it, that trend started earlier), meaning the amount of code very quickly being corrected or discarded is rising. And more code is being added to projects, rather than updated or (re)moved, indicating a trend towards bloat (my words). The latter is mentioned in the report I downloaded as worsening the asymmetry between writing/generating code and time needed for reading/reviewing it. This increases downward quality pressure on repositories. I use GitHub Copilot myself, and like Github itself reports it helps me generate code much faster. My use case however is personal tools, not a professional coding practice. Given my relatively unskilled starting point CoPilot makes a big difference between not having and having such personal tools. In a professional setting more code however does not equate better code. The report upon first skim highlights where benefits of Copilot clash with desired qualities of code production, quality and team work in professional settings.
Via Karl Voit

To investigate, GitClear collected 153 million changed lines of code,
authored between January 2020 and December 2023….. We find disconcerting trends for maintainability. Code churn — the
percentage of lines that are reverted or updated less than two weeks after
being authored — is projected to double in 2024 compared to its 2021,
pre-AI baseline. We further find that the percentage of “added code” and
“copy/pasted code” is increasing in proportion to “updated,” “deleted,” and
“moved” code.

Gitclear report

I imported hundreds of Amazon e-book purchases into my book notes, using a script I wrote with the assistance of GitHub Co-Pilot.

As a home-cooking coder, coding things often takes me a long time. I know how to cut things up in order to be able to code the pieces. I know a bit of my default coding language PHP, and can read it to see what it does. But actual coding is a different thing. It’s more like a passive fluency rather than an active one. Mostly because I don’t do it often enough to become actively fluent, even though I have been coding my own things since the early 1980s. So coding for me means a lot of looking up how statements work, what terms they expect etc., which is time consuming.
Over time I’ve collected pieces of functionality I reuse in various other projects, and I have a collection of notes on how to do things and why (it’s not really a coding journal, but it could grow into one.) Yet it is usually very time consuming.

Earlier this year I took a subscription on Github Co-pilot. I installed two plugins in Visual Studio Code, the text-editor I use for coding: Co-pilot and co-pilot chat. I thought it might help me make it easier to create more personal tools.
It took until yesterday before I both had the urge and the time to test that assumption.

I am backfilling different types of information into my Obsidian notes. Such as my Google calendar items from 2008-2020, earlier.
Another is ensuring I have a book note for every book I bought for Amazon Kindle. I’ve bought just over 800 books since December 2010 for Kindle (vs 50 physical books since 2008, as I usually use local bookshops for those). For a number of them I have book notes in Obsidian, for others I don’t. I wanted to add notes for all Kindle books I bought over the years.
And this gave me my first personal tool project to try Co-pilot on.

The issue is having a list of Amazon Kindle purchases (title, author, date) and a list of existing book notes, where the title is usually shorter than the one on the Amazon list (no sub title e.g.). I set out to make a script that checks every existing book note against the Amazon list, and writes the remaining Amazon purchases to a new list. Then in a next step that new list is used to create a book note with a filled out template for each entry.

Using Co-pilot and especially the chat function made the coding quick and easy. It was also helping me learn as the chat provides reasons for its suggestions and I could go back and forth with it to understand various elements better. A very useful effect was that from having to write prompts for the chat bot and following up on the answers allowed me to much better clarify to myself what I was trying to do and coming up with ideas how to do it. So it sped up my thinking and creation process, next to providing helpful code suggestions that I only needed to tweak a bit for my use case (rather than find various solutions on stack-overflow that don’t really address my issue). It also helped me make useful notes for my coding journal and code snippet collection.
It was still time consuming, but not because of coding: data cleaning is always a big chore, and will remain so because it needs human inspection.

I now have a folder with some 475 automatically made book notes, in the right structure, derived from the 800 Kindle book purchases over 13 years using my existing book notes as filter.
Next projects to have a go at will be the physical book purchases through Amazon (50), and my old Calibre library of all books I owned before 2012 (over 1000, when we did away with most of them, after I scanned their barcodes all into a database.)

I am pleased with how helpful GitHub Co-Pilot was for me in this. It energises me to think of more little coding projects for personal tools. And who knows, maybe it will increase my coding skills too, or have me branch out in programming languages I don’t know, like python, or help me understand other people’s code like in WordPress plugins I might want to tweak.

In reply to Creating a custom GPT to learn about my blog (and about myself) by Peter Rukavina

It’s not surprising that GPT-4 doesn’t work like a search engine and has a hard time surfacing factual statements from source texts. Like one of the commenters I wonder what that means for the data analysis you also asked for. Perhaps those too are merely plausible, but not actually analysed. Especially the day of the week thing, as that wasn’t in the data, and I wouldn’t expect GPT to determine all weekdays for posts in the process of answering your prompt.

I am interested in doing what you did, but then with 25 years of notes and annotations. And rather with a different model with less ethical issues attached. To have a chat about my interests and links between things. Unlike the fact based questions he’s asked the tool that doesn’t necessarily need it to be correct, just plausible enough to surface associations. Such associations might prompt my own thinking and my own searches working with the same material.

Also makes me think if what Wolfram Alpha is doing these days gets a play in your own use of GPT+, as they are all about interpreting questions and then giving the answer directly. There’s a difference between things that face the general public, and things that are internal or even personal tools, like yours.

Have you asked it things based more on association yet? Like “based on the posts ingested what would be likely new interests for Peter to explore” e.g.? Can you use it to create new associations, help you generate new ideas in line with your writing/interests/activities shown in the posts?

So my early experiments show me that as a data analysis copilot, a custom GPT is a very helpful guide… In terms of the GPT’s ability to “understand” me from my blog, though, I stand unimpressed.

Peter Rukavina

Over drie weken, op 18 december vindt de PKM / Obsidian gebruikers meet-up plaats! De Digitale Fitheid community en de PKM groep van de Nederlandse Vereniging van Informatieprofessionals (KNVI) zijn de gastgevers. Vanaf 19:00 duiken we bij S2M Utrecht in hoe we persoonlijk kennismanagement doen, en hoe ieder van ons dat al wel of nog niet in Obsidian implementeert. Elkaars werkwijzen zien en bespreken is altijd enorm inspirerend, en levert nieuwe ideeën op hoe je je eigen PKM-flow kunt tweaken en hoe je je Obsidian gereedschap scherper slijpt.

We willen veel tijd nemen om elkaar dingen te kunnen laten zien. We gebruiken daarom in ieder geval 2 projectieschermen naast elkaar, zodat we ook dingen kunnen vergelijken en we sneller meer mensen iets kunnen laten tonen.

Leidend is telkens de vraag “Hoe doe jij X in je PKM systeem, en hoe heb je dat geïmplementeerd in je Obsidian set-up?”
Waar X een onderwerp kan zijn als:

  • hoe zoek en vind je in jouw PKM systeem?
  • hoe begrens jij jouw PKM systeem, hoort productiviteit/GTD er bij, of alleen leren? Is jouw PKM flow generiek, of gericht op bepaalde thema’s?
  • naar welke outputs / resultaten werk je in je PKM toe (schrijven, vertellen, creatieve ideeën etc.)?

En daaromheen zijn er interessante thema’s als:

  • Welke overgangen van analoog naar digitaal en vice versa zitten er in jouw systematiek? Hoe speelt analoog een rol in je leren?
  • Welke visuele elementen spelen een rol in je PKM systeem?

Tot slot, omdat Obsidian werkt op lokale bestanden, is ook een onderwerp:

  • Hoe gebruik je vanuit andere programma’s of werkwijzen die lokale files buiten Obsidian om?

We vragen je iets dat jij graag wilt laten zien uit jouw PKM systeem in Obsidian voor te bereiden. Aan het begin kijken we wie graag bepaalde voorbeelden wil zien of laten zien. Vanuit dat overzicht van onderwerpen gaan we dan aan de slag.

Gratis aanmelden kan op Digitale Fitheid. Tot 18 december!