I imported hundreds of Amazon e-book purchases into my book notes, using a script I wrote with the assistance of GitHub Co-Pilot.

As a home-cooking coder, coding things often takes me a long time. I know how to cut things up in order to be able to code the pieces. I know a bit of my default coding language PHP, and can read it to see what it does. But actual coding is a different thing. It’s more like a passive fluency rather than an active one. Mostly because I don’t do it often enough to become actively fluent, even though I have been coding my own things since the early 1980s. So coding for me means a lot of looking up how statements work, what terms they expect etc., which is time consuming.
Over time I’ve collected pieces of functionality I reuse in various other projects, and I have a collection of notes on how to do things and why (it’s not really a coding journal, but it could grow into one.) Yet it is usually very time consuming.

Earlier this year I took a subscription on Github Co-pilot. I installed two plugins in Visual Studio Code, the text-editor I use for coding: Co-pilot and co-pilot chat. I thought it might help me make it easier to create more personal tools.
It took until yesterday before I both had the urge and the time to test that assumption.

I am backfilling different types of information into my Obsidian notes. Such as my Google calendar items from 2008-2020, earlier.
Another is ensuring I have a book note for every book I bought for Amazon Kindle. I’ve bought just over 800 books since December 2010 for Kindle (vs 50 physical books since 2008, as I usually use local bookshops for those). For a number of them I have book notes in Obsidian, for others I don’t. I wanted to add notes for all Kindle books I bought over the years.
And this gave me my first personal tool project to try Co-pilot on.

The issue is having a list of Amazon Kindle purchases (title, author, date) and a list of existing book notes, where the title is usually shorter than the one on the Amazon list (no sub title e.g.). I set out to make a script that checks every existing book note against the Amazon list, and writes the remaining Amazon purchases to a new list. Then in a next step that new list is used to create a book note with a filled out template for each entry.

Using Co-pilot and especially the chat function made the coding quick and easy. It was also helping me learn as the chat provides reasons for its suggestions and I could go back and forth with it to understand various elements better. A very useful effect was that from having to write prompts for the chat bot and following up on the answers allowed me to much better clarify to myself what I was trying to do and coming up with ideas how to do it. So it sped up my thinking and creation process, next to providing helpful code suggestions that I only needed to tweak a bit for my use case (rather than find various solutions on stack-overflow that don’t really address my issue). It also helped me make useful notes for my coding journal and code snippet collection.
It was still time consuming, but not because of coding: data cleaning is always a big chore, and will remain so because it needs human inspection.

I now have a folder with some 475 automatically made book notes, in the right structure, derived from the 800 Kindle book purchases over 13 years using my existing book notes as filter.
Next projects to have a go at will be the physical book purchases through Amazon (50), and my old Calibre library of all books I owned before 2012 (over 1000, when we did away with most of them, after I scanned their barcodes all into a database.)

I am pleased with how helpful GitHub Co-Pilot was for me in this. It energises me to think of more little coding projects for personal tools. And who knows, maybe it will increase my coding skills too, or have me branch out in programming languages I don’t know, like python, or help me understand other people’s code like in WordPress plugins I might want to tweak.

In reply to Creating a custom GPT to learn about my blog (and about myself) by Peter Rukavina

It’s not surprising that GPT-4 doesn’t work like a search engine and has a hard time surfacing factual statements from source texts. Like one of the commenters I wonder what that means for the data analysis you also asked for. Perhaps those too are merely plausible, but not actually analysed. Especially the day of the week thing, as that wasn’t in the data, and I wouldn’t expect GPT to determine all weekdays for posts in the process of answering your prompt.

I am interested in doing what you did, but then with 25 years of notes and annotations. And rather with a different model with less ethical issues attached. To have a chat about my interests and links between things. Unlike the fact based questions he’s asked the tool that doesn’t necessarily need it to be correct, just plausible enough to surface associations. Such associations might prompt my own thinking and my own searches working with the same material.

Also makes me think if what Wolfram Alpha is doing these days gets a play in your own use of GPT+, as they are all about interpreting questions and then giving the answer directly. There’s a difference between things that face the general public, and things that are internal or even personal tools, like yours.

Have you asked it things based more on association yet? Like “based on the posts ingested what would be likely new interests for Peter to explore” e.g.? Can you use it to create new associations, help you generate new ideas in line with your writing/interests/activities shown in the posts?

So my early experiments show me that as a data analysis copilot, a custom GPT is a very helpful guide… In terms of the GPT’s ability to “understand” me from my blog, though, I stand unimpressed.

Peter Rukavina

Over drie weken, op 18 december vindt de PKM / Obsidian gebruikers meet-up plaats! De Digitale Fitheid community en de PKM groep van de Nederlandse Vereniging van Informatieprofessionals (KNVI) zijn de gastgevers. Vanaf 19:00 duiken we bij S2M Utrecht in hoe we persoonlijk kennismanagement doen, en hoe ieder van ons dat al wel of nog niet in Obsidian implementeert. Elkaars werkwijzen zien en bespreken is altijd enorm inspirerend, en levert nieuwe ideeën op hoe je je eigen PKM-flow kunt tweaken en hoe je je Obsidian gereedschap scherper slijpt.

We willen veel tijd nemen om elkaar dingen te kunnen laten zien. We gebruiken daarom in ieder geval 2 projectieschermen naast elkaar, zodat we ook dingen kunnen vergelijken en we sneller meer mensen iets kunnen laten tonen.

Leidend is telkens de vraag “Hoe doe jij X in je PKM systeem, en hoe heb je dat geïmplementeerd in je Obsidian set-up?”
Waar X een onderwerp kan zijn als:

  • hoe zoek en vind je in jouw PKM systeem?
  • hoe begrens jij jouw PKM systeem, hoort productiviteit/GTD er bij, of alleen leren? Is jouw PKM flow generiek, of gericht op bepaalde thema’s?
  • naar welke outputs / resultaten werk je in je PKM toe (schrijven, vertellen, creatieve ideeën etc.)?

En daaromheen zijn er interessante thema’s als:

  • Welke overgangen van analoog naar digitaal en vice versa zitten er in jouw systematiek? Hoe speelt analoog een rol in je leren?
  • Welke visuele elementen spelen een rol in je PKM systeem?

Tot slot, omdat Obsidian werkt op lokale bestanden, is ook een onderwerp:

  • Hoe gebruik je vanuit andere programma’s of werkwijzen die lokale files buiten Obsidian om?

We vragen je iets dat jij graag wilt laten zien uit jouw PKM systeem in Obsidian voor te bereiden. Aan het begin kijken we wie graag bepaalde voorbeelden wil zien of laten zien. Vanuit dat overzicht van onderwerpen gaan we dan aan de slag.

Gratis aanmelden kan op Digitale Fitheid. Tot 18 december!

I have lots of images in my Flickr account, over 35k uploaded from April 2005 until now. Searching them by date isn’t made easy: you first have to do a search and then in the advanced filters set a date range. The resulting search URL however is straightforward, and contains my user id, and both the start and end date to search in unix time.

To make it easy for myself I made a tiny tool that has a locally hosted webform where I enter a start and end date to search, and turns that into the right search URL which it shows as clickable hyper link. Meaning with two clicks I have the images I searched for by date in front of me (if they exist).

The small piece of code is shown below (save it as a .php file, insert your Flickr user ID, and run it on a local webserver on your laptop, add its URL to your bookmark bar in your browser)

/* form om te zoeken in Flickr */
if ($_POST) {
$dvan = $_POST['van'];
$dtot = $_POST['tot'];
$uvan = strtotime($dvan);
$utot = strtotime($dtot);
$searchurl=$base.$uvan."&max_taken_date=".$utot."&view_all=1"; echo "<a href=".$searchurl.">".$searchurl."</a>";
/* begin form */
<form name="input_form" method="POST" action="<?php echo $_SERVER['PHP_SELF'] ?>">
Search Flickr (date format yyyy-mm-dd)<br/>
From <input type='text' size='20' name='van'><br/>
To <input type='text' size='20' name='tot'><br/> <br/>
<input type="submit" name="submitButton" value="Zoek">

I realised I had an ical file of all my appointments from the period I used Google Calendar from January 2008 when I started as an independent consultant, until February 2020 when I switched to the calendar in my company’s NextCloud.
I never search through that file even though I sometimes wonder what I did at a certain moment in that period. After a nudge by Martijn Aslander who wrote on a community platform we both frequent about back filling his daily activities into Obisidian for instance based on his photos of a day through the years in his archive, I thought to import that ical file and turn it into day logs listing my appointments for a date.

I tried to find some ready made parsers or libraries I could use in PHP, but most I could find is aimed at importing a live calendar rather than an archived file, and none of them were aimed at creating an output of that file in a different format. Looking at the ical file I realised that making my own simple parser should be easy enough.

I write a small PHP script that reads the ical file line by line until it finds one that says BEGIN:VEVENT. Then it reads the lines until it finds the closing line END:VEVENT. It then interprets the information between those lines, lifting out the date, location, name and description, while ignoring any other information.
After each event it finds, it writes to a file ‘Daylog [date].md’ in a folder ./year/month (creating the file or appending the event as a new line if the file exists). It uses the format I use for my current Day logs.
Let it repeat until it processed all 4.714 events in my calendar from 2008 to 2020.

A screenshot of all the folders with Daylogs created from 2008-2020

Screenshot of the newly created Daylog for 20 February 2008, based on the appointments I had that day, taken from the calendar archive. This one mentions a preparatory meeting for the open GovCamp I helped organise that year in June, which kicked off my work in open data since then.

I and my team at The Green Land are looking for a self-hosted version of event organisation tools like MeetUp.com or Eventbrite. Both for small scale events as part of projects, such as meet-ups of citizen scientists, as well as for ourselves, such as small gatherings we organise around AI ethics with our professional peer network.

We don’t want to use Meetup.com or things like Eventbrite because we don’t want personal data to be handed over to US based entities, nor require the participants to do so just because they want to attend a local event. We also notice a strong hesitancy amongst participants of events when it is needed to create yet another account on yet another service just to let us know they will be joining us for something.

Nevertheless we do want an easy way to announce events, track registrations, and have a place to share material before, during and afterwards. And I know that events are hard in terms of discovery, because although there are a plethora of events, for most participants as well as event organisers they’re incidents (years ago I came across a blogpost describing this Events Paradox well.). Additionally, for us as professionals it is usually more logical to host our own events than find one that fits our needs.
So we need a way to announce events where we can assure participants there’s no need to hand over personal information, and where material can be shared.

There seem to be two FOSS offerings in this space. Mobilizon by Framasoft and Gettogether. In the past weeks my colleague S and I tried to test Mobilizon.

Mobilizon is ActivityPub based, and there’s a Yunohost version which I installed on our VPS early last month. Mobilizon promises several strong points:

  • Fully self-hosted, and able to federate with other instances. There aren’t many visible instances out there, but one NGO we frequently encounter in our network does run its own instance.
  • you can maintain different profiles in your account, so that for different parts of your life you can subscribe to events, without e.g. your historical re-enactment events showing up amongst your professional events in a public profile.
  • People can register for an event without needing an account or profile (using e-mail confirmation)

Working with Mobilizon turned out less than ideal at a very basic level. Accounts couldn’t log in after creation. As an administrator I could not force password resets for users (that couldn’t log in anymore). Not being able to do user admin (other than suspending accounts) seems to be a deliberate design choice.
I still had access through my Yunohost admin account, but after an update yesterday of the Mobilizon app that stopped working too. So now both instance admins were locked out. Existing documentation wasn’t much help in understanding what exactly is going on.

I also came across an announcement dat Framasoft intends to shift development resources away from Mobilizon by the end of the year, and thusfar there’s little momentum in the developer community to pick up where they intend to leave off.

For now I have uninstalled Mobilizon. I will reach out to the mentioned NGO to hear how their experiences are. And will look at the other tool, although no Yunohost version of it exists.

I’m open te hear about other alternatives that might be good to try.