I mentioned wanting to automatically add related blogposts based on comparing embeddings. When Dries Buytaert blogged about it, I wanted to follow his example. However I want to use those embeddings not just on this site, but also in my notes that include a local blog archive. That way I can quickly find related postings or notes with what I am writing/thinking. Embeddings are a numerical vector representation of the contents of a blogpost. You then find related blogposts by comparing those numbers across postings. Two weeks ago I set up the blog archiving.
Today I used the local archive of blog posts to calculate the embeddings. I wanted to calculate the embeddings on my local machine. When I write a blogpost and it gets stored in my local archive that should trigger calculating the embedding. Which I can then re-use in my notes or on my site.
There are multiple models available for local use to calculate embeddings, and I wanted to test which one seems most effective.
I installed three models in Ollama locally, and ran my complete blog archive through them:
- Nomic-embed-text-v1.5, which took 5 minutes to get through my blog archive, a US model by Nomic
- bge-m3, which took 12.5 minutes, a Chinese model by BAAI, Beijing Academy for AI
- qwen3-embedding:0.6b, which took 25 minutes, a Chinese model by Alibaba Cloud
The processing time difference across all three models is a factor 5, but that is mostly only relevant this first time when I run the entire archive (or when I need to redo that). From now on, only newly written or edited existing postings will have to have their embeddings (re)calculated. The time used for a single posting is insignificant whichever model I use (the slowest model took about 0.4 seconds for a blogpost, the fastest about 0.1).
Using Nomic I noticed that it handles language as a dominant distinction, meaning a Dutch post will only ever have Dutch language related postings, and German postings only German language ones. While there are topics that will likely indeed stay within the context of a single language, many of my Dutch postings are talking about things I also write about in English a lot. So Nomic is off the list. The other two models were created with multi-linguality in mind.
To compare the remaining two, I looked at 25 or so postings, and compared the first 10 related postings both models gave me. I intend to only ever show at most 5 related postings, if any, but looking at the first 10 gives an opportunity to see what else the embeddings calculate as relevant. Bge-m3 was slightly better at showing indeed related postings in the top 5. Qwen3-embedding worked well enough in the top 5 too and was stronger further down the list, where Bge-m3 started showing more clearly unrelated postings further down. There the Qwen model would find relations to things mentioned in longer blogpostings that Bge missed but I see as relevant. So I can use either for showing the most likely related blogposts on my blog, with bge-m3 being the smaller and faster model to calculate embeddings.
On the other hand, in my notes when writing, the Qwen model would work better if looking at entire blogposts. However, I don’t intend to make comparisons there to full length blogposts, but to paragraphs in long blogposts. There are 264 blogposts longer than 800 words (their average is 1200 words. Thirteen over 2000 words, the longest 2850 words, often talk transcripts) Then the difference between the two models will dissolve again. For now I am going forward with Bge-m3.
A concern is there might be model drift, even if always nominally using the same model. This can only happen if I update the model locally, and then it is needed to check against known results whether there has been drift. I have the current embeddings, so occasionnally recalculating a few and compare as well as logging a few good results to see if they still hold would be enough. The remedy for such drift when it influences results too much is to recalculate all embeddings again. That, as shown above, can be done fairly quickly. Something to add to the maintenance note for this set-up.
The next stage is building a way of storing / pushing the embeddings to this blog and in my notes, and write the php script (based on what Dries created) that does the comparisons both on my site and across my notes.
