Threading My Thoughts_

🇺🇦 Resources to help support the people of Ukraine. 🇺🇦
November 02, 2022 @14:00

It has been almost 3 years now since I built Thoughts — the microblogging platform and I'd say it's been a success. Over the years I've tweaked it a bit for cost and usability, integrating some quality of life features into the posting interface (I swapped out my own editor for Trix), changing the attachment processing pipeline to support videos, and making a rudimentary swing at rich link previews (that does need a revisit). As it stands today it costs me between 9¢ and 11¢ per month to run. One of the things that landed in the ~/TODO after using it for a while was to add the ability for me to reply to an existing Thought. Sometimes I throw something out there and later on want to follow up to it but there wasn't a great way to do that. Thoughts are generally speaking immutable and independent — and while this is an intended feature it lead to situations like this.

Alerting in Grafana is so trash that I'm seriously about to write Nagios/Icinga plugins to just watch values from InfluxDB.  They should just throw the alert system in the trash and let some software fit for purpose do the work.

Followed later in the feed, and potentially on a different page by...

So yeah, after whining about how bad Grafana's alerting system is and how I was about to rage-write some Icinga checks... Yeah.. I did it.  I replaced a whole suite of rules with a 262 line Python script.

I also went and replaced a bunch of Azure Function Monitor rules and alerts with a few calls to check_http.

I've tried several things in various Thoughts to try to make it obvious that one is a reply to another, usually involving linking back but netnews and e-mail have had threading since the 1980s so why the hell don't I?

It is useful at this point to remind ourselves of the original design requirements of the Thoughts system as those have not changed. In fact they informed several decisions I had to make along the way.

Happily the first is easy as the system that periodically renders Thoughts to files on disk isn't going away, it'll just have to be updated to support whatever API changes come along. The next two were accomplished by simply taking the ID of the parent Thought and thinking of that as a thread ID, then coming up with a way to redirect child Thought IDs to the parent thread ID. The final consideration is posting and that was largely solved by thinking about how I wanted to handle threading in general. Since Thoughts are meant to be idempotent we can take inspiration from the way netnews and e-mail work. Since we can assume that the poster doesn't have access to change the data that is already there all the information needed to post a reply must be a part of the existing Thought, hence the use of the ID attribute which is the time of the post in POSIX epoch seconds format.

If you want to get really in depth, I kept the notes I took while I was designing and building the changes.

Thought Changes

Again, taking inspiration from decades old Internet technology I chose to create an in_reply_to field in the Thought table storage. All the threading is constructed from this by the new Thought object abstraction that I created. ThoughtService was then converted to leverage the Thought object instead of directly interacting with the Entity objects from the Table Service. At first generating these relationships took a long time and so several iterations and refactors were required to produce something performant enough to happen on every call. Happily we got there, reducing the call time from around 3 seconds for the 25 entry RSS feed to about 800 ms. Further performance was gained by caching the RSS feed as it's the same for everyone and only needs to be regenerated when I post something new. I was able to leverage Blob metadata to provide a simple cache layer that I was able to plumb into the RSS function.

Timeline Changes

Given it was written in Python, the RSS timeline was the easiest to update. The function was taught about the new API and then given the ability to render the HTML needed to nest children inside <blockquote> elements. The Thought object was given a UUID property that is generated deterministically from the thread ID and the highest ID of any child thought. This way it changes when the Thought is replied to. Most RSS readers use on the GUID tag in the feed XML as the key to store the read/unread state. This way when I reply to a Thought it will create a new item in the feed. Using the same API the method was adapted to JavaScript for the web display portion. It's a little more complex because there are several pieces interacting, between the API client, the logic on the page, and the logic inside the various Web Components In the end what you see is mostly driven from thoughts.js.

Single Thought Display to Thread Display Changes

One of the trickiest parts was deciding how to handle the individual Thought display. It seems useless to keep the old single Thought display mechanism as it would destroy the entire purpose of showing the context of related Thoughts. I did want to simplify the URLs a bit so I didn't have to render what would end up being duplicates over and over for parents and their children, so I decided to change the format from /thoughts/<id>.html to /thoughts/<thread id>.html#id<child id>. This means that the URL becomes It was simple enough to teach the JavaScript about this new format but I had to devise a method to make sure the legacy URLs all work. This is where Apache's mod_rewrite comes to the rescue. I already lean on it for the single Thought display, internally serving a file called single.html for any un-rendered Thought. The Apache configuration for that looks like this.

RewriteCond     %{REQUEST_URI}  '^/thoughts'    [NC]
RewriteCond     /var/www/{REQUEST_URI} !-f
RewriteRule     '^/thoughts/(\d+)\.html' \
    '/thoughts/single.html' [L]

This fragment says if a request comes in for something in /thoughts that looks like <a bunch of numbers>.html check for it on disk in /var/www/<a bunch of numbers>.html. If it exists serve that — otherwise internally rewrite the request to /thoughts/single.html and serve that. The client is none the wiser (that is the [L] bit) which is important because the JavaScript uses window.location to figure out what Thought to load.

I updated the static Thought builder to write out a map file that points each child Thought ID back to it's parent ID. I then added the following prior to this bit which causes Apache to redirect the browser to the new format URL if a mapping is found in the file.

RewriteMap  thoughts "txt:/var/www/"
RewriteCond %{REQUEST_URI}  '^/thoughts' [NC]
RewriteCond ${thoughts:$1}  '\d+'
RewriteRule '^/thoughts/(\d+)\.html' \
    '/thoughts/${thoughts:$1}.html#id$1' [R=301,NE]

The Apache RewriteMap documentation may be helpful in explaining what is going on here.

The rest was JavaScript and CSS changes to make the display pretty and was mostly done as part of getting the timeline view working. The nice part of the Web Components is that most of the code is shared.

Posting Interface Changes

This ended up being about the easiest of all the different bits and pieces. The JavaScript function was just modified to know about a URL hash parameter.

async function post(evt)


    // Try to get the In-Reply-To from the location.
    const hash = window.location.hash.slice(1).split('=');
    let in_reply_to = null;

    if (hash[0].toLowerCase() == 'in-reply-to') {
        in_reply_to = hash[1];


        /* According to the table at:
         * The JSON type for Edm.Int64 is... STRING!
        const row = {
            'PartitionKey': 'thought',
            'RowKey': id.toString(),
            'id@odata.type': 'Edm.Int64',
            'id': id.toString(),
            'message': nl2br(msg)

        if (attachments) {
            row['attachment'] = JSON.stringify(attachments);

        if (link) {
            row['link'] = JSON.stringify(link);

        if (in_reply_to) {
            row['in_reply_to@odata.type'] = 'Edm.Int64';
            row['in_reply_to'] = in_reply_to;

        await connections.table.insert(row);

In my view of Thoughts I get an extra control that provides a link to the posting interface with the hash parameter set, so I can simply browse to a Thought that already exists and reply to it.

A sample Thought


To look at the amount of changes that this took it is useful to think of the system as two parts. Vociferate is the Azure Functions, Table Storage and Blob Storage bits (mostly the Function as far as where code lives) and Thoughts is the web application parts that live here on

vociferate changes

 .funcignore                         |   1 +
 .gitignore                          |   2 +
 Makefile                            |   5 +-
 README.txt                          |  13 +-
 TODO                                |   4 -
 cli/              |   2 +-
 lib/                     |   5 +-
 lib/                    | 103 +++++++++++
 lib/                   |  31 ++++
 lib/                      | 235 +++++++++++++++++++++++++
 lib/               | 339 ++++++++++++++----------------------
 list/                    |   9 +-
 lib/ => parent/ |  55 ++++--
 parent/function.json                |  17 ++
 pypeteer/                |   9 +-
 requirements.txt                    |   7 +-
 resolver/                |  18 +-
 rss/                     | 114 ++++++++++--
 tests/                   |   1 +
 tests/               |  96 ++++++++++
 tests/        |  69 ++++++++
 thought.html                        |   2 +-
 thoughts/                |  25 ++-
 thumbnailer/             |  21 +--
 24 files changed, 889 insertions(+), 294 deletions(-)

Thoughts changes

 js/classes.js                      |  62 +++++++++++++++++++++--------------------------
 js/lave.js                         |   6 +----                   |  49 +++++++++++++++++++++++++++++++------
 templates/components/large.html    |  20 +++++++++++----
 templates/components/small.html    |   8 +++++-
 templates/thoughts/index.html      |   2 +-
 templates/thoughts/onethought.html |  10 +-------
 thoughts/css/thoughts.css          |  35 +++++++++++++++++++++------
 thoughts/js/components.js          | 164 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------
 thoughts/js/thoughts.js            | 184 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------------------------
 10 files changed, 373 insertions(+), 167 deletions(-)

Most of the heavy lifting was quite happily done in Python on the Function end of the equation. There is no reason you couldn't do it the other way around and just return JSON with the in-reply-to properties in them and do your threading in the JavaScript if you are more comfortable with it. Hell if you really wanted to Functions supports JavaScript as a language so you could eschew Python entirely if you happened to be some sort of masochist.


I pushed the changes live on the 1st of November and I've been happy with it so far. I went back and tagged a few threads and have posted a few replies to Thoughts with the new interface and all in all it seems to work pretty well.

Comment via e-mail. Subscribe via RSS.