Reading my Thoughts with JavaScript

April 19, 2020 @13:45

Last Friday I deployed my new Azure Functions based Thoughts application to this website and wrote about the Python bits of it. Towards the end of that entry I mentioned that quite a bit of JavaScript and some Web Components technology went into pulling all this together. I figured I'd talk a little bit about the JavaScript side of things. Since there is much of it I will start with the reading side of things, it being the more straightforward part.

Basic Thought Retrieval Flow

There are four different ways Thoughts get displayed. Three of which I will talk about here since one of the ways is the RSS feed which comes straight out of the /api/rss function. The basic overview is that the Thoughts come from the /api/get function in JSON format and all the media are directly served from the Azure Blob Container.

The Home Page (index.html)
Embedded Thoughts
The Thoughts page (/thoughts/)

The Home Page

For the homepage I didn't want any of the Thoughts stuff to be anywhere in the rendering critical path. I want the landing page to be as fast loading and simple as can be, and I want it to not break if a dependency fails. All the loading is managed by /js/lave.js, specifically the thoughtLoader() function.

function thoughtLoader() {
    if (window.location.pathname !== '/') {
        return;
    }

Since lave.js is included in the template for every page and has several utility functions, thoughtLoader() will get called every page. This simply keeps it from running anywhere but the home page.

    const thought_start = new Date();
    const thoughts = new Thoughts();
    const abase = 'https://thoughtsassets.blob.core.windows.net/';

    // Create a loading throbber that we will replace upon load.
    const tEl = document.createElement('div');
    tEl.classList.add('wrapper');

    const loadEl = document.createElement('span');
    loadEl.classList.add('thinking');
    tEl.appendChild(loadEl);
    tEl.appendChild(loadEl);
    tEl.appendChild(loadEl);
    document.getElementById('thoughts').appendChild(tEl);

If we are going to run the first thing we do is create the animated loading throbber. This way the visitor has an indication that something may happen since under very low load the Azure Function can take over a second to start up and return data. We also instantiate the Thoughts object which does all the actual work and is contained in /js/classes.js. Thoughts is used by all of the JavaScript getters to interface to the Azure Function.

    thoughts.load(1).then((thought) => {
        if (Array.isArray(thought)) {
            thought = thought[0];
        }

        Array.from(tEl.childNodes).forEach((el) => {
            el.remove();
        });

        const aDiv = document.createElement('div');
        const cDiv = document.createElement('div');
        const dDiv = document.createElement('div');
        const mDiv = document.createElement('div');
        const pDiv = document.createElement('div');
        dDiv.classList.add('date');
        pDiv.classList.add('push');
        cDiv.appendChild(mDiv);
        cDiv.appendChild(pDiv);
        cDiv.appendChild(dDiv);
        tEl.appendChild(aDiv);
        tEl.appendChild(cDiv);

        if (Object.prototype.hasOwnProperty.call(
            thought, 'attachment')) {
            const img = document.createElement('img');
            img.src = abase + 'thumbs/' + thought.attachment;
            aDiv.appendChild(img);
        }

        const content = (new DOMParser()).parseFromString(
            thought.message, 'text/html'
        );

        Array.from(content.body.childNodes).forEach((el) => {
            mDiv.appendChild(el);
        });

        dDiv.appendChild(document.createTextNode(thought.date));

        tEl.addEventListener('click', (evt) => {
            evt = evt || window.event;
            const src = evt.target || evt.srcElement;
            if (! src || src.tagName === 'A') {
                return;
            }

            evt.stopPropagation();
            evt.preventDefault();
            window.location.href = window.location.origin + '/thoughts/';
        });

Once the Thoughts.load Promise resolves we clear the throbber elements and replace them with the Thought DOM elements. This pattern is generally repeated throughout but I don't use any custom elements or templates here.

        const thought_end = new Date();
        postThoughtTiming(
            thought.id,
            thought_start,
            thought_end
        );
    });
}

Finally I post some anonymized timing metrics to the same InfluxDB instance I use to collect the timing metrics for the rest of the site that I previously wrote about. Those go into a dashboard so I can see how long everything takes to load. This gets used by the embedding system as well.

Embedded Thoughts

I don't hate the crafting aspect of Animal Crossing New Horizons as much as I thought I would but seriously, can we please get rid of tool durability?
04/11/2020 @09:57

I wanted a way for myself and others to embed an individual thought elsewhere. Much like I don't want the API and JavaScript to be in the critical path of the home page I don't want it to be in the critical path of... well wherever an embedded Thought ends up. With that goal in mind the embed code you are given when you click on the forward arrow on a Thought contains the actual text content of the thought already in it. It does this by cloning part of the Shadow DOM that is used to render the Thought. I add a small piece of JavaScript that on load will find all the Thoughts on the page and add some styling to them. It is this script that also reports back the load time metrics for remotely embedded Thoughts.

const __going_flying_thought_loader = (async () => {
    const config = {
        'css': 'thoughts/css/embed.css',
        'guardattr': '__going_flying_thought_embed_configured',
        'pingurl': 'https://ssl.ub3rgeek.net/metrics/thought',
    };
    const pings = [];
    const start = new Date();

    if (Object.prototype.hasOwnProperty.call(window, config.guardattr)) {
        return;
    }

    const thoughts = document.querySelectorAll('.goingflying-thought');
    if (! thoughts) {
        return;
    }

I create an asynchronous function that I attach to an event listener that looks for the document's readyState to reach 'complete'. This is probably unnecessary but since I don't know what the page these will be included in is doing I figured I would try to play it safe. I also make an attempt to only run the script once even if more than one Thought is embedded on the same page (which would mean multiple <script> tags, unless the person edited them out).

    const url = thoughts[0].getAttribute('data-base-url');

    // Include my stylesheet once.
    const cssEl = document.createElement('style');
    cssEl.setAttribute('type', 'text/css');
    cssEl.innerHTML = '@import url("' + url + '/' + config.css + '");';
    thoughts[0].appendChild(cssEl);

Now that we've decided to run we attach the style sheet to the DOM. Like the script itself I try to only do this once as well since it should style all embedded Thoughts on the page.

    Array.from(thoughts).forEach((t) => {
        const id = t.getAttribute('data-id');

        // Include attribution section.
        const attrib = document.createElement('div');
        attrib.classList.add('attribution');
        const attribA = document.createElement('a');
        attribA.href = url;
        attribA.target = '_blank';
        attribA.appendChild(
            document.createTextNode('www.going-flying.com')
        );
        attrib.appendChild(
            document.createTextNode('A thought from ')
        );
        attrib.appendChild(attribA);
        t.appendChild(attrib);

        // Don't phone home if we are embeded on our own sites...
        if (window.location.host === 'www.going-flying.com') {
            return;
        }

        const stop = new Date();

        // Now that we are all rendered up, we phone home.
        const p_p = new Promise((resolve, reject) => {
            const pingdata = {
                'id': id,
                'loadtime': stop.getTime() - start.getTime(),
                'location': window.location.toString(),
            };

            const xhr = new XMLHttpRequest();
            xhr.open('POST', config.pingurl);
            xhr.setRequestHeader(
                'Content-Type',
                'application/json;charset=UTF-8'
            );
            xhr.addEventListener('readystatechange', (e) => {
                if (e.target.readyState == 4) {
                    resolve(e.target.status);
                }
            });
            xhr.addEventListener('error', (e) => {
                reject(e);
            });
            xhr.send(JSON.stringify(pingdata));
        });
        pings.push(p_p);
    });

I now send the load events back for each Thought on the page. As you can see I only collect the Thought ID, the time it took to load and the page upon which we are running. This is all I really need and really don't want more information than this. I talk a lot more about my concerns and why I made these choices in the post where I originally designed metric collection for the website.

    await Promise.all(pings).catch((e) => { console.error(e); });
    window[config.guardattr] = true;
});

After all of this runs we set an attribute on the global Window object so hopefully we won't run more than once.

The dashboard that this all rolls up into is a slightly re-configured copy of of the general timing dashboard.

Grafana Dashboard

The Thoughts Page

This is the most interesting one overall. Underneath there is one HTML page that gets redirected to internally by Apache. I detect how we get called and either display a list of Thoughts or a larger single Thought. I will omit most of the class below since it deals with things like displaying errors and prepping the DOM.

    constructor() {
        let loader;
        const loc = document.location;
        const idRe = /^\/thoughts\/([0-9]+).html/;
        const idM = loc.pathname.match(idRe);

        this.root = document.getElementById('thoughts');
        this.thumburl = 'https://thoughtsassets.blob.core.windows.net/';
        this.thoughts = new Thoughts();
        let loadEl;

        if (! idM) {
            loadEl = document.createElement(
                'goingflying-thought-small'
            );
            loader = this.thoughts.load();

        } else if (Array.isArray(idM) && idM.length > 1) {
            loadEl = document.createElement(
                'goingflying-thought-large'
            );
            loader = this.thoughts.loadById(idM[1]);

        } else {
            this.displayError();
            return;
        }

The important part here is detecting how we are called and selecting the proper custom element to use. The URL format I settled on was /thoughts/<id>.html so we simply parse that out of the current location and if we do not get an ID we assume we are to display a list of all the thoughts. We then lean again on the Thoughts class to get the JSON from Azure.

        this.root.appendChild(loadEl);

        loader.then((result) => {
            this.clearThoughts();
            if (! Array.isArray(result)) {
                const el = document.createElement(
                    'goingflying-thought-large'
                );
                el.addEmbedListener((evt) => {
                    this.createEmbed(evt);
                });
                el.addLinkListener((evt) => {
                    this.createLink(evt);
                });

                this.root.appendChild(el);
                this.createThought(result, el);
                return;
            }

            result.forEach((thought) => {
                const el = document.createElement(
                    'goingflying-thought-small'
                );
                el.addEmbedListener((evt) => {
                    this.createEmbed(evt);
                });
                el.addLinkListener((evt) => {
                    this.createLink(evt);
                });

                this.root.appendChild(el);
                this.createThought(thought, el);
            });
        });

        loader.catch(
            (err) => {
                console.error('Thoughts() threw: ' + err);
                this.displayError();
            }
        );
    }

Next we make sure there is an animated throbber displaying and wait for the loader Promise to resolve. Depending on if we got an array of Thoughts or a single Thought we create the appropriate elements and insert them into the DOM after clearing the animation.

The next interesting part is the creation of the embed code. I described the result above but this is the function that actually does it.

    createEmbed(evt) {
        evt = evt || window.event;
        const src = evt.target || evt.srcElement;
        evt.preventDefault();

        const domHost = src.parentElement.parentNode.host;
        const id = domHost.getAttribute('data-id');

        const embedEl = document.createElement('blockquote');
        embedEl.classList.add('goingflying-thought');
        embedEl.setAttribute('data-id', id);
        embedEl.setAttribute('data-base-url', window.location.origin);

Since this is an event callback I first have to figure out what element I was actually called on. Then I can get the information needed to create the embed code.

        const wrapDiv = document.createElement('div');
        const attachDiv = document.createElement('div');
        const contDiv = document.createElement('div');
        wrapDiv.appendChild(attachDiv);
        wrapDiv.appendChild(contDiv);
        wrapDiv.classList.add('wrapper');
        embedEl.appendChild(wrapDiv);

        // Insert the JavaScript for styling.
        const scriptEl = document.createElement('script');
        scriptEl.setAttribute('async', '');
        scriptEl.src = window.location.origin + '/thoughts/js/embed.js';
        embedEl.appendChild(scriptEl);

Next I prep all the elements I will need including the JavaScript fragment.

        // Clone the content into the content wrapper
        const thoughtEl = src.parentElement.parentNode.host;
        Array.from(thoughtEl.getContent().children).forEach((el) => {
            contDiv.appendChild(el.cloneNode(true));
        });

        // Wrap the date in a <a> with a link back to myself.
        const dateEl = contDiv.querySelector('.date');
        const linkEl = document.createElement('a');
        linkEl.href = domHost.getLink();
        linkEl.innerHTML = dateEl.innerHTML;
        Array.from(dateEl.childNodes).forEach((el) => {
            el.remove();
        });
        dateEl.appendChild(linkEl);

My custom element has a getContent() method which returns the underlying elements from the Shadow DOM so I don't have to reach into it from outside. Those elements then get appended to the DOM fragment that I'm building up.

        // Embed attachments.
        const attachments = domHost.getAttachments();

        if (attachments) {
            Array.from(attachments).forEach((a) => {
                const img = document.createElement('img');
                img.src = this.thumburl + 'thumbnails/' + a;
                attachDiv.appendChild(img);
            });
        }

Finally I add the attachments if there are any. The attachments are stored in the custom element and fetched with another custom method, getAttachments().

        const inputEl = document.getElementById('embedCopy');
        inputEl.value = embedEl.outerHTML;
        try {
            this.displayStatus('Generating embed code...');
            inputEl.select();
            if (! document.execCommand('copy')) {
                throw Error('copy command denied');
            }
            this.dismissStatus();
            this.displayStatus('Copied embed code!');
        } catch (error) {
            this.dismissStatus();
            this.displayStatus('Failed to copy embed code!');
            console.error('Failed to copy: ' + error);
        }

        window.setTimeout(() => { this.dismissStatus(); }, 1000);
    }

This part leverages the most portable way I could find to cause the browser to copy the string into the user's clipboard. It basically displays a brief modal dialog over the entire screen which has an <input> element with the string in it. It then asks the browser to highlight and copy that. At the moment it seems like the Clipboard API is poorly supported and may even be in flux so while this method is a bit hacky it at least has the virtue of working.

Custom Elements

The most interesting part of the Web Component stuff that I ended up using is the <template> tag which let me define all the styling and markup once and re-use it as many times as I needed. As an example the small Thought used in the list view looks like this.

    <template id='goingflying-thought-small'>
        <style>
        .attachments img {
            max-width: 120px;
        }

        .content {
            display: block;
            position: relative;
            width: 100%;
        }

        .controls {
            background-color: inherit;
            border-bottom: 1px dotted #00dd00;
            display: flex;
            flex-direction: row;
            justify-content: flex-end;
            padding-bottom: 0.5em;
        }

        .controls > div {
            background-size: contain;
            border: 1px solid #00dd00;
            cursor: pointer;
            height: 24px;
            margin-left: 0.5em;
            width: 24px;
        }

        .controls > .embed {
            background-image: url('/thoughts/images/share.png');
        }

        .controls > .link {
            background-image: url('/thoughts/images/link.png');
        }

        .date {
            bottom: 0;
            color: #3acc85;
            display: block;
            font-size: 8pt;
            position: absolute;
            right: 0;
            text-align: right;
        }

        .push {
            height: 1em;
            width: 100%;
        }

        .thinking {
            animation: thinkingAnim 0.75s ease-in infinite alternate;
            background: linear-gradient(
                .25turn, black 0%, #00dd00 20%, 60%, black 80%
            );
            background-size: 400% 100%;
            display: inline-block;
            height: 1em;
            width: 100%;
        }

        .thinking-date {
            float: right;
            width: 25%;
        }

        .thought {
            background-color: inherit;
            box-sizing: border-box;
            color: #0acf00;
            display: flex;
            flex-direction: row;
            justify-content: flex-start;
            margin-right: 0.5em;
            padding: 0.5em;
            text-align: left;
            width: 100%;
        }

        .thought:hover {
            cursor: pointer;
        }

        .thought a {
            color: #3acc85;
        }

        .thought .attachment img {
            padding-right: 0.5em;
        }

        @keyframes thinkingAnim {
        0% {
            background-position: 0% 50%;
        } 50% {
            background-position: 50% 50%;
        } 100% {
            background-position: 100% 50%;
        }
        </style>

        <section class="thinking">
            <span class="thinking"></span>
            <span class="thinking"></span>
            <span class="thinking thinking-date"></span>
        </section>

        <section id="thoughts" class="thought">
            <div class="attachments">
            </div>
            <div class="content">
                <div class="content" id="content"></div>
                <div class="push"></div>
                <div class="date" id="date"></div>
            </div>
        </section>

        <section class="controls">
            <div class="link"
                id="createLink"
                role="img"
                title="Get link this Thought"></div>

            <div class="embed"
                id="createEmbed"
                role="img"
                title="Get code to embed this Thought"></div>
        </section>
    </template>

It is attached to a JavaScript class GoingflyingThoughtSmall which then provides all the behavior required for the element.

class GoingflyingThoughtSmall extends HTMLElement {
    constructor() {
        super();
        const shadowRoot = this.attachShadow({mode: 'open'});
        const template = document.getElementById(
            'goingflying-thought-small'
        ).content;
        shadowRoot.appendChild(template.cloneNode(true));
        this.attachments = [];
        this.baseurl = 'https://thoughtsassets.blob.core.windows.net/';
        this.shadowRoot.addEventListener('click', (evt) => {
            evt = evt || window.event;
            const target = evt.currentTarget;
            if (! target || evt.target.tagName === 'A') {
                return;
            }

            evt.stopPropagation();
            evt.preventDefault();
            window.location.href = target.host.getLink();
        });
    }

Most of the constructor is boilerplate stuff but the click handler took a bit of playing around to work right. Unlike a normal click event on an element the event.target does not a handle to anything particularly useful, instead it is in event.currentTarget. We can then determine if the click event was on an <A> and bail since we do not want to override any links in the markup nor in the content itself.

    addEmbedListener(func) {
        const el = this.shadowRoot.getElementById('createEmbed');
        el.addEventListener('click', (evt) => {
            evt.stopPropagation();
            func(evt);
        });
    }

    addLinkListener(func) {
        const el = this.shadowRoot.getElementById('createLink');
        el.addEventListener('click', (evt) => {
            evt.stopPropagation();
            func(evt);
        });
    }

These are just some helpers to allow attachment of event handlers onto elements within the Shadow DOM without having to reach in from the outside.

    getAttachments() {
        return this.attachments;
    }

    getContent() {
        return this.shadowRoot.querySelector('div.content');
    }

    getLink() {
        if (this.link) {
            return this.link;
        }
    }

Some attribute getters, the getContent() one is used to create the embed code, again so I can get information out of the Shadow DOM without doing nasty things from the outside. All information is known by and accessed through the custom element's API.

    loaded() {
        const throbber = this.shadowRoot.querySelector(
            'section.thinking'
        );
        throbber.style.opacity = 0.0;
        throbber.style.display = 'none';
        throbber.style.visibility = 'hidden';
    }

    setAttachment(url, size) {
        this.attachments.push(url);
        const attachments = this.shadowRoot.querySelector(
            '.attachments'
        );
        const attachEl = document.createElement('div');
        const imgEl = document.createElement('img');
        attachEl.classList.add('attachment');
        imgEl.src = this.baseurl + 'thumbnails/' + url;
        imgEl.setAttribute('data-size', size);
        attachEl.appendChild(imgEl);
        attachments.appendChild(attachEl);
    }

    setContent(id, date, message) {
        this.dateStr = date;
        this._id = id;
        this.link = document.location.origin + '/thoughts/' + id + '.html';
        this.shadowRoot.host.setAttribute('data-id', id);

        const contEl = this.shadowRoot.getElementById('content');

        /* message might contain HTML so parse that out so we can
         * append it into the DOM correctly (innerHTML makes me
         * itch in uncomfortable ways).
         */
        const content = (new DOMParser()).parseFromString(
            message, 'text/html'
        );

        Array.from(content.body.childNodes).forEach((el) => {
            contEl.appendChild(el);
        });

        const dateEl = this.shadowRoot.getElementById('date');
        dateEl.appendChild(document.createTextNode(this.dateStr));
        this.loaded();
    }
}

Most of the rest of this stands on its own, it is pretty common JavaScript stuff, adding elements to the Shadow DOM conditionally, hiding elements, etc. The most interesting bit is setContent() which takes the Thought message returned by the Thought class and turns it into a set of DOM Nodes that get attached to the Shadow DOM that is created by the constructor of custom element.

Conclusion

I learned a lot throughout this process. The home page display is very reminiscent of how I have been writing JavaScript powered web pages for years now and the Thoughts page stuff is much more modern. I had a pretty hard time finding good examples of complex Web Components online while I was building this -- I suspect largely because the most popular way to interact with this technology is through higher level frameworks which abstract away the mechanics. If you are like me and want to understand the mechanics underneath the frameworks I hope this helps.

At some point I'll write about the posting interface which is quite a bit more complex but really is just building upon the concepts here.