When I sat down and put together the requirements for my micro-blog thing that I've dubbed "Thoughts" I decided that I wanted to provide a simple and clean way to link to and embed individual Thoughts. I explain a bit in a previous post about how the embed works and the importance of keeping my infrastructure out of the critical display path. When you click the embed button on a thought you get a bunch of standard HTML that includes all of the Thought. The only thing the JavaScript does is to apply a custom stylesheet. It can be omitted (or blocked by something like CSP) and you will still get a decent looking representation of the Thought as you can see below.
Hm, so I've had Little Brother in my Amazon wishlist for years now so instead I decided to get it and the two sequels all at once on Kickstarter. 🧐 Bonus points for not sending any of the money to Amazon.
The Problem
You can probably imagine my shame when I texted a link to one of my Thoughts to a friend only to be greeted with the generic link bubble.
Obviously this cannot stand. 😆 Apple has a technical report that describes how iOS/macOS generates the link previews and it leans on Open Graph. Most of this website already has the machinery to insert the appropriate Open Graph tags in the pages as they are generated but the Thoughts pieces do not. Since the iOS/macOS bits refuse to run any JavaScript so I cannot add the Open Graph tags dynamically, so I will have to put them into the static Thought renderer.
Exploring a Solution
Early on I decided that I'd really like it if the image used could be a representation of the embedded version of the Thought. I know that there are several mechanisms available to use either Firefox or Chromium as a rendering engine to emit an image file of a webpage. A bit of searching revealed that someone else had already looked into this and landed on a JavaScript / AWS Lambda based solution. Being that the rest of this is in Azure Functions and written in Python I set about to see if sticking with that was even possible.
So first, can I even run a browser in an Azure Function? Looks like yes so we can move forward there. The next hurdle was that Google's Puppeteer is a JavaScript API and while Azure Functions supports JavaScript, I really don't like it so I kept looking around and found pyppeteer which provides a Python port of the Puppeteer API. With those two predicates satisfied it is time to start work.
Create Some HTML (and JavaScript)
I want to render a Thought using the embed styling I have already designed so I took the HTML that gets generated from the embed button, in-lined the stylesheet that the embed.js adds dynamically, and added a small amount of inline JavaScript to fetch the actual Thought data from the API.
const a = document.getElementById('attachment');
const c = document.getElementById('content');
const d = document.getElementById('date');
const a_url = 'https://thoughtsassets.blob.core.windows.net/';
const r_url = 'https://vociferate.azurewebsites.net/api/get';
const t_id = new URLSearchParams(document.location.search);
const xhr = new XMLHttpRequest();
xhr.open('GET', r_url + '?id=' + t_id.get('id'));
xhr.addEventListener('readystatechange', (evt) => {
if (evt.target.readyState !== 4 || evt.target.status !== 200) {
return;
}
const r = JSON.parse(evt.target.response);
c.innerHTML = r['message'];
d.appendChild(document.createTextNode(r['date']));
if (r['attachment']) {
const i = document.createElement('img');
i.src = a_url + 'thumbnails/' + r['attachment'][0]['name'];
a.appendChild(i);
}
});
xhr.send();
This file gets uploaded to an Azure Blob Container so my Function can get to it.
Create Some Python
The next part is to actually pull this stuff together and get Chromium to generate a properly cropped screenshot of the Thought.
async def screenshot(url: str) -> bytes:
ua = 'vociferate-screenshot-bot/1.0 (+matt@going-flying.com)'
browser = await launch()
screenshot = None
try:
page = await browser.newPage()
await page.setJavaScriptEnabled(True)
await page.setUserAgent(ua)
await page.setViewport({'height': 800, 'width': 800})
await page.goto(url, {'waitUntil': 'networkidle0'})
element = await page.querySelector('blockquote')
screenshot = await element.screenshot()
except Exception as e:
logging.exception(e)
finally:
await browser.close()
return screenshot
This took a little bit of work, Chromium wanted to either crop nothing at
all or to chop off the border. There was also a little bit of trouble getting
the browser to wait long enough for the JavaScript to complete, especially if
a function involved in the request was cold starting (which adds around a
second to the execution time). What worked out in the end was to use
networkidle0 for the waitUntil argument to goto() and to take a
screenshot of the container element using the querySelector API.
Fonts, Fonts, Fonts.
The last hurdle was getting the correct fonts to render. Whatever fonts are included in the Linux image used by Microsoft under the hood do not render quite right. Most noticeably the emojis were either not rendered or were very plain. Obviously this would not look great so I had to figure out if there was a way to fix this.
When you run func azure functionapp publish the entire directory is sent to
Azure and put into a squashfs image that gets mounted at /home/site/wwwroot
upon Function startup. I created a fonts directory in the root of my project
placed a fonts.conf file and the .ttc from
emojione
in there.
fonts/fonts.conf
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<dir>/home/site/wwwroot/fonts/</dir>
<cachedir>/tmp/fonts-cache/</cachedir>
<config></config>
</fontconfig>
Finally I set the FONTCONFIG_PATH environment variable prior to calling
pyppeteer in my Function's main() as you can see below.
async def main(req: func.HttpRequest) -> func.HttpResponse:
if not req.params.get('id'):
return func.HttpResponse('Not Found', code=404)
id = req.params.get('id')
url = f'https://[REDACTED]/thought.html?id={id}'
os.environ['FONTCONFIG_PATH'] = '/home/site/wwwroot/fonts'
png = await screenshot(url)
if png is None:
return func.HttpResponse('Server Error', code=500)
return func.HttpResponse(png, mimetype='image/png')
Now Chromium has access to a reasonably pretty emoji font.
Finally, Success
The last piece was to wire this up into the static Thought renderer. Every
hour or so my server fetches any new Thoughts from the API and renders them
to disk. This way any hits after the first hour or so are served without
hitting the API. It also keeps any link preview lookups or other Open Graph
lookups (like search engine crawls) from hitting the screenshot API and causing
a copy of Chromium to launch needlessly. All I do is fetch the API using
the excellent Requests API
and save the resulting PNG. If that worked I add the og:image META tag
to the page and send it off to Jinja2
to create the page, just like the rest of the website.
try:
resp = requests.get(
self.ogImgApi,
params={'id': thought['id']}
)
resp.raise_for_status()
fn = os.path.join(
self.dirbase,
f'img/{thought["id"]}.png'
)
with open(fn, 'wb') as fd:
fd.write(resp.content)
page.setCustomOg(
'og:image',
f'{self.urlbase}/img/{thought["id"]}.png'
)
except Exception as e:
print(f'Error writing {thought["id"]}: {str(e)}')
The Azure Function bits are still a bit in flux so they are in a separate git repository where I keep some experiments. I feel like running a web browser to make an image is a bit heavy handed but it gets the job done quickly and easily. If nothing else it is another example of being able to get something out the door quickly that solves the problem. 🌦