It turns out that describing my new Thoughts system has turned into a three part series. You probably want to go back and read the previous two articles before reading this one.
So whereas the reading side of the system depends on Azure Functions to actually read the Thoughts out of Azure Table Service (and construct the urls to load media from Azure Blob Storage) the posting side communicates directly to the Azure REST APIs for both Table Service and Blob Storage. As I described in the first post there is a Function based workflow that gets triggered, however the trigger is from the completion of a Blob upload so it isn't directly called by any of the client side code and runs on its own.
Posting flow
The posting client is, as bemoaned in the original post a simple HTML 5 web page hidden behind some Apache ACLs with a bunch of JavaScript bound to some event handlers. The JavaScript simply does a Put Blob into a container (if there is an attachment in the post) and then does an Insert Entity into the table. Thankfully the only difficult bit to figure out was properly computing the Shared Key signature for the Authorization header. The method is described by the Azure documentation however it's a bit esoteric feeling. Especially when the transport must be HTTPS one wonders why you end up signing so much information in the request. Honestly it feels like the request is decrypted somewhere in the chain (perhaps after their load balancer) and so they are trying to make sure that the request isn't modified in transit. There are actually two different signature methods, one used for Blobs, Queues and File Services, and one used for Tables. They differ in which headers must be present in the request and included in the signature.
Shared Key for Blob, Queue, and File Services
async setAuthHeader(len) {
/* Format of the string we have to sign:
StringToSign = VERB + "\n" +
Content-Encoding + "\n" +
Content-Language + "\n" +
Content-Length + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Date + "\n" +
If-Modified-Since + "\n" +
If-Match + "\n" +
If-None-Match + "\n" +
If-Unmodified-Since + "\n" +
Range + "\n" +
CanonicalizedHeaders +
CanonicalizedResource;
CanonicalizedHeaders: The x-ms-* headers, stored as key:value
all lowercased.
CanonicalizedResource: /<account>/<uri path> <query string>
Query string is a k:v\n pair of all query parameters sorted
and lowercased.
*/
const headerList = [
'Content-Encoding',
'Content-Language',
'Content-Length',
'Content-MD5',
'Content-Type',
'Date',
'If-Modified-Since',
'If-Match',
'If-None-Match',
'If-Unmodified-Since',
'Range'
];
// This must be sorted lexagraphically.
const msHeaderList = [
'x-ms-blob-cache-control',
'x-ms-blob-content-type',
'x-ms-blob-type',
'x-ms-date',
'x-ms-version'
];
// Need this to be signed.
this.setRequestHeader('Content-Length', len);
I start by setting up lists of headers that need to be signed. The JavaScript interpreter in most browsers will throw an error message, refusing to set the Content-Length header on the request because it is not allowed (the XMLHttpRequest object will compute it and set it internally) but we need to have it in our mapping so it can be signed. I could filter it out before I actually pass all this to XMLHttpRequest but at the moment the error message appears to be benign.
let s = this.verb + '\n';
// Standard headers only get set as value.
for (const hdr in headerList) {
if (Object.prototype.hasOwnProperty.call(this.headers, headerList[hdr])) {
s += this.headers[headerList[hdr]] + '\n';
} else {
s += '\n';
}
}
// CanonicalizedHeaders
for (const hdr in msHeaderList) {
if (Object.prototype.hasOwnProperty.call(this.headers, msHeaderList[hdr])) {
s += msHeaderList[hdr] + ':' + this.headers[msHeaderList[hdr]] + '\n';
}
}
const url = new URL(this.url);
const qs = new URLSearchParams(url.search);
// CanonicalizedResource
s += '/' + this.account_name + url.pathname;
for (const p of qs) {
s += '\n' + p[0].toLowerCase() + ':' + p[1];
}
Now we just convert all of the headers, the HTTP verb and the query string parameters to a string that we can sign. We then have to dispatch it off to get signed using the internal browser crypto APIs.
const b64sig = await this.signString(s);
this.setRequestHeader(
'Authorization',
'SharedKey ' + this.account_name + ':' + b64sig
);
}
Shared Key for Table Service
/**
* Variant of setAuthHeader() for the Table Service version of the
* signature string.
*/
async setTableAuthHeader() {
/* Format of the string we have to sign:
StringToSign = VERB + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Date + "\n" +
CanonicalizedResource;
CanonicalizedResource: /<account>/<uri path> <query string>
Query string is a k:v\n pair of all query parameters sorted
and lowercased.
Also note:
If the request sets x-ms-date, that value is also
used for the value of the Date header
*/
const headerList = [
'Content-MD5',
'Content-Type',
];
As you can see this is very similar to the above, but with a much shorter list of headers we care about.
let s = this.verb + '\n';
// Standard headers only get set as value.
for (const hdr in headerList) {
if (Object.prototype.hasOwnProperty.call(this.headers, headerList[hdr])) {
s += this.headers[headerList[hdr]] + '\n';
} else {
s += '\n';
}
}
if (Object.prototype.hasOwnProperty.call(this.headers, 'Date')) {
s += this.headers['Date'] + '\n';
} else if (Object.prototype.hasOwnProperty.call(this.headers, 'x-ms-date')) {
s += this.headers['x-ms-date'] + '\n';
} else {
s += '\n';
}
const url = new URL(this.url);
const qs = new URLSearchParams(url.search);
// CanonicalizedResource
s += '/' + this.account_name + url.pathname;
for (const p of qs) {
s += '\n' + p[0].toLowerCase() + ':' + p[1];
}
const b64sig = await this.signString(s);
this.setRequestHeader(
'Authorization',
'SharedKey ' + this.account_name + ':' + b64sig
);
}
Again we construct a string as per the specification and then sign it. In
both cases we call the signString() method to actually do the work since
the actual mechanism is the same for both. It is just the input string that
changes.
Signing your request string
As I mentioned in the first post of this series I didn't want to use any third party libraries if I could help it. I feel like it is not as useful for me to learn how to program in insert your library of choice instead of learning how the language and underlying browser runtime environment works. I initially started re-learning JavaScript (in the post-Netscape Navigator days) primarily learning jQuery and it took years to unlearn the shortcuts and features it provides. So while there are several nice libraries to do crypto either in pure JS or leveraging the browser APIs I decided to roll my own signing function.
When the AzureRESTClient class is instantiated (which all these methods
are part of) it fetches some configuration information from a JSON file that
is stored in a protected path on my web server. This includes the actual
token that I'm signing the requests with (that has permissions to write to
my Azure Storage account). That token is stored on the instance as token
so that is where this.token is coming from below.
/**
* Sign string str with the key stored in self.token. Returns the
* a Promise that will resolve to the base64 encoded result.
*/
async signString(str) {
const encoder = new TextEncoder();
const encS = encoder.encode(str);
const rawKey = Uint8Array.from(
atob(this.token),
c => c.charCodeAt(0)
);
The key is provided by the Azure portal as a base64 encoded UTF-8 string so
we need to convert that into an ArrayBuffer to pass into
importKey.
let key;
try {
key = await window.crypto.subtle.importKey(
'raw',
rawKey.buffer,
{ name: 'HMAC', hash: { name: 'SHA-256' }},
true,
['sign']
);
} catch(error) {
console.error('importKey failed ' + error);
throw error;
}
I then import the key so it can be used in the signing function. I also set the information on what the key is being used for (in this case to sign using HMAC-SHA-256 as per the Azure spec).
const sig = await window.crypto.subtle.sign('HMAC', key, encS);
return btoa(String.fromCharCode(...new Uint8Array(sig)));
}
Finally we pass the key and the string we constructed to the
sign
function and await on the result, which is eventually fulfilled with an
ArrayBuffer. We then have to convert that back to a UTF-8 formatted string
and base64 encode it. et violà! It took a bit of fiddling to manage
all the asynchronous calls required to make sure the posting code didn't
just batch all the requests and plow ahead but thankfully modern JavaScript
interpreters in basically all modern browsers support
async/await
as a way to resolve
Promises. Also it is important to understand that I'm the only
person who will be posting to this so I really only have to support Safari on
macOS and iOS.
Conclusion
That's basically it. This was a fun project and I've already found myself more apt to post random things to it than I am to this blog. I would say it was worth the time investment and has given me a look into the Serverless Computing ecosystem a bit from the lens of actually trying to build an app with it.
If you want to look at the AzureRESTClient class and the supporting bits
you can find it here and the bits that actually
manage the posting page are in post.js.
🍻