List-Unsubscribe handling with mutt_

🇺🇦 Resources to help support the people of Ukraine. 🇺🇦
April 09, 2022 @20:45

Mutt, running inside tmux, somewhere on the Internet I can't imagine it will surprise any regular reader that my preferred MUA is mutt(1), and has been for close to 20 years now. I only switched to using Outlook at my current job a few years ago when they decommissioned the IMAP bridge and forced everyone into it. One of the things I've found myself having to deal with more lately is unsubscribing from marketing e-mails. Typically this was a trivial matter of finding the unsubscribe link and visiting it; however, it seems that lately all links in those e-mails come wrapped in click trackers that are blocked by one or more of my proxy server, DNS configuration, or content filter extensions, making unsubscribing difficult. Luckily RFC 2369 seems very well adhered to by even illegitimate e-mail marketing campaigns so I set out to try to solve the problem in the usual way. Angrily writing software.

Extracting List-Unsubscribe headers

The first step was to extract the List-Unsubscribe headers from a mail message. A little bit of Python was able to do the trick here. While I was in there I decided to handle RFC 8058 one click unsubscribe headers myself.

#!/usr/bin/env python3
''' mutt-unsubscribe.py (c) 2022 Matthew J. Ernisse <matt@going-flying.com>
All Rights Reserved.

Redistribution and use in source and binary forms,
with or without modification, are permitted provided
that the following conditions are met:

    * Redistributions of source code must retain the
      above copyright notice, this list of conditions
      and the following disclaimer.
    * Redistributions in binary form must reproduce
      the above copyright notice, this list of conditions
      and the following disclaimer in the documentation
      and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
'''
import email.parser
import email.policy
import os
import re
import subprocess
import sys
import requests
from urllib.parse import parse_qs, urlparse

# This program gets called with the URL as the argument if we do not
# handle the unsubscribption method ourselves.  You may want something
# like open(1) on macOS or xdg-open(1) on Linux.
URL_HANDLER = 'remote-url-open'


def extract_urls(header):
    ''' Given a header value encoded as described in RFC-2369,
    extract any urls and return as a list.
    '''
    url_re = re.compile(r'\s*\<([^>]+)\>,?\s*')

    matches = []
    for match in url_re.finditer(header):
        matches.append(match.group(1))

    return matches


def try_one_click(url, parsed):
    ''' Given the parsed headers object try to perform a one-click
    unsubscribe as described in RFC 8058.  If we cannot for some
    reason return None.  Otherwise return True.
    '''
    # Note this does not check if this meets the DKIM requirements
    # of ยง4 of the RFC.
    if parsed.get('List-Unsubscribe-Post'):
        body = parsed.get('List-Unsubscribe-Post')
        if body != 'List-Unsubscribe=One-Click':
            print('Invalid one-click header', file=sys.stderr)
            return None

        data = {'List-Unsubscribe': 'One-Click'}
        headers = {'User-Agent': 'mutt-unsubscribe/1.0'}
        try:
            resp = requests.post(url, data=data, headers=headers)
            resp.raise_for_response();
        except Exception as e:
            print(
                f'Exception sending POST to {url}: {e!s}',
                file=sys.stderr
            )
            return None

        print(f'One-Click Result: {resp.content}')
        return True


if __name__ == '__main__':
    msg = email.parser.HeaderParser(policy=email.policy.default)
    parsed = msg.parse(sys.stdin)

    urls = []
    for url in extract_urls(parsed.get('List-Unsubscribe')):
        parts = urlparse(url)
        urls.append((parts.scheme, url))

    types = [x[0] for x in urls]
    # Prefer mailto: unsubscription since it's more likely not to trip
    # my content filters.
    if 'mailto' in types:
        url = [x[1] for x in urls if x[0] == 'mailto'][0]
        subprocess.run(f"{URL_HANDLER} '{url}'", shell=True)

    # If we are doing a https url, check to see if it is one-click
    # and handle it directly if it is.  See RFC-8058
    elif 'https' in types:
        if not try_one_click(url, parsed):
            subprocess.run(f"{URL_HANDLER} '{url}'", shell=True)

    elif 'http' in types:
        subprocess.run(f"{URL_HANDLER} '{url}'", shell=True)

    else:
        print('No supported unsubscription method found')
        for url in urls:
            print(x[1])

Handling mailto: links

The other key here is the URL_HANDLER script, which in my case is another, more generic Python script I wrote ages ago to handle opening URLs. I run mutt on a different computer than the one I am generally working on so I have to handle all sorts of things remotely. Things like URLs, attachments, printing, etc. If you run mutt on your system you can likely get away with whatever the normal method of opening things from the command line is (usually something like open(1) or xdg-open(1)), though you probably will still want to look at the script below for inspiration of handing mailto: links since most systems do not register mutt as the system mail handler.

#!/usr/bin/env python3
''' remote-url-open (c) 2015-2022 Matthew J. Ernisse <matt@going-flying.com>
All Rights Reserved.

Redistribution and use in source and binary forms,
with or without modification, are permitted provided
that the following conditions are met:

    * Redistributions of source code must retain the
      above copyright notice, this list of conditions
      and the following disclaimer.
    * Redistributions in binary form must reproduce
      the above copyright notice, this list of conditions
      and the following disclaimer in the documentation
      and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
'''
import os
import subprocess
import sys
import tempfile
from urllib.parse import parse_qs, unquote, urlparse

if __name__ == '__main__':
    # Process mailto: urls locally.
    url = sys.argv[1]
    parts = urlparse(url)
    if parts.scheme == 'mailto':
        args = ['/usr/bin/mutt']
        body = None
        qs = parse_qs(parts.query)

        for field in [('bcc', '-b'), ('cc', '-c'), ('subject', '-s')]:
            if qs.get(field[0]):
                args.append(field[1])
                for item in qs.get(field[0]):
                    args.append(unquote(item))

        if qs.get('body'):
            body = tempfile.NamedTemporaryFile(delete=False)
            for item in qs.get('body'):
                body.writeline(unquote(item))

            args.append('-i')
            args.append(body.name)

        args.append(parts.path)
        subprocess.call(args)

        if body:
            os.unlink(body.name)

        sys.exit(0)

    # All other urls goto the terminal I'm logged in from.
    fn = os.path.expanduser('~/.ssh_remote_host_addr')
    if not os.path.exists(fn):
        print('Cannot find remote terminal.')
        sys.exit(1)

    with open(fn) as fd:
        remote_addr = fd.readline().strip()

    cmd = '/usr/bin/ssh'
    subprocess.call([cmd, remote_addr, 'open', sys.argv[1]])

Only the first part of the script is really relevant, it handles converting the RFC 6068 mailto URI into arguments for mutt. It will then launch mutt and let you send the e-mail normally.

Configuring Mutt

Now that the pieces are all in place we just need to plumb it into mutt. Thankfully this is a simple pair of macros put into .muttrc (or a file sourced by it).

macro index <esc>U <pipe-entry>~/git/dotfiles/mutt/mutt-unsubscribe.py<enter> 'Unsubscribe from this mailing list.'
macro pager <esc>U <pipe-entry>~/git/dotfiles/mutt/mutt-unsubscribe.py<enter><exit> 'Unsubscribe from this mailing list.'

Conclusion

Now I can just hit ESC U and let the machinery unsubscribe me from things. If you want to look further, the scripts are in my dotfiles git repository.

Enjoy!

Comment via e-mail. Subscribe via RSS.