Battling with the UART on my Raspberry Pi_

... or how I stopped worrying and learned to love Device Tree.

I have been looking around for a long time for a green CRT Wyse terminal to replace one I had many many years ago but got rid of in a move. I finally found a decent WY-60 on EBay so I picked that up at the end of last year. Now that I've had some time on my hands I set about getting it to do something useful.

RS-232 to... ?

For those who don't know a terminal isn't a computer. It's a peripheral designed to be plugged into a computer. Usually you would either do this via a RS-232 cable or via a modem. The terminal is known as a DTE or Data Terminal Equipment and is wired to be connected to a DCE or Data Communications Equipment. If you were using a modem a straight through cable would suffice as the modem itself is a DCE, but if you are connecting to a computer you need a 'null modem' cable which swaps the pins to connect the DTE to the computer which itself is a DTE. Since the terminal needs to be connected to something I decided to try to use a Raspberry Pi. Honestly other than the fact that they are small and low power they are not particularly well suited for this as they are really designed for hobbyists to fiddle around with so most of the hardware features needed to connect a terminal like this aren't actually enabled. You see in the year 2020 most serial ports and cables have omitted many of the pins used to be present. In fact a fully featured RS-232 DB-9 serial port contains the following pins:

RS-232 DB-9 Pinout

Flow control

Of those in most modern implementations all you get a RXD, TXD and GND. This is because in most cases everything is fast enough to process data at whatever the connected line rate is, but in the old days this was often not the case. Consider a modem connection to a remote computer. The modem used the DTR/DSR lines to let the computer know it was ready. This usually just meant it was powered on. It then used the RTS and CTS lines to tell the computer about the state of the remote system out on the other end of the phone call. Looking at it another way you might have a nice 'modern' serial port capable of a blazing 38,400 baud that is plugged into your modem but the computer you dialed up was stuck with an old V.32 modem chugging along at 9,600 baud. Obviously this would cause the buffer memory (if there was any) to fill up in the modem if you tried to transmit or receive faster than its connection to the remote host so it would toggle the CTS line to tell the computer to wait.

In the case of a terminal device it worked a little different. They usually used the DTR and DSR lines to indicate to the device that it was busy and back in the day these systems usually only had just enough memory to draw whatever text was on the screen at the time with maybe a few pages worth of buffer. If we keep our modem scenario but upgrade our modem to a V.34 with a blazing speed of 33,600 baud and drop our serial port connection down to 19,200 baud you can see how now perhaps the local connection can get overwhelmed so instead of the modem toggling CTS to say slow down the terminal can toggle DTR to tell the modem to slow down (this would often be signaled to the far end's modem which would then toggle CTS to have the remote host pause while you caught up).

Beating some sense into hobby hardware

So now we have to figure out how to emulate this in 2020 since my 1993 terminal needs flow control between itself and the computer. (As a quick aside, the terminal communicates at 38,400 baud and since it uses '8n1' that means the RS-232 frames are 8 data bits, no parity bits and 1 stop bit. RS-232 has a mandatory 'start' bit so that means that 10 bits are transmitted for every 8 data bits meaning an effective rate of 3,840 bytes per second. An 80x24 screen is 1,920 bytes so it takes half a second to transmit an entire screen and the terminal cannot keep up with that.) Our friend the Raspberry Pi by default only has the RXD and TXD pins enabled. It turns out the Broadcom SoC that the Raspberry Pi is based on has a mostly-real ish serial port built in but it just isn't setup right by default. Firstly on the Raspberry Pi 3 it is connected to the Bluetooth module. Since I don't need that a simple line can be added to the config.txt file to disable the Bluetooth and connect the serial port to the 40-pin GPIO header.

dtoverlay=pi3-disable-bt

But now we have to figure out how to enable the rest.

Device Tree

The harder part is to enable flow control. Looking at the pin out of the Raspberry Pi header you will notice that pin 36 and pin 11 have an alternate function of UART0 CTS and UART0 RTS. Further investigation notes that Linux doesn't seem to do DTR/DSR flow control, instead using CTS/RTS. Thankfully the meaning of the signals are roughly the same in our case so it should be a simple case of making a proper cable. Before we get that far though we need to figure out how to turn these pins on. A few years ago jwz blogged about this very problem and one of the commenters seems to have even done a bunch of research that helped me along. The only downside is that the solution proposed required running a program to turn those pin functions on, which didn't feel like the right way to do it given that ultimately I wanted to run a getty on the serial port on the Pi so I could login via my terminal. It turns out that the Pi has this configuration language it uses to describe the hardware. Like most embedded devices without a proper BIOS, Device Tree is used to communicate the state of the hardware to the Linux kernel so it can setup and load the proper drivers. That dtoverlay line from above is just adding a fragment to the larger description that changes the pin behaviors at boot.

Off to the official documents I went and began searching to try to find how exactly to come up with the correct incantation of bizarre symbols required to tell the system to do what I want. Thankfully I ran across someone who had already done the work and created the device tree overlays needed. How this tells the SoC to enable alternate function 3 on the GPIO pins, I don't know, but it works.

/dts-v1/;
/plugin/;

/*
   Enable /dev/ttyAMA0 RTS and CTS signals on 40-pin GPIO header
   CTS is available on pin 36 of J8 header (GPIO 16)
   RTS is available on pin 11 of J8 header (GPIO 17)
*/

/ {
    compatible = "brcm,bcm2708";

    fragment@0 {
        target = <&uart0>;
        __overlay__ {
                pinctrl-names = "default";
                pinctrl-0 = <&uart0_gpio14 &uart0_ctsrts_gpio16>;
        };
    };
};

They even had the compiled files ready to go so I could plop uart-ctsrts.dtbo into /boot/overlays and add dtoverlay=uart-ctsrts to config.txt and get a working serial port.

Logic Levels

As noted in JWZ's post the serial port on the Pi is what is commonly called 'TTL level' (though it is actually using CMOS level signaling) so it uses a digital voltage level to signify on and off whereas the RS-232 standard actually uses a positive and negative voltage level to help with noise rejection over longer distances. This means that if I just plugged the terminal into the Pi directly it would most assuredly blow up the port on the Pi by shoving +/- 12 volts into pins designed to go from +3.3 to 0. Thankfully for about as long as there have been serial ports this has been a problem so there are special chips designed to convert these voltage levels and I bought a module with one on it. A little bit of wiring and I had myself a nice working RS-232 serial port with... the wrong flow control pins.

Frankencable

One thing I didn't think I'd be doing in 2020 was making up a custom serial cable but here we are.

Basically I crossed over the RXD and TXD as described in the 'null modem' cable bit but instead of crossing over DTR and DSR I swapped DTR and DSR on the terminal side to CTS and RTS on the PC side. This enables the Linux driver (which you may recall only uses CTS/RTS) to talk to my terminal.

systemd oh systemd

Now that everything seems to work how do I get a nice login: prompt on my terminal? Well back in the more sensible days of the dark ages you could just add a line to a file in /etc/, usually called inittab that said something along the lines of

s1:12345:respawn:/sbin/agetty -L ttyS0 9600 vt100

This told the system to start a getty (the thing that gives you the text login: prompt) on ttyS0 at 9600 baud and set the terminal type to vt100. Now it seems the magical incantation is to make a file like the one below in /etc/systemd/system by copying /lib/systemd/system/serial-getty@.service and modifying the ExecStart line to contain the proper baud rate and terminal type for your application. Then you get to run systemctl daemon-reload and systemctl enable serial-getty@ttyAMA0 && systemctl start serial-getty@ttyAMA0. In my case the terminal is capable of a screaming 38400 baud and is a wy60 terminal type. Obviously given all the work we did to enable flow control you will want to enable it here as well.

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Serial Getty on %I
Documentation=man:agetty(8) man:systemd-getty-generator(8)
Documentation=http://0pointer.de/blog/projects/serial-console.html
BindsTo=dev-%i.device
After=dev-%i.device systemd-user-sessions.service plymouth-quit-wait.service
After=rc-local.service

# If additional gettys are spawned during boot then we should make
# sure that this is synchronized before getty.target, even though
# getty.target didn't actually pull it in.
Before=getty.target
IgnoreOnIsolate=yes

[Service]
ExecStart=-/sbin/agetty --keep-baud 38400,19200 --flow-control ttyAMA0 vt100
Type=idle
Restart=always
UtmpIdentifier=%I
TTYPath=/dev/%I
TTYReset=yes
TTYVHangup=yes
KillMode=process
IgnoreSIGPIPE=no
SendSIGHUP=yes

After all of that, it actually worked. I was pretty thrilled. Of course in proper scope creep fashion I wanted to put this whole thing into a nice case so it wasn't a little bump away from ripping all the wires off the board so I first set about and designed a small marshaling board that would allow me to connect the RS-232 level shifter to the Raspberry Pi. I also wanted to add a real time clock module since bizarrely the Pi has always lacked one. While I was doing that I noticed that I had a LM75 temperature sensor laying around so I threw that on the board too.

I bought the LM75s originally because they are supported by the Linux lm-sensors framework, being found on many motherboards back in the day. I had always planned on putting one in a Raspberry Pi but never actually got around to it. I found a post where someone not only did this but got the Device Tree stuff working. In their case they recompiled the entire Device Tree for the board so I simply turned their changes into an overlay source file and compiled it.

// Enable the LM75 Sensor on the Marshalling Board
/dts-v1/;
/plugin/;

/ {
    compatible = "brcm,bcm2835";

    fragment@0 {
         target = <&i2c_arm>;
         #address-cells = <0x00000001>;
         #size-cells = <0x00000000>;
         status = "okay";
         __overlay__ {
             lm75@48 {
                 compatible = "lm75";
                 reg = <0x48>;
                 status = "okay";
             };
         };
    };
};

Of course I wrapped all of this up into an ansible role similar to the one I made for my ADS-B feeder and let it rip on a default install of Raspbian Buster Lite. After a run and a reboot the getty appeared on the terminal and the RTC and sensors appeared on the i2c bus.

Shockingly given the current state of affairs the PCBs took only a week to get here from JLCPCB so I was able to assemble and test the final version.

Final Changes

My terminal doesn't have the glorious 80x60 portrait layout or moody phosphor sustain as JWZ's has but it brings me back to the good old days when I actually had a Wyse VGA monitor version of one of these and taught myself Turbo Pascal on a NEC PowerMate 286/12...

After assembling everything I ran into a couple changes that needed to be made. Firstly the Linux wy60 terminal profile seems wrong, it was acting like it was dropping characters and doing strange things to my prompt so I dug out an old RS-232 tester to make sure the flow control lines were in fact being used, and they are.

Since it didn't seem like a communication error I figured it must be the fact that the WY-60 terminal profile was less popular than good old VT-100. The WY-60 itself can speak VT-100 (and a bunch of others it turns out) so I switched to that and everything worked great.

Conclusion

Honestly, I'm just happy to have this thing working. It should be easier to get real serial ports these days but maybe I'm just too fondly remembering the dark ages of computing when RS-232 was king. 🍸