Modular Monitor: Serial Terminals, Hardware Bugs, and You

Connecting the 6502 to a Serial Terminal Turned Out to be Harder Than Expected.

When I designed my minimal 6502 development board, I made plenty of concessions to keep the project feasible. A lot of my grandiose ideas for it turned out to be impractical, but I’m still using it. Why not? It’s still a (mostly) functional 6502 computer.

One of those design decisions was to use a basic UM245R USB to FIFO board as the only means of I/O. This is because a serial terminal is the lowest common denominator when it comes to computer I/O. USB works on modern computers and doubles as a power supply.

While my first “Hello World!” program needed very little work, making an interactive interface is considerably harder. There’s a lot of logic to keep track of, and very little structure to build around. Not everything is well documented, and the 6502 is a real chore to work with sometimes.

Then there were a bunch of outstanding hardware bugs. Those didn’t help either. In fact they straight up stole the show and became the main focus of this article.

Rather than suffer alone, I figured I’d explain the rather long journey I took to get just the bare bones of an interactive system set up. It’s a tale of intrigue, deception, and a single misplaced wire that took a really long time to find.

Terminal I/O

A terminal is a character oriented device that transmits typed characters to the host computer and receives characters sent from said host computer. This isn’t really how things work, but it’s a sufficient explanation for now.

Logically, the two data streams are separate. Terminal drivers can therefore be split into sending and receiving parts.

Typical terminals will talk in ASCII. While it’s close to a universal standard, non-ASCII encodings definitely still exist (e.g. EBCDIC, FIELDATA, Baudot). Variations on ASCII also exist. Everything I’m going to do relies on the input being plain US ASCII.


The receiver half of the driver must do a lot of work. Data must first be copied from the serial port to memory in a timely manner. That’s the easy part.

What comes out of the serial port is a raw stream of bytes. That includes control characters that indicate various formatting instructions. It’s up to software to interpret them. There are two main ways to do this:

  • Forward a stream of bytes, only interpret the bare minimum of control characters.
  • Forward whole lines, interpreting control characters as they come up.

Both are useful in different situations. Simple software prefers formatted lines. Advanced software likes having direct access to control characters. A good terminal driver should be able to switch between the two.

For now, I chose to implement a simple interpreter that buffers a whole line and handles the two most common control characters: Line Feed (LF, $0A) and Backspace (BS, $08). Since I don’t have any advanced software to interpret those control characters, my hands were already tied.


Sending is much simpler. It can be reasonably assumed the incoming line is already formatted. Control characters are interpreted by the receiver, so the sender has no need to care about them.

The main job of the sender is to simply copy the character to the serial port in order. Very simple- but there’s a catch. If the serial port can’t accept a new character, the sender should pause. We call that flow control. Receiving also has to deal with flow control sometimes, but the UART (UM245R included) can usually handle it on their own.

UM245R’s handle flow control mostly automatically. The only trick is that it has a finite input queue, so new characters shouldn’t be written if it can’t accept them. UARTs have a very similar interface, so porting from the virtual serial port to an actual serial port should be pretty easy.

6502 Terminal Driver

Writing the terminal driver was supposed to be fast and easy. In many ways it was. But, as anyone who ever wrote any software knows, “almost working” and “always working” can be on two different planes of existence.

Part of the problem is my development board was thrown together to get a 6502-based system up and running as quickly as possible. I didn’t take nearly as much care as I should have. With this project complete, I have hopefully found all the hardware bugs.

Establishing a Baseline

First step: check if the 6502 board is still working. I haven’t touched it in maybe a year now, and it took a lot of work to get even a peep out of.

My initial test was not successful, but I did take the ROM out at some point. Maybe I got it mixed up?

When I looked at the ROM, it was full of corrupt garbage. Random bytes were given random values. There was enough there to prove this was the ROM I used in the 6502 board, but the random crap all over is a mystery. My only explanation is the AT28C64B EEPROM got unlocked somehow. If the 6502 crashed then it could potentially write garbage to random bytes.

I reflashed the ROM with my “Hello world!” program. No issues there, so it’s (probably) not the ROM dying. A little fiddling gave the expected result:

Hardware Bug Squishing

Before going any further, I feel the need to point out various problems with the hardware I had to fix.

I fixed the UM245R before the “Hello world!” test, but the ‘HC125 and resonator came after.

You would not believe how many wiring errors a computer can have before it reliably crashes! The biggest hardware bug I’ve found is still yet to reveal itself.


Serial terminals are typically treated as two logical devices: keyboard input, visual output. For all intents and purposes, they are totally separate.

This separation causes the first problem the terminal driver has to solve. When you press a key, the character should appear on the screen. We call that echoing. Terminals sometimes have a “local” echo mode that handles this in the terminal. Local echoes are undesirable, since they hide whether the computer is actually working.

In order to echo the key press, the receiver must immediately send the byte back to the terminal. This isn’t always a good idea with control characters, but for now I feel they can be echoed too.

SER_DAT  EQU $CBFF ;Serial port is at a fixed address

;===== Main program body =====
   BIT SER_FLAG  ;Check flags
   BVS READ      ;BIT sets V, which is RXF
   BRA READ      ;Endless loop

Notice that READ waits in a tight loop, endlessly checking the serial port for new characters. We call that polled I/O. Polling is simple but inefficient. When the CPU has nothing else to do, polling is acceptable.

Receive Interrupt

Where polling is completely unacceptable is an interactive environment. You don’t want processing to screech to a halt because the user pauses for a bit.

Interrupts are the solution. Instead of constantly asking the serial port for new characters, we wait for the serial port to tell us when it’s ready. I wired the new character signal on the UM245R to the 6502 IRQ input just for this.

SER_DAT  EQU $CBFF ;Serial port is at a fixed address

;===== Main program body =====
   LDA #$80      ;Mask and enable RXF interrupt
   CLI           ;Turn on interrupts
   BRA MAIN      ;Nothing to do- yet

;===== IRQ routine =====
   BIT SER_FLAG  ;Check flags
   BVS EXIT      ;BIT sets V, which is RXF
   BRA RECEIVE   ;Endless loop
   RTI           ;Return to MAIN

;===== 6502 interrupt vectors =====
    DW RECEIVE   ;Hook IRQ

Despite the underlying program changing drastically, the user experience didn’t change much. The only notable difference is the green IRQ LED lights.

Also there was some odd latency, and missed characters. It was a warning sign that shouldn’t have been ignored.

Line Buffer

Thus far, the terminal driver is operating strictly in character mode. Line mode requires a whole line of text be buffered, with control characters transparently executed.

First, we have to find a place to put a whole line. It must be in read-write memory for obvious reasons. I chose $0200-$02FF. That’s the first RAM region beyond the zero page and stack.

RECEIVE is modified to store characters in the buffer, using Y as the index. A static copy of RECEIVE’s Y has to be added to the zero page. Additionally a zero page semaphore is added to indicate a new line is ready.

Control characters are isolated, interpreted, and executed. I chose to implement BS and LF. That’s the minimum needed for a functional interface. Everything else is currently ignored.

SEND simply copies the buffer out to the serial port. It terminates when it sees LF, which is the end of line marker.

;Test program to get serial I/O up and running
;========== Constants and Variables ==========
;Fixed addresses
;ASCII control chars
BS EQU $08
;ZP variables
NEW_LIN EQU $00 ;Simple semaphore
INT_Y   EQU $01 ;Keeps Y safe for interrupt indexing

;========== Initialization ==========
	ORG $0000	;ROM relative! Actually starts at $E000	
START:		 	;Run once initialization
	LDA #$3E        ;Print '>' just to show we're up and running
	LDA #$7F
	CLI		;Interrupts on
MAIN:			;Set up for a brand new line
	LDA NEW_LIN ;Wait on semaphore
	LDY #$00
SEND:			;Send chars using LF terminator
	BEQ MAIN 	;Guard against infinite loop

;========== Receive interrupt ==========
	LDY  INT_Y  ;Load interrupt Y
RLOOP:			;Read all available chars from serial port
	BIT SER_FLAG;Get flags, bit 6 is data available signal
	BVS REXIT	;V set means no new data
	STA SER_DAT ;Echo char
	CMP #32 	;Printing chars >=32
CHR_BS:			;Backspace: move back one space. Simple as that
	CPY #$00 	;When Y = 0 nothing more can be backspaced
CHR_LF:			;Line feed: clear Y, set NEW_LIN
	LDY #$00
CHR_PRINT:		;Printing chars go in the buffer
REXIT:			;Exit from int
	STY INT_Y	;Store interrupt Y
NMI:			;If we ever get here, we need to leave
;========== Hardware vectors ==========
	ORG $1FFA					
	DW START + ROM_START		;ROM is last 8K, but address space is 64K

This software isn’t that well optimized, nor is it properly encapsulated. I’ll have to add more functionality later, so there’s no sense in putting in much effort yet.

My Most Embarrassing Hardware Bug Yet

Getting the line buffer to work was several days in the making. The software was easy, I had the basics done in less than an hour- including various breaks.

What followed was about twelve hours of intense debugging spread out over three days. I tried dozens of little variations, adding a debug print here, tweaking an address there. Nothing. At best I had a stream of garbage bytes endlessly cascading down the terminal window.

If the software doesn’t work, even a little bit, you need to consider if the hardware is working properly. Probing showed no issues. Not even my logic analyzer revealed much of interest.

Eventually I got to the barbaric part of hardware debugging: yanking parts. I had narrowed things down to the RAM chip. When I pulled it- while live mind you- nothing happened. Pulling the RAM out of a working computer should be an instant crash. I had no change whatsoever.

Almost as if the RAM wasn’t being written to. Like it had been wired up wrong from the beginning. Surely not?

Well, there’s the problem and the solution. After patching in the new link everything worked perfectly. Something I could have done while building the board. Should have done.

Moral of the story: read those datasheets carefully!

The strange thing is my interrupt routine seemed to work fine, if strangely slow and unreliable. How?!

Uninitialized EEPROM is guaranteed to be $FF. Looking at the R65C02’s instruction set, $FF is BBS7. This branches if bit 7 in a zero page location is set. A string of $FF, $FF, $FF bytes means “check ZP location $FF, then move back -127 instructions if bit 7 = ‘1’”.

Since the RAM wasn’t hooked up, the read to $FF would return something random. A long string of BBS instructions is effectively a NOP. By sheer luck the 6502 would eventually stumble back upon working code. Using the stack would result in something similar, so an RTI would end up here too.

Second moral of the story: completely broken software/hardware can still produce correct results, at least some of the time!

What’s Next

Figuring out how to get the 6502 to talk to a serial terminal is the first step to getting it to do something interesting. It’s much harder to do that when your hardware is very subtly broken. I intended to write a software-centered article based around the ins and outs of serial terminals. What I ended up with was a long, difficult hardware bug hunt.

This is the first time I’ve touched the RAM. Hence, I never realized it was wired wrong for such a long time. Removing the incorrect link required some circuit surgery but I managed to do it without breaking anything else.

I still have no good explanation for why my ROM got corrupted. It took new data just fine, and hasn’t got corrupted since. Something to keep on watch, I guess.

While re-programming the ROM, the inevitable happened: I plugged it in backwards. Amazingly the AT28C64B took this remarkably well. It seems to work just fine, even though it got super hot in the thirty seconds or so it was on. Normally that’s instant death for an IC. I suspect USB current limiting saved me. Consider it extra motivation to get a self-hosting system ASAP.

I’m really, really hoping this is the end of serious hardware bugs. In addition to the aforementioned bugs, I transposed the RXF and TXF flags on the UM245R interface. Interrupts work just fine, it’s the data lines I swapped around. RXF was supposed to go on D7, but ended up on D6. I can fix that in software using BIT, but that’s slower than a simple LDA that would work if the was flag on D7.

When I started writing the terminal driver, I expected it to be done in a day or two. I didn’t expect to have such an intense bug removal session. It was so tiring, I decided to wrap this up without digging into the software much.

Next time I’ll go into the first big 6502 program I’m tackling: the ROM monitor. Assuming of course the hardware isn’t broken enough to prevent me from doing that. We’ll see.

Have a question? Comment? Insight? Post below!

Discover more from Some Disassembly Required

Subscribe now to keep reading and get access to the full archive.

Continue reading