Monday, April 2, 2018

retro-B5500 Emulator Release 1.05

Nigel and I are pleased to announce that version 1.05 of the retro-B5500 emulator was released on 18 March 2018. All changes have been posted to the repository for our GitHub project. The hosting site has also been updated with this release.

This is a primarily a bug-fix release, containing the collection of corrections and enhancements that have accumulated since 1.04 was released in September 2016.

Enhancements in Release 1.05


Apple Safari Pop-up Restrictions


This first item could be qualified as a bug fix, but it's really a new feature. The emulator had been operating quite well in the three major modern workstation browsers -- Google Chrome, Mozilla Firefox, and Apple Safari. I recently updated my seldom-used MacBook Air to macOS 10.13 (High Sierra), and with that update came Safari 11.0.

When I tried to run the retro-B5500 emulator in Safari after this update, the emulator crashed after opening its first pop-up window. I quickly discovered that my retro-205 and retro-220 emulators suffered the same fate. I had Safari configured to allow pop-ups, so what was the problem?

Some research revealed that this new version of Safari places additional restrictions on pop-up windows. Specifically, with pop-ups enabled it would always allow the first pop-up, but if subsequent pop-ups were opened before some amount of time had passed, Safari would inhibit the pop-up. Reports varied on the amount of time that had to elapse between opens, ranging upward from 500ms.

The emulator opens pop-ups for the Console Panel and each of its peripheral devices during its initialization. All of these elements are implemented as Javascript objects, and as part of their instantiation, each one opens, sizes, and positions its own pop-up window. Trying to schedule appropriate delays in the initialization process that instantiates these objects was going to be difficult, so I decided instead to centralize the mechanism for opening pop-ups.

Now, each object calls a common "open pop-up" function in the B5500Util.js module, passing the parameters necessary to open and configure the window. The new mechanism maintains a queue of these open requests, and inserts delays between servicing of the queued requests. If an open request fails (i.e., the Javascript window.open() function returns null), the request is reinserted at the head of the queue and the servicing delay is increased.

This new scheme works extremely well. It automatically adjusts the delay between opens based on the individual browser's behavior. The initial timeout value is 500ms and increments by 250ms on each iteration of a failed open. My tests indicate that Safari 11.0 starts making nice when the timeout reaches 1000ms. Chrome and Firefox both cruise through pop-up opens, because their open attempts do not fail; thus nothing ever gets queued and their opens proceed without delay.

Deimplement the Application Cache


The Application Cache (or AppCache) is a feature that allows a browser to download and store all of the scripts, images, and other resources for a web-based application and store them on your workstation. You can then run the application locally, without access to a network connection to the host web server. When a network connection is available, the browser checks during application initialization for updates. You would see messages in blue displayed briefly at the top of the emulator's home page reflecting this activity.

This feature seemed ideally suited to the emulator, since once it is loaded by the browser, the emulator needs no further communication with the host web server. Alas, this feature ran afoul of a segment of the developer community who thought it should be something different than what it was, some of whom were quite nasty about it. As a result, the Application Cache has been withdrawn as a draft web standard. Both Google Chrome and Mozilla Firefox have deprecated the feature in their browsers and announced that it will be removed in a future release.

In anticipation of this, I have removed from the emulator the things that enabled the Application Cache. The home page now displays a brief message indicating the feature has been deimplemented. This message will be removed in a future release. The first time you load the emulator's home page for this new release (assuming you have an active network connection), the AppCache will be deleted in your browser, but you will still be running the prior release. That is normal AppCache behavior. Simply reload the home page to use the new release.

For those who would still like to be able to run the emulator off-line, the only remaining option is to run a web server on your workstation. You only need a minimal web server in order to do this. As described in the Getting Started wiki page, I have had good success with the free, open-source mongoose web server, which is available in versions for Windows, Mac OS, and Linux/Unix.

Emulator Release Zip Archives



For those of you running your own web server, I have decided to discontinue creating zip archives of the emulator files and placing them on the project's Google Drive. Generating such an archive can be done directly on GitHub. On the project home page, look for the green Clone or download Code [updated 2022-05-07] button:

Download Source ZIP Archive


Click that button and then the Download ZIP link. You can also browse the complete list of commits by clicking the small clock-face icon with the number on its right, just below the green Code button on the project home page and download an archive from a specific release from that list.


Console Panel Annunciator Lamps


The small annunciator lamps along the bottom of the emulator's Console Panel that show the system's I/O channel, interrupt, and peripheral device activity have a slightly different look. Before, the lamps simply appeared in white when they were lit and disappeared when they weren't. Now each annunciator shows in dark gray when not lit. This allows you to see what all of the annunciators are, and it more closely approximates the back-lit effect the annunciators are intended to emulate.

New Console Panel Annunciators
 Note that these annunciators did not exist on a real B5500 console. They are an artifact of the emulator, present simply to give you an idea of the system's activity. Their presence can be toggled on or off by clicking the Burroughs logo on the panel. The annunciator mnemonics are explained on the Using the Console wiki page.

Asynchronous Callback Management


There has always been a significant exchange of ideas, techniques, and code among my Javascript-based emulator projects. One common module among the projects is something called the setCallback() function. It it part of the mechanism that regulates performance, trying to make the emulated Processors and peripheral devices operate at the speed of a real B5500.

Like all Javascript applications running in a web browser, the emulator executes on a single thread of the Javascript run-time system. The appearance of multiple, simultaneous activities is achieved in the same way that most operating systems appear to run multiple programs at once -- by giving each activity a short time slice and switching rapidly among the slices. The way you do that in Javascript is through callback functions that are executed asynchronously. The run-time system assures that only one of these callbacks executes at a time.

User interface events (e.g., mouse clicks) and some DOM APIs use callback functions to signal the emulator, but the primary way in which the emulator regulates performance is by keeping track of when Processor activity and I/O completions should be occurring in real time and inserting delays to allow real time to catch up with emulator time. The way it does this is through the Javascript setTimeout() function. You pass it a reference to a callback function, a number representing a delay in milliseconds, and an optional list of parameters. The run-time calls that function, passing any parameters, after the specified delay. The emulator typically has several of these asynchronous function calls scheduled at a time, and their interleaved execution is what gives the appearance of multiple simultaneous activities in the system.

There are two big problems with setTimeout(), though:
  1. The delay is almost always at least a little longer than you asked for. Since everything is single-threaded, some other function may be executing when your delay expires, and there may be other functions whose delay has expired ahead of yours, waiting in the queue for their turn on the single thread. In most browsers, background activities such as screen painting and memory garbage collection also run on the single thread, so these can also inflate the delay before a callback is actually called.
  2. Modern browsers limit the minimum delay to at least four milliseconds when callbacks are scheduled out of callbacks. There are a couple of reasons for this, but the original one was to inhibit poorly-behaved web pages from engaging in intensive CPU activity that drained the batteries of laptops.
The emulator tries to be well-behaved, but both of these problems interfere with its goal of running the Processors and peripheral devices at real B5500 speeds. The 4ms limit was particularly bedeviling for Processor performance throttling, which requires delays of less than 4ms. There is no mechanism for scheduling shorter delays, but I did find one that could produce a near-zero delay -- what in operating systems is termed a yield. This is based on the Javascript postMessage() API. It involves the emulator sending a small message to itself and attaching a callback function to the onMessage event.

This discovery lead to the development of setCallback(), which is used wherever you would normally use setTimeout().  If a particular call requires a delay of at least 4ms, settCallback() uses setTimeout(); if the delay is less than 4ms, it uses postMessage/onMessage instead. In the first case, the actual delay will be somewhat longer than requested, and in the second case it will usually be much shorter. To compensate for that, once the delay expires, the mechanism measures the actual delay that occurred and accumulates the deviation between the actual delay and the one originally requested.

A portion of that cumulative deviation is then applied to the next call on setCallback(), adjusting the requested delay up or down depending on the sign and magnitude of the deviation. If the cumulative deviation is positive (delays are running longer than requested), future delays will be reduced by some amount; if the deviation is negative (delays are running too short), future delays will be increased by some amount. The magnitude of the cumulative deviation is also reduced by these adjustments, with the goal of driving it toward zero.

Thus, instead of trying to make the individual callback delays accurate, the goal of this mechanism is to have the deviations offset each other to produce delays that are accurate in the average.

This mechanism was originally developed for the retro-B5500 emulator, and has been ported more or less as is to the other emulator projects. The portion that adjusts requested delays has required a lot of tuning to make it stable and reduce jitter, however, and that has been an on-going effort in all of the emulators. Last year, the retro-205 emulator exposed an instability in the adjustment algorithm, and I ended up completely redesigning it for that emulator. This release of retro-B5500 now includes that redesign.

Sending a message to yourself always seemed to me to be a bit of a hokey way to generate a yield, and there are potential cross-site security issues with postMessage(). It worked well enough, though, and it was the only approach I had been able to find. Then, while working on the tape drive implementation for the retro-220 emulator last Fall, I started using a relatively new mechanism for handling asynchronous function calls, Promises. Basically, a Promise is an object to which you can attach one or more callback functions. At some point, the Promise is "resolved" (i.e., the thing that was "promised" is reported as being fulfilled) by calling one of the Promise's methods. The Promise then calls each of the attached callbacks, one at a time, in the order they were attached.

This may not seem like much, but Promises turn out to be a very useful abstraction in asynchronous programming. They have two features that really caught my attention:
  1. After attaching a callback function to a Promise, the function will not be called until after the attaching code has returned to the Javascript run-time dispatcher.
  2. It is possible to create a Promise that is in resolved immediately upon creation. Any callbacks that are attached to a resolved Promise will still be called, but only after the code exits back to the Javascript dispatcher.
The fact that a Promise's callbacks are only called after returning to the Javascript run-time means that any other queued callback functions would be called first. That in turn means that a Promise should be able to generate a minimum-delay yield. Some experiments proved this to be true, and apparently with lower overhead and fewer security implications than postMessage()/onMessage(). Thus, the setCallback() implementation in this retro-B5500 release also incorporates the new Promise-based mechanism from the retro-220 project.

Minor Refinements

  • Rotational latencies for the Head-per-Track disk units are now computed from the high-precision performance.now() timer instead of the random number generator used previously.
  • The emulator's internal bindMethod() function, used to apply an object context to functions, has been replaced by the standard Javascript Function.bind() method.
  • The emulator's custom utility routines for manipulating a DOM element's className attribute have been replaced by standard DOM methods for the attribute's classList object.

Corrections and Bug Fixes


Two Nasty Bugs



The retro-B5500 emulator has always had a problem running a heavy mix of jobs. We first noticed this when we tried to recompile the Mark XIII system software, running several large compilations at once. The symptoms were Flag Bit and Invalid Address errors. Usually these errors aborted jobs in the mix. The Invalid Address error would occasionally result in an MCP "punt," typing "INVALD ADRSS" on the SPO and halting the system. The errors occurred at random times and in random locations in random programs.

Since this problem seemed to exhibit itself only during a heavy mix, and a heavy mix usually implies significant memory segment overlay activity due to Presence Bit interrupts, it seemed likely that the errors were somehow related to memory overlay or the handling of Presence Bit interrupts. I had tried several times over the past few years to trap these errors as they occurred and to isolate their cause, but without success.

Since I had to dig into the emulator to fix the Safari pop-up window problem anyway, I decided to make yet another run at resolving this long-standing problem. In previous attempts, I had tried to trigger memory dumps that captured the state of the system, but had been unable to generate one that revealed anything useful. Determining the source of the Flag Bit errors proved to be especially elusive.

The Flag Bit is the high-order bit in a B5500 word. Flag Bits mean nothing special in Character Mode, but in Word Mode they distinguish data words (operands) from control words. If the Flag Bit is a 1, it's a control word; if 0, it's a data word. The Processor generates a Flag Bit Interrupt in Word Mode when it fetches a word from memory that has a Flag Bit different from what it is expecting. This occurs in two cases:
  1. In the Operand Call (OPDC) instruction, if an indexed data descriptor references a non-operand word, i.e., one with its Flag Bit set.
  2. When exiting a procedure, if the Return Control Word addressed by the F register has its Flag Bit reset.
Interrupts for condition #1 are definitely errors. In researching this, however, I learned something about the B5500 architecture that I hadn't realized before -- interrupts for condition #2 are most often both intentional and beneficial.

Virtual memory on the B5500 (there's actually nothing virtual about it, but that is a subject for another post) is implemented using control words. Data areas are addressed through data descriptors, and code is addressed through program descriptors, also known as Program Control Words (PCWs). Data descriptors contain the physical memory address of the start of a data area, and PCWs contain the physical memory address of a procedure entry point or a cross-segment branch destination. Both words also contain a Presence Bit that indicates whether the associated data or code segment is physically present in memory or not.

Attempting to access memory through a descriptor having a zero Presence Bit generates a Presence Bit interrupt, which is the B5500 equivalent of a page fault. The MCP reacts to this interrupt by allocating space in memory (overlaying other areas if necessary to make room), making the referenced data or code present in memory, fixing up the descriptor with the physical memory address, setting the descriptor's Presence Bit to 1, and restarting the interrupted instruction. All of this is well known.

There are two other control words, however, that also address code, the Return Control Word (RCW) and Interrupt Return Control Word (IRCW). Both are stored in the stack by the hardware and hold return addresses -- the RCW as a result of a procedure call and the IRCW as a result of an interrupt. Both have a similar format, but neither one has a Presence Bit. So what will happen when a procedure exits back to its caller, but the caller's code segment has been overlaid while the called procedure has been executing?

This seems to be an aspect of the architecture that is less-well understood -- I certainly had not appreciated it before digging into the problem at hand, and have not found any mention of it in the documentation. It turns out that the MCP treats the Flag Bit of an RCW or IRCW somewhat like a Presence Bit. When a code segment is overlaid, the MCP not only resets the Presence Bit in any PCW that references that segment, it also resets the Flag Bit in any active RCW or IRCW that references that segment. If the hardware attempts to return to the address in one of these "absent" RCW or IRCW words, a Flag Bit Interrupt will occur. If the syllable that caused the interrupt is one of the return operators, the MCP will handle the interrupt similar to a Presence Bit Interrupt; otherwise the MCP will DS (abort) the job with a Flag Bit error.

Thus, my first attempts to capture the source of the problem, by trapping every Flag Bit Interrupt, led to somewhat comical results -- a tsunami of memory dumps caused by perfectly normal and legitimate page-fault activity. What I needed was a way to trap the problem at the point where the MCP has determined the Flag Bit Interrupt was not a pseudo Presence Bit Interrupt. I did not want to modify the Mark XIII MCP to do this, so the approach I chose was to modify the emulator to detect entry into the MCP's TERMINALMESSAGE procedure, which generates the program abort message that gets typed on the SPO. TERMINALMESSAGE is a so-called "save" procedure, meaning that it has a fixed memory address and is not overlayable. For the Mark XIII MCP as released, its entry point is physical address 999 decimal.

That approach worked. After running my standard heavy mix batch of jobs a few times and analyzing the resulting Flag Bit dumps, it became apparent that all of the interrupts were occurring at the point a program returned from a Character Mode procedure back into Word Mode.

With that clue in hand, the problem was easy to find -- the emulator was handling both Character and Word Mode exits using the same common routine, but a closer inspection of the Processor Flows and the B5281 Processor Training Manual showed that while Character and Word Mode returns are very similar, they are not the same. The common routine was handling Word Mode returns properly, but not Character Mode returns. In particular, it was not invalidating the A register before checking the Flag Bit in the RCW. Leaving the A register occupied could result an extra word in the stack if the interrupt was triggered, which could throw off the MCP's addressing of the stack while handling the interrupt. Of course, the interrupt was most likely to be triggered during a heavy mix when the probably was high that many code segments would have been rolled out.

Rewriting the procedure return mechanism in the emulator so that Word Mode and Character Mode returns were handled separately completely eliminated the problem with Flag Bit aborts. That left the Invalid Address aborts, which had been occurring a lot less frequently than the Flag Bit aborts, but were still taking place.

Invalid Address Interrupts occur under two conditions:
  1. When the Processor attempts to access a memory location that does not exist. This could be due to a memory module that was not installed or one which had been powered off or switched to local status.
  2. When a Normal State program attempts to access a physical address less than 1000 octal. The first 512 decimal words of memory are reserved for the MCP, and can be accessed only when the Processor is in Control State.
I was running a configuration with a full complement of 32K words, so condition #1 did not apply. In reviewing how the emulator handles condition #2, I realized that the original implementation distributed responsibility for detecting this condition between the Processor and Central Control. In fact, that detection was a matter entirely internal to the Processor. In the process of correcting that, it suddenly struck me that while the emulator was checking for Normal State in the detection of condition #2, it wasn't checking for it in other cases -- an invalid address occurring in Control State should be ignored and not generate an interrupt. It was clear the emulator had been generating interrupts in Control State, and that was the problem, not a bad address.

After fixing that really dumb error, my heavy mix batch now runs successfully and repeatedly. It has taken almost five years to nail these two nasty bugs, but finally it's done.

Miscellaneous Corrections

  1. Disable the word count field in I/O Descriptors for card reader/punch operations. I/Os for both devices did not use this field -- they always access 10 words of memory to transfer 80 characters of data.
  2. Fix a bug when using the RUNOUT button on the card punch to empty the stackers. The code was making an invalid property reference that caused a Javascript error to be thrown.
  3. Implement separate jump mechanisms for Word and Character Mode to correct edge-case errors in branch logic.
  4. Remove extraneous white space from B5500FramePaper.html. This file provides the initial content for the <iframe> elements that represent "paper" in peripheral devices such as the line printer, card punch, and SPO. The extraneous white space produced additional blank lines of text when the contents of the <iframe> were saved to disk or copied to the clipboard.
  5. Correct top-of-form handling in the line printer. Certain combinations of carriage control would interfere with top-of-form handling.
  6. Correct the animation of the tape reel in the tape drive for backward motion.
  7. Make a slight correction to clock counting in Single-Precision Add.
  8. Add P1 S and F register values to the dump caption record in tape images for internal memory dumps to tape (i.e., those generated by clicking the NOT READY lamp on the Console Panel). Also update and improve the dump tape's label records.
  9. Correct the method to "focus" the Console Panel window after the SPO initializes and becomes ready.
The problem resolved in #1 above was causing the Cold- and Cool-Start decks to function improperly. These decks are used to initialize and repair the disk filesystem and to load MCP code files. Both decks consist of the Cold- or Cool-Start program deck, followed by their parameter cards, followed by the MCP Tape-to-Disk loader program deck, followed by its parameter cards.

The Cold- and Cool-Start programs are designed to bootstrap the Tape-to-Disk Loader program automatically when they finish. I had not realized this, and thought you had to halt/load each program individually. The only way I could get that to work was to insert a blank card just before the Tape-to-Disk Loader program.

What was really happening was that the Cold- or Cool-Start program was attempting to read the first card of the loader program into memory, but unlike the MCP, was not setting the word count field in the I/O Descriptor. Thus the card was physically being read, but because the emulator was incorrectly relying on the word-count field, none of the card's data got transferred to memory, so there was no code to execute when the Cold- or Cool-Start program branched to it. Adding the blank card to the deck caused the Cold- or Cool-Start program simply to "eat" that card, allowing a subsequent manual halt/load to boot the loader program starting with its first card.

With this correction in place, that blank card now gets in the way. In this release it has been removed from the COLDSTART-XIII.card and COOLSTART-XIII.card decks available in the tools/ directory of the project repository. Any decks you may have that are based on these two should be checked. Any card between the STOP card that terminates the Cold- or Cool-Start program's parameters and the binary bootstrap card for the Tape-to-Disk Loader should be removed.