Emulator Release 1.03 and Browser Disk Space Usage

Nigel and I are pleased to announce that version 1.03 of the retro-B5500 emulator was released on 22 August 2015. All changes have been posted to the repository for our GitHub project. The hosting site has also been updated with this release, and for those of you running your own web server, a zip file of the source can be downloaded from the GitHub repository [updated 2022-05-07].

This is a minor release containing corrections for a few issues we have discovered since version 1.02 was released in June. The most significant correction addresses excessive disk storage usage for emulated B5500 disk units.

Excessive Browser Disk Usage Fixed

The emulator uses an HTML5 API known as IndexedDB to provide persistent storage for B5500 disk devices. IndexedDB is a non-relational (NoSQL) database mechanism. It stores Javascript objects, indexed either by an external key value, or by possibly multiple data items internal to the object being stored. Instead of tables, IndexedDB has "object stores," which serve much the same purpose.

The emulator creates a small IndexedDB database named "retro-B5500-Config" to store system configuration data. It creates one additional IndexedDB database for each emulated disk system created by the configuration interface. Each of these subsystem databases contains a "CONFIG" store for its internal configuration, plus an "EUn" store for each disk Electronics Unit in the subsystem. Within an EU store, each disk sector is represented as an object, indexed by its zero-relative sector number. An SU is modeled simply as the range of sector numbers it represents.

The Problem

One of the puzzling (and quite disappointing) things several of us have noticed about the emulator is the very large amount of physical disk space it uses to store B5500 disk sector data. The amount of physical disk space required for the IndexedDB database is typically at least 30 times the size of the data being stored.

Each browser manages IndexedDB data internally in its own proprietary way, and nether of the two browsers that are known to support the emulator, Mozilla Firefox and Google Chrome, make it easy to see from the outside what's there or how it's stored. Firefox implements IndexedDB on top of SQLite. It is easy to see the EU structures in the SQLite database, but the objects themselves are very opaque, probably because they are compressed. Chrome implements IndexedDB using LevelDB, an open-source database manager developed by Google and based on its BigTable technology. Its internal structures are opaque from the outside, also due to compression.

In the past few months, several of us who have larger disk subsystems have been having increasing problems with Firefox accessing those subsystems. The first symptoms were that B5500 operations would simply grind to a halt in the middle of running ordinary work. The MCP was still active, but one of the Disk File Control Units (DFCU, i.e., DKA or DKB) would be hung, along with one of the I/O Units. A halt/load would bring the system back up, but often the system would lock up again fairly quickly. If you had multiple disk subsystems, deleting one of them would usually bring the system back to normal operation. This behavior was seen only with Firefox, not with Chrome.

After a couple of weekends of very frustrating investigation, I finally discovered that (a) Firefox was aborting a disk I/O due to a "Quota Exceeded" error, and (b) aborts are not reported to the IndexedDB onerror event, but rather to its onabort event, and the emulator was not catching the onabort event. The aborted I/O resulted in that I/O never being reported as complete, so the IO Unit stayed busy, and the MCP considered the I/O to still be in progress. Additional I/Os for that DCHA or EU queued up behind the aborted I/O, and eventually everything in the system went idle waiting for I/Os to complete. Not detecting the onabort event in the disk driver was a design error on my part.

The quota error reports that the emulator's "source" (web site) is using more local disk storage than it is allowed. I have not been able to find out exactly what that storage limit is in Firefox, plus, either the limit or the way it is enforced has been changing over the last few Firefox releases. Based on what a few of us have observed, the limit appears to be somewhere in the range of 500MB to 1GB of disk usage. With a 30X inflation factor, that relates to 30-60MB of B5500 data, which means that just loading the SYSTEM, SYMBOL1, and SYMBOL2 tape images would put you in range of the limit.

Knowing we were hitting a quota error and approximately where that error is triggered was useful, but it did not help in knowing what to do about it. My working assumption was that the method we were using to represent sector data was responsible for the 30X inflation factor, and was starting to think about different representations in order to reduce that factor.

The situation came to a head about two weeks ago when Firefox 40.0 was released. It refused to open most of my disk subsystems. I couldn't even halt/load the MCP in order to dump the files to tape. Fortunately, I had held off upgrading to FF 40 on my main development system until I found out about this problem, so was able to dump its disk subsystem while still on FF 39.

At this point something obviously had to be done about the way the emulator was storing disk sectors, so last weekend I started to build a testbed to evaluate different ways of representing sector data. As reported in more detail in this post on the forum, I used the old B5500ColdLoader script as a basis for my testbed, and decided to start by creating a disk subsystem using the current sector representation to obtain a baseline disk space usage.

To generate that baseline, I simply loaded the entire SYSTEM tape image, which is 12.4MB in size. When I checked the size of the resulting IndexedDB database to determine its inflation factor, I found -- surprise, surprise -- it wasn't inflated at all! In fact, it was only 10MB -- 80% the size of the raw data.

That result made me start looking very carefully at the way the ColdLoader and the emulator's disk driver stored data to IndexedDB. I had thought they were the same, but found there was one of those differences that you wouldn't believe could cause a problem until the evidence pushes the realization in your face.

The short story is that each IO Unit has an internal 16KB buffer it uses to convert between the 6-bit character codes in B5500 memory and the 8-bit ASCII codes used by the device drivers. The buffer is an HTML5 Uint8Array, which is a form of TypedArray object. The IO Unit passes this buffer during calls to the drivers, along with a length that indicates how much of the buffer is valid.

Since each disk sector is modeled as a separate object in IndexedDB, each sector must be stored to IndexedDB separately. Therefore, the disk driver must extract each sector of a multi-sector write from the IO Unit buffer and store it with a separate IndexedDB put() call. It implemented that extract using the subarray() method of Uint8Array:

eu.put(buffer.subarray(bx, bx+240), segAddr);

Alas, the resulting object that gets passed to put() is not a 240-byte Uint8Array, but rather an object that consists of the underlying array plus the starting and ending index values, as if it were something like this:

{data: buffer, start: bx, end: bx+240}

Thus, what got stored in the EU object store for that sector was not 240 bytes as I had assumed, but a copy of the entire 16KB IO Unit buffer, plus the two index values. No wonder we were seeing at least a 30X inflation factor in the data stored.

Firefox obviously does some compression on this object, but the degree of compression depends largely on what is in the IO Unit buffer. That buffer initially contains all zero bytes, and is simply overwritten by the data for each I/O. Thus, while the end of the buffer could be expected still to contain zeroes, the extent to which the front of the buffer had been overwritten by non-zero data, and therefore how much it could be compressed, would depend on the length of previous I/Os.

While storing these inflated sector objects had thus far only caused a problem in Firefox, they had the same effect in Chrome, and the sizes of Chrome's LevelDB databases were comparable to that of Firefox's SQLite databases. Chrome computes its quota threshold differently, however, based on the workstation's total available disk space. Apparently no one running the emulator has yet hit the Chrome quota limit.

The Solution

The solution to this problem is quite simple -- stop dragging that 16KB buffer around and storing it as part of every sector object. I did that by declaring a 240-byte Uint8Array object local to the disk driver, extracting sector data from the IO Unit buffer into that local object, and doing the IndexedDB put() on that local object, thus:

var sectorBuf = new Uint8Array(240);

...
while (segAddr <= endAddr) {
    for (x=0; x<240; ++x) {
        sectorBuf[x] = buffer[bx++];
    }
    eu.put(sectorBuf, segAddr);
    ++segAddr;
}

where bx is the current offset into the IO Unit buffer, segAddr is the current disk sector address, endAddr is the ending sector address for the I/O, and x is just a local index variable.

This approach introduces a little extra overhead to copy the bytes of each sector from the IO Unit buffer to the local buffer, but that overhead should be more than offset by the elimination of the 16KB IO Unit buffer in the stored object, the need to compress that buffer, and the extra I/O overhead to read and write an inflated sector object.

The nice thing about this solution is that it can be used with existing disk subsystems. Sector objects already present in the subsystem will be big and fat, but will still work. When a sector is written, it will stored in the new, more space-efficient format. Rewriting a sector will release some space to the underlying database's available pool, but it will not reduce the total amount of disk space used by the subsystem database. To do that, some sort of compaction process needs to take place, such as the vacuum command for SQLite. LevelDB will apparently compact its databases gradually over time automatically.

Experience to date with this solution indicates that my initial testbed experience is holding up. By completely dumping a disk subsystem to tape images, deleting and recreating it (to get an empty subsystem database), and reloading the dump, I am seeing reductions in disk space usage of 30-60X. In Firefox, the physical size of the reloaded databases continues to be about 80% of the size of the tape images used to load the databases. To cite one concrete example, the disk subsystem I use for most of my emulator development and testing went from 1.9GB down to 48MB, a reduction of almost 40X.  The dump tapes used to reload the subsystem totaled 58.7MB.

Even though this solution was easy to implement, works with existing disk subsystems, and adequately resolves the problems we have been having with recent versions of Firefox, I have not entirely given up on the idea of a new representation for sector data. There is a lot of overhead in the IO Unit involved in translating between 6- and 8-bit characters, packing and unpacking words, etc., and it may be that another approach will yield both better space and better processing efficiency.

Implementation

Since the fix for IndexedDB disk space usage in 1.03 is compatible with existing disk subsystems, there is nothing you need to do in preparation for moving to this release. If you are already having trouble with Firefox quotas, or have large disk subsystems and want to avoid having problems in the future, then you should consider dumping and reloading your disk subsystems. This section describes how to do that.

If you have already upgraded to Firefox 40 and cannot access your disk subsystems at all, then dumping the data is going to be problematic. See the next section for some ideas on things you can try and some background on where IndexedDB data is stored.

Assuming you can halt/load from your disk subsystem, however, then here are the steps to shrink the amount of disk space that subsystem is using.

1. Dump the B5500 disk data: 
1. Mount a blank tape in one of your tape drives. Make sure it is write-enabled (it will be by default) and the drive is in REMOTE status. 
2. On the SPO, enter the following command: "?DUMP TO MYDUMP =/=; END". You may substitute any other tape name for MYDUMP, up to seven characters long.
3. After the drive rewinds, click LOCAL and UNLOAD on the drive, then indicate you want to save the tape image.
4. You can either save the page directly in Firefox (make sure you save it as type Text, not HTML), or copy/paste the text of the image to a text editor and save it from there. Make sure your editor is set not to trim trailing spaces from lines in the file or to replace spaces with tabs.
5. If Library/Maintenance is not finished, mount another blank tape and repeat steps 3 and 4 above as necessary.
2. Start the emulator, but do not power it on.
3. Delete and recreate the disk subsystem:
1. Enter the System Configuration tool by clicking the B5500 logo on the Operator Console panel.
2. Select the appropriate system configuration, then click the EDIT button next to the Storage name for that configuration.
3. In the Disk Storage Configuration window that opens, click the yellow DELETE button. Click through the "are you sure you want to do this" prompts. The Disk Storage Configuration window should close.
4. Back on the System Configuration window, click the NEW button next to the Storage name field. The name of the subsystem just deleted will probably still display in the pull-down list. Enter the name of the disk subsystem you just deleted in the pop-up dialog that displays.
5. Configure the new disk subsystem as desired in the resulting Disk Storage Configuration window. Click SAVE on that window when finished, then click SAVE on the System Configuration window.
4. Cold-start the new disk subsystem. See the instructions for doing this in the Getting Started wiki page. You can either use the standard SYSTEM tape image to do this, or the first tape of the dump you just created. In that latter case, you will need to modify the cold-start deck to use the tape name of the dump.
5. Once the system has halt/loaded, enter on the SPO, "?LOAD FROM MYDUMP =/=; END". Library/Maintenance will refuse to overwrite a few system files, such as the running MCP. Do not forget to CI the Intrinsics file once it has been reloaded.

This may read like a lot of work, but it usually goes quickly. You should repeat this process for each of your disk subsystems.

Recovering from Quota Exceeded Problems

If Firefox considers your disk storage already to have exceeded the quota limit, then you may not be able to run the emulator in order to dump the data. There are a couple of things you can try, however.

• If you have multiple disk subsystems, try moving all but one of them out of the folder where Firefox maintains IndexedDB databases. That may reduce the disk space usage below the quota threshold, and allow you to halt/load from the remaining disk subsystem and dump it to tape. You may then be able to swap in the remaining disk subsystems one at a time and dump them as well.
• If some disk subsystems are too large for the first suggestion to work, you can try downgrading temporarily to an earlier version of Firefox, and see if you can halt/load and dump the subsystem using that version. You can download older Firefox releases from https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/.

On Windows 7, Firefox stores its IndexedDB data in the following location:

\Users\<user>\AppData\Roaming\Mozilla\Firefox\Profiles\<profile ID>\storage\default\<source>\idb\

where <user> is your Windows user name, <profile ID> is the profile name Firefox assigns (e.g., 0qus6gtz.default), and source is the host name of the web site from which you have loaded the emulator, e.g.,

http+++localhost
http+++www.phkimpel.us

Within the idb\ folder, each IndexedDB database is represented by a directory and an SQLite file. The names of these files are derived from the disk subsystem name as follows:

• The first part of the name is a 10-digit number, probably derived from a numeric hash of the name.
• The second part of the name is the disk subsystem name, but with the letters rearranged. This part consists of the characters from the first half of the name, with the remaining characters inserted between them in reverse order. For example, B5500DiskUnit is converted to 1182897429Bt5i5n0U0kDsi.
• The directory name has the extension ".files" and the SQLite file has the extension ".sqlite".

If the database is presently open and being updated, there may be additional SQLite files with the same name but different extensions. 

I have found that you can move the file and directory for a database into and out of the idb\ folder, and Firefox will adapt to that, but it is best to do this when Firefox is completely shut down.

For Ubuntu Linux, the IndexedDB data is stored in this path under your home directory. Most Linux distributions will probably be similar:

~/.mozilla/firefox/<profile ID>/storage/default/<source>/idb/

For Apple Macintosh OS X, the IndexedDB data is stored in this path under your home directory:

~/Library/Application Support/Firefox/Profiles/<profile ID>/storage/default/<source>/idb/

Prior to Firefox 38, IndexedDB data was stored in the .../storage/persistent/ folder instead of .../storage/default/ for all host systems, and the directory for the database did not have the ".files" extension.

Other Corrections in Release 1.03

In addition to the disk space usage fix described above, this release has the following:

1. Added onabort traps in B5500DiskUnit to catch QuotaExceeded errors. These are reported to the MCP as unit not-ready conditions, which will cause an error message to be printed on the SPO.
2. Modified the delay-deviation adjustment mechanism in B5500SetCallback to avoid oscillating between positive and negative cumulative deviations. This should improve the responsiveness of the emulator's internal timing and multi-threading.
3. Corrected tape reel angular motion in B5500MagTapeDrive, especially during reverse tape movement.
4. Fixed a bug with reporting memory parity error during tape I/O (should that error ever occur, which at present it won't, as the emulator does not generate memory parity errors).
5. Reset the Algol Glyphs option for card punch CPA in the default system configuration template. Newly-created configurations will no longer have this option set by default.
6. Fixed tools/B5500LibMaintDecoder to examine an entire .bcd tape image file instead of just the first 64KB.
7. Added USE SAVEPBT to the default cold-start options in tools/COLDSTART-XIII.card. This will cause printer-backup tapes to be marked as saved when they are released and not printed automatically.
8. Eliminated the extraneous "schema update successful" alert when altering a disk subsystem configuration. This means there is one less dialog box you must click through when changing that configuration.
9. Commited minor corrections supplied by Richard Fehlinger to source/B65ESPOL/SOURCE.alg_m.

Emulator Release 1.02 and the Move to GitHub

Nigel and I are pleased to announce that version 1.02 of the retro-B5500 emulator was released on 21 June 2015. All changes have been posted to the repository for our GitHub project. The hosting site has also been updated with this release, and for those of you running your own web server, a zip file of the source can be downloaded from the GitHub repository [updated 2022-05-07].

This is a minor release containing corrections for several issues we have discovered since version 1.01 was released. In addition, this release contains the documentation and web site changes made necessary by the move to GitHub.

Project Now Hosted on GitHub

As some may already know, Google announced in early 2015 that it would be discontinuing its Google Code project hosting service. All projects will become read-only in August, and the service will be shut down completely in early 2016.

Google Code served us extremely well, but obviously we had to find a new home for the project. After looking at a few alternatives, Nigel and I settled on GitHub, easily the largest and most popular code project hosting service available today. Public projects like this one can be hosted for free, which was an important consideration for us. Plus, the site does not pester visitors with advertisements. Google provided a service to migrate Google Code projects to GitHub, and we took advantage of that to move the emulator's repository, issues, and wiki pages to their new home.

The new URL for the project site is https://github.com/pkimpel/retro-b5500/.

The former Google Code site for the project now redirects to the GitHub URL above. All further emulator development will be posted to the new GitHub site.

Note that this move affects only the open-source project site. This blog, the hosting site, and the forum are unaffected and remain at their former URLs. References to the project site in previous blogs will be updated to refer to the new site.

One big change in this move concerns the repository. On Google Code, we used Subversion to maintain an archive of the code base. On GitHub, the only option is Git, the distributed version control system developed by Linus Torvalds for the Linux kernel project. I liked Subversion a lot, and there are things about Git that I don't care for, but it is certainly adequate for our purposes.

Interestingly, GitHub provides a Subversion interface to its Git repositories, so it is possible to use Subversion client tools (such as the standard svn command-line client and TortoiseSVN, the Windows Explorer plug-in) to check out GitHub repositories and make commits to them. I'm continuing to use Tortoise with GitHub, but the branching and merging model of Git is significantly different from that of Subversion, so eventually I may have to abandon Subversion and become a git head myself. I have tried a few Git GUI clients, and thus far the one I like the best is Atlassian's SourceTree, which is free, but requires you to register with Atlassian first.

For those of you interested in obtaining a copy of the source code, viewing the commit history, or forking the emulator, GitHub offers some improved capabilities over Subversion:

• You can download a zip file of the latest version of the source code by clicking the Download ZIP button on the project's home page at the GitHub link above.
• You can clone the project using a Git command similar to the following:
  git clone https://github.com/pkimpel/retro-b5500 retro-b5500
• You can use a Subversion client to check out a copy of the code as well, e.g.,
  svn checkout https://github.com/pkimpel/retro-b5500/trunk retro-b5500
• As a distributed version control system, Git has some powerful forking and merging features that allow you to create a separate copy of the repository, make commits to it, and then pass the result of those commits to other distributed copies of the repository. We have no plans at present to open up the emulator to other committers, but the forking capabilities of Git may be the solution we have been seeking for the community to store and maintain a library of B5500 source code and other software we are gradually recovering. Whether that library should be a part of this project or a separate one is an open question, and one on which we would appreciate hearing feedback, preferably in the forum.

You will notice that the wiki pages have a considerably different look on GitHub than they did on Google Code. The markup notation used to compose the pages, known as "markdown," is also different from the one used by Google Code. The text and images are all there, though, and the links on the hosting site have been updated to point into the new wiki on GitHub.

The GitHub project has an issue tracker, similar to the one we used in Google Code. You are welcome to post issues (suspected bugs, new feature requests, documentation errors, etc.) to the project, but in order to do so you must have a GitHub account. Anonymous issue submission is not supported. Registration for an account is free, and can be initiated from the GitHub home page. You do not need any special permissions -- any GitHub account can submit issues to this project.

Please do not submit items of general discussion or questions on installation, configuration, or use of the emulator as issues. Those should be posted on the forum instead.

Changes in Release 1.02

This release incorporates the following fixes and enhancements over 1.01:

1. Memory Dump Tape Image

Since version 0.18 the emulator has supported a built-in dump of processor state and memory. You obtain this dump by clicking the red MEMORY CHECK lamp on the console panel. The emulator opens a temporary window and formats the processor and memory state into the window as plain text. You can view this data in the window or copy/save it for later analysis. The dump is just a snapshot and does not affect operation of the system. It can be generated at any time, and as often as you wish, even while the MCP is running.

This is a nice capability, but the resulting memory dump is just raw octal, and it is not easy to analyze. The B5500 MCP had a memory dump mechanism that was invoked using the DP MT SPO command, which would pause normal operations, copy the contents of memory to a magnetic tape and resume normal operations. You could then run the DUMP/ANALYZE program to read the tape and produce a formatted dump with a lot of the MCP information broken out in a much more readable manner.

In order for this dump mechanism to work, of course, the MCP must be up and running. To support a similar dump mechanism independently of the MCP, the emulator now supports an in-built tape dump. If you click the white NOT READY lamp on the console panel, the emulator will open a temporary window and format the contents of memory into this window, much as it does when clicking the MEMORY CHECK lamp. The difference is that the data is output as a tape image in the format used by the DUMP/ANALYZE program. Simply copy/save this tape image to a file on your local system, the same way you would for a tape image that is created when you unload a tape drive, and run the analyzer program against that tape, For example, from the SPO:

    ?RUN DUMP/ANALYZE; FILE MDUMP=MEMORY/DUMP001; END

Note that DUMP/ANALYZE is for the Datacom MCP. If running under the Timesharing MCP, you must use TSDUMP/ANALYZE. Also, it appears that the dump analysis utilities may have been quite new in Mark XIII, because there are complete source file replacements for both of them on the Mark XIII SYSTEM tape under PATCH/DUMPANL and PATCH/TSDUMP. You may wish to compile and use those newer versions instead of the compiled versions on the SYSTEM tape.

2. Capturing Printer and Punch Output

The emulator's implementation of the B5500 line printer works under the idea that to save the printed output, you would save the line printer text area or copy/paste its contents into another program for saving. In early releases this worked quite well, but starting with 1.00, the line printer implemented optional "green-bar" shading of the output. That shading works by grouping every three lines into a <pre> element and applying the shading to every other element. Disabling the shading simply removes the green color style from the elements -- the grouping into <pre> elements occurs regardless.

Alas, when copying or saving the contents of the printer's text area, some browsers (particularly Firefox) insert a blank line between each of the <pre> elements when rendering them as plain text. This makes the saved or copied text appear as if every third line is double-spaced.

To eliminate this problem, the line printer now supports a new way of capturing the contents of its text area, copied from the approach used with the SPO and datacom terminal. If you double-click anywhere in the printed text, the emulator will open a temporary window, render the entire contents of the text area into that window as plain text (without the extra blank lines), and clear the entire contents of the text area. You can then copy/save the contents of that temporary window as you would any other. When you are finished with the temporary window, simply close it. One advantage of this new mechanism is that you can do it at any time, even while the printer is in a ready state and printing.

The old method of clearing the print area by making the printer not ready and triple-clicking the FORM FEED button still works. There was a bug in the implementation of this method that was introduced in release 1.00, however. It has been fixed in this release.

This mechanism of double-clicking in the output text and opening a temporary window to receive the contents of the text area has also been implemented for both stackers on the card punch.

3. SPO Double-Click Problem

Double-clicking the INPUT REQUEST button on the SPO appeared to enable the SPO for input, but no characters were accepted from the keyboard. The problem could be resolved by clicking END OF MESSAGE and starting over.

What was actually happening was that the first click sent the keyboard-request interrupt to the system and the MCP responded by issuing a read to the SPO device. That usually happened fast enough that by the time the second click registered on the button, the read had already been issued and the input text box in the SPO paper area had been enabled and given the focus. The second click simply moved the focus to the INPUT REQUEST button, which of course ignored any keystrokes sent to it. The problem could also be resolved by clicking in the input box to give it back the focus.

The SPO driver now detects when the INPUT REQUEST button is clicked while the SPO is in the middle of a read operation. In such an event it returns the focus to the input box.

Also, the ability of the SPO to wrap output messages longer than 72 characters to a new line was lost in release 1.00. That ability has been reinstated in this release.

4. Miscellaneous Changes

1. The disk driver will now report a not-ready error if a read or write transaction is aborted by the IndexedDB API in the browser. See the next section on disk quota problems in Firefox for more information.
2. The disk driver now reports in the result descriptor for a Read Interrogate operation whether the addressed EU has Model IB (also known as "bulk" or "slow") Storage Units attached to it.
3. The handling of Ctrl-B (break), Ctrl-D (disconnect request), Ctrl-E (WRU), Ctrl-L (clear input buffer), and Ctrl-Q (alternate end-of-message) keystrokes by the datacom terminal was not working. These keystrokes now generate the appropriate status indicators sent to the I/O Units.
4. Sensing of the Beginning-of-Tape (BOT) marker for a magnetic tape by a Write Interrogate operation was being reported in the result descriptor in a way the MCP did not like. This was causing problems in purging blank tapes. Write Interrogate no longer reports BOT status, which seems to satisfy the MCP.
5. The X register in the Processor was not being set properly at the end of some arithmetic operators. The X register is cleared at the end of each operator in Word Mode, so this did not affect operation under the normal user interface, but it did cause problems in the B5500SyllableDebugger interface. The guilty operators have been corrected.
6. The B5500SetCallback.js module, which controls the synchronous interleaving and timing of all emulator activities, has a mechanism that adjusts future timing behavior based on the deviation between the execution delays it requests and those it actually receives. Tuning this mechanism has been an on-going process, and further tuning adjustments were made in this release.
7. The ESU card in the standard cold-start card deck (tools/COLDSTART-XIII.card) has been changed to indicate the system has two disk EUs instead of three. When cold-starting a disk subsystem, you should always change this card to reflect the actual number of EUs configured for the disk subsystem.
8. Preliminary decks for cool-starting and loading an MCP from tape to disk have been created in the tools/ folder as COOLSTART-XIII.card and MCPTAPEDISK-XIII.card, respectively.
9. The B5500ColdLoad.html cold-start script (which is now deprecated) could not open the IndexedDB database for the disk subsystem in Google Chrome. This was caused by a differing interpretation between Firefox and Chrome of the second parameter to the IDB open() method, and has been fixed.
10. Most of the disk subsystem utility scripts in the tools/ folder (B5500DiskDirList.html, etc.) would work only with the default disk subsystem, named B5500DiskUnit. These scripts now support a "db" query string parameter that specifies the name of the disk subsystem. For example, you can now invoke these scripts with a URL of the form: ".../tools/B5500DiskDirList.html?db=MyNewDisk".

Firefox Disk Quota Woes

Beginning with release 1.00, the emulator supports multiple disk subsystems. Using the configuration interface (entered by clicking the B5500 logo on the console panel while the emulator is in a powered-off state), you can easily create and switch among disk subsystems to, for example, move between datacom and timesharing MCPs or operate multiple versions of the MCP.

Several users have found multiple disk subsystems to be a nice thing to have, but its use has recently brought a problem to light when running under Firefox.

I have encountered the problem most often after setting up a second or third disk subsystem and during the process of loading system or symbol files to the new subsystem. Suddenly the tape load stops and the console shows the DKA lamp and one of the IOUn lamps lit. The rest of the system quickly ceases operating as well, except that the TIMR lamp for the interval timer interrupt continues to blink. A halt/load will often bring the system back up, but attempts to continue loading files from tape, or doing anything that uses more disk space, quickly results in the system locking up again.

It took a while to track down the cause of this. It was clear fairly early on that an IndexedDB write transaction never reported completion. Without that completion event, the emulated B5500 I/O never finished, so the Disk File Control Unit (DKA) and the I/O Control Unit (IOUn) remained in a busy state. Additional disk I/Os in the system quickly queued behind the write that never completed, and being denied further disk access, the MCP and its tasks quickly came to a standstill.

After several frustrating attempts to trace the I/O causing this problem, I finally discovered what was happening, which in turn revealed a rather serious bug in the emulator's disk driver. The IndexedDB API in Firefox was reporting a Quota Exceeded exception and aborting the write transaction. Unfortunately, the API reports this condition through the transaction's onabort event handler, not its onerror handler, and even more unfortunately, I had not implemented an onabort handler for either disk reads or writes, so the quota error went undetected by the emulator.

On abort event handlers have been implemented in release 1.02. They simply terminate the I/O operation, reporting the EU as not ready. That does not get around the quota problem, but it does cause a message reporting the not-ready condition to be printed on the SPO, so at least you will now know that there is a problem with the disk subsystem. The system as a whole will still stop operating. No further data can be inserted into any of the disk subsystems.

I have not been able to find any documentation on disk quotas for Firefox, and in fact the MDN web site clearly states there isn't one. The best I have been able to determine after a good deal of searching is that Firefox currently has a fixed 2GB upper limit on the amount of off-line storage that can be used by applications from the same source (i.e., same website). That limit appears to be hard-coded in Firefox and cannot be changed in the browser configuration. The only solution to the quota limit I have found is to delete one of the disk subsystems so that total off-line storage will fall below the quota limit.

Google Chrome also has quota limits for off-line storage used by web sites, but its limits are based on the amount of free disk space available on the workstation.

2GB is a lot of disk space, and should be sufficient to support many large B5500 disk subsystems, but I've also found that with the Uint8Array structure currently used to represent data in a disk sector, Firefox is storing that data in an extremely inefficient fashion. The overhead appears to be on the order of 30 times the size of the raw data being stored. I have a disk subsystem that contains perhaps 50-60 million (B5500) characters. Each six-bit B5500 character is stored as an eight-bit byte. The SQLite database used by Firefox for that IndexedDB instance currently occupies 1.8GB on disk. That means I'm already close to the 2GB limit, and a second disk subsystem does not need to get very large to exceed it.

The real problem here is the bloated storage overhead. My guess is that instead of storing a Uint8Array(240) disk sector image as simply a string of 240 bytes, Firefox is storing it as hash table containing 240 index:value pairs.

I am hopeful that we can find another data representation that will not carry so much storage overhead. This will require some research. If we are successful in reducing the storage overhead, however, that will invalidate all disk subsystems currently out there, and we will need a plan and some tooling to covert the existing subsystems to whatever new format we come up with.

For now, the best advice I can offer to those experiencing this problem is to try to keep your disk subsystems fairly small, or not to create multiple of them. Note that the quota limit is per website, so if you can split your emulator environments across multiple web sites, each will have a separate quota limit. Also note that a website is determined by the host portion of a URL, so for example, http://www.phkimpel.us/B5500/ and http://phkimpel.us/B5500/ are considered to be separate web sites by browsers, even though both URLs go to exactly the same place.

As always, if you have questions or need help with emulator issues, please raise them on the forum, so everyone can participate and share in the knowledge.

Emulator Version 1.01 Released

Nigel and I are pleased to announce that version 1.01 of the retro-B5500 emulator was released on 8 February 2015. All changes have been posted to the Subversion repository for our Google Code project GitHub project. The hosting site has also been updated with this release, and for those of you running your own web server, a zip file of the source can be downloaded from the GitHub repository [updated 2022-05-07].

 This is a minor release containing corrections for a few issues we have discovered in the four months since version 1.00 was released. In addition, we have changed the way the operator Control Panel is displayed and implemented a different home page for the emulator's web application.

New Control Panel

In previous versions of the emulator, you started by loading the B5500Console.html page into your browser window or tab. This page displayed the operator Control Panel from which you power on the emulator, load the MCP, and monitor the status of the running system:

Emulator Operator Control Panel

The problem with this approach was that the Control Panel occupied only a small portion at the top of the page, leaving the remainder of the page unused. In order to bring the Control Panel into view, the entire home page had to come to the top of the screen, which tended to obscure the other windows opened by the emulator, particularly the window for the SPO.

In this release, we have split the Control Panel from the home page. It now displays in its own window, which is sized just large enough to show the entire panel. This smaller window is much easier to position and use without obscuring the other windows on the screen.

That change left the B5500Console.html home page completely empty, so in order to make some use of that space, we have implemented a proper home page for the emulator site:

Emulator Site Home Page

This is the page that will now greet you when you load the emulator's B5500Console.html URL. The two blue buttons below the image on the page allow you to open the Control Panel and optionally power up the emulator.

• Clicking the Start & Power On button will open the Control Panel and power up the emulator, as if you had clicked the panel's POWER ON button. This is the option you will probably use most frequently.
• Clicking the Start - Powered Off button will simply open the Control Panel. This option is useful if you want to start by modifying the emulator configuration, which can only be done in a powered-off state.

Both buttons will be disabled when either of them is clicked, and will remain disabled until the Control Panel window is closed. Attempting to close the Control Panel while the emulator is powered on will result in the display of a warning dialog. If you choose to confirm the warning and close the panel's window, the system will be halted and the emulator implicitly powered off.

Note that in some browsers, particularly Firefox, closing the window for the new home page, minimizing it, or switching to a different tab in the same window may cause the emulator to run slowly. The reason is that Firefox tends to impose a minimum timeout delay of one second on hidden windows, whereas the emulator typically generates timeout delays in the low tens of milliseconds. The difference between the requested and imposed delays results in very slow performance.

Other Changes

The following enhancements and corrections have been implemented in this release:

1. The scroll-back buffer for the SPO and Datacom devices has been increased from 1500 to 5000 lines.
2. Switching on and off the green-bar styling for the Line Printer device did not work in Chrome due to the way the styling was being applied. This has been fixed.
3. A number of improvements have been made to the emulator's setCallback() mechanism, which is used to regulate performance and time-slice among the processors and I/O devices. These changes were imported from the Datatron 205 project, discussed below.
4. Some optimizations to the bit-field isolate/insert routines have been made. These were also imported from the Datatron 205 project.
5. A letter "B" has been added to the tape reel image used by the Magnetic Tape device. This makes the image less rotationally symmetric, and less likely to beat with the refresh rate of the screen.
6. The wiki pages have been updated for the Control Panel and home page changes in this release.

Related News

With the B5500 emulator in a stable state, I have started a new project to emulate the ElectroData/Burroughs Datatron 205 computer system. This was a vacuum-tube, drum-memory computer designed in the early 1950s. It competed against the IBM 650 and Bendix G-15. It was fairly popular in its day -- several hundred were built -- and it continued to be used into the mid/late 1960s.

The primary goal of this project is to build a web browser-based emulator that will run 205 software at its actual speed -- which is very slow by even B5500 standards. A secondary goal is to reconstruct the Algol-58 compiler written for the system by Donald Knuth in 1960 and to get that working. An assembly-language listing of the compiler survives in the papers Knuth has donated to the Computer History Museum.

Tom Sawyer of Minnesota (US) has done considerable research on the 205 and on ElectroData, which was purchased by Burroughs in 1956 and became the foundation of its commercial computer business. He has an interesting web site and a blog devoted to the 205 and its history. I contacted Tom when starting this project in the Fall of 2014; he has been enormously helpful in my work on an emulator and has shared with me a number of technical resources that are difficult to find. Tom has also written an emulator for the 205, and reconstructed a different version of the Algol-58 compiler, which he discusses in a recent blog post.

At present, my emulator is functional, but supports only paper-tape and Flexowriter (teletype) I/O. I am currently working on the Cardatron, a buffering and reformatting device that interfaced the 205 to IBM punched-card tabulating equipment. A couple of weeks ago, I posted an article on Tom's blog describing the current version of the emulator and how to use it.

As with the B5500 emulator, my 205 emulator is an open-source project hosted on Google Code. If you are interested, the emulator can be run directly from my hosting site.