DMPP Cymus – CIA emulation library

Now that I have a somewhat working M68000 emulation, I can finally focus on the rest of the Amiga system. Currently most of the system is stubbed out with a couple of dummy functions so I can see when something interesting happens.

I have reached now reached a point, where I can’t fake out the CIA functionality anymore, because the Amiga OS tries to use the timers and the TOD (Time-of-Day clock). Apropos TOD – in german the word “TOD” means “death”, so using that abbreviation in the source code gives me the shivers. In UAE’s CIA source code, I even found the words “TOD Hack” – which sounds to me like “chop to death” – (brrr ! horrible !).

However, I’ll stick with that word since “tz” (for “Tageszeit”) does not seem to be any better. I grabbed the MOS CIA 6526 datasheet linked from the Wikipedia article and the information from the Amiga HRM. As an additional reference, I am also looking into the CIA implementation of the MAME project (as I have done for the M68000 implementation). MAME  is a pretty large project, but from what I have seen so far, I really like the overall code quality and the CIA implementation is no exception.

From the combined information, I created a first class design:

 

I find creating UML diagrams during software development very useful. I can verify and review the software design on different levels. When designing, I first create a rough draft and then try to verify it with a prototype (usually written in Ruby or Python) to get quick feedback, so I can feel how well it works. I then iterate a couple of times until the design seems to meet the requirements.

A while ago, I had created the project “Dream Machine Preservation Project”, which I currently use as an incubator to experiment with a couple of ideas, which will hopefully one day lead to a full Amiga emulation.

The CIA library is the first thing I checked into the public git repository on Sourceforge. The implementation is still incomplete, because features are added as they are needed in the Amiga emulator. It was originally written in Scala, but when I decided to make a separate library out of it, I changed the implementation language to Java. This was mainly because I did not want to force users of the CIA library (assuming that there could be someone else besides me) to have to add the Scala library or having to know Scala in order to compile it.

Birth of a New Amiga Emulator

Do not take the title serious. As long as there is no release, this is strictly Vaporware(TM). Or something like that…. However, with the release of Mahatma68k, the likeliness of such a release has become somewhat more probable.

When I started the project, which has the current working title “I Heart Amiga”, it actually started as an attempt to make WinUAE’s features available in Unix versions. I have to admit it, I do not like C when it comes to writing a virtual machine. The abstraction level is simply too low and I like to look at a virtual machine as a system of components (a job for an object-oriented language). There is no sense for me to write a lot of extra code that has nothing to do with the problem. You can get it done in pure C, but then you could also use Notepad to create whole web sites.

I then had this crazy idea that the JVM might actually be fast enough nowadays to at least reach the performance of an Amiga 500 (something that has yet to be proven) and so I just kind of started hacking together some code in Scala. The nice thing about doing side projects is that you can just choose what you think suits the problem and I wanted a static JVM language (for performance), which allows me to avoid writing tons of boilerplate and “syntactical noise”. Scala seemed to be much more difficult than Java at first, but when I saw that the “difficult parts” were actually in the library and not part of the language, I realized that it is actually simpler. I like functional languages, but after having done the Z-machine in Erlang, using a object-oriented language to implement a VM feels more natural to me.

The emulator is still in its early stages and I do not know how fast I will progress. However, I wanted to share some details of my current design, that might be useful to anyone who is trying to implement an Amiga.

The Amiga is the most fascinating computer system ever created. That’s why I am trying to implement it in software. You might think differently, but ask yourself: can you think of any other system that was years (released in 1985) ahead of its competition and yet those competitors slowly incorporated all of its features and eventually surpassed it in the nineties. I have a feeling that if the Amiga would have been an Apple brand, it would have probably been much more successful, but then, the competition would also have tried harder to catch up quicker. The way the Amiga was marketed, it did not really pose a threat to the established competition. That’s sad, but there are lessons to learn from that. We see everyday that technological excellence is not everything, and great marketing and business development can make up for not having the best product.

I digressed. I actually wanted to show the current system design of my emulator. Here is an UML diagram of it:

Current System Design

I like to keep it simple and my approach for this project is to only add new components when it seems to be necessary. On ZMPP I admittedly got a little overboard and created more classes than I actually wanted to, which has to do with the TDD approach I took and especially my inexperience with TDD when I started ZMPP. This time, I did not use TDD (I did use a couple of unit tests to enhance my confidence on some tricky parts, though), but entirely focussed on understanding the problem and keeping it simple.

As you can see, this is currently a CPU and an address space which maps to the various system components.The Amiga uses memory-mapped I/O, and one of the most important concepts I had to learn as a non-Hardware person was “incomplete address decoding”. As I have documented in the diagram, some system components respond outside of the documented addresses (e.g. Chip Memory if you have less than 1 MB and the Custom Chips). You could learn this by looking at UAE’s memory map, but I actually found out by looking at a disassembled Amiga Exec. Finding the disassembled (and commented !) Exec source code was almost like having found the Rosetta Stone for me and I am grateful to Markus Wandel (I noticed that a lot of Amiga fans seem to be german !) for doing this hard work. So far, this has been one of my most important tools (besides the Hardware Reference Manual). I just kind of progressed by stepping through the ROM listing. I currently reached a point where I am outside of the commented part (probably dos.library and graphics.library). Getting this far gives me some confidence that my CPU emulation seems to work pretty well and that my memory mappings are working.

Some time ago, I purchased the Cloanto DVD set: “Amiga Forever”. Looking at the videos, I am fascinated by the passion, enthusiasm and the incredible skill that the original Amiga team had and I am inspired by that. Reading through the Amiga OS assembly code is an exciting experience as well. This is the beginning of a wonderful journey into computing history, and into the internals of a computer system that is unlike any other that has come before or after it.

Emulating the Motorola 68000 on the JVM

I am currently trying to write an emulator for the Amiga, which is powered by a Motorola 68000 CPU. I initially started in C++, but switched to the JVM, for various reasons:

• development on the JVM is much faster than in C++ and the performance penalty might be acceptable here compared to other non-native platforms
• the license of the CPU emulation library I used was not flexible enough for me

I actually expected that I would find a suitable M68K emulation library for Java, but there seemed to be nothing that really fit my requirements. Because there is at least one person with a strong need for such a library (me), I decided to create one myself with the main goals:

• very liberal license (BSD)
• fast and simple to use
• easy to maintain

I have created that project on Sourceforge, and did my first checkin today.

There are no released files and the source is available in a git repository. It is basically a generator written in Ruby with a database containing information about the instructions (decoding, timing, output, execution), which is mostly defined as Ruby hashes. This approach is inspired by the emulation libraries as they are found in UAE or Musashi. The advantage is that the step for decoding the instruction is replaced by a simple lookup in an array, so that the CPU only has to evaluate the parameters and execute the instruction.

Execution times (which are necessary for my emulation) are also mostly pre-calulated: Because addressing modes are available at generation time, the generator can simply calculate the number from the timing database and put it in the generated source code. There are only a couple of exceptions, which are related to conditional execution (e.g. Bcc, Dbcc, privileged instructions…)

Another nice feature is that the decoded instruction objects “know” how to print themselves, which can be used to disassemble sections of code. This feature is optional, because it makes the code much larger.

With a generator approach I also can reduce one of my main pain points with the Java language: boilerplate. Writing tons of structurally similar code would tire me out and I’d be exhausted (and bored to death) before I actually write my Amiga emulator.

Currently the emulation can emulate 58 of the 76 instructions and I am adding a couple each day. There is some exception handling and support for traps and supervisor mode, and interrupts will be added shortly (because I need them).

I love the 68000 CPU, I really do. I am not a hardware expert, but I think, when it was released (1979), this processor must have been quite revolutionary. I just love how easy it was to program it (assembly code that did not make you go “cuckoo”), with a flat 24 bit address space and working in 32 bit internally. It’s nice that this project gives me an opportunity to study its design and history.

It is my hope that other people find this emulator useful in some way, I have tried to provide some help to make it easy to integrate in projects (maven build, example program, simple public interface).

Challenges in updating ZMPP’s screen model

So now ZMPP has a dramatically simplified setup and execution model, those changes were actually much quicker to do than I thought and each took about a day. On the other hand I did not really make changes to the core, but simply wrapped it in a different way so it can be easier reused in different contexts.

The real challenge is the rework of the screen model – in fact there are three:

1. “classic” screen model (V1-3)
2. “modern” screen model (V4-8 except 6)
3. V6 screen model

1. The “classic” screen model

What I call the “classic” screen model is the screen model used by the majority of Infocom text adventures, such as the Zork trilogy, “Leather Goddesses of Phobos” etc. It is characterized by a main text window with a standardized status bar at the top. This model is very simple to implement with standard GUI components and even transfers well to an HTML interface. Unfortunately, aside from the Infocom adventures, this screen model is not frequently used by modern Interactive Fiction.

2. The “modern” screen model

This is the screen model which is by far the most popular of the current Z-machine screen models. A top window rendered in a fixed pitch font, without scrolling and word wrapping and a bottom window for main text display and input which can have variable pitch font, scrolling and word wrapping. The top window’s number of rows can be specified by the game and the bottom window will shrink accordingly so that the sum of the heights of both window add up to the total height of the window.

The popularity of this particular screen model can be explained by the availability of one of the best authoring systems for Interactive Fiction: Inform. Inform’s output formats are usually V5 or V8 (or Glulx, which is a different virtual machine for Interactive Fiction with larger available memory).

This screen model does not translate itself easily to standard user interface components or web interfaces, which is mainly due to the fact that a few games use the behavior of the screen model that the content of the windows are not erased after the top window was resized. So, in “Curses” by Graham Nelson, the top window is used to render a book citation in box (because of the fixed pitch font) and shrinks so that the former content of the top window now belongs to the bottom window (and will be cleared away). This behavior is difficult to implement with standard UI components or in a web interface, because rendering into other component’s regions is usually not desired. 

3. V6 screen model

This is the most complicated screen model and neither widely used by Interactive Fiction authors nor commonly implemented by many Z-machine interpreters. It uses 8 windows which can contain graphics or text and can use an interrupt mechanism to implement word wrapping. ZMPP implements it, but currently has a number of issues due to lack of testing. I will probably describe it in more detail when I get to work on that.

For the near future, reimplementing the classic and modern screen models is the main focus. The requirements I have distilled from user requests and my own ideas are:

• use JTextPane for the main text window if possible
• support selection, cut, copy, paste
• support resizing

Since I do not have enough in-depth experience using the styled text components of Swing in the way described (that’s why I ended up with custom rendering for the current versions – it was quicker to implement), I will create a number of prototypes in the next days to see what is possible.

 

Back to the Z-machine

In the last two years, I have only made smaller fixes and changes to ZMPP. The reasons were various: learning to be a father, tinkering with TinyUML and playing with Z-machine implementations in Ruby and Erlang. While working on the other pet projects, I noticed that ZMPP is still the most successful of them, simply because it is the one with an existing user base besides myself.

ZMPP is ready for a major overhaul. There are several pain points I want to tackle:

• the public interface between the core and the user interface is too complicated and makes implementing different front ends a big pain.
• all currently released ZMPP versions follow a game-loop style scheme, which means that the Z-machine runs in its own thread, until it stops, timed input is realized by threading as well. Besides making the execution and input logic too complicated, there are reportedly deadlocks which are almost definitely a result of this design decision. However so far, ZMPP is probably the Java Z-machine where timed input works best
• top priority and the largest challenge is the rework of the screen model. On Java, ZMPP has the most complete screen model implementation, but it is still far from what I want it to be. I will elaborate on that later.

So, with that said, it means that I am waving good-bye to TinyUML, Smeagol and Schmalz in order to focus on ZMPP entirely. As a single, it is easy to find time to do a lot of different projects, but once you get married and have children, programming in the free time while still important, should not exactly be the top priority.