Migrating ZMPP to git

I did it. Finally I moved from Subversion to git. Until today, ZMPP was the

only larger project that was still in a Subversion repository.

To be fair, in the two years I used it for my projects, I have never really experienced

larger problems. It felt exactly like it was intended to be: A better, more modern CVS,

and it has great tool support as well, something where git could still improve.

Still, after using git in parallel to svn and cvs for about a year now, nothing really

beats the comfort working with a DVCS.

I did it. Finally I moved from Subversion to git. Until today, ZMPP was the only larger project maintained by me that was still in a Subversion repository. To be fair, in the two years I used it for my projects, I have never really experienced larger problems. It felt exactly like it was intended to be: A better, more modern CVS, and it has great tool support as well, something where git could still improve.

Still, after using git in parallel to svn and cvs for about a year now, nothing really beats the joy working with a DVCS, even when used solely through the command line. Personally, the way git handles branching and merging is perfect for me: it really invites one to make wild experiments. Another nice thing is that due to its decentralized nature, it is much easier for me to add ZMPP to my Hudson setup, work on my local copy and just push the changes to Sourceforge when I think they are good enough.

Porting ZMPP to Android in one day

Being sick at home has its advantages and drawbacks. On one hand you do not really get to see to many people (which is for their own good) and suddenly you feel pretty lonely. One the other hand you suddenly get time for stuff that you always wanted to do. One of those things for me was to try writing an Android frontend for ZMPP. The modest goal was being able to run Trinity, Curses and Minizork on Android. I reserved myself a total of one day for that: half a day for trying out how to implement the screen model with the Android user interface API and half a day for implementing the rudimentary user interface (with some generous amount of sleep in between).

I had already changed the ZMPP core to compile on Android last year (thanks to Sandy McArthur who looked into the code and pointed out the parts that were incompatible) and pushed a lot of the screen model handling into the core, so what was really to do was the plumbing into the Android interface. Luckily, Android’s user interface library is pretty flexible, so porting from Swing to Android Views seems relatively simple. Well guess what, I luckily reached my goal within the time budget:

Trinity on ZMPP/Android

This is Trinity, one of Infocom’s greatest classics (and out of my “Masterpieces of Infocom” collection that I purchased for a fortune at the beginning of the Z-machine Preservation Project). While these classics are more than 20 years old, in my opinion good Interactive Fiction is timeless like good literature.

I actually could run Curses and a good number of Version 3 games (Zork I-III, Minizork…) as well, but I guess that there are still quite a bit of issues, given the short time I had for implementing it. One huge issue I instantly noticed here is the performance: For Curses, response times are pretty horrible, for Infocom games and Minizork it is pretty ok. I can only imagine that a modern piece of Interactive Fiction written with Inform 7 would take minutes between each turn. Infocom Z-code is tighter than Inform code because the computers in the 80’s used to be much more limited than our cell phones today and they were written with a compiler which seemingly produced pretty good Z-code. ZMPP is a “VM-in-a-VM” approach, still I am a little surprised of the slowness on Android’s Dalvik VM compared to Java SE.

Well, you don’t know what you don’t measure. I guess that calls for a profiling session – some time…

A new ZMPP design

The parts for a new release of ZMPP are slowly coming together. Pichuneke, a spanish ZMPP user, contributed a spanish translation for the user interface, which is now the second user initiated translation (the first being French, done by Eric Forgeot).

This is a point that makes an Open Source project fun: the involvement of its users. At some point I have realized that ZMPP has turned from being my personal pet project to being software that I believe now belongs to its user community which has exceeded my initial goal (being a reference implementation of the Z-machine in Java) by far.

I also realized that “Z-machine Preservation Project” to me means not the Java implementation itself, but it is more my personal idea how a Z-machine could be implemented on a variety of platforms. Having done ports in Ruby and Erlang has deepened my understanding of the general problem of implementing the Z-machine in an implementation language which is on a higher abstraction level than C/C++. The Subversion trunk now contains an extract of the lessons learned mainly during the implementation of the Erlang port (and I am still amazed how well some things can be done in Erlang). The illustration below shows the updated design as it currently exists in both the Erlang and Java versions:

The ZMPP reference design

As can be seen, an ExecutionControl object controls decoding and execution of instructions, which in turn work on the Machine object’s and screen model’s state. The Z-machine core now runs single threaded only, which simplified the control logic quite a bit, especially the implementation of interrupts, which are now controlled in the user interface. Also, there are now much less core objects the user interface needs to deal with, which in combination with the pause/resume execution facilitates the integration in different contexts.

The most apparent difference in the new ZMPP is the new screen model implementation, which finally uses a JTextPane as it should have been from the beginning. It should be noted that Zinc has taken that route years ago. The way it is implemented is totally different from what I wanted it to be, which is why I chose custom rendering in the first place.

New ZMPP screen model with "Bronze"

This time around, I took more time to analyze the Z-machine screen model more thoroughly, which led to the decision of implementing the screen model view in a way I think it should be implemented:

This was now done by having two Swing components to represent the upper (a fixed text grid) and the lower (a flexible text area) window. As opposed to Zinc, but like in Zoom, I wanted to have a scroll bar, which spans both the upper and lower window, but only controls the lower window.

This is not easily done using the standard AWT layout managers, so ZMPP implements its own which manages the components in the following way: The main view is a JLayeredPane and the layout manager puts the bottom window, which always spans the whole layout area, below the top window. The upper window is used as a flexible and transparent overlay over the lower window so its output can be rendered overlappingly with the lower window, and its size can be controlled by split commands.

Overall, I happily trashed a good portion of the original ZMPP code, which I credit to a better understanding of the Z-machine. Switching to JMock 2.5 (which is a huge improvement over JMock 1.x) also helped greatly in simplifying the test classes.

Why was this all necessary ? As a short-term goal, I want to deliver on some promises I made: A screen model supporting selection, cut and paste, more reader-friendly margins and resizable windows for example.

Medium-term goal is V6 support, which is still incomplete and currently deactivated for Version 1.5. It was one of the main drivers for the new design, which will hopefully make it easier to overcome some of the problems how the old ZMPP handles V6 games. V6 support is one of the things which I really feel ZMPP should do well, to deliver on the “Preservation” in ZMPP’s name.

Ultimately, the changes made were done in order to provide a baseline for future improvements. I always want to try my best to support this goal. Change is the only constant in our world and as Extreme Programmers (I do not consider myself one) say: “Embrace Change”.

To ZMPP’s users: Thank you for all your support, your suggestions, improvements and criticism. Without you, the changes made and which are going to be made in the future would not have been possible.

Challenges in updating ZMPP’s screen model

So now ZMPP has a dramatically simplified setup and execution model, those changes were actually much quicker to do than I thought and each took about a day. On the other hand I did not really make changes to the core, but simply wrapped it in a different way so it can be easier reused in different contexts.

The real challenge is the rework of the screen model – in fact there are three:

1. “classic” screen model (V1-3)
2. “modern” screen model (V4-8 except 6)
3. V6 screen model

1. The “classic” screen model

What I call the “classic” screen model is the screen model used by the majority of Infocom text adventures, such as the Zork trilogy, “Leather Goddesses of Phobos” etc. It is characterized by a main text window with a standardized status bar at the top. This model is very simple to implement with standard GUI components and even transfers well to an HTML interface. Unfortunately, aside from the Infocom adventures, this screen model is not frequently used by modern Interactive Fiction.

2. The “modern” screen model

This is the screen model which is by far the most popular of the current Z-machine screen models. A top window rendered in a fixed pitch font, without scrolling and word wrapping and a bottom window for main text display and input which can have variable pitch font, scrolling and word wrapping. The top window’s number of rows can be specified by the game and the bottom window will shrink accordingly so that the sum of the heights of both window add up to the total height of the window.

The popularity of this particular screen model can be explained by the availability of one of the best authoring systems for Interactive Fiction: Inform. Inform’s output formats are usually V5 or V8 (or Glulx, which is a different virtual machine for Interactive Fiction with larger available memory).

This screen model does not translate itself easily to standard user interface components or web interfaces, which is mainly due to the fact that a few games use the behavior of the screen model that the content of the windows are not erased after the top window was resized. So, in “Curses” by Graham Nelson, the top window is used to render a book citation in box (because of the fixed pitch font) and shrinks so that the former content of the top window now belongs to the bottom window (and will be cleared away). This behavior is difficult to implement with standard UI components or in a web interface, because rendering into other component’s regions is usually not desired. 

3. V6 screen model

This is the most complicated screen model and neither widely used by Interactive Fiction authors nor commonly implemented by many Z-machine interpreters. It uses 8 windows which can contain graphics or text and can use an interrupt mechanism to implement word wrapping. ZMPP implements it, but currently has a number of issues due to lack of testing. I will probably describe it in more detail when I get to work on that.

For the near future, reimplementing the classic and modern screen models is the main focus. The requirements I have distilled from user requests and my own ideas are:

• use JTextPane for the main text window if possible
• support selection, cut, copy, paste
• support resizing

Since I do not have enough in-depth experience using the styled text components of Swing in the way described (that’s why I ended up with custom rendering for the current versions – it was quicker to implement), I will create a number of prototypes in the next days to see what is possible.

 

Back to the Z-machine

In the last two years, I have only made smaller fixes and changes to ZMPP. The reasons were various: learning to be a father, tinkering with TinyUML and playing with Z-machine implementations in Ruby and Erlang. While working on the other pet projects, I noticed that ZMPP is still the most successful of them, simply because it is the one with an existing user base besides myself.

ZMPP is ready for a major overhaul. There are several pain points I want to tackle:

• the public interface between the core and the user interface is too complicated and makes implementing different front ends a big pain.
• all currently released ZMPP versions follow a game-loop style scheme, which means that the Z-machine runs in its own thread, until it stops, timed input is realized by threading as well. Besides making the execution and input logic too complicated, there are reportedly deadlocks which are almost definitely a result of this design decision. However so far, ZMPP is probably the Java Z-machine where timed input works best
• top priority and the largest challenge is the rework of the screen model. On Java, ZMPP has the most complete screen model implementation, but it is still far from what I want it to be. I will elaborate on that later.

So, with that said, it means that I am waving good-bye to TinyUML, Smeagol and Schmalz in order to focus on ZMPP entirely. As a single, it is easy to find time to do a lot of different projects, but once you get married and have children, programming in the free time while still important, should not exactly be the top priority.