Now that I have a somewhat working M68000 emulation, I can finally focus on the rest of the Amiga system. Currently most of the system is stubbed out with a couple of dummy functions so I can see when something interesting happens.

I have reached now reached a point, where I can’t fake out the CIA functionality anymore, because the Amiga OS tries to use the timers and the TOD (Time-of-Day clock). Apropos TOD – in german the word “TOD” means “death”, so using that abbreviation in the source code gives me the shivers. In UAE’s CIA source code, I even found the words “TOD Hack” – which sounds to me like “chop to death” – (brrr ! horrible !).

However, I’ll stick with that word since “tz” (for “Tageszeit”) does not seem to be any better. I grabbed the MOS CIA 6526 datasheet linked from the Wikipedia article and the information from the Amiga HRM. As an additional reference, I am also looking into the CIA implementation of the MAME project (as I have done for the M68000 implementation). MAME  is a pretty large project, but from what I have seen so far, I really like the overall code quality and the CIA implementation is no exception.

From the combined information, I created a first class design:

I find creating UML diagrams during software development very useful. I can verify and review the software design on different levels. When designing, I first create a rough draft and then try to verify it with a prototype (usually written in Ruby or Python) to get quick feedback, so I can feel how well it works. I then iterate a couple of times until the design seems to meet the requirements.

A while ago, I had created the project “Dream Machine Preservation Project”, which I currently use as an incubator to experiment with a couple of ideas, which will hopefully one day lead to a full Amiga emulation.

The CIA library is the first thing I checked into the public git repository on Sourceforge. The implementation is still incomplete, because features are added as they are needed in the Amiga emulator. It was originally written in Scala, but when I decided to make a separate library out of it, I changed the implementation language to Java. This was mainly because I did not want to force users of the CIA library (assuming that there could be someone else besides me) to have to add the Scala library or having to know Scala in order to compile it.

I had this actually running for some time now: It is a graphical debugger which helps me understand how the 68000 instructions in the Kickstart ROM affect my Amiga emulator’s state. Here is a screen shot:

It is currently mainly a view to the Mahatma68k CPU, the two CIA-8520’s and the custom chip registers. It can execute a defined number of steps and any changes to the chip registers or CPU state will show there. I am currently implementing a function which displays a tooltip for each custom chip register to show a little more information than just its numerical value. The CIA’s and the CPU have only a few registers, so it is still feasible to display the role of each flag in the main view. The sheer amount of custom chip registers however, made it even hard to get them all in one frame.
By the way, compliments to the developers of Scala Swing – these are very nice wrappers ! I started with conventional Java Swing components and then realized that Scala Swing makes programming user interfaces much easier. As a long time user,  Swing does really seem to be all that difficult to me, still I like it when the code looks as tidy as it does when using Scala Swing.

What else is new today ? I have made my second file release to Mahatma68k today. As its main addition, it now comes with the Javadocs the distribution archive and made sure only to have the methods and classes in there that matter to the user of the library.

I decided to build a package of the current code in the Mahatma68k repository and upload it with the little example program that comes with it. It can now be found on its project website. I am aware that not everyone likes or knows maven and so I provided an ant build file for the binary distribution. I like Maven a lot, and have therefore standardized all my builds on it, but it has a steeper learning curve.

I could have provided projects for Netbeans, Eclipse and IDEA, but I guess that this task is simple enough. For Mahatma68k, I wrote everything in Emacs and Vim by the way.

Oh yes, and finally, I have the most important reference manuals together that I will need for my emulation project:

Amiga Reference Manuals

Of these, the “Amiga Hardware Reference Manual” is the most important, of course. I got the Exec Manual and “Libraries and Devices” as well (DOS and Intuition are coming, too), because I might be able to clarify some details in case I get stuck.

Do not take the title serious. As long as there is no release, this is strictly Vaporware(TM). Or something like that…. However, with the release of Mahatma68k, the likeliness of such a release has become somewhat more probable.

When I started the project, which has the current working title “I Heart Amiga”, it actually started as an attempt to make WinUAE’s features available in Unix versions. I have to admit it, I do not like C when it comes to writing a virtual machine. The abstraction level is simply too low and I like to look at a virtual machine as a system of components (a job for an object-oriented language). There is no sense for me to write a lot of extra code that has nothing to do with the problem. You can get it done in pure C, but then you could also use Notepad to create whole web sites.

I then had this crazy idea that the JVM might actually be fast enough nowadays to at least reach the performance of an Amiga 500 (something that has yet to be proven) and so I just kind of started hacking together some code in Scala. The nice thing about doing side projects is that you can just choose what you think suits the problem and I wanted a static JVM language (for performance), which allows me to avoid writing tons of boilerplate and “syntactical noise”. Scala seemed to be much more difficult than Java at first, but when I saw that the “difficult parts” were actually in the library and not part of the language, I realized that it is actually simpler. I like functional languages, but after having done the Z-machine in Erlang, using a object-oriented language to implement a VM feels more natural to me.

The emulator is still in its early stages and I do not know how fast I will progress. However, I wanted to share some details of my current design, that might be useful to anyone who is trying to implement an Amiga.

The Amiga is the most fascinating computer system ever created. That’s why I am trying to implement it in software. You might think differently, but ask yourself: can you think of any other system that was years (released in 1985) ahead of its competition and yet those competitors slowly incorporated all of its features and eventually surpassed it in the nineties. I have a feeling that if the Amiga would have been an Apple brand, it would have probably been much more successful, but then, the competition would also have tried harder to catch up quicker. The way the Amiga was marketed, it did not really pose a threat to the established competition. That’s sad, but there are lessons to learn from that. We see everyday that technological excellence is not everything, and great marketing and business development can make up for not having the best product.

I digressed. I actually wanted to show the current system design of my emulator. Here is an UML diagram of it:

Current System Design

I like to keep it simple and my approach for this project is to only add new components when it seems to be necessary. On ZMPP I admittedly got a little overboard and created more classes than I actually wanted to, which has to do with the TDD approach I took and especially my inexperience with TDD when I started ZMPP. This time, I did not use TDD (I did use a couple of unit tests to enhance my confidence on some tricky parts, though), but entirely focussed on understanding the problem and keeping it simple.

As you can see, this is currently a CPU and an address space which maps to the various system components.The Amiga uses memory-mapped I/O, and one of the most important concepts I had to learn as a non-Hardware person was “incomplete address decoding”. As I have documented in the diagram, some system components respond outside of the documented addresses (e.g. Chip Memory if you have less than 1 MB and the Custom Chips). You could learn this by looking at UAE’s memory map, but I actually found out by looking at a disassembled Amiga Exec. Finding the disassembled (and commented !) Exec source code was almost like having found the Rosetta Stone for me and I am grateful to Markus Wandel (I noticed that a lot of Amiga fans seem to be german !) for doing this hard work. So far, this has been one of my most important tools (besides the Hardware Reference Manual). I just kind of progressed by stepping through the ROM listing. I currently reached a point where I am outside of the commented part (probably dos.library and graphics.library). Getting this far gives me some confidence that my CPU emulation seems to work pretty well and that my memory mappings are working.

Some time ago, I purchased the Cloanto DVD set: “Amiga Forever”. Looking at the videos, I am fascinated by the passion, enthusiasm and the incredible skill that the original Amiga team had and I am inspired by that. Reading through the Amiga OS assembly code is an exciting experience as well. This is the beginning of a wonderful journey into computing history, and into the internals of a computer system that is unlike any other that has come before or after it.

I am currently trying to write an emulator for the Amiga, which is powered by a Motorola 68000 CPU. I initially started in C++, but switched to the JVM, for various reasons:

• development on the JVM is much faster than in C++ and the performance penalty might be acceptable here compared to other non-native platforms
• the license of the CPU emulation library I used was not flexible enough for me

I actually expected that I would find a suitable M68K emulation library for Java, but there seemed to be nothing that really fit my requirements. Because there is at least one person with a strong need for such a library (me), I decided to create one myself with the main goals:

• very liberal license (BSD)
• fast and simple to use
• easy to maintain

I have created that project on Sourceforge, and did my first checkin today.

There are no released files and the source is available in a git repository. It is basically a generator written in Ruby with a database containing information about the instructions (decoding, timing, output, execution), which is mostly defined as Ruby hashes. This approach is inspired by the emulation libraries as they are found in UAE or Musashi. The advantage is that the step for decoding the instruction is replaced by a simple lookup in an array, so that the CPU only has to evaluate the parameters and execute the instruction.

Execution times (which are necessary for my emulation) are also mostly pre-calulated: Because addressing modes are available at generation time, the generator can simply calculate the number from the timing database and put it in the generated source code. There are only a couple of exceptions, which are related to conditional execution (e.g. Bcc, Dbcc, privileged instructions…)

Another nice feature is that the decoded instruction objects “know” how to print themselves, which can be used to disassemble sections of code. This feature is optional, because it makes the code much larger.

With a generator approach I also can reduce one of my main pain points with the Java language: boilerplate. Writing tons of structurally similar code would tire me out and I’d be exhausted (and bored to death) before I actually write my Amiga emulator.

Currently the emulation can emulate 58 of the 76 instructions and I am adding a couple each day. There is some exception handling and support for traps and supervisor mode, and interrupts will be added shortly (because I need them).

I love the 68000 CPU, I really do. I am not a hardware expert, but I think, when it was released (1979), this processor must have been quite revolutionary. I just love how easy it was to program it (assembly code that did not make you go “cuckoo”), with a flat 24 bit address space and working in 32 bit internally. It’s nice that this project gives me an opportunity to study its design and history.

It is my hope that other people find this emulator useful in some way, I have tried to provide some help to make it easy to integrate in projects (maven build, example program, simple public interface).

Contrary to my regular habits, I bought the latest update to Mac OS X, “Snow Leopard” on the first day of release. Our local Apple store opened early (I think around 8-ish) on the 28th and I picked up my copy as soon as I saw the doors opened.

The truth is, since the days of the Amiga, I have not been as excited for an operating system release. My confidence in the quality that comes out Apple is also almost limitless, so I just popped the DVD into my computer and waited about an hour for the install to finish, hoping that there would be 7 GB magically reclaimed from my hard disk. I was wrong – it turned out to be a whopping 14 GB, which I thought was pretty impressive.

The next thing was for me to look for apparent problems, but apart from my “A Lot of Water” screen saver stopping to work (some changes to OpenGL in Snow Leopard might have caused this *sniff*), I did not notice any problems, just the “it just works” that I am used to from Apple. Well – until I started to do some development in Java. The Snow Leopard installation seems to remove  an installation of Java SE 5 and instead link the Java SE 6 version to it.

So what’s the problem, Java SE 6 is compatible to Java SE 5, some would say. Yes, almost. Unfortunately there are some incompatible changes in JDBC (in my case, the Connection interface) that requires me to work on a Java SE 5.

Luckily someone had the same problem and has published a solution for that problem: The bottom line is to download a Leopard version of Java 5 and redirect the symbolic links to that downloaded version.

After following the instructions, I could continue working on my Java project as before.

Update: I had another issue with maven even though I had already changed the version in Java Preferences. This could be fixed by adding a line

JAVA_VERSION=1.5

to the  ~/.mavenrc file.

I have started to use Scala occasionally since last year, it is less scary than it initially looked to me and thanks to good tool support (specs comes to my mind), it is even a joy to program in Scala.

What I initially found irritating though was the fact that the Scala library (despite running on a platform with one of largest standard libraries in existence) is huge – almost 3 megabytes in size – reminds me of my first “Hello world” written in Eiffel 15 years ago (that was about 4 MB). For applications to be run on the server one could simply say: who cares ? Still, I do write quite a bit of applications that are distributed over the Internet, be it applets, Webstart or Android applications and I strongly believe Scala is good for the server as well as the client side. For downloadable client applications, actually size does matter (at least to me).

Fortunately, there are tools that can strip out unused code from class files and further optimize the size of the generated jar files. ProGuard, which is Open Source, is such a tool and it even works for Android. Since I plan to replace Java with Scala as my main static JVM language (haven’t settled on the dynamic one, but so far, it looks good for Clojure), I fiddled a bit with a maven POM, that I can use as a template for my projects and thought I might share the relevant pieces that took me a bit of experimentation to get working. Maven 2 has now become my standard Java build tool, simply because I can generalize my builds a lot when working with JVM languages and they integrate well into my Hudson/git/jira/Eclipse environment.

I like my Java applications to be started with a simple java -jar <jarfile>, so I configured the maven-assembly-plugin to set the main class into the manifest file and linked it to the package phase in order to avoid that the assembly plugin would add my class files to the jar file twice (ProGuard complains about this) – therefore the assembly:single phase is invoked.

...
      <!-- Assembly: jar-with-dependencies -->
<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-assembly-plugin</artifactId>
	<version>2.2-beta-4</version>
	<configuration>
	  <descriptorRefs>
	    <descriptorRef>jar-with-dependencies</descriptorRef>
	  </descriptorRefs>
	  <archive>
	    <manifest>
	      <mainClass>com.boxofrats.App</mainClass>
	    </manifest>
	  </archive>
	</configuration>
	<executions>
	  <execution>
	    <id>make-assembly</id>
<phase>package</phase>
	    <goals><goal>single</goal></goals>
	  </execution>
	</executions>
      </plugin>
...

ProGuard is linked to the package phase as well and runs after the assembly goal. <includeDependency> is set to false to avoid warnings about duplicates, because we have already pulled the dependencies into the jar file in the assembly goal:

...
      <!-- ProGuard -->
<plugin>
	<groupId>com.pyx4me</groupId>
	<artifactId>proguard-maven-plugin</artifactId>
	<executions>
	  <execution>
<phase>package</phase>
	    <goals><goal>proguard</goal></goals>
	  </execution>
	</executions>
	<configuration>
	  <includeDependency>false</includeDependency>
	  <injar>${project.build.finalName}-jar-with-dependencies.jar</injar>
	  <outjar>${project.build.finalName}-small.jar</outjar>
<proguardInclude>${basedir}/proguard.conf</proguardInclude>
	</configuration>
      </plugin>
...

My proguard.conf looks like this, note that this is a setup, for Mac OS X, where the JDK’s jar files are at a different location than on Windows, Solaris and Linux (they should be in $JAVA_HOME/lib/rt.jar:$JAVA_HOME/lib/jsse.jar):

-dontwarn
-dontskipnonpubliclibraryclasses
-dontskipnonpubliclibraryclassmembers
-libraryjars /System/Library/Frameworks/JavaVM.framework/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Classes/jsse.jar
-keep class com.boxofrats.App {
  public static void main(java.lang.String[]);
}

When building application with mvn package, there is a significant reduction in size of the resulting jar file, which is exactly what I wanted. “Hello world” is about 3-4 K, but that’s of course not representative. The bottom line is: we do not have to carry around 3 megs of dead baggage, we just take what we need. Being able to create smaller applications makes development in Scala much more reasonable for mobile platforms.

I did it. Finally I moved from Subversion to git. Until today, ZMPP was the only larger project maintained by me that was still in a Subversion repository. To be fair, in the two years I used it for my projects, I have never really experienced larger problems. It felt exactly like it was intended to be: A better, more modern CVS, and it has great tool support as well, something where git could still improve.

Still, after using git in parallel to svn and cvs for about a year now, nothing really beats the joy working with a DVCS, even when used solely through the command line. Personally, the way git handles branching and merging is perfect for me: it really invites one to make wild experiments. Another nice thing is that due to its decentralized nature, it is much easier for me to add ZMPP to my Hudson setup, work on my local copy and just push the changes to Sourceforge when I think they are good enough.

I have often played with the idea of writing Mac and iPhone applications in Scheme, but as a result of not knowing the right compiler to embed (and lack of experience in Scheme), I could not execute on it. While programming in Objective-C is definitely doable (and of course you can use Objective-C++), I like to have more options for native application development.

Luckily, this month, there was one blog entry about Gambit-C on the iPhone which was a great starting point (and James has since put out more interesting posts on the topic). In addition to the fantastic advice found there, I wanted a better integration within Xcode, since for now, it seems impossible to develop (legally) for the iPhone outside of the IDE (personally, I prefer setting up builds based on cmake and make them as cross-platform as possible). In particular, I wanted a build phase to compile from Scheme to C without having to invoke the Scheme compiler manually. I have created a template for this purpose for later use and have put it on github in case other people might be interested to use it. The source is released under the new BSD license, so feel free to experiment !

The principle used is similar to the one described in James blog – all Scheme modules defined in the project are compiled to C files and linked into a single executable. When using the Gambit-C compiler, one has to pay attention to the code that it generates: when linking statically, we need to:

• provide the -link switch to the compiler command
• include all the Scheme source files we want to link into the executable

If we would compile each file separately with the link option there would be a flat link file for each Scheme module and we would get linker errors due to duplicate symbols. Calling gsc with all the needed Scheme modules provides it with the necessary information to generate a single flat link file. The template assumes that all the Scheme files are in the scheme sub directory. In order to invoke the compiler with the correct parameters, I added an external target (a small Ruby build script) that scans the scheme directory and feeds it to the gsc command and it also includes a clean action that removes the generated C files.

To use the template in a different environment, the user simply needs to set the variable GAMBITC_BASEPATH, which could be as simple as /usr/local. This can be set either in the .MacOSX/environment.plist or the bash .profile script in the home directory. After a change to these, the user needs to log out and log in again so Xcode can recognize the change to the user settings. I personally prefer to install development tools somewhere in my home directory, which has the advantage that I can use different configurations and don’t pollute my system directories.

The contents of my personal GAMBITC_BASEPATH look like this:

$(GAMBITC_BASEPATH)/Debug
$(GAMBITC_BASEPATH)/Release
$(GAMBITC_BASEPATH)/Debug-iphonesim
$(GAMBITC_BASEPATH)/Release-iphonesim
$(GAMBITC_BASEPATH)/Debug-iphoneos
$(GAMBITC_BASEPATH)/Release-iphoneos

These are the directories that are the parameters to the –prefix option in the configure call when compiling gambit-c (see James’ blog). In fact, I simply built a version for each platform (Mac OS X, iPhone Simulator and iPhone Device) and symlink’ed the directories to reflect the layout above, since I do not need separate Debug and Release versions for Gambit-C.

In the build rules for the main target, I added a dummy rule which specifies the output files of the “compile scheme” target. This is the only part that I find a little ugly, because whenever the list of Scheme modules changes, this list needs to be changed, so the C compiler can include these files in the builds. I still find this solution better than checking in generated files into version control or including C files in the project tree which do not exist.

The github repository contains templates for a Mac command line tool and one for an OpenGL ES application on the iPhone. Since I am a game developer, the OpenGL ES template is most useful to me, but it is pretty easy to adapt the setup for other kinds of iPhone (or Mac OS X) applications. The initialization point in my template differs from the one described in James Long’s blog: the main.mm is unmodified, instead, the Scheme system is initialized when the view is initialized, because that is the place where I currently pull up my engine.

Happy Hacking

I am currently working on the core mechanics for my current game project. I had the idea that it could be used as a base for a quick implementation of an air hockey game. A search for “air hockey” in AppStore returned 14 air hockey games. Oh well, that was too simple…