[COFF] Dragon Quest Disassembly

[segaloco via COFF]

Howdy all, I've made some mention of this in the past but it's finally at a point I feel like formally plugging it.  For the past few years I've been tinkering on a disassembly of Dragon Quest as originally released on the Famicom (NES) in Japan.  The NA release had already gotten this treatment[1] but I wanted to produce something a little more clean and separated: https://gitlab.com/segaloco/dq1disasm

Another reason for this project is I've longed for a Lions's Commentary on UNIX-quality analysis of a scrollplane/sprite-driven console video game as I find generally analysis of this type of hardware platform lacking outside of random, chance blog posts and incredibly specialized communities where it takes forever to find information you're seeking.  In the longer term I intend to produce a commentary on this codebase covering all the details of how it makes the Famicom tick.

So a few of the difficulties involved:

The Famicom is 6502-based with 32768 bytes of ROM address space directly accessible and another 2048 bytes of ROM accessible through the Picture Processing Unit or PPU.  The latter is typically graphics ROM being displayed directly by the PPU but data can be copied to CPU memory space as well.  To achieve higher sizes, bank swapping is widely employed, with scores of different bank swapping configurations in place.  Dragon Quest presents an interesting scenario in that the Japanese and North American releases use *different* mapper schemes, so while much of the code is the same, some parts were apples to oranges between the existing and new disassembly.  Where this has been challenging (and a challenge many other Famicom disassemblers haven't taken) is getting a build that resembles a typical software product, i.e. things nicely split out with little concern as to what bank they're going to wind up on.  My choice of linker facilitates this by being able to define separate output binaries for different memory segments, so I can produce these banks as distinct binaries and the population thereof is simply based on segment.  Moving items between banks then becomes simply changing the segment and adding/removing a bank-in before calling out.  I don't know that I've ever seen a Famicom disassembly do this but I also can't say I've gone looking heavily.

Another challenge has been the language, and not necessarily just because it's Japanese and not my native language.  So old Japanese computing is largely in ShiftJIS whereas we now have UTF-8.  Well, Dragon Quest uses neither of these as they instead use indices into a table of kana character tiles, as they are ordered in memory, as the string format.  What this means is to actually effectively manage strings, and I haven't done this yet, is one needs a to-fro encoder that can convert between UTF-8 Japanese and the particular positioning of their various characters.  This says nothing of the difficulty then of translating the game.  Most Japanese games of this era used exclusively kana for two reasons: These are games for children, so too much complicated Kanji and you've locked out your target audience.  The other reason is space; it would be impossible to fit all the necessary Kanji in memory, even as 8x8 pixel tiles (the main graphic primitive on these chips.)  Plus, even if you could, an 8x8 pixel tile is hardly enough resolution to read many Kanji, so they'd likely need 16x16, quadrupling the space requirement if you didn't recycle quadrant radicals.  In any case, what this means is all of the strings are going to translate to strictly hiragana or katakana strings, which are Japanese, but not as easily intelligible as the groupings of kana then have to be interpreted into their meanings by context clues often times rather than having an exact definition.  The good news though is again these are games for children, so the vocabulary shouldn't be too complicated.

Endianness occasionally presents some problems, one of which I suspect because the chip designers didn't talk to each other...So the PPU exposes a register you write two bytes to, the high and low byte of the address to place the next value written to the data register to.  Problem is, this is a 6502-driven system, so when grabbing a word the high byte is the second one.  This means that every word has to be sent to the PPU in reverse when selecting an address.  This PPU to my knowledge is based on some arcade hardware but otherwise was based on designs strictly for this console, so why they didn't present an address register in the same endianness I'll never know.  It tripped me up early on but I worked past it.

Finally, back to the mappers, since a "binary" was really a gaggle of potentially overlapping ROM banks, there wasn't really a single ROM image format used by Nintendo back in the day.  Rather, you sent along your banks, layout, and what mapper hardware you needed and that last step was a function of cartridge fab, not some software step.  Well, the iNES format is an accommodation for that, presenting the string "NES" as a magic number in a 16-byte header that also contains a few bytes describing the physical cartridge in terms of what mapper hardware, how many banks, what kind, and particular jumpers that influence things like nametable mirroring and the like.  Countless disassembles out there are built under the (incorrect) assumption that the iNES binary format is how the binary objects are supposed to build, but this is simply not the case, and forcing this "false structure" actually makes then analysis of the layout and organization of the original project more difficult.  What I opted towards instead is using the above-described mechanism of linker scripts defining individual binaries in tandem with two nice old tools: printf(1) and cat(1).  When all my individual ROM banks are built, I then simply use printf to spit out the 16 bytes of an iNES header that match my memory particulars and then cat it all together as the iNES format is simply that header then all of the PRG and CHR ROM banks organized in that order.  The end result is a build that is agnostic (rightfully so) of the fact that it is becoming an iNES ROM, which is a community invention years after the Famicom was in active use.

Note that this is still an ongoing project.  I don't consider it "done" but it is more usable than not now.  Some things to look out for:

Any relative positioning of specific graphic data in CHR banks outside that which has been extracted into specific files is not dynamic, meaning moving a tileset around in CHR will garble display of the object associated.

A few songs (credits, dungeons, title screen) do not have every pointer massaged out of them yet, so if their addresses change, those songs at the very least won't sound right, and in extreme circumstances playing them with a shift in pointers could crash the game.

Much of the binary data is still BLOB-afied, hence not each and every pointer being fixed.  This will be slow moving and there may be a few parts that wind up involving some scripting to keep them all dynamic (for instance, maps include block IDs and such which are ultimately an index into a table, but the maps make more sense to keep as binary data, so perhaps a patcher is needed to "filter" a map to assign final block IDs, hard to say.)

Also, while some data is very heavily coupled to the engine, such as music, and as such has been disassembled into audio engine commands, other data, like graphic tiles, are more generic to the hardware platform and as such do not have any particular dependencies on the code.  As such, these items can exist as pure BLOBs without worry of any pointers buried within or ID values that need to track indices in a particular table.  As such these flat binary tiles and other comparable components are *not* included in this repository, as they are copyright of their respective owners.  However, I have provided a script that allows you to extract these binary components should you have a clean copy of Dragon Quest for the Famicom as an iNES ROM.  The script is all dd(1) calls based on the iNES ROM itself, but there should be enough notes contained with in if a bank-based extraction is necessary.  YMMV using the script on anything other than the commercial release.  It is only intended to bootstrap things, it will not properly work to extract the assets from arbitrary builds.

If anyone has any questions/comments feel free to contact me off list, or if there is worthwhile group discussion to be had, either way, enjoy!

- Matt G.

[1] - https://github.com/nmikstas/dragon-warrior-disassembly