Thursday, July 30, 2009

ROM Trojaning the Super Cassette Vision

Hi.

Each time I do something geeky I tell myself "now THIS is THE most geeky thing I did in my life". But whatever happens, this is a phrase I tell myself month after month working on chipsounds....

Which brings us to the wonderful topic of ROM Trojaning!
But nah, lets put things in context first. :)

The idea behind chipsounds is not to put in VSTi form what people have been doing with gameboys for nearly a decade now with LSDJ/NL. The idea is to analyze, salvage and reproduce the entire crunchy 80's aural history from oblivion.

Early 80s sound chips are unique because they were EXERCISES. Nobody (except for Yannes) had a clue of what they were doing. The engineers that were given the task to come up with a soundchip were seldom musicians. So they tried and invented as they went along. Some things worked, others were dogey, but we fell in love with them anyway. Some chips were hugely popular, others faded, in most cases it was more a question of the success of their console and games more than their sound quality.

The Epoch SCV was crushed by the famicom a mere year after its launch... However, reading about  its specs, I was instantly curious about its obscure sound generator. Some sound bytes taken from the net, including this one made one thing clear, this thing didn't just generate square waves!

So a month ago I bought a PAL version of this console (renamed YENO in France), with three ROMS which I started playing with and recording right away (to get a rough idea of the spectrum of sounds it could create).
Here is some of that.

Safe to say It definitely warranted its place in chipsounds :)

But now that's where the fun parts starts. Nearly nothing is known about its soundchip, the NEC uPD1771C. Here is a picture of mine:



The only known research comes from three talented Japanese people: Mr Enri, Mr 333 and Takeda Toshiya.

Their research provided the world with a nice emulator for the console, and tons of reverse engineering details.... including a disassembler for its obscure NEC cpu.

However, the BIOS Rom for that machine (4096 Bytes) is stored inside the main cpu (NEC D7801G or uPD7801G). And its nowhere to be found online. This BIOS code holds the key to how the console talks to the chip, and without it I would have lost many (more) days to try to extract all the raw data i need for analysis. (perhaps i would have abandoned the idea altogether) To be fair, no one can blame them for not providing the bios rom online. It is indeed is a breach of copyright law! (you wont get it from me directly either)

The Mr Henri proposed a solution to grab the rom which requires a battery backed ram cartridge and some "thing" to read it back which I have no clue is what (especially with babelengrish). However I have a fascination with the trojaning research some people, including GURU have done for the MAME/MESS projects...

So I came up with a VERY LAME solution, but one that didn't require modding my rare console, or buy yet more special equipment. I however really need to give Compute's Gazette magazine some credit for my idea (MLX) ... hehe

The idea is to transform the 4096 rom bytes in something that can be printed and humanly retyped without errors, using a checksum after each line typed.

SO

1)I wrote a ROM for the console which spewed out the ENTIRE thing onto 30 PAL frames with a running check sum on each line. (You have to realize that there is NO DIRECT ASSEMBLER FOR A D7801G; only for D7810s which only as about 75% of the same byte code. Even a simple instructions like RET is different)

2)Placed the ROM on a W27C512 chip then onto a 16Kb Cart PCB:
(an electrically erasable eprom is handy when you do a hundred trials)



3)Recorded my hacked SCART/RBG/PAL output to a NTSC Canon MiniDV camera whose composite input -somehow- seem to accept its signal)

4)Transferred to the computer using Firewire

5)Saved each 30 frames separately. Here is the first one:



(note I really wanted full HEX code, but I had to be lazy somewhere right? .. so value=input-'0';

6)Batch-Massaged the picture a bit. (contrast mostly)

7)Tried to OCR the bloody thing for fun (I knew it wouldn't do any good).

8)Last resort... Back at work Monday morning, I used the whole company (we are 5 now) for a one hour intelligent typing challenge!

9)We got 'a' BIOS ROM:
MD5 : 635a978fd40db9a18ee44eff449fc126
SHA1 : 6e89d1227581c76441a53d605f9e324185f1da33
CRC32 : 7ac06182

Tried it in the eSCV emu... and it worked!!!!

I could NOW start the real work

I disassembled the BIOS and looked for the simple code that the console plays when pressing the pause button... this gave me exactly what I needed to know in order to generate the lists of values to be sent to the real chip in loop, (to check for waveform pattern changes, pitch limits etc).
The D1771C on that end is completely different from the typical !WR/!CS address/d0-d7 thing...
And its more like MIDI, in a sense that its a state machine of bytes. 4 bytes (with interrupt based ACK) to play a tone, and 10 bytes for a noise message. A pain.

Im still capturing waveforms and analyzing them, but I figured there was enough for a long overdue post on this blog.

Thanks for the Plogue Team. (Seb/Max had a tie... Seb finished first but had one checksum error. While Max finished last but had no errors)... I was pretty pathetic myself... Finished second with 7 errors... cant be good at everything :)

Here is the rom to CAPTURE the BIOS, (including the ASM file to modify it).

EDIT: Ive updated the zip file with more comments and example decoding code for anyone else attempting to dump the BIOS (including the NTSC one)

Friday, July 10, 2009

A new format to log them all.

While doing my Chip research I stumbled upon many different formats to store “chip music information”.

Some of them contain the full assembly code used by the original console/video game
-SID (C64 6510 assembly)
-NSF (NES)
-SAP (Atari 8bit computers with POKEY chips in them)
(etc)

Others only contained the actual register values WRITES along with their timestamps:
-VGM (SN, YM2612, YM2151 etc)

Both have pros and cons, But I do prefer the VGM approach, since it abstracts the chip from its processor, it allows much easier re-purposing of the music data streams. For instance, in order to use a direct PIC interface, parallel port, arduino, what-have-you to play the data back on a real chip.

For my continuing analysis and cross checking, I need to do bunch of tests on chip register data (especially for SID and POKEY).
I also need to edit entries, filter some registers, add comments to specific writes, play the result, record, analyze in a sound editor, using various home made tools.

I usually also had to type or procedurally generate long lists of chip register commands in order to hear how some chips behave under specific scenarios, and also to record a long series of chromatic notes for a sampler mapping (SID combined waveforms in chipsounds using SFZ mapping) etc, without having to delve into this or that system's native assembly each time. (I had to do that too many times that I dont miss it much)

I thought of using the VGM format and extending it but:

1)Its binary only
2)Only logs writes at 44100 Hz, (while some emulators run at full master clock of 3.57MHz or more, so you DO lose information)
3)Its been patched a lot through revisions, and is hard to maintain.
4)NIH :)

So really I just went with the simplest thing I could come up with.
VGMX!! (Extended VGM or VGM XML) .. how original

The format is very simple it doesn't require a DTD (especially since most of us use modified TinyXML source trees right?)
It goes something like this:




Note: clock and rate are two different things. Clock is the actual oscillator frequency feeding the chip, while rate is the time base for the writes. (emulator timebase in many case)

This is very preliminary, but I hope that this interest somebody.

In closing here are a few examples of real files and rendered results from my protoboard+pic setup

1)7800 Ballblazer (VGMX) (MP3)

2)C64 Great Gianna Sisters Doom (VGMX) (MP3)