Sprung stuff

I seem to be getting in conversations about Sprung a lot lately. I’ve definitely written about it plenty in the past but those blog entries are no longer available and probably not written with the sort of voice I’d like these days, so I figure it’s time to revisit what I worked on, 100 hours a week1, 16 years ago.

So, it was June, 2004. I’d been out of grad school for about a year and was having trouble finding the sort of job I wanted to, and had been doing IT consulting while living back at my parents' house. At the time I was incredibly focused on being a graphics programmer, and so the only real choices available to me were scientific visualization (which meant mostly government work) and the games industry.

I didn’t really want to work in the games industry, but I came across an interesting listing for a new studio/spinoff/subsidiary of Ubisoft2, based in Manhattan, NY, and I figured that was as good a way to launch my career as anything. After doing well on their coding assessment, they flew me out to the studio for an interview, and they showed me a prototype for a game which had me hooked.

What they had at the time was running on a J2ME cellphone (a Nokia 6600, as I recall), and it was a simple conversation simulator. I believe the concept was that you were trying to get the phone number of a girl you were interested in. I had heard of Japanese dating games but had never played one at the time, and there was one underlying concept to the game which I was absolutely fascinated with: rather than it being built out of a simple conversation tree, the conversational flow had what the lead programmer called “gauges,” basically hidden stats that informed the way the story progressed. It gave the conversation a feeling of flexibility and freeform progress, and almost felt “alive;” you could arrive at the same part of the conversation in different ways, but the same choices would have different results, because of the hidden state.

Anyway, the goal was to get this game turned into a launch title on the Nintendo DS, to get a bigger player base. While they had some of the early DS developer kit hardware (the original single-screen Nitro board, not even the “blue box” that most would be familiar with) they had nobody who knew how to work with it, and all of the development so far had been for the cellphone. And there wasn’t that much of it done yet, either. And where I’d come in would be to write the graphics and platform code to make it work.

So, I accepted the job and moved to New York City a week later, at the beginning of July.

The team was pretty small; in-house there were three developers (including myself), a small staff of writers (whose background was entirely in film and stage), and a producer. Artwork was being outsourced to a couple of animation studios.

At this point there was very little game code written; aside from the cellphone prototype (which was written in Java) and the conversation editor (which was written in Visio) there wasn’t a whole lot. The other two developers were working on porting the conversation engine to Windows, and I immediately got started with developing on DS hardware.

Dev tooling

The dev tools available to me were the single-screen Nitro board and Ensata, the official, cycle-accurate-ish, and very slow emulator. I’d had enough experience with emulation and demo coding to manage my expectations; the other developers, on the other hand, had very different ideas of what the emulator was and how fast it should run. In particular, since my graphics prototypes ran at around 2 frames per second on the emulator, they believed that it would only run at 2 frames per second on the real hardware, especially since our desktop PCs were many orders of magnitude faster. But I got it working on the single-screen board, and demonstrated that it did in fact run at the full 60 frames per second, even if only on a single screen. I also wrote up a quick raster-scan “profiler” to demonstrate that the emulator was going for accuracy rather than speed, and there were very good reasons for the emulator to work this way.

The raster-scan profiler trick is one of those lost-art sorts of things which only makes sense to a certain generation of graphics programmer. Back in this era, the display was almost never a direct-mapped true-color buffer; instead you had paletted graphics, where each pixel was an index into the palette table, and at raster time the rasterizer would read the index, then look that up in the palette table to generate a color.

So, the basis of the trick is: you can modify the palette while the raster scan is taking place. This is how a lot of historical demo and game effects happen; for example, any time you see horizontal waves on the screen, or a split-screen effect where only part of the screen scrolls (or different parts scroll at different speeds), something similar is happening, only with the scroll position instead. Retro Game Mechanics Explained actually just posted a video explaining this, and it’s worth a watch.

Anyway, in this case, instead of changing the scroll offset, the colors are being changed mid-screen, and so if you have, say, a stripe of palette color 0 along the right side (or filling the background), and you set color 0 at different parts during the render, you can see how much time is being taken up by each function, and where in the frame raster process that function is running.

Showing this on both the real hardware and on Ensata, and how the color bars more or less lined up, satisfied the other engineers well enough. And keeping this profiler in my back pocket became very useful later on.

Graphics stuff

So, the Nintendo DS graphics aren’t too dissimilar from the GBA, which aren’t too dissimilar from the SNES. There’s a lot of facets which are different, of course, but the overall design of how you make graphics work on each of them is largely the same.

In the case of the DS, there were several different modes available with different combinations of flat and tiled layers, and with different memory mappings. Given the requirements of Sprung, the best option seemed to be a mode with two flat buffers (one for background, one for the foreground) and two tile buffers (one for UI elements, and one for text). Unfortunately, these options were only available on one of the two screens, with the other limited to only a handful of modes, mostly ones which were intended for a heads-up display.

Fortunately, one of the modes was designed to support fully-featured dual-screen operation: you could render to one screen while simultaneously capturing it to a backbuffer, while the other screen displays the previous backbuffer. So, you render to the top screen while the previous frame displays on the bottom screen, and vice versa. This meant you could get full functionality on both screens, with the downside of only being able to render effectively at 30 frames per second instead of the full 60.3 Fortunately, our animation was only around 10 frames per second, so this didn’t matter.

The bigger problem, though, was that the Nintendo DS wasn’t designed to have full frames of graphics sent to the screen on every frame, and we had a lot of frames of graphics to deal with, which also took a lot of memory.

In my demo coding days I had learned a trick for how to do sprite transparency without having to do per-pixel transparency checks; basically, instead of storing every pixel, you store alternating spans of transparent and opaque pixels; transparent spans are just a number, and opaque spans are a number followed by that many pixels. Generally you just store each count as one byte (as the number of bytes to skip or send), and then the pixels as the raw bytes. In our case, since we were loading a full frame every time, the “transparent” count was just the number of key (i.e. 0) pixels to write, then the number of color pixels, and then the span of pixels gets copied over. If there was ever a span larger than 255, we could just send a span of 255 followed by en empty span of the other type (so for example, 290 opaque pixels could be sent as 255 opaque, then 0 transparent, then 35 opaque).

However, sending pixels one at a time via the CPU was incredibly slow; unaligned 8-bit writes were much slower than aligned 16-bit ones, and also doing it all in the CPU was a terrible idea as well. Fortunately, the DS also had a DMA transfer function which could send 16-bit aligned writes in the background with no CPU involvement.

But, there was another problem; doing it as 16-bit writes with 16-bit alignment meant needing to use a 16-bit span size and pair-of-pixel spans. I could (and in retrospect, should) have just done this, but I wanted to squeeze every single byte possible out of it. Also, at the time we were discussing the possibility of having some frames just be incremental updates on top of another frame, which would have been much more difficult to do if we had to consider the previous frame’s pixels in it. (We didn’t end up doing this but I wanted to keep the possibility open.)

So, I figured out how to keep the vast majority of the reads and writes 16-bit-aligned:

  • If the source and destination span both had the same alignment, send the span directly (send a single byte if the start was unaligned, then send all of the aligned words, then send a single byte if the end was unaligned)
  • If the destination was aligned but the source was not, take a byte from the source and save it for later, then send the aligned words, then send the saved unaligned byte
  • If the source was aligned but the destination was not, send the byte from the end of the source span, then send the aligned words for the rest of the span

This worked really well, and I could also use DMA transfers to schedule sending spans that were longer than a certain length (basically experimenting to see the smallest size of DMA transfer whose DMA setup overhead was lower than how long it’d take to send the equivalent number of pixels via the CPU).

There was a bit more fiddling than the above (like, how to deal with odd vs. even span lengths) but it made for a pretty reasonably fast blitter that let us just barely send an entire frame of animation ahead of the raster scan.

For the longest time, DS emulators had a lot of trouble with Sprung. Now you know why — the game was doing a lot to push massive amounts of graphics in ways that emulators never expected!

Also, a full animation frame had to be drawn 60 times a second; because each screen was drawing a different character (and sometimes different scene, in the case of phone calls), there was nowhere to save the previous frame in video RAM.

Later down the road we ran into some usability problems with using a tile buffer for text; the common complaint was that 8x8 characters were too small to be readable on the DS’s relatively-high-density screen. 8x16 text would have taken too much screen space, though (and I believe the DS didn’t provide enough tile memory to fit a double-tall font anyway), so we decided to go with an 8x12 font. I considered dynamically rendering the characters to the tiles, but by this point we were under so much crunch and I didn’t want to think about it too much, so I ended up just putting the text on the flat layer and stored the character art as tile memory. Fortunately, the span-transfer algorithm still worked like that, but it ended up taking a bit more space and raster time due to the increased span discontinuities around the edges of the characters. It also meant that I had to be a lot more careful about the render order of UI elements, because at many points the raster scan was now catching up to the render process.

Fortunately, I still had the raster profiler, and was able to reorganize the frame loop around it; I believe the final order was to draw any top-of-screen text (which happens in a handful of scenes), followed by the characters, followed by the dialogue text, and then run the game logic. Pretty simple in retrospect but at the time it was starting to seem like it might not be possible to do all of our rendering when it needed to happen.

Oh, and as far as UI elements go, we had a chunk of VRAM that was only available to the layer where the text box background lives, and that chunk happened to be the right size for a complete box-drawing setup, where I could have all 16 possible border/junction combinations (of a line going up, down, left, or right from the center) and background cells for any of the four corners. I also organized the palette entries such that there could be boxes of two background colors4. Originally I built the entire in-game UI to use it, but eventually we replaced the inventory screen with a baked background bitmap, and most of the remaining UI only uses a single window color. I think the only place where the two window colors still exist is on the music player, and there it’s very subtle (mostly I used it to give the background a slight checkerboard texture).

Art assets

Most of the character art was produced by an animation company. They had no idea how to make stuff for games, and they certainly didn’t understand the limitations of the DS. For each character we could pick 255 colors out of 32768, and we had only on-or-off transparency.

So I had to write a lot of custom art processing stuff.

The initial step involved gathering all of the animation frames of each character and having Photoshop derive a 24-bit 255-color palette for them. Then I wrote a macro to have Photoshop apply threshold transparency and spit out all of the animation frames as 256-color-with-transparency, diffusion-dithered art.

After this I had to postprocess the palettes to the 15-bit palette that the DS wanted.

In the end I’d have preferred to implement the dithering myself to have better control over the color space and the differences between 24- and 15-bit color, but I had to pick my battles given the extremely limited time available. I think I converted the palette to 15-bit-equivalent first before having Photoshop apply the dithering, at least, but hey, it was 16 years ago. I can’t remember everything.

The inventory item art was done by a contractor, but he worked from our office and I was able to work much more closely with him. I forget if we settled on a single global 255-color palette for item art or if we made use of the DS’s per-sprite palette functionality; I don’t remember doing anything fancy with palettes so it was probably the first one. In any case he was able to do all of the major asset conversion for me, and I only had to do some basic tile conversion stuff (which I’d already written and automated for the character art).

UI/UX

Originally the game was designed to only be played with the d-pad and buttons, but marketing decided that since it was a DS launch title it had to use touchscreen features. Unfortunately, Nintendo required that if anything was touchscreen-accessible, then everything had to be, in that you couldn’t require that people switch between the touchscreen and the d-pad. Which makes perfect sense! But it meant that a lot of UI had to change; the inventory screen (originally on the top screen) moved to the bottom screen and had to become touch-friendly, but more importantly there also had to be touch access to text selections. This was a major pain in the ass to do, and the screen just wasn’t precise enough to make touch-based selections, and I also had to add an entire touch UI for accessing the pause/quit menu. Reviewers (rightfully) said we shouldn’t have even bothered with touch, which is exactly what I was saying during every step of this process5. Oh well.

I did have three Easter eggs in the UI, of which two remained in the shipping game.

The first one was the visual transition editor; I thought it was fun to play with and so I hid it behind a complex button sequence. Unfortunately, one of our testers managed to stumble across this button sequence and flagged it as a bug that this dev tool was “accidentally” left in. So I had to take it out.

The second one was a complex button sequence that puts a ♥︎ in the corner of the main menu. I don’t know of anyone who’s come across this one (or cared to document it anyway), and unfortunately I don’t remember what the button sequence even is. If someone else finds it, please post a comment here so it can be remembered!

The final one is one I’d actually forgotten about until just now when I tried to remember how to trigger the heart: if you hold down both shoulder buttons while on the file select screen, it shows you the scene number that the save game is on. This was less an Easter egg and more a debug tool that I forgot to remove. I’m amazed QA didn’t find and flag that one.

On the note of visual transitions, those are all very basic palette effects. I gave the writers the ability to do some simple palette manipulation things, such as changing the brightness, contrast, and saturation, and I purposefully allowed those to go negative to get various other weird effects going on. I felt like it got used pretty effectively.

Music

Because of the experimental design of the game, I thought it would be fun to be experimental with how the music was built. I designed a semi-generative music system where music could be split up into separate layers of “riffs” and have the hidden conversation state drive a Markov chain that would assemble the riffs together into a song, and I wanted to work with a bunch of my musician friends to make the riffs. I couldn’t get buy-in from the producer, though, and so we hired Tom Salta to do the music. I did still work closely with him to split the music up into “moods” and used the Markov chain system to assemble it into the game’s soundtrack, and you can still access that system in the music screen.

Unfortunately, it turns out the writers weren’t actually using the hidden state for most of the game, so very little of the soundtrack actually ends up playing during a normal playthrough!

There were a bunch of other challenges in doing the music; the DS’s music engine is pretty limited, and composing music for it involved either using a proprietary tracker-style tool, or a MIDI converter. Tom was used to working in MIDI so that’s what we used, but the MIDI implementation was pretty limited as well, so we (meaning mostly I) had to do a lot of post-processing on the files to make it work right. For example, the volume curve was linear rather than logarithmic, and there was no support for a sustain pedal.

We also had a very limited audio sample RAM budget, and the design of the player system made it difficult to dynamically load samples as-needed, so we had to get all the samples to fit into memory at once. This meant compromising on the sample quality quite a lot, and we (again, I) spent a lot of time tweaking the sample rates and loop points to try to get them as small as I could manage. But in the end, the DS’s software-based music player degrades the audio quality so much6 that it didn’t really make much of a difference.

None of this was anything Tom had run into before, since almost all of his game music work was done on streaming audio systems like the Playstation and he’d never worked on a tracker-style system at all. So, this was a source of frustration for both of us.

Another annoying thing is that the music player runs at a different interval than the game itself; I think it runs at 192Hz instead of 60. And the music player was designed to play tracks either looping or one-shot, with no way for it to signal “hey I’m finished playing a riff, time to start another one,” so with the riff-based thing I had to do a whole bunch of drift compensation stuff and wonky inter-processor communication to try to minimize the audible gaps between one riff ending and another one beginning. This still sticks out like a sore thumb to me whenever I listen to the game.

In retrospect, a thing that I thought would make it easier to author a lot of dynamic music ended up being more difficult than it was worth, and it was mostly unused anyway. It would have been better off just going with traditional authored music and calling it a day. Oh well!

There’s a weird “film noir” scene. I abused the DS’s extremely-limited audio effect engine to try to make it sound old-timey. It sorta worked in testing although I don’t think it really comes across in the final game.

Sound

We had literally no budget or design for sound. I got tired of there being no sound, so one day I brought in my laptop and a microphone and I occupied an empty meeting room to record a bunch of sound effects. Any sound effect you hear in the game probably came from my mouth.

When reviews came out, a couple of them mentioned the lackluster and limited sound. The producer’s reaction: “We had sound?!”

Oh well!

The joys of doing launch-window games

As I said above, when I first started on the game, the only dev hardware we even had was the early single-screen Nitro board. We didn’t get one of the blue boxes until much later. And we only got one. As the platform engineer (and the one using it the most) it mostly sat on my desk, which meant any time someone wanted to test on the real hardware I’d get a visitor.

We also didn’t have any idea what the cartridge interface would be; the SDK still gave the impression that the cartridge would be memory-mapped into the address space (like it was on the GBA), rather than the serial interface that they eventually went with.

I had built the internal filesystem around the memory-mapping setup (where building the cartridge would just map the files into ROM data and then generate a symbol table that we could use to map the actual asset into memory), and the SDK continued to support the notion that even over the serial connection it would be virtualized and memory-mapped in some way. But then at nearly the last minute before gold master we got a new, final SDK drop that required that we do an explicit read from the cartridge over serial, using their own filesystem — and the filesystem itself was extremely limited in terms of what things could be named (I think it was limited to 8.3 filenames and a certain number of files per directory), and our asset names violated that. So, I whipped up a quick-and-dirty filesystem wrapper that lived inside of a single file within the DS filesystem.

The filesystem I designed was very much a get-it-done-fast thing. It wastes memory (IIRC, the entire file table gets loaded into memory at startup and stays there forever) and I think I just store the file table as a flat, name-sorted list of file offsets and do a binary search on the full path or something. It was also 16 years ago. Someone who’s datamined the game would probably remember more about how it works than I do. I believe I wrote the entire filesystem and converted the whole game to use it in a manner of, like, four hours. A day tops.

It was the eleventh hour and I had to Get It Done. That’s what happens sometimes.

“Cut” content

The Cutting Room Floor’s writeup lists a whole bunch of stuff which was “cut.” The reality is that nothing was cut, it was just forgotten.

Basically, we had no test/validation pipeline, and our QA was overwhelmed with just finding game-breaking bugs. We had no overall narrative plan or test plan, and it’s only at the last minute that anyone even managed to play the whole game the whole way through.

If something was “cut” it was just because we forgot to add its trigger in.

Most of what went missing was just fun stuff scripted to a bunch of the inventory items, where said items were never given to the player. There’s also an entire scene that’s not reachable, and an unlockable artwork that never got its trigger placed into a level.

The inventory system was pretty much an afterthought that came very late into the process; there was a general feeling that just having a conversation tree wasn’t enough of a game, and there needed to be more to it. Inventory items as a result ended up becoming either like “hey I’ll randomly give you this item which gets immediately used in the next scene” or “hey I’ll randomly give you this thing and it’s never useful except as a way to trigger something funny that usually results in a game over.” The writers specifically added the pepper spray as being a funnier way of doing a “restart level” action (it’s never anything other than a game over), for example.

Bugs

You can pepper spray people over the phone! Of the few memes to come from Sprung, this is my favorite. It was also a complete oversight; phone calls were hacked in, pepper spray was hacked in, neither hack was tested much on its own much less together. And it results in something wonderful and memorable that people talk about fondly.

The memory tooling on the DS SDK was (like the DS itself) very limited, and the allocator was designed for loading a large static level and then throwing it out before moving to another level. We could have used it correctly, but due to team inexperience that didn’t end up happening. So, I wrote a pretty invasive memory profiler that would keep track of every dynamic allocation and watch out for memory leaks, and in debug builds would inform the user of double-deallocs and so on.

Unfortunately, one of the programmers didn’t trust this, and decided at the last minute (as in, after code freeze) to free up some memory which he didn’t think was being freed. And he didn’t check it in the memory profiler (which he didn’t trust), nor did he tell anyone else about this “bug fix” until we’d already shipped off the gold master to Nintendo. After which I demonstrated to him exactly where the memory was being properly freed. And then he tried actually triggering the “fixed” code on said gold master build, and it did in fact crash.

This double-dealloc bug is in the achievement system. Getting two achievements during a single play session has a high chance of corrupting the heap and crashing the game. All for an allocation that was only something like 1 byte, anyway! (Why was it allocated on the heap to begin with? Who knows. I didn’t write that code.)

There’s also a couple of one-off characters which weren’t properly integrated into the conversation engine, and there’s a crash bug that happens if you use an inventory item on them if there’s no scripted action for that item (main characters all had generic fallbacks for unscripted interactions). One that comes to mind is that if you use the mittens on the old lady, the game crashes. Unfortunately, the first we’d heard of this was in someone’s comment on a forum.

DS games weren’t patchable.

Oh well!

Some meta stuff

There’s an odd myth about Sprung, that it was developed in Halifax, and that any of the random cultural references to places that players have recognized must be a coincidence. But no, it was written and developed in NYC, the writing staff primarily being NYU film and theater students, and a lot of the obscure local references came from their own experiences growing up around the country.

The Halifax belief is probably because the studio did eventually move to Halifax, but that was years after Sprung was completed. Only around half of the Sprung team was still even at the studio by that point. Amazingly, most of them are still there!

Publicly the name has been known under two titles: Crush (the working title) and Sprung (the release title). However, there were a lot of working titles for it, including Hooked Up (the original title) and Match. Many of the lines in the game are a title drop [WARNING: TV Tropes link] for titles that were long gone.

We had a really hard time coming up with a title that worked and which didn’t have existing trademark issues. “Sprung” was a writer’s joke that stuck, somehow (and I believe it was a rare case of the title being in the game’s dialog before it became the title, making the title drop retroactive). I have no idea how that got past marketing. Or even how it passed legal review, considering some of the names we couldn’t use.

Incidentally, the original “Hooked Up” name lives on in one of the unlockable artworks, which was also used in our marketing material and got prominent center placement on the game box art in the US. (In Europe it got moved to the corner for some reason.)

Also, the game only barely squeaked by with a T rating in the US, and PEGI 12 in Europe. There were a few jokes which had to be cut because the ESRB reviewer thought they were drug references. Oddly enough, way worse jokes (including actual drug references) made it through.

My understanding is that this was the first time the ESRB had to review a dialog-heavy game and they were completely ill-equipped to do so. The ratings process involved us sending them all of our art and all of our lines of dialog. The art was easy, but the dialog was something like 50,000 lines of text.

Oh, also, there’s marketing around Colleen McGuinness having written it. My understanding is she came up with the character treatments and some of the overall plot, but the vast majority of the writing was our in-house writing staff (Sam, Ed, AJ, and Desiree), and her involvement with the game ended before full production started.

Legacy

I believe Sprung was the first commercially-released visual novel for Western audiences, at a time before the term “visual novel” even meant anything outside of Japan. The only term we really had for it here was “dating simulator” and even that wasn’t well-known. It’s not the first VN for a game console, of course (Phoenix Wright for the GBA came out in 2001, only in Japan), and in Japan VNs had a long and rich history on PCs by that point. But I haven’t found any examples of VNs developed in the West before Sprung, much less for release on a console.

This was the first commercial game any of us had worked on (aside from the studio head and the musician), and for many of us the first game any of us had worked on at all. I had a background of homebrew/experimental hobby game dev7, and one of the other programmers had a degree from DigiPen (back in 2004 when they were mostly a game art college and had only the barest beginnings of a programming track). This was also the lead programmer’s first programming job at all (he was lead by merit of being there two months longer than anyone else and therefore being the only one there to take on the role).

As I understand it, the game started being prototyped (for cellphones) in early 2004, and full game production had only started a couple weeks before I took the job. So, June 2004 at the earliest. And we were done by the end of October. So, a completely inexperienced team managed to make a launch title for one of the most anticipated game consoles in a matter of 5 months! It’s pretty amazing we managed to do what we did.

Its sales were abysmal and the only people who even remember it see it as a cult classic that they ironically enjoy. But it’s amazing what we got done in a short time on a shoestring budget8 with no planning or experience.

I can’t help but be weirdly proud of it, even to this day.

Even though my deadname is in the credits.9

Comments

Before commenting, please read the comment policy.

Avatars provided via Libravatar