In spring 2013 I took a course on Physical Computing at PNCA. Physical computing is the use of objects or sensors as input or output for a computer. The idea behind it is that the computer should not dictate the artistic experience as much as become a new medium with which to work from. Since going to Dorkbot meetings and getting involved with Suspect Devices, I’ve wanted to make art that was in a more technological space, while still drawing from my own experiences and life themes. I used this piece as both my final for the class and as an entry to BarCamp Portland.

About

My original premise was to create an installation piece using ambient sound that generates and activates 8-bit graphics. The piece would have several microphones in different parts of a room. Each microphone would plug into a custom-built NES (Nintendo Entertainment System) cartridge with a microcontroller that would analyze the sound from each microphone and recompile a ROM (memory chip) with suitable graphics and color palette. I only had eight weeks to finish the project, so much of my original ideas had to be scrapped in order to have a presentable piece.

The piece now has two characters on a screen that dance based on a single microphone input.

Process

I broke the piece down into three seperate problems. I ended up doing most of the sound stuff at the same time.

• Creating a NES cartridge
• Sound analysis from source
• Turning analysis into an input

NES emulation

I briefly looked into NES cartridge building but realized that the turnaround time to get the circuit to be correct would be longer than eight weeks, so I focused on the software. I started learning about programming assembly for the 6502 chipset, which is the heart of the Nintendo console of the 80s. I didn’t get very far when I read about a neat little project called PyNES, which will create compiled ROMs from Python scripts. I immediately downloaded the source and started poking around, trying to understand the example scripts that it came with.

Getting a ROM together was much harder than it seemed. I had to learn to edit CHR “files”, which are the character banks from which NES graphics are stored. I used Mac OS in developing this project, which was notorious for not having a complete NES toolchain for quite some time. However, I managed to find the NES CHR Editor made by Squirrel. It is not an open-source tool though I did contact the author to have the CHR file be editable without being included in a ROM. (Previously you needed to have a pre-compiled ROM to edit from.) I used the CHR from Zooming Secretary as a reference point for how the sprites are stored and arranged.

Sound as an input

I found a seven band graphic equalizer CMOS chip from Mixed Signal Integration that seemed easy enough to work with. Sadly, I found out about the chip too late and didn’t receive the sample in time to put it into the piece. (I received the sample the day before BarCamp.) The microphone that I used was a Blue Yeti, which is USB powered, so I focused on using the stereo signal from that to get an analogue value that would be read via an Arduino Micro. I then translated these signals to keypresses. A word of caution - having a microcontroller pretend to be a keyboard (a HID device) can cause some very funny things to happen…

After I realized exactly what happened and wiped the coffee-spittle from my computer screen, I added an on-off switch to the circuit so that I could control when the keyboard would be on or off.

Putting it all together

Now that I had an NES emulator whose buttons were pressed from the microphone, then went through several iterations of compilation, I started to play around with the sprites. I had tried to use patterns that I drew in the CHR Editor, but couldn’t figure out how to background and tile multiple sprites (or a single sprite, for that matter). I settled on building up characters from Zooming Secretary and suddenly got nostalgic for my college girlfriend, the way they danced around the blue screen. I then styled the characters to look like we did at prom and added some cute stars that buzz around. Sadly, when I took the setup to BarCamp Portland for the Mini-Maker Expo, I found that the NES emulator crashed frequently duing demonstration.

I would like to revise the piece using either actual hardware or perhaps learning more about writing assembly for the Nintendo. I would also like to incorporate the audio processing chips. As a finished art piece it was a failure, but as a prototype for an art piece it was somewhat successful.