Them Old Ideas Are Buried Here is an interactive art installation that turns an album into a physical place you explore. It is an interactive digital map that links place, story and sound. Visitors sit down at a hand-built wooden console, turn a crank to travel across an illustrated Wild West map, and stop at real and imagined places: a homestead, a saloon, a graveyard, the moon. Each holding its own scene: an original illustration, its song, and a piece of the history behind it.

Link to GitHub repository.

Last year, I was inspired to illustrate a musical body of work with the intent to do something novel, something interactive. How can use a creative body of work to push the bounds creatively on how one’s vision can come to life outside of radio spins and music video views? I didn’t have the language then, but I knew I wanted to create something that people could experience outside of passive consumption; my mind never connected the dots until I was introduced to p5.js.

Instead of settling for static illustrations, I chose to enhance (or remix). With this project, users are able to experience the "Cowboy Carter" album sonically while taking in the visual art that was inspired by it. Each song now takes place in a specific point on the United States rail map, each song has its own story, each song has its own visual interpretation in the form of an illustration.

Process

Originally my idea was to build a side-scrolling game. I started illustrating each board: the background signage, the platform, the subway itself. I soon realized that in order for me to do it justice, I needed more than a week’s time to pull it off. I soon after made the decision to pivot my idea into something that was both interesting and time-permitting. 

Once I knew I wanted to build a storytelling device, I immediately brainstormed the right UX for getting in and out of various scenes. Because there are many different places specified throughout the story, the invitation to explore and discover needed to be immediate. The map was the first thing I tested with users, and the feedback I got during those testing rounds was paramount. I learned that the initial map was confusing and the way the points connected didn’t make it clear to users where they were going next. I updated the map to show only connection points between places in succession.

I didn’t have the actual illustration scenes lined up and implemented at that point, but I knew how to present the information on the screen; I did not know how to switch between scenes. This is where AI came in handy and where I asked Claude to assist me in making a clear experience that evoked a sense of discovery and play.

I also hit a real, unusual problem: the newer version of p5's sound library had been substantially rewritten, so play() silently ignored the start-time and duration I passed it, so I had to call jump() separately to actually seek to the right position. Instead of guessing, I utilized print methods directly in the console to find out what was really happening.

On the hardware side, I also had to work through a few snags: the rotary encoder initially produced two events per physical click, pins not set to INPUT_PULLUP, and getting the encoder to feel precise rather than loose and jumpy, which eventually meant building a proper quadrature decode table instead of relying on simple debouncing.

Technical

I combined my Arduino with p5.js (v2) for this project, so the Web Serial API is the glue that holds this all together. I also utilized the p5.sound library to bring a key component of the experience to life: the music. I wanted users to control their experience by having them use a handmade controller device to navigate. Because the project’s theme is reminiscent of the Wild West, I want the device to look and feel like something a train conductor would use on the job. 

I utilized custom functions, arrays and for loops to build out this experience. I felt that that was the most logical way to organize the same key pieces of information across multiple assets; that structure became the driver for the information that was presented on screen. Because I was on the v2 of p5.js, I also had to figure my way around breaking changes from older tutorials/docs (the [preload()] removal, [keyCode] → [code]).

Check out a code snippet that lays the foundation for this creative experiment:

Until I put this together, the project felt incomplete. I always envisioned the illustrations and sounds as pairs, and with the limitations of today's social media platforms, it was hard for me to replicate the full vision through the restricted audio tools copyright hoops. This project helped me realize the completion of a vision.