City Drift
Translate map data and movement into sound and visuals, creating a continuous feedback loop shaped by interaction.
City Drift
Try it
How it works
City Drift turns map exploration into sound and visuals.
As the user moves through the map, their position and movement are translated into values like speed and intensity. Different areas, such as green spaces, water, or urban zones, influence the tone and behavior of the system. The system is always active. There are no fixed actions to trigger. Instead, the user continuously shapes the output by exploring:
As the user moves through the map, their position and movement are translated into values like speed and intensity. Different areas, such as green spaces, water, or urban zones, influence the tone and behavior of the system. The system is always active. There are no fixed actions to trigger. Instead, the user continuously shapes the output by exploring:
1. Interacting with the map:
When you move across the map, you’re exploring different parts of the city. Each area has its own color patterns, and those colors are translated into sound. So as you drag, the system reads the visual data in real time and turns it into a changing soundscape.
2. Gesture control:
Your body becomes the controller. Your hand movements are tracked and mapped to sound parameters like rhythm, intensity, or effects.
Background
First thing first, City Drift is not a polished project. It is a series of attempts. It started from a very simple intention: I wanted to build something interactive. Most of my past work is visual or system-based, but not something people can directly play with. This project was my way to explore that.
At the same time, I have always liked maps. I collect maps and vintage globes as I’m drawn to their visual language. Instead of just looking at them, I was curious what it would feel like to play them. So my first idea was: What if people can interact with a map and generate music from it? 🤔
The idea also came from a simple observation: Cities are not static. They shift, pulse, and reorganize over time. In my previous project Urban Spheres, I already explored this through time-based patterns. City Drift moves one step further:
From time → interaction
From visualization → control
From observer → participant
The interface is no longer just representing the city. It becomes something you can play with.
From time → interaction
From visualization → control
From observer → participant
The interface is no longer just representing the city. It becomes something you can play with.
So... Why music? Why lo-fi? Why???
At first, I did think about using real-world sounds. For example, green areas could trigger birds, water could play flowing sounds, and dense urban areas could have traffic or city noise. It felt more literal and directly connected to the map. But the more I thought about it, the more it felt too predictable. It turns the experience into something you already know.
I chose lo-fi music instead because it creates a different kind of connection. It feels calm and slightly abstract. Small changes are still noticeable, and MOST IMPORTANTLY it doesn’t require perfect precision. 😉 Even if the system is slightly messy, it still sounds intentional. You’re not just hearing the city, you’re interpreting it. That made it a good fit for an experimental interaction.
At first, I did think about using real-world sounds. For example, green areas could trigger birds, water could play flowing sounds, and dense urban areas could have traffic or city noise. It felt more literal and directly connected to the map. But the more I thought about it, the more it felt too predictable. It turns the experience into something you already know.
I chose lo-fi music instead because it creates a different kind of connection. It feels calm and slightly abstract. Small changes are still noticeable, and MOST IMPORTANTLY it doesn’t require perfect precision. 😉 Even if the system is slightly messy, it still sounds intentional. You’re not just hearing the city, you’re interpreting it. That made it a good fit for an experimental interaction.
Interaction Layers
(Layer 1) From Color to Sound: Color Classification and Sound Mapping
To translate the map into sound, I created a simple color-based system.
Each color on the map represents a different type of environment:
To translate the map into sound, I created a simple color-based system.
Each color on the map represents a different type of environment:
roads, buildings, and dense city areas
Controls: drums, bass, rhythm
parks, forests, and natural spaces
Controls: warmth, texture, chord softness
Controls: warmth, texture, chord softness
rivers, lakes, and water areas
Controls: reverb, delay, stereo width
(Layer 2) Gesture as Control: X and Y Gesture Logic
Movement is tracked across two directions: X and Y. The system reads the position of the hand and converts it into values like HAND_X and HAND_Y. These values are then used to influence the sound.
Movement is tracked across two directions: X and Y. The system reads the position of the hand and converts it into values like HAND_X and HAND_Y. These values are then used to influence the sound.
The X-axis (horizontal movement) mainly affects the overall level of the sound, such as volume.
The Y-axis (vertical movement) affects the energy and intensity, like density or activity.
Process: Trying, Testing, Adjusting
Spoiler alert: it is a series of back-and-forth steps
I started with a customized map from Snazzy Maps.
Instead of using live data at first, I exported a styled map as a PNG and used it as a base to test the system. This made it easier to control everything.
I tried different color combinations and adjusted the map multiple times. At the beginning, the system couldn’t read the colors properly. Some areas were too similar, and the values were not clear enough.
So I pushed the contrast and simplified the palette. After a few iterations, I finally got a version where the system could clearly recognize the color groups. ✌️
So I pushed the contrast and simplified the palette. After a few iterations, I finally got a version where the system could clearly recognize the color groups. ✌️
Once the color system was working, I started defining how colors translate into logic. Each color range becomes a parameter, and these parameters control the sound, such as tone, rhythm, and intensity.
Later, I also added waveforms to reflect the output in real time. This makes it easier to see how the system reacts. Together, they create a clearer feedback loop between input and output.
Later, I also added waveforms to reflect the output in real time. This makes it easier to see how the system reacts. Together, they create a clearer feedback loop between input and output.
Something Unexpected:
But while testing, I noticed something unexpected. The pixels were constantly changing and reacting in real time. It felt quite alive and dynamic. So instead of keeping it as a hidden debug layer, I decided to keep it and turn it into part of the UI. Now it helps users see what the system is “reading” and how their movement affects it.
Implementing Google Maps (failed attempt):
After testing with my own customized PNG map, I tried to bring the project closer to a “real map”.
I used the same Snazzy Maps styling and applied it through the Google Maps API inside the project. The idea was to keep the same color system, but use real map data instead of a static image, and combine real geography with my sound system.
But... this is where things started to break:
After testing with my own customized PNG map, I tried to bring the project closer to a “real map”.
I used the same Snazzy Maps styling and applied it through the Google Maps API inside the project. The idea was to keep the same color system, but use real map data instead of a static image, and combine real geography with my sound system.
But... this is where things started to break:
Visually, it looked correct, but technically it didn’t work as expected.
The map is rendered in tiles and changes depending on zoom and resolution. Because of this, the colors are not consistent at the pixel level.
So even if an area looks like one color, the system reads many slightly different values. This made the color detection unstable, and the sound output unpredictable in a way that felt wrong.
The map is rendered in tiles and changes depending on zoom and resolution. Because of this, the colors are not consistent at the pixel level.
So even if an area looks like one color, the system reads many slightly different values. This made the color detection unstable, and the sound output unpredictable in a way that felt wrong.
In the end, I chose to drop it for now. I went back to using my own PNG map, where I had full control over the colors and structure. It was simpler, but much more stable and expressive.
Ongoing iteration:
After dropping Google Maps, I went back to refine the system:
Water and Green were already quite stable, but I felt Urban could be more detailed. I added different weights to make the system feel more layered and alive:
After dropping Google Maps, I went back to refine the system:
Water and Green were already quite stable, but I felt Urban could be more detailed. I added different weights to make the system feel more layered and alive:
At the same time, I began experimenting with another interaction layer.
I added gesture control using hand tracking, allowing the user to shape the system through movement. Instead of triggering fixed actions, the interaction continuously influences the sound and visuals. This made the system feel more dynamic and responsive, and brought the idea of “drifting” more to life.
I added gesture control using hand tracking, allowing the user to shape the system through movement. Instead of triggering fixed actions, the interaction continuously influences the sound and visuals. This made the system feel more dynamic and responsive, and brought the idea of “drifting” more to life.
Other Improvements:
I then moved to a mobile version, adding gesture controls like drag and pinch.
I also made a few small adjustments, such as refining the layout, simplifying the UI, and improving how the system responds to interaction.
I then moved to a mobile version, adding gesture controls like drag and pinch.
I also made a few small adjustments, such as refining the layout, simplifying the UI, and improving how the system responds to interaction.
This project did not go the way I first imagined, but that is what made it valuable for me.
I started with a very clear idea: using real maps and turning them into sound. I thought working with real data would make the project stronger. But after many attempts, I realized it was limiting me instead. Letting go of that idea was not easy, but it helped me focus on what I actually cared about, which is interaction and feeling.
Throughout the process, I was constantly trying, testing, and adjusting. Many things did not work at first, but each problem helped me understand the system better. Instead of seeing them as failures, I started to treat them as part of the process.
Working with AI also changed how I approach design. It made things faster, but it also forced me to be clearer in my thinking. I had to learn how to guide it, how to break problems into smaller parts, and how to decide what to keep or remove.
This project helped me feel more comfortable working without a fixed plan. I learned to trust the process, to explore more freely, and to make decisions along the way. It reminded me that design is not only about the final result, but also about how you get there.