A web-based interactive music video for Ty Segall.
👾 play it >
📽 watch a playthrough >
💻 project code on github >
What is it?
Manipulator is a web-based interactive music video for the musician Ty Segall's title track to his album of the same name. It was a collaboration between myself and director/producer Matt Yoka.
When the video plays, the user can click anywhere on the video and change it. From the sky to what's on the television to Ty's brain, the elements in the frame can be 'manipulated' while Ty sings the song through.
It draws its collage aesthetic heavily from Bosch, Rauschenberg, Monty Python, and a bunch of magazines we cut up and scanned.
What I did
Role: Designer and Coder
- Interaction Design
- Art Direction & Graphic Design
- Front-end Development
- HTML/CSS/JS and Photoshop
The main team was myself and Matt. I handled everything related to the webpage, while Matt handled all filming. We collaborated heavily on ideation, testing, and asset production.
This project had a steep learning curve. This was my first time coding js into HTML5's canvas element, my first time doing live video processing, and it had to be completed within 3 weeks.
We launched successfully (after a small delay) in early September, 2014.
- Adobe Suite
- Video and Green Screen
- Processing (for early prototypes)
- Old Magazines and Scissors
Research, Inspiration, and Early Ideas
Aesthetically, we were heavily influenced by collage the collage aesthetic in pop culture and art (such as Monty Python and Robert Rauschenberg). We drew heavily from Bosch's The Garden of Earthly Delights; we were inspired by just how much is conveyed in one paiting and how the visuals add to this almost overwhelming sense of activity.
We wanted to blend the aesthetic and tone of Bosch's video with the silliness by edgy combination of Monty Python's Flying Circus' animated transitions. If you listen to the song itself, it has a retro, low fi feel, which matches well with the aesthetic. We also wanted to give the feel of stop-motion without it actually being stop motion. The one last piece of the puzzle was this image that Matt found. It was just a bedroom, but it had a grainy texture and a retro feel that we wanted. We modified it heavily, but it's clearly the room in the first scene of the video.
For the interaction, we very literally interpreted the name of the track, "Manipulator", and this notion of manipulating a few collage scenes, while Ty sang throughout the song, was born.
We didn't have much time for design iterations on this project, but we ran tests using the technical solutions that doubled as playlists.
Matt took charge of doing some early concepts on paper while I tackled the main technical issues that would have killed the project. Namely, how do we do green screen? We did come up with a solution by doing pixel processing frame-by-frame on each video file as it was running. The code of which can be found here if you'd like to take a look at it.
Once the tech worked, it was full on-production. While Matt filmed Ty Segall on a green-screen, I set about coding up the whole thing. Once we had a full run-through with green screen function working, we just piled on whatever content we could until launch.