Manipulator

A web-based interactive music video for Ty Segall.

Play it or watch a youtube playthrough
What is it?

Manipulator is a web-based interactive music video for the musician Ty Segall's title track to his album of the same name. It was a collaboration between myself and director/producer Matt Yoka.

When the video plays, the user can click anywhere on the video and change it. From the sky to what's on the television to Ty's brain, the elements in the frame can be 'manipulated' while Ty sings the song through.

It draws its collage aesthetic heavily from Bosch, Rauschenberg, Monty Python, and a bunch of magazines we cut up and scanned.

What I Did

Role: Designer and Coder

  • Interaction Design
  • Art Direction & Graphic Design
  • Front-end Development
  • HTML/CSS/JS and Photoshop

The main team was myself and Matt. I handled everything related to the webpage, while Matt handled all filming. We collaborated heavily on ideation, testing, and asset production.

REFLECTIONS & THOUGHTS

Main Challenges

Timing.
It was tight. Very tight. From start to launch was 4 weeks. It was extended by 2 but only a few days before launch, but by that time, due to a lot of crunch time and hiring someone to help with asset production, we were able to put it together.

Technical: Green-Screen and HTML5 Video.
At the time of creating the project, there was no way to have a video play on a website with transparency, so our original plan of simply keying out the green in Premier or After Effects wouldn't work. So we had to come up with a way to do it live, while the video was playing, in javascript. We figured a way, even if it was clunky.

Skills.
I didn’t know how to use javascript for web apps. I had used it for Unity3D and I knew some basic js and jquery for websites, but not for something of this scale. But you gotta learn what you gotta learn to get the job done.

What went right?

I'm pretty proud of this project. It was just a lot of fun, and Matt and I collaborated really well. This was my first major project that was worked on remotely; Matt and I knew each other beforehand but we never met in-person while working on Manipulator.

We got it launched despite crazy crunch times (Matt was asked to create a completely separate, more standard music video "just in case" this one didn't work), we were invited to speak at SxSW. And on top of that, I learned a bunch of new things, we got some good press, and it was a weird project in a great way.

What, well, didnt?

Technical issues were our biggest problem, and I underestimated the importance of optimizing things like slow servers, long load times, etc. They weren't things I knew how to do, and I should have at least asked a back-end expert how I might solve them, even if we didn't have the budget to hire someone.

I do believe that if we would have solved that issue, we would have gotten way more web engagement then we did.

And while we did launch on such a short timeline, it did take a toll on me. Working crunch for 4 weeks is a bad idea, no matter how amazing the project.

More Project Details

Launch

We launched successfully (after a small delay) in early September, 2014.

Tools Used
  • Adobe Photoshop
  • HTML/CSS/JS
  • Video and Green Screen
  • Processing (for early prototypes)
  • Old Magazines and Scissors

Project Code available on GitHub.

Recognition

Recognition

Select Coverage

Research, Inspiration, and Early Ideas
Bosch's The Garden of Earthly Delights

Aesthetically, we were heavily influenced by collage the collage aesthetic in pop culture and art (such as Monty Python and Robert Rauschenberg). We drew heavily from Bosch's The Garden of Earthly Delights; we were inspired by just how much is conveyed in one paiting and how the visuals add to this almost overwhelming sense of activity.

Monty Python's Flying Circus Stop Motion Animations

We wanted to blend the aesthetic and tone of Bosch's video with the silliness by edgy combination of Monty Python's Flying Circus' animated transitions. If you listen to the song itself, it has a retro, low fi feel, which matches well with the aesthetic. We also wanted to give the feel of stop-motion without it actually being stop motion. The one last piece of the puzzle was this image that Matt found. It was just a bedroom, but it had a grainy texture and a retro feel that we wanted. We modified it heavily, but it's clearly the room in the first scene of the video.

The Low Fi, Retro Bedroom Image that Decided Our Aesthetic

For the interaction, we very literally interpreted the name of the track, "Manipulator", and this notion of manipulating a few collage scenes, while Ty sang throughout the song, was born.

Design Iterations

We didn't have much time for design iterations on this project, but we ran tests using the technical solutions that doubled as playlists.

An early tech test

Matt took charge of doing some early concepts on paper while I tackled the main technical issues that would have killed the project. Namely, how do we do green screen? We did come up with a solution by doing pixel processing frame-by-frame on each video file as it was running. The code of which can be found here if you'd like to take a look at it.

Editing all the interactive elements together in one screen.

Once the tech worked, it was full on-production. While Matt filmed Ty Segall on a green-screen, I set about coding up the whole thing. Once we had a full run-through with green screen function working, we just piled on whatever content we could until launch.