Manipulator | Simon Wiscombe

What is it?

Manipulator is a web-based interactive music video for the musician Ty Segall's title track to his album of the same name. It was a collaboration between myself and director/producer Matt Yoka.

When the video plays, the user can click anywhere on the video and change it. From the sky to what's on the television to Ty's brain, the elements in the frame can be 'manipulated' while Ty sings the song through.

It draws its collage aesthetic heavily from Bosch, Rauschenberg, Monty Python, and a bunch of magazines we cut up and scanned.

What I did

Role: Designer and Coder

Interaction Design
Art Direction & Graphic Design
Front-end Development
HTML/CSS/JS and Photoshop

The main team was myself and Matt. I handled everything related to the webpage, while Matt handled all filming. We collaborated heavily on ideation, testing, and asset production.

This project had a steep learning curve. This was my first time coding js into HTML5's canvas element, my first time doing live video processing, and it had to be completed within 3 weeks.

Project Details

Launch

We launched successfully (after a small delay) in early September, 2014.

Tools Used

Adobe Suite
HTML/CSS/JS
Video and Green Screen
Processing (for early prototypes)
Old Magazines and Scissors

Coverage and Awards

Recognition

Select Coverage

Bosch's The Garden of Earthly Delights

Research, Inspiration, and Early Ideas

Aesthetically, we were heavily influenced by collage the collage aesthetic in pop culture and art (such as Monty Python and Robert Rauschenberg). We drew heavily from Bosch's The Garden of Earthly Delights; we were inspired by just how much is conveyed in one paiting and how the visuals add to this almost overwhelming sense of activity.

Monty Python's Flying Circus Stop Motion Animations

We wanted to blend the aesthetic and tone of Bosch's video with the silliness by edgy combination of Monty Python's Flying Circus' animated transitions. If you listen to the song itself, it has a retro, low fi feel, which matches well with the aesthetic. We also wanted to give the feel of stop-motion without it actually being stop motion. The one last piece of the puzzle was this image that Matt found. It was just a bedroom, but it had a grainy texture and a retro feel that we wanted. We modified it heavily, but it's clearly the room in the first scene of the video.

The Low Fi, Retro Bedroom Image that Decided Our Aesthetic

For the interaction, we very literally interpreted the name of the track, "Manipulator", and this notion of manipulating a few collage scenes, while Ty sang throughout the song, was born.

Design iterations

We didn't have much time for design iterations on this project, but we ran tests using the technical solutions that doubled as playlists.

An early tech test

Matt took charge of doing some early concepts on paper while I tackled the main technical issues that would have killed the project. Namely, how do we do green screen? We did come up with a solution by doing pixel processing frame-by-frame on each video file as it was running. The code of which can be found here if you'd like to take a look at it.

Editing all the interactive elements together in one screen.

Once the tech worked, it was full on-production. While Matt filmed Ty Segall on a green-screen, I set about coding up the whole thing. Once we had a full run-through with green screen function working, we just piled on whatever content we could until launch.