3D / Time Delay Study

Having achieved a ‘base level’ slit-scan effect, I have been thinking about how I might create links between audio/MIDI input and the video time-delay effect. One idea is to create a ‘3D imprint’ in the moving image every time a particular audio/MIDI cue is given. The idea being that the image shown in a ‘hole’ created in the video is a delayed image slice that becomes less delayed as the hole shallows, eventually synchronising with the main image as the hole disappears. As with the slit-scan effect, this took me quite some time to create in Quartz Composer and I confess I almost gave up having used up many hours. I tried complex texture mapping onto bespoke mesh objects (the ‘proper’ way of handling 3D image mapping) but still could not get what I wanted. Finally I had a last stab at using Quartz Composer’s ‘simple 3D’ and with a little extra sleight of hand had a late-night break through. I guess I can put all the hours down to getting up to speed with Quartz Composer, not the most comprehensively documented composition platform, but certainly a very powerful tool in the realm of real-time video manipulation.

The 3D delay effect, a sample of which can be seen below, uses a MIDI trigger to create a square-shaped intrusion into the video playback surface. The mechanics are pretty much as described in the initial idea above. In the execution of the idea, I found that the video delay aspect doesn’t really work that well unless the ‘frame queue’ (ie the number of video frames to delay by) is kept very small, possibly between 5 and 10 frames rather than the 500 or so used in the previous slit-scan experiments. More than a short video delay in the surface of the intrusion only serves to confuse the overall image – it all seems too much to take in, especially when combined with the 3D effect.

While thinking about the exploration of time, I came up with the idea of using a simple Drum and Bass loop to run with the visuals. Drum and Bass seems appropriate, firstly for the sense of frenetic energy which seems to match that of the spider. Secondly – Drum and Bass is built on timestretching of slower drum loops to create fast rhythmic cut-ups, sometimes dropping to half speed where the timbre of the original loop is better revealed. This audio form of playing with time seemed like a good match for the video material.

The clip below runs in sync with the beats per minute of the audio loop, which drops from 170 to 85 twice. The 3D intrusions broadly match the sound of the kick drum heard in the initial loop, although I have simplified the pattern which was otherwise too confusing when visualised. I have also introduced some 3D rotation of the entire scene which helps to accentuate¬†intrusions depth. The position of each intrusion is semi-random but within a confined area. The fill on the left, right, top and bottom faces of each intrusion is a 1 pixel wide strip taken from the edge of the image placed on the back face. This technique produces an interesting elasticity effect between the ‘normal’ plane showing the current frame and the intruded surface showing the time delayed frame. In a sense, the study is as much about spatial displacement as it is about temporal shift.

I really like the result but am not sure it adequately accentuates the temporal phenomena I have sought to explore in the current module. So I shall try a few other ideas out before firming up a direction to develop for the final piece.

Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s