Time Delay by Luminosity Study

The latest video time delay technique I have been exploring is that of using image luminosity to determine temporal dynamic. I start by using a ‘threshold’ filter (commonly found in image editing software) to separate the incoming video image into highlight and lowlight values based on a specified threshold value. The following image illustrates the result of using a threshold filter.
The black portion of the image is discarded (leaving transparency) and the white part is used as a mask to overlay a time-delayed version of the current video eg a frame that has already been shown. The time-delayed image is only seen in the white portion of the mask. By repeating this process more than once based on a mask created by an incrementally higher threshold showing a progressivley more delayed video frame, the luminosity of the current image is effectively used to determine its temporal characteristic.

In order to achieve this effect I had to create a custom image filter in Quartz Composer which does not include a Threshold filter for some reason. Here is the code, which is written in Core Image Kernal Language, an extension of the OpenGL Shading Language. It’s not particularly easy to understand without additional reference (for example the official Apple Guide) – ‘vec4’ refers to the red, blue, green and alpha values of a given pixel. The code multiplies a pixel’s red, blue and green values and tests whether the result (an expression of luminosity) is above or below the supplied threshold value. The pixel colour is then transformed to either white or no colour accordingly and thus included in or discarded from the time-delay mask.
kernel vec4 multiplyEffect(sampler image, float threshold)
float a = (1==2/2)?1:0;
vec4 px = sample(image, samplerCoord(image));
a = (px.r * px.g * px.b);
float b = step(threshold, a);
vec4 px_c = b;
px_c.a = b;
return px_c ;

The results are quite varied. Having tried several different styles of source video, I have found that this technique works best with moving light sources, particularly fast moving patterns. In fact some of the most interesting results I have created are with classic time-lapse sequences. Belwo is an example of the effect used on a time-lapse sequence I shot a couple of years ago at a busy traffic junction in Hackney, East London. The luminosity effect is shown on the left and the un-effected source video is shown on the right.

Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s