Test Event #1

I ran a closed event at Slack Space, Colchester, on Friday the 22nd of February. Despite being ill and the weather being bitterly cold, I managed to complete some basic exercises in set up and content testing that have proved to be hugely useful.

The following still shows the basic set up – projector positioned as high as possible, large screen and stand-mounted webcam.


I couldn’t place the projector high enough to clear the capture field of the webcam which needs to sit around 1m above the ground – the projector light could be seen and was overpowering the capture image.

For testing purposes I mitigated this problem by rotating the webcam to point to the side of the projector’s field of vision. This was an effective quick fix but resulted in a certain amount of image distortion, which can be seen in the content stills further below.

The following diagram represents the ideal set up where the projector throw and webcam field of vision do not coincide, therefore the webcam can point directly back at the viewer as he/she faces the projector screen.


It can also be seen from the diagram that the viewer needs to kept back from the webcam by some distance to allow the webcam to capture an image of the viewer from head to foot. In practice this distance may be around 4 metres and is a significant factor in staging of the installation.

Moving on from staging – I next spent some time experimenting with the resolution and aspect ratio of the projected image, although was unable to get the resolution up to the target of 1080p due to not having a suitably long HDMI cable. In terms of aspect ratio, 4:3 seemed to work better than 16:9 and this is probably because the native aspect ratio of the webcam is also 4:3. Why am I trying to achieve 1080p? 1920 x 1080 is a common HD standard and is also the resolution of each individual projector used as part of the multi-projector Fusion gallery.

Performance is a major issue for this project – by this I mean computer performance and stability. I am going to need 3 individual computers to run the 3 final screens at Fusion and each one must be capable of processing a webcam input in real time and outputting the final HD image while all the time maintaining a fluidity of behaviour (ie not ‘freezing up’). My test machine at the moment is a mid-spec 2010 Mac Mini 2.4 GHz Intel Core Duo II with 8 Gig of RAM running OS x 10.8. Interestingly enough, using this machine I found that just running a live HD webcam input to 1080p resolution through the creative development host platform I am using (Quartz Composer) results in a framerate of circa 13fps. From a user experience perspective, I believe this framerate is acceptable in an installation environment but probably near to the lower limit of acceptability. Why is the framerate so low? This is due to sheer number of pixels being input by the webcam (1600×1200 = 1,920,000) being scaled up to a greater number of pixels (1920 x 1080 = 2,073,600) to be displayed every frame. The primary means of increasing framerate in this situation would be to decrease resolution but the entire premise of the Fusion gallery is that is is HD and therefore one of the major technical goals of this project will be to achieve stable performance at HD resolution.

Concerning actual content, I ran a range of test pieces to try out a number of principal techniques. Firstly, the full screen slit-scan techniques that I have previously used at lower resolutions proved to be unusable at HD resolution, or at least so restrictive that the final effect was poor. I am referring to the technique where the screen is divided into horizontal or vertical slices, each showing a slightly delayed portion of live video captured by the webcam. The greater the number of slices and larger the range of delay, the better the effect but the greater the demand on the host machine. I can rule this technique out straight away based on plummeting performance.

The next content type I tried was what I have referred to previously as the ‘history slice’ effect – where the live image is sampled at the halfway line and repeated as vertical or horizontal slices, each progressively delayed from  the next. The result was promising, particularly using horizontal slices running down from the centreline. In the following image you can see my 5-year-old son playing with this effect, the webcam can be seen at the foot of the image. The image is at an angle due to having to rotate the webcam to avoid the projector throw (as described above).


The effect is enhanced by the colourful street art backdrop, which happened to be in the room I was using for the test – certainly, another important consideration is what lies behind the viewer from the webcam point of view. Why does this idea work for me? I like the instantly responsive, playful aspect of the interaction.

Edge detection is another real time technique I have been exploring, and in the test this worked particularly well in conjunction with the Quartz Composer ‘video trails’ patch which normally creates a long-exposure effect. When used in conjunction with edge detection, the result is a more defined edge for static objects and less defined for moving objects. A viewer moving into shot looks like an apparition until he/she stands still for a few seconds after which point they are treated as a static object.


Why do I like this idea? I like the way the viewer is forced to remain still to make an appearance. I also like the ‘chalkboard’ aesthetic. One idea is to try to pre-sample the background and then subtract it from the image, thus just leaving the subject. I’m not sure how successful this will be, but certainly worth a try.

I tried out a few more techniques with varying success – face recognition and 3D height based on luminance to name 2 –  both have some potential but I’ll need to work up concrete examples to test more extensively.

Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s