There are now 2 weeks to go before a key test session at the exhibition venue and just over 5 weeks before the exhibition itself is due to open.
I have 2 primary test candidates which I will refer to as ‘Neon Dance’ and ‘Outline Sparks’. Neon Dance relies on the ‘luminance difference’ technique I have developed (and detailed here) to detect areas of changing image which are interpreted as viewer presence. The Quartz Composer Carasuelo plug-in is then used to create a shape around the detected presence which is further manipulated to create a series of ‘neon outlines’.
Outline Sparks uses a simpler outline detection routine that renders to monochrome. A frame combine effect makes the image appear more solid when there is no movement and more ghost-like when there is. The Optical Flow plug-in is used to generate ‘sparks’ that emit from areas of movement and appear to flow in the direction of that movement.
I am currently in the process of using these 2 test pieces to address a number of critical technical and operational elements of the piece with a view to minimising the risk of failure during the gallery test session. These elements are –
During much of the creative development process I have made use of the development machine’s built-in webcam – an iMac iSight capturing at a resolution of 1280 x 1024 pixels. I have also purchased and tested a standalone Agent V6 webcam (resolution 1600 x 1200) which gives good results and critically a has mounting accessory which can receive a standard ½” camera / microphone mount, which I intend to use in the gallery space. The Agent also features a manual focus ring which I intend to use to set focal range to the ‘sweet spot’ i.e. the optimum distance from the webcam that the viewer should stand to see a full sized image of themselves at best quality.
One downside to the Agent is paradoxically its high resolution – the sheer processing power required to manipulate nearly 2 million pixels of incoming video every frame is a significant burden for the host machine. In the case of Neon Dance, framerates have been dropping down to below 10fps when using the Agent. This has worried me as I have always tried to maintain a minimum of around 15fps and have been careful to watch out for symptoms that the host machine is labouring excessively – i.e. I am attempting to preserve processing overhead.
I tried ‘downsampling’ the incoming video at source and this made some improvement but I came to the suspicion that even this operation was taking its toll on the processor and thus impacting framerate. In actual fact, Neon Dance relies on a massively downsampled video input in any case, so I wondered if a lower-resolution webcam might actually be better. Consequently I bought and tried a Neptune 640 x 480 webcam which did improve performance visibly – Neon Dance immediately became more responsive to movement. But, I quickly discovered that as I moved away from the Neptune webcam towards the sweet spot (of around 4-5 metres with this webcam), Neon Dance ceased to pick up my outline. The Neptune webcam is incapable of delivering the necessary definition at the required range. Thus I now need to mitigate the slower performance of the Agent with the higher video capture quality required for the pieces to work at gallery distance.
Another concern is that the pieces I am working on will not run without fail for the 7 hours of gallery opening time. In order to evaluate robustness I have created a test presentation that alternates between the 2 candidate pieces every 10 seconds. I have successfully run the presentation for 3 hours without glitch which is a positive indicator. However, a full 7 hour test is certainly required particularly once I have the entire set-up including peripherals in place
Yet to do: the webcam furthest from the exhibition control room requires a 20 metre USB extension to deliver the video feed from the webcam to the host machine. This is a significant potential point of failure and must be tested asap. If it does become an issue, I will have to consider an alternative positioning of machines, peripherals and routing of cables. A 20m active USB extension is now on order.
Calibration in situ
All of the set-up testing to date has made me realise exactly how important calibration of the hardware and software will be in situ. Equally important will be optimisation of the gallery space itself particularly in terms of lighting characteristics ie which light sources need to be masked and where extra lights may need to be positioned.
I plan to run the audio on a separate machine which will act as a master – cueing the video machines using MIDI commands. I now need to develop a basic sound design for each of the test candidates and try this set-up out during the gallery test.