Thursday, April 5, 2012

Digital Media Interaction with MS's Kinect

Photo of "The Treachery of Sanctuary" on digital media interaction with MS's Kinect
Above is a photo of Chris Milk's installation at The Creators Project show in San Francisco titled "The Treachery of Sanctuary," held a few weekends ago. You can read more about the synopsis here:

It uses light projection to create a silhouette which is then read by a Kinect sensor from Microsoft's XBox 360 gaming platform (Kinect now commonly available as an open source developer tool, read about it here.) and allows the viewer/performer to participate with digital imagery of varying bird scenes. This bares uncanny resemblance to my following sketch, dating back 2 years ago in a post where I posit (digital) Interaction Design as a physical and spatial field, imagining a projection / silhouette / projection combination allowing users to interact with the digital imagery in a literal, physical manner.

That post is found here:

Ironically, I didn't pursue this installation upon the advent of the Kinect, having originally conceived the use of a simple webcam. Thus I am delighted that very technology was used in the development in this interactive, silhouette-morphing piece for The Creators Project SF visit. Those squiggles in the sketch are imagined as birds flying onto the human figure and transforming the image into (other), able to be 'shaken off' by the participant. Very similar to the second panel of Milk's installation where birds, albeit a tad violently, pick apart their user's figure. As a collaboratively funded project by VICE and Intel, there was no shortage of programming knowledge resource, for sure.

But this is not the end-all to the Interaction Design-into-physicality experiment. Continuing on with the human obsession to occupy our virtual environment as a Tron Grid-like space (definition), we can envision launching from past virtual reality environments, bending our 2-D projection screen into a tunnel, thus expanding into three dimensionality, and introducing the actual human as a participatory character in the image itself, not just their silhouette.

University of Michigan's MIDEN used in visualizing digital 3d environments

To do our due diligence, it is worth understanding the current state of CAVE technology, an immersive room    where a stereoscopic 3d jiter is projected making 'the walls disappear' and a digital spatial environment come alive to a single user with a game pad for navigating and glasses for seeing. Not what I'm proposing at  all, as this environment expresses the extreme interiority fascination, either a hyper-Banham-like complex or complete the opposite. Alas, understanding is the key to differentiating, so learn more about CAVE technology, and Michigan's MIDEN here:

What I propose is not an abstraction, much like I see Milk's installment as not a representation of a space that doesn't exist. Instead these objects, with people and digital projection, are the thing itself, interacting in a sort of unscripted performance art. The characters are one or many humans, one or many projected images, and one or many projected silhouettes of the human characters, all interacting, all existing, and all being realized at once, in real time. What comes next is some representation of this cyber/real life - mimetic device. Stay tuned for that! This will be a multi-part series.