re-blogged from Digital Arts
One of this year’s up-and-coming creative trends is ‘experiential’ design. It’s an umbrella term for innovative projects that encompass not just visuals, but also sound, touch and even smell – all driven by real-time feedback generated by the actions of viewers and the world around them. Michael Burns explores how you get start creating your own experiences.
You can think of it as interactive design that has outgrown the computer or cyberspace and entered the real world, where the input driving the experience can be anything from the reading on a temperature gauge to the movement of the ‘participant’ in front of a Microsoft Kinect. Meanwhile the output can range from projecting responsive visuals onto an object to triggering a camera to capture shots at exactly the right moment. Here’s some recent examples of the wide variety of projects this field encompasses from Leviathan’s groundbreaking tour visuals for Amon Tobin to a hanging garden installation.
Experiential design is an exciting field that’s still in its infancy and one that is, in principle, relatively easy to get into using a mix of inexpensive hardware and free software (alongside standard creative software). “Projects using devices like the Arduino tend to lead to unique, engaging and highly visually creative [work],” says Adrian Campbell of Belfast studio The Design Zoo (thedesignzoo.co.uk). “This is due to the fact that [such work] allows a deeper level of engagement with the viewer and also requires you to keep the engagement levels high through your visuals, animations and interactions.
“For us, the Arduino opened up a brand new medium for our designs to be shown on, and in a uniquely interactive way,” he adds.
That said, projects aren’t confined to producing rich interactive experiences for attendees at a live event (or users on the web). Experiential work can also be a creative tool, enabling you to produce non-interactive art or designs that would otherwise be impossible.
What binds all experiential projects together is a keen interest on the part of designers to channel their inner geek to hack and adapt open-source hardware and software, all in the name of furthering the cause of design.
“There’s always been an undercurrent of experimentation and geekiness running through the digital-creative scene,” says Dave McDougall, head of digital at the London offices of cutting-edge agency Studio Output (studio-output.com). “I remember hacking one of the first Lego robotics projects many years ago and working with some quite cool technology then: in-store installations of sound-responsive projections and very early video social networking made by hacking together a heap of technology.
Perspective Lyrique by 1024 Architecture uses an old theatre in Lyon as a ‘screen’
“But what we’re now starting to see is this technology maturing and forming part of larger, more prominent campaigns. Clients and brands are really embracing it, and that’s where we’re seeing some great things happen.”
With clients excited by the technology and, more importantly, willing to commission projects that use it, it’s no wonder that experiential design is seeing such a surge in activity. One area where there have been some high-profile projects already is projection mapping – a hybrid of live, VJ-like visuals and 3D modelling.
The central idea here is to sculpt video content to match the geometry of the surface onto which it’s being projected. VJ software such as the Mac-only VDMX (vidvox.net) and the newer MadMapper (madmapper.com) can be called upon to handle the visuals. Both applications are often used in conjunction with Quartz Composer, Apple’s visual programming environment for processing and rendering graphical data.
Chicago-based animation, VFX and motion design house Leviathan (lvthn.com) has been involved in several large-scale projection-mapping projects, ranging from marketing events for Dodge cars to dazzling visuals for musicians like Amon Tobin. Executive creative director Jason White feels there will be even more projects in this genre this year. “It’s fresh to so many because, if executed correctly, it captivates audiences and pushes the boundaries of what video can do.”
Projection mapping is only one of the multiplicity of possible toolsets within experiential design and – as with much in this field – you don’t have to be involved in ‘event’ productions to get in on the act.
“You can create a huge public installation that uses expensive, advanced technology or do amazing, innovative things running a project off a single Arduino kit and a network connection,” says Dave of Studio Output. “As the technology is relatively easy to learn and the entry costs are low, the only limit is your imagination.”
Dave says open-source software and toolkits such as Processing (processing.org) and Open Frameworks (openframeworks.cc) are “reasonably straightforward to pick up”, adding that as with Arduino, “there’s a massive and very open community of programmers, artists and creatively minded people sharing information about these technologies, so there’s always help and inspiration around the corner”.
VVVV (vvvv.org) is another open-source toolkit that creatives can make good use of. It’s “great fun” too, says one of its firm fans, Stuart Dearnaley, 3D artist and innovation developer at Manchester-based agency The Neighbourhood (the-neighbourhood.com). “It’s great for prototyping an idea before really getting down into the design.”
The Neighbourhood is still best known for motion design projects, such as the title sequence for BBC show The Magicians (learn how it was created in our After Effects tutorial). It was only a few months ago that the agency ventured into the world of interactivity.
“We started using a head-tracking set-up with a converted webcam to track an infrared source, then used that data to control a virtual camera,” Stuart says. “The virtual camera would mimic the viewer’s head movements, causing the image on screen to appear to [be] 3D.”
Stuart also managed to get the rig working using stereoscopic 3D, which he says greatly increased the depth perception of the ‘virtual window’. “As with the Christmas tree project, another of our internal R&D experiments, the goal was to discover and learn new techniques and apply another creative layer to make it more of a piece of art than a technical demo,” he says.
The Christmas Tree Project – aka the Cubic Light Tree (above) – was a 3D projection-mapping project which The Neighbourhood ran over the recent festive season. It used a stack of white boxes as a screen onto which the designers projected their animated creations.
“I used VVVV to map out the faces of the cubes and in real time we could test out different image resolutions, colours and so on,” Stuart explains. “It was a great help to be able to work like that – flexibility is key to any type of innovative project where the goal may not be completely defined at the start.”
The team is now working towards creating an interactive installation, using projection mapping to display details of an architectural development in a physical space. “We plan to use an iPad for the interactivity and control,” says Stuart. “It’s a great device, and we will no doubt be using it in a few of our future projects and experiments. There are already a few free apps available that we can use to get input data across into VVVV.”
At the opposite end of the hacking scale is Twine, a little green box from US startup Supermechanical (supermechanical.com/twine). This low-power module can connect to a Wi-Fi network and thence to the Internet, allowing you to control it using a simple web app. It also has sensors to monitor the world, and its connectivity allows it to feed output back to the world via email or Twitter. Having enjoyed a successful crowdfunding programme via Kickstarter (kickstarter.com), Twine boxes should start shipping in May.
Another box that’s generating a lot of interest is Berg’s Little Printer, which uses the creative agency’s forthcoming BergCloud platform (bergcloud.com) to send information ranging from to-dos to a breakdown of your last exercise session. The Little Printer is fun if limited in scope, but BergCloud has the potential to add interactivity to all manner of objects. However, Berg isn’t ready to discuss the platform yet, an oddity in a community where sharing how projects were created is inherent in being part of it.
Alongside boxes expressly designed for engineered projects, there’s a growing trend to incorporate consumer electronics products – some of which boast sophisticated interactive technology – into hacked projects. Leviathan has customised the Microsoft Xbox 360 Kinect controller for control and visualisation purposes in the Amon Tobin shows and at last year’s Leo Burnett Leovative innovation event.
These aren’t isolated examples – the design world is brimming with people exploiting Kinect’s capabilities. One wily programmer has created a tool to motion-capture animation from Kinect devices to drive real-time character animation in Autodesk’s MotionBuilder application.
Michael Burns
Read our informed and inspiring features as soon as they’re published — click here follow @digital_arts on Twitter.
The Bohemian Home Journal 