Back in 2014, when Japanese media artist Nobumichi Asai‘s real-time face-mapping video Omote hit the internet, it quickly went viral. At over 7 million views and counting, it features a model’s face transforming into a canvas for a series of surreal and captivating state of the art animated masks, all done through light projection. Morphing from cyborg to isometric patterns to a reflected surface that looked like liquid metal, all expertly mapped to the contours of the model’s face.
Now Asai and his team at WOW have created their newest face-mapping project. It’s another groundbreaking work, again using real-time face tracking and projection mapping, but this time bringing in an advanced 1,000 fps projector and ultra high speed sensing.
The project is called Inori-Prayer and is a collaboration with the Ishikawa Watanabe Laboratory at the University of Tokyo and the dancing duo AyaBambi. AyaBambi are used as the canvases for this piece. Set to music the pair perform a choreographed dance routine while their faces begin to stream with black tears, turn into skulls, then freakishly grinning faces inspired by Noh masks, along with a variety of other designs and patterns.
Asai notes that lots of the time when it comes to face-mapping, as you might expect, one of the biggest issues is the latency that comes from the equipment. “It’s a careful balancing act between the performers choreography and the limitations of the current technology,” explains Asai to Creators. “Initially we had to find the right speed for the movement of the performance that would allow AyaBambi to dance somewhat freely while still being able to track their movements.”
The projector, developed by Ishikawa Watanabe Laboratory, was crucial. “The result is very dramatic, [so] projections can follow much higher speed movements with greater accuracy allowing for more artistic expression,” notes Asai. The team also adopted a new approach for the projection designs themselves: a projector that doesn’t produce color. Instead of projecting “makeup” as it were, Asai notes, they played with light and shadow. Sound familiar? He says it’s a makeup concept and technique known as shading, which allows you to sculpt the face using shadows.
“In our performance, the expressions of the skulls and the Noh masks are created by adding shadows to change the shape of the face,” the artist explains. “‘White’ is treated not as ‘white,’ but rather ‘light,’ so the real color of the performers’ skin is expressed, and the result is a look that doesn’t feel monochrome.”
Asai worked as creative and technical director. CG director Shingo Abe from WOW completed the visual production and programming. Aya Sato was the choreographer, while the Ishikawa Watanabe Laboratory made the project into a video.
For the Noh masks Abe mixed the traditional Japanese designs with some stylings from Venetian masks too. “Accompanied by the overwhelming performance of AyaBambi, a visual synchronization of black tears, skulls, faces being severed, Noh Masks of agony and the Heart Sutra have sublimated into a single piece of work,” says Asai. The visuals were inspired by the music, from which Asai took on the theme of ‘radioactive,’ hence the skulls. “Radioactive wields destructive power, and from that brings ‘death,’ ‘suffering,’ and ‘sadness,’ then the ‘opportunity’ to overcome those things.”
The result, using such a high speed projector combined with the high speed sensor, means the visuals are indistinguishable from the faces. There’s no discernable lag, and it also means every second counts. The making-of video notes that it took three months of trial and error to reduce the tracking speed just a few milliseconds. But it made all the difference.
“The human face has always been a very effective form of communicating and from an artist’s perspective it is interesting to be able to change and alter this to create a new dialogue.” says Asai. “Humans have always found ways to alter their image in one way or another, the desire [to] look and feel different is nothing new. Face-mapping and future technologies are an extension of this idea and one that could allow humans to communicate in new ways, previously seen as impossible. This is very exciting place to be working in from an artist’s perspective.”
Check out the making-of:
Find out more about the work of Nobumichi Asai at his website here. Find out more about WOW’s work here. You can see more of AyaBambi’s performances at their website and keep up with the work of Ishikawa Watanabe Laboratory here.
