Immersive Projection Techniques: 360° Projection Mapping and Real-Time Tracking

Immersive. If you are an event professional or someone who is indirectly involved in creating events, or you interested in technology, it is a buzzword that you have surely encountered countless of times over the past year. We are going to explore how immersive technology elevates projection mapping to new heights, and give you specific examples into the products that make it all possible.

With immersive technology you can deliver a unique experience that creates the illusion of blending the physical world around us with virtual reality. This can happen through sound and video technology, for instance – and in this particular case, through projection mapping. Immersive projection has become increasingly popular, and it is no wonder: its ability to transform any space simply by providing the audience with incredibly realistic sound and imagery and transporting them to a completely different atmosphere and location – be it the depths of the ocean, a medieval cathedral, the middle of a sports stadium, or even inside the Sydney Opera House – rivals only that of highly advanced LED technology.

In our previous article we illustrated with a few examples how immersive projection mapping can be done, and in this post we are going to dive deeper into two very specific types of this technology: 360°projection mapping, and real-time projection tracking, both of which work with elements of mixed reality in order to create a life-like illusion without having to use a green screen or LED.

Immersive projection mapping

Before we go any further, let us help define what we mean by immersive technology: it is any kind of technology that extends reality, or creates a new  is about: any technology that merges the physical reality with a digital or simulated one. The main difference between experiences created on a device with augmented reality (AR) or virtual reality (VR), and immersive technology that uses elements of mixed reality (AR+VR) is that the “magic” is no longer restricted to a phone or a headset of a user, it exists in the physical world. Combined with immersive sound, the experience is able to transport the audience into any reality, place, or time.

Going beyond the scope of classic video mapping, immersive projection utilizes any and all surface types in 360° to take the audience on a journey through a spellbinding experience. It also encompasses 360° projection, real-time tracking that enables you to project on fast-moving objects, and much more. The use of these immersive technologies is on the rise, and recently they have made their way into interactive exhibitions, for instance, where museum goers have been subjected to mind-blowing experiences in immersive environments.

In this article we are going to explore in more detail the two above-mentioned immersive projection mapping technologies: 360°projection and real-time projection tracking, both of which are designed to create a visual illusion that is unlike any other.

360° projection mapping

What is 360° projection mapping?

This technology utilizes all surfaces within a given space, projecting images on either walls or three-dimensional objects that will thus be transformed into interactive displays. Mapping these surfaces is a complex feat that involves projection from all directions by a large number of projectors in order to transfer seamless images onto the surroundings. Creating the immersive, HD content happens with 360° camera systems and multiple camera rigs. This type of projection mapping technology can be used on the walls of buildings, industrial spaces, interior walls, theatrical backdrops, etc. 

Beyond the cameras and the projectors a specific, high-precision software, such as disguise‘s OmniCal, is also necessary to create the 360°projection mapping. The software spatially maps the object or surface and then digitally recreates the environment in which the projection will happen.

360° projection mapping can serve great many purposes, as its impact on audiences is profound and long-lasting, and that kind of experience is immeasurably valuable for any brand. With cultural institutions, global associations, non-profits and multinational companies all implementing immersive technology to wow their target audiences, 360° mapping is becoming the new trend in this post-pandemic landscape. And rightfully so: with travel still being restricted to varying degrees, people can be easily transported to any part of the world without having to jump on a plane, from the rain forests of the Amazon to the majestic palaces of Kyoto, or even the depths of the Mediterranean sea – all in massive spaces, surrounded by moving, living-breathing 360° images.

Barco presents immersive projection mapping for museums and venues

How can it be used?

360° projection mapping has proven to be a useful and popular “edutainment” – entertainment and education – tool in corporate and social events, as well as in the workplace for the purpose of employee training and development. It has also gained momentum in the recent years with a growing number of schools, colleges, and universities incorporating this technology in their education and L&D toolkit, for example introducing mixed reality elements with the VRLS (Virtual Reality Learning Space).  

Planetariums, theaters, exhibitions, and museums are also utilizing 360°projection mapping, and the number of such immersive exhibitions has been continuously on the rise. For example: the global Van Gogh: The Immersive Experience, the Once Upon a Time, van Gogh, Monet, Renoir… exhibition in Prague (supported by our local member company AV Media Events), or the Gustave Klimt: The Immersive Experience, one of the highlight projects delivered by our technology partner Barco.

Our AV Alliance members have also been using 360° projection mapping for various events in the recent years. One of the best examples for this type of immersive projection is our French member VIDELIO Events‘ projects called JAM Capsule, in collaboration with their long-standing partner Barco. This immersive mapping project involved the 360° projection of over 2,500 m2 of floor and wall surfaces with ten Barco UDX 40,000 lumen video projectors.

Projection mapping on a new level: amazing visual effects provided in real time on fast moving objects by Panasonic’s High Speed Tracking technology

Real-time tracking and projection mapping

It sounds exciting for sure, but what is real-time tracking, you may ask? Well, what it does is that it detects the positions and movements of the target object that is in motion, even at high speed, and then project content onto or around them. The target object can be a fast-moving car, a dancer in motion, or even a background that interacts with the movements of the object. This emerging technology is a game changer in how the presenter or performer, their audience, and the environment all interact and engage with each other, and it delivers a showstopping creative element to any event. Until now this technology was almost exclusively used in concerts of the biggest international bands and performers, or during half-time shows at the Super Bowl, but now it has become part of the offering of select manufacturers that are used in corporate events as well.

Those who are familiar with the technical aspects of this topic will find it interesting that this interactive technology, which is also known as dynamic projection mapping, works with very low latency, meaning minimal delay (lag) between detecting an object’s position and transmitting images. The lower the latency, the smoother the projection mapping will look like. Right now the lowest latency used in professional projection, which at an incredible 0.0016 seconds is just 1/10 of regular devices, can be achieved by Panasonic’s real-time tracking and projection mapping technology. Of course, real-time tracking is only achievable with a compatible projector, but these days most manufacturers have such devices available in their portfolio.

Panasonic’s high-speed Real-Time Tracking and Projection Mapping System put to the test on a dancer

Examples for real-time tracking

So what type of events can this stunning real-time technology be used for? The simple answer is: any and all. The possibilities are vast for sporting events, opening ceremonies, award shows, live entertainment, trade fairs, and yes, even corporate events! Real-time tracking for spatial rendition (uniform projection of video onto different surfaces) offers creative solutions for any situation and venue where projection is an option. With this technology you can take video mapping to a whole new level by projecting images onto, say, a race car in motion on the track, a fast-moving ball on a court or field, a group of dancers on stage, an aerial artist mid-performance, or virtually anything you have your mind set on. With immersive experiences on the rise, more and more corporate clients, artists, and venues are looking for options that make them stand out from the crowd in terms of technology, and real-time tracking for projection mapping is fast gaining popularity.

With the postponed Tokyo Olympics set to kick-off in less than a month, it is worth mentioning the stunning real-time tracking project that Panasonic delivered during the “1 Year to Go” ceremony in July 2019 that kicked off what was supposed to be the one-year countdown to the Olympic and Paralympic Games 2020 (later delayed by the outbreak of the pandemic). Panasonic’s technology was used for a video performance that incorporated a the projection of colors and effects that followed a fast-moving dancer performing a dynamic routine with gymnastic elements. As an Official Worldwide Olympic Partner, Panasonic is no stranger to bringing mind-blowing projection mapping elements to the table, having also impressed global audiences, for instance, during the Opening Ceremony of the Olympic Games in Rio in 2016.


Real-Time Tracking Software Development Kit by Panasonic

Panasonic’s solution for real-time tracking in projection mapping for the entertainment and event production industry makes it possible to project images in real-time, with pinpoint accuracy, onto fast-moving objects using a Real-Time Tracking Software Development Kit (ET-SWR10) for the Panasonic PT-RQ35K projector. In combination with an infrared emitter, camera, marker and a content server, the projector creates a Real-Time Tracking Projection Mapping System that is ideal for use by production companies and at live entertainment events, theme parks, museums and sporting events.

The kit enables the 30,500 lumen, 3-Chip DLP laser projector to accept and output 240Hz HD signals delivering enhanced images with a latency of just 5ms. The addition of dedicated software from the kit onto the content server enables images to be easily mapped to the moving object or performer using a single cable connection between the projector and content server (HDMI/Display Port).

To learn more about the technical aspects, the necessary equipment, and the related costs of projection mapping, we invite you to get in touch with us and find out what our AV Alliance members can do for you and your event!