GIANTSTEP is a renowned advertisement production company in Korea, and its studios are used mainly for producing advertisements and product launches. The company had a unique journey recently in setting up its XR Studio.
GIANTSTEP established its GX-LAB in 2016, started its R&D for Video Effects (VFX) using a real-time engine, and released ‘Vincent’ in 2019, a ‘Virtual human’ based on AI and Real-Time Engine technology. Choi Seong Ho, the chief technology officer of GIANTSTEP, states, “From this process, I became interested in the virtual space where ‘Virtual Human’ can work. We felt that the virtual space we focused on requires a space where real humans and ‘Virtual Humans’ can interact, which is different from gaming. traditionally used in the VFX industry. To implement and test our concept, we built our A.I-ONE Studio 1 in 2019. In the studio, we tested placing an actual human in a virtual space and, conversely, a space in which a “Virtual Human” exist simultaneously in real space and communicates with each other.”
“In this way, we verified the usability and validity of the real-time engine in the chroma key-style XR studio, and then decided in 2020 to make the LED-based XR studio, which we called the next-generation XR studio,” added Mr Choi.
GIANTSTEP’s XR studio features multiple high-resolution LG displays with a centre LED screen at 11,520 x 2,700 pixels, a top ceiling display at 1928 x1538 pixels, a bottom display at 2816 x 1408 pixels, and a right side display of 1920 x 1080 pixels. These displays power up with just one Analog Way Aquilon C+ image processor and one Analog Way Picturall Pro Mark II media player.
What were the challenges GIANTSTEP faced when setting up the XR Studio?
“To be honest, almost every process from design to operation was difficult,” says Mr Choi. “Let me break it into three specific areas – the studio design, installation and operation. We met up with several well-known LED manufacturers, did various test images with our camera, and compared their performance. This helped us with our studio design, as we wanted the optimal studio while simulating the camera’s angle of view and movement in full 3D. The most difficult point in the design stage was what criteria to use to select the LED. We formed a TFT to simulate various content creations and prepared for six months the items to be tested and to analyse the results. We tested various image qualities such as colour, brightness, colour reproducibility and sharpness the LED can express. We also measured the latency from real-time engine to LED data expression and found that LG Electronics’ LED was the most suitable for our studio.”
Though GIANTSTEP had decided on the LG LED display, difficulties still had to be overcome during the installation process. “During the LED installation process, even if a small module was slightly warped, there were problems in the image quality of the warped part of the camera screen. We also had to have proper plans in case of fire or blackout and cater for large-capacity power consumption and air conditioning for LED heat generation. We had to solve problems that are not required in a typical studio,” added Mr Choi.
Regarding the operational aspects, Mr Choi highlighted that in an XR Studio, the various systems, such as cameras, lighting and sound, as well as real-time engines, rendering systems, media control equipment, camera tracking systems, and network systems, had to act as one solution.
“Among them, the most difficult thing was to render dozens of high-resolution contents with a real-time engine and synchronise them as if it were a single screen. Because it is such a complex system, it required a lot of learning time for operators of various occupations such as the cameraman, system operators, content creation artists, and programmers to run the studio.”
How did GIANTSTEP overcome these challenges?
Mr Choi highlights that various issues during the installation process were discussed closely with LG Electronics, where ideas were exchanged, which resulted in the hardware and software of the LED products being modified and even the installation method changed. GIANTSTEP also constantly re-photographed with the camera and shared the results with LG for better outcomes. Though it took longer than usual to set up the studio, the CTO felt that GIANTSTEP got the best quality and gained a lot of experience from the process.
Kim Dong Pil, vice president of LG Asia, B2B Business, comments, “LG Electronics is extremely pleased to collaborate with GIANTSTEP. We offer differentiated value by providing dedicated ecosystem solutions such as cameras, media servers, camera tracking and rendering software. As cameras can be unforgiving in production, it is of utmost importance that the LED panels are of high quality. In addition, the processors driving the LED panel and the media servers used for real-time content need to be of very high standards. Viewers should not realise that the production was shot in an XR Studio when the end product is broadcast. That is the ultimate goal, and I believe we have helped GIANTSTEP achieve that goal.”
During the operation, the rendering results in the real-time engine were simultaneously output to about 10 LEDs and had to be perfectly synchronised. “We performed tests such as maximum resolution, transmitting and receiving speed, supported codecs, and data consistency on multiple LED screens with products from various companies expressing rendering data on a large screen. In the end, Analog Way showed the best performance in all these aspects,”
Mr Choi adds that Analog Way’s technical support and cooperation completely solved the complex problem where output quality had to be perfectly synchronised while supporting 4K resolution.
Guy De Souza, vice president of Asia, Analog Way, adds, “When we were given the chance to work with GIANTSTEP on an XR studio in Korea, we immediately looked at it as an opportunity to help them with the challenges they faced by doing what we do best which is managing several high-resolution screens as a single canvas. Now in a broadcast setting like XR, there are many other factors to consider as well, such as the amount of latency incurred when transmitting large amounts of high-resolution videos to the actual rendering of these videos onto the screens. We are glad to know that all the tests performed on our equipment by GIANTSTEP met their high expectations.”
GIANTSTEP’s XR Integrated controller
The XR integrated controller is a solution that GIANTSTEP created for use in their studio environment. The controller was recently named ‘Helios’, the sun god from Greek mythology, to use it specifically for the XR Studio.
Mr Choi comments, “The XR Studio is much more complex than a normal studio and all the
equipment, software, and sensors must move like a single creature. However, since the manufacturer of each gear is different and the purpose of the product is slightly different, controlling each of them independently is inefficient. For example, suppose you need to set the real-time engine settings in various places, such as the LED wall controller, media server, video processor, and camera tracker. In that case, you must move the data one by one to each system. So, we thought that it should be possible to control the entire data flow from the camera to the real-time engine in one system.
Another reason we tried to develop our own without using other XR solutions is that we felt that the existing solutions were inconvenient and had many limitations. Since most solutions are configured in such a way that various functions are integrated by embedding Unreal Engine in their own solution, the solution engine is in the old version even if the real-time engine is updated or the functions supported by the real-time engine are not supported in the solution.
With the help of LG Electronics and Analog Way, the ‘Helios’ project, which we are currently developing, has completed the primary development to integrate all functions of the real-time engine, video controller, media server, and LED wall controller to enable control.
We are currently verifying usability and performance in A.I-ONE Studio 2. We will continue to upgrade the performance and necessary functions while using it ourselves.”
What benefits has the XR Studio brought for GIANTSTEP?
In terms of business, besides commercial advertising, where GIANTSTEP has a competitive edge, a new business area of broadcasting and entertainment content production has emerged, which GIANTSTEP is able to take advantage of with its XR Studio. “The benefit of the XR Studio is the ability to create life-like backgrounds and transport the actors and viewers to another space. Creativity and imagination is the only limitation. The XR Studio is also an enabler when cost, travel and even health-related constraints restrict a production as we are able to recreate whatever scene is necessary without anyone feeling it was shot in an XR studio,” added Mr Choi.
Where is the XR Studio heading?
“I would like to discuss two points. First, the LED-based XR technology is not a finished technology. It is still a developing technology, so it cannot be said that any product or technology is standard. XR Studio is a comprehensive set of technologies that must harmonise. The point is that although the LED is a large and good product, XR content is not created correctly just by using a famous solution. System engineers, Programmers, 3D artists, Cinematographers, and Lighting experts can only create content when they cooperate.
Second, although XR Studio makes many things possible that were previously impossible, it is not a silver bullet that solves all problems. In some cases, it may be more effective to use chroma key XR than LED XR, considering the cost. Alternatively, it may be more effective to do post-production in the traditional way of the past or post-production with VFX. Of course, XR can be an effective alternative if a large-scale filming team is filming overseas or if a large-scale set is needed. However, it is not effective in all cases.
I think that XR technology will probably provide a high-quality virtual space indistinguishable from reality. This is because the underlying real-time engines, LEDs, video processors, and camera trackers are rapidly evolving. However, the creators who can create a realistic virtual space indistinguishable from reality will have a competitive advantage.”
If you missed Part One you can read it HERE.
You can watch the Video Interview HERE.
