Tag: integrate
Immerse yourself in the Olympics this summer
Tokyo will see many firsts when it hosts the Games of the XXXII Olympiad. Not least, that the 2020 Games are being held a year later than planned and less than one year before the 2022 Winter Olympics in Beijing.
The worldwide pandemic also means that there will be no international visitors in Tokyo, but Olympic Broadcasting Services (OBS) will build on virtual-reality (VR) technology introduced in Rio in 2016 and PyeongChang’s 2018 Winter Olympics to make TV viewers feel a part of the Games from the comfort of their own homes.
OBS is the host broadcast organisation for the Olympic, Youth Olympic and Paralympic Games. It was created by the IOC in 2001. It provides broadcast content for use by all rights-holding broadcasters (e.g. the BBC, Eurosport) around the world, and it also helps broadcasters prepare for the Games; OBS oversees the fit-out of the International Broadcast Centre (IBC), which is the home of the broadcasting operations of the Games. It also prepares the compounds at each competition venue, where OBS and rights-holding broadcasters’ production and technical facilities are located and from where international television and radio signals will be produced.
OBS expects to produce 56 live feeds and nearly 9,500 hours of content during the 17-day event, which consists of 339 events across 50 sporting disciplines. It has a team of nearly 8,000 people from more than 70 countries, with specialities and skills in different sports broadcasting.
Tokyo will be the first Olympic Games to be natively produced in ultra-high definition (UHD) – or 4K – and high dynamic range (HDR). UHD content has a resolution of 3,840 x 2,160 pixels, four times the number of pixels of full HD, to give more detail. HDR technology improves the contrast between black and white pixels for an accurate picture with more colour shades. Coverage of previous Games has been done in parallel coverage with broadcasters but this year will be the first time the native broadcast coverage for the OBS world feed will be produced directly as UHD and HDR across all competitions and ceremonies of this year’s Games.
Until recently there was no universal standard for UHD and the application of HDR was not a foregone conclusion, says Yiannis Exarchos, chief executive officer of OBS. He places great store on HDR in particular. “It brings, especially in outdoor sport, a level of detail, both in colour and light rotation, which really makes the images considerably more realistic than you get in HD; it’s not just about resolution.”
OBS has developed a standard which derives the best possible HDR output out of the UHD solution, making it more efficient and sustainable because OBS does not have to double up broadcast resources.
The closing ceremony will be broadcast in 8K to make the most of what are traditionally vibrant spectacles. 8K doubles 4K’s number of pixels to 7,680 x 4,320 and is 16 times greater than HD. Other sports (athletics, gymnastics, judo, and some > < swimming events) will be available in 8K in Japan, although other broadcasters may pick up the feed and experiment with trial transmissions. There are already plans for 8K broadcast by Chinese broadcasters for the 2022 Winter Olympics in Beijing.
Another Tokyo first will be immersive audio. All venues will deliver an immersive audio feed as discrete channels in a 5.1.4 format. This is five microphones placed front left, right and centre and either side or behind for surround sound, with one dedicated bass channel and four speakers above the source of noise. There will be 85 separate 5.1.4 audio feeds available for national broadcasters.
Immersive audio brings a three-dimensional experience to viewers or listeners. Exarchos says: “I am a huge audio fan because I believe that audio, in some sense, is the carrier of emotions... so I am happy that for the first time, across all sports, we will do 5.1.4 immersive audio.” Countries such as Japan, China, the US, Brazil, and most of Europe (via Discovery and Eurosport channels) will be able to experience 5.1.4 audio transmission.
Bringing 3D quality of the Games to viewers from events held in arenas, OBS has partnered with Intel to bring its True View technology to Tokyo. TrueView is based on directors using images from virtual cameras all around a venue to deliver perspectives that cannot be seen by physical ones. It allows viewers to select the angle from which they want to see the camera, with options for three or six degrees of movement. It uses an array of high-resolution cameras positioned to capture the entire field of play, connected to on-site servers, based on Intel’s Xeon processors. Data from the cameras is sent to the cloud to be synchronised, analysed, and processed.
In the production suite, engineers can use virtual stationary and tracking cameras to create content focused on particular points in the game, maybe the most exciting action or sequences, for analysis by commentators. Images from the virtual cameras are rendered and converted into compressed digital video in the cloud. Up to 200 terabytes of raw data is processed per event, including height, width, depth, and relative attributes to create high-fidelity 3D video.
TrueView supports the common industry-standard video codecs (H.264, H.265, MPEG and AAC for audio) for use on different platforms and devices. The encoded video is converted into bit streams for live streaming. The bit streams are converted by the rights-holding broadcaster to decompress the video into a series of images, which are rendered sequentially and broadcast.
The output of volumetric content allows viewers to see all perspectives of the game, or to follow a particular player or see the play from any position on the field – including the referees.
In Tokyo, TrueView will be used for basketball games. It is, says Exarchos, one of the sports where it could be really outstanding, because of the three-dimensional nature of the sport. “It is also a very fast sport, that moves in many different directions and there is significant vertical movement.” OBS will produce five to seven volumetric replay clips for each game, which will be made available for directors to integrate into live coverage. “I’m pretty sure that some of these clips will go viral on social media platforms,” he adds.
Tokyo is the first step as broadcasters explore how to create exciting content. It could be someone ‘walking’ into the field of play, turning and watching the athletes around him or her, suggests Exarchos. He expects to see this develop and is looking forward to what will be broadcast at the 2024 Olympics in Paris.
An added benefit is that True View saves costs, and carbon footprint, by not needing a camera crew to travel to the venue.
‘Covid, travel bans and a postponement are challenges that should be used to advance broadcasting.’
While Exarchos is sworn to secrecy and cannot reveal details of the opening ceremony, he is able to confirm that it will be broadcast using multiple cameras for an immersive, virtual-reality experience.
VR is not new for the Olympic Games; it was used in 2016 at Rio and in 2018’s Winter Games in South Korea. For the opening ceremony, there will be six 180° camera systems and one 360° VR system. There will also be experimentation with 5G technology during the ceremony to ensure that content is quickly received at high resolutions and turned around for the worldwide audience.
“I believe that 5G can be a big game-changer for broadcasting,” says Exarchos. “It can help us get rid of a lot of the technical constraints, equipment and regulatory needs, as well as permits for RF transmissions... We need to find a way that we can use all these interesting technologies in a way which is equally effective as we do with tools we have now.” VR has a huge bandwidth and 5G’s reduced latency could address synchronisation issues around live events. “5G provides ample bandwidth and also defeats the digital problems of latency to a very big extent,” he continues. There will also be opportunities for progress using mobile phone screens and the computational capabilities; “hence our collaboration with Intel,” Exarchos says.
The VR experience will be “significantly upgraded” compared to that used at PyeongChang, with coverage of 47 live events as well as between 50 and 100 pieces of VR highlights. “We have noticed that people tend to prefer to experience highlights on VR,” says Exarchos. “For Beijing, our common goal, with Intel, is to produce an 8K VR product.”
The 2018 Winter Games introduced Intel True VR, whereby OBS broadcast 30 events with live and video-on-demand VR events. There will be three to six camera viewpoints recording each event for content compatible with most commercially available VR headsets. Viewers can watch an immersive ‘VR Cast’, seen from different angles as well as graphics and picture-in-picture overlays. True VR can also be used for post-event highlights, and production teams can overlay statistics or picture-in-picture content.
The quest for stats by viewers has also led to Intel’s 3D Athlete Tracking (3DAT) being used at this year’s Games. Originally a coaching tool, it uses AI to identify 20 skeletal points on an athlete. Replaying the race these points indicate where pressure or stress is being exerted or to analyse moments of acceleration and deceleration. In Tokyo it will be used post-races for viewers to analyse the race in detail.
Intel’s director of sports performance technology, Jonathan Lee, explains how much data processing is involved. “We will track all eight or nine sprinters, so that’s nine videos at 60 frames per second for 10 to 12 seconds. It is not just the detection of athletes but tracking skeletal points... We use AI because the athletes are next to each other, so we may not be able to see the arm of the sprinter in lane five, for example,” he says. AI is used to differentiate which sprinter is which at any given moment.
Data that is of interest to an athlete and coach may be too detailed for a viewer but it is likely that statistics such as top speed, when a sprinter reaches that top speed and when they decelerate will be shown. 3DAT allows athletes and coaches to review a race and see that they maintain their top speed for say 20 or 30m and then start to decelerate at the 70m mark. “To be honest, to be able to digest all that information quickly, you’ll see colour indicators, almost like a heat map,” Lee says. The classic view of the sprinters racing towards the finish will be embellished with colours indicating their speed, moving from yellow to red and darker as they go faster, explains Lee. This in itself is a visual representation but post-production teams can overlay when a particular sprinter hits their top speed and what speed that is.
“We will work with broadcasters to understand how they use this type of content... perhaps some of the things coaches and athletes find interesting, we may find that fans find them compelling too, and we might start to weave them in as well,” suggests Lee. The technology is designed to complement the coverage of rights-holding broadcasters. The 3DAT clips with overlaid ‘heat maps’ can be used by commentators to examine what has happened.
Intel’s graphics partner receives the video from OBS, ingests the 3DAT data and renders the clips. AI processing happens in the cloud on Xeon scalable processors. It is turned around in less than 60 seconds.
“To be able to have volumetric replay, potentially also with analysis, and to be able to use it as a fourth or fifth replay after a race is a big deal,” says Exarchos. “I’m sure it will go faster and faster... and hopefully, we will do more in Beijing.”
How to be there, without being there
The Tokyo Games are expected to be popular around the world as people emerge from a pandemic hibernation. Exarchos expects viewing figures to exceed Rio’s 3.3 billion worldwide. (The 2012 London Olympics was watched by 3.6 billion people worldwide, topped only by the 2008 Beijing Games’ 4.4 billion viewers).
The absence of foreign visitors to Tokyo will create a different experience for viewers, as much of the Olympic vibe is derived from the crowd reactions. Flying the flag and the crowd roaring on their favourite is part of many events, and OBS has plans to allow spectators and fans to show their virtual support. It is providing rights-holding broadcasters with digital tools to enable end users to cheer for athletes or countries or the Games in general. These cheers will be shown in the venues.
This is a last-minute development, concedes Exarchos. “We decided to fast-track it, given the situation in Tokyo,” he says. This remote engagement by fans will be a virtual presence – they will be prompted to cheer or post a selfie celebrating a win. “They’re not just passive watchers, they will actually have a virtual presence in the field,” he explains.
“We made a commitment to deliver the same Games [regardless of the Covid pandemic]. We will provide exactly the same level of coverage. Since the [initial] postponement of the Games we have been thinking about ways we can address the situation. You never want to let a challenge go unused,” he adds.
Tokyo’s innovation team is planning a VR booth in a location outside Tokyo to allow fans to don VR headsets and be part of the Games virtually using 360° projections and in co-operation with local production companies. There are similar plans in China; all, however, are dependent on permission by the local broadcasters.
Microsoft closes deal to produce AR headsets for US Army
The company’s 2019 HoloLens 2 headset is a head-mounted display, designed for enterprise applications using technology previously developed for consumer-focused hardware.
The HoloLens 2 displays hologram-like images overlaid on the wearer’s real-world field of vision, aiming to provide useful information (such as instructions or communications) and improve efficiency for workers whose hands are occupied with physical tasks. The HoloLens can be controlled with gestures or voice commands.
The military contract will see Microsoft move into the production phase of technology based on the HoloLens 2 and backed by its Azure cloud computing services. The Pentagon described the technology as an “Integrated Visual Augmentation System” to boost soldiers’ awareness of their surroundings in order to spot targets and dangers.
According to the US Army, soldiers tested the gadgets last year at Fort Pickett, Virginia. It said the system could help troops gain an advantage “on battlefields that are increasingly urban, congested, dark, and unpredictable.” The US Army first started to trial prototype headsets through a $480m (£350m) contract in 2018, and concluded from this that the hardware could have applications for both training and combat.
In January, Congress passed a defence bill which would cut funding for the headset scheme; it is unclear how this contract stands in relation to this legislation.
In February, Microsoft president Brad Smith told the Senate’s Armed Services Committee that the headset could integrate features such as thermal night vision, facial recognition, and “real-time analytics” for use in combat. He also said that the HoloLens could help plan hostage recovery operations by generating a digital twin of the target building.
Microsoft told Reuters that the new contract is worth up to $21.88bn (£15.87bn) over the next decade, with a five-year base agreement that can be extended for another five years. Under the agreement, Microsoft will be able to mass-manufacture Hololens 2-like units for more than 120,000 soldiers in the Army Close Combat Force.
Microsoft also told Reuters that the headsets will be manufactured in the US.
Following the announcement of Microsoft’s initial HoloLens contract with the US Army, at least 94 employees petitioned company leadership to walk away from the deal and stop development of “any and all weapons technologies” that could turn real-world combat scenarios into video games.
The headsets deal is just one partnership between Microsoft and the Pentagon; last year Microsoft was confirmed as its partner in the JEDI cloud computing project worth $10bn ($7.3bn). This deal has been contested by Amazon, whose representatives have made claims of a flawed bidding process and filed a lawsuit over the contract.
Latest Intelligence
