In the late 20^th century, museums underwent a transformation, shedding their image as places with roomfuls of static exhibits in glass cabinets featuring information-heavy labels but little context. Today most have some interactivity elements, whether it is a touchscreen display or virtual reality, as used by the Tate Modern in London to depict Modigliani’s Paris studio from 1919 in the artist’s retrospective exhibition.

In June 2021, the Muséum National d’Histoire Naturelle in Paris went one step further and introduced Revivre, which offered visitors the chance to mingle with 3D animals that are either extinct or in danger of extinction.

The museum partnered with SAOLA, an augmented creation studio, to create the augmented-reality (AR) experience. SAOLA, founded by film maker Jeremy Frey and producer Florent Gilard, includes a naturalist, a scenographer and a team dedicated to 3D creation. It drafts in specialists in wildlife, organic, architecture and design according to a project’s needs.

The 3D modelling and animation process took six months and included several visits to scan the entire room so that the AR did not jar with the architecture.

“We had to make sure that the thylacine (Tasmanian tiger) jumps onto the window and not next to it, or that the Angolan flightless beetle rolls its dung on the floor of the room,” says Marie Wacrenier, head of museography and technical service at the museum.

The AR experience takes visitors through the room, guided by an AR passenger pigeon (itself extinct since the beginning of the 20^th century). Wearing Microsoft’s HoloLens 2 AR glasses, visitors can stop in front of a display where a life-sized 3D hologram becomes animated. They can experience each creature in its natural habitat, whether that is African grasslands or under the sea.

The museum says augmented reality was chosen over virtual reality because it provides an immersive environment in which visitors can interact with others in the group while enjoying the sights and sounds.

Binaural and spatialised sound create 3D audio as the visitor meets a creature. Or visitors may only be aware of the tiger to the side of them when they hear it roar!

The creators used the museum’s present collection, its archives and contemporary descriptions and sketches of animals to model 3D creatures and animate them within the historic architecture of the room.

Revivre was conceived to highlight both extinct animals and those in danger of extinction and to explore the causes of their demise. Some of the ‘exhibits’ have been extinct for centuries, like the dodo, but the rythine or Steller’s sea cow (similar to a manatee but 10m in length and eight to 10 tonnes in weight) was eradicated by 18^th-​century hunters in less than 30 years after first being sighted in the Bering Sea.

As well as highlighting threats to some species today, Revivre also elaborated on ‘good practices’ of indigenous peoples to preserve others and their habitats.

The Revivre exhibition ran from June 2021 until the end of December 2022, though you can still find a promotional video on YouTube. Possibly a restricting factor for such exhibitions is the cost of the headsets – Microsoft’s HoloLens 2 currently costs well over £3,000. Unfortunately it will probably take a substantial reduction in the cost of this hardware for such exhibitions to become more feasible on a large public scale.

The Cooper Hewitt, Smithsonian Design Museum is housed in the Carnegie Mansion in New York City and is dedicated to historic and contemporary design. When it reopened in 2014, following renovation of the mansion, it introduced an immersion room to help visitors browse its wallpaper collection. Using the interactive table in the middle of the room, visitors can view a wallpaper pattern at scale as it is projected onto the walls via two projectors communicating with the table in real time. Visitors can also draw their own patterns on the table which are displayed as wallpaper.

The museum also wanted to encourage visitors to engage with the design on display and commissioned the Cooper Hewitt Pen. A New York consultancy identified Sistelnetworks’ vWand, an inventory control pen used in healthcare, as a suitable model. GE developed a sleeker form factor and Sistelnetworks modified the circuitry and electronics. A sensor read the data in the NFC tags embedded in the object labels. The Pen’s onboard memory stored the data, which could be displayed on the interactive tables. It could also suggest similar exhibits that might be of interest with an image and catalogue information. When issued, the pen was paired with the visitor’s entry ticket. On its return a receipt with a unique code was issued, which the recipient could use to access all of the images and data collected during their visit, via the museum’s website.

Despite its usefulness, the Covid pandemic meant that the Pen was withdrawn from use due to health concerns.

Exploring art – really exploring it – can make a painting come alive. This is the approach taken by Bild Studios, a production company which uses large video canvases and LED screens with mixed reality, visual effects and real-time rendering for performances and installations.

In 2020, Bild used its MARS (Mixed Augmented Reality Studio) in West London to recreate Edward Hopper’s 1942 painting Nighthawks as a virtual 3D presentation. Using an in-camera LED wall, LED ceiling, cameras and real-time rendering, it created a virtual production studio allowing the art historian Dr James Fox to ‘step into’ the depicted diner and ‘explore’ his surroundings.

Bild mapped Nighthawks onto a 3D space and used 3D reconstruction to recreate the diner’s interior and its occupants. AI was used to analyse Hopper’s brush strokes, which were applied to the whole environment, filling in the gaps which cannot be seen in the 2D original. Using the immersive, mixed-reality stage, Fox was able to get a closer look at the diners seated around the bar.

With this 3D recreation, the viewer can explore possible answers to questions often asked about the painting, such as what is the relationship between the man and the woman sitting next to each other but not speaking? It also allows a series of alternative paintings to be created, experimenting with different light conditions, for example. How would the same scenario look at midday? Or with rain reflected from puddles in the pavement outside?

These three projects illustrate how readily available technology can be used to transform exhibits in museums and galleries into an object that causes someone to look and wonder. From there, they can begin to uncover more knowledge, branching out into interest areas in an accessible way and in an easily digested format. 

Check out how British art historian, BBC presenter and BAFTA-nominated broadcaster Dr James Fox was placed inside a virtual 3D representation of Edward Hopper’s 1942 painting Nighthawks through extended reality.

[embedded content]