I’m inside a chic, modernist, glass-fronted building in Santa Monica, just a few blocks from the beach, but there’s no ocean view. The entire property—Headspace Studio, an immersive audio company—is covered in lush foliage, protecting it from curious passers-by.

Inside, I’m getting a behind-the-scenes look at Wes Anderson’s Isle of Dogs in virtual reality. The movie—an animated adventure about a pack of dogs living at a garbage dump who help a 12-year-old boy find his lost dog, Spots—hits theaters today, but you can also get a sneak peek in VR. There are several official cuts/mixes out there. One was shown at CES, another at Sundance; there’s a special Daydream edit exclusively for Pixel phones, and the final longer full cast version, which I saw.

As you might expect from Hollywood, there are a lot of partners credited on this one, including the movie’s production team, Félix & Paul Studios, FoxNext VR Studio, Fox Searchlight Pictures, and Google Spotlight Stories.

I met with Viktor Phoenix, Partner and Head of Audio at Headspace Studio LA, who handed me a Google Daydream View and a pair of Bose noise-cancelling headphones to experience Isle of Dogs in VR.

What’s it like? Well, it’s visually stunning, as you’d expect from Wes Anderson. You really are inside his world as soon as the 4K video imagery kicks in. The narrative is very funny; a sly take-off on DVD extras, but these actors have fur.

Stop-animation dogs shuffle up to the camera, sometimes invading your personal space as a real dog might, to tell you about their role. Duke (Jeff Goldblum) does a Duke Ellington jazz scat riff, Nutmeg (Scarlett Johansson) explores her motivation, and Chief (Bryan Cranston) reveals his sensitive side. Tilda Swinton and Edward Norton, inside their canine characters, avoid plot spoilers while engaging in amusing non sequiturs.

Isle of Dogs

Then there’s Boss (Bill Murray). I don’t know about you, but I’d happily pay good money for a VR experience that lets me hang out with Murray in a stop-motion ski lift for a while. These are trying times.

The imagination behind the VR short is brilliant, but the technical expertise is, too. Phoenix told me to look out for “magic window” moments, where you can engage with the dogs talking to you/the camera OR you can turn around and see the animators, set designers, technical staff—at time-lapse speed—doing their jobs. It’s a great back-and-forth experience. And, because the VR setup uses eye tracking, it knows the position of your head and can adjust the action to your gaze in real time.

Isle of Dogs

The key, though, is innovative use of audio. Jean-Pascal Beaudoin, co-founder and Head of Sound at Headspace Studios, told us how it came to be, via email, from Montreal.

How did Headspace Studios get the gig to do sound on Isle of Dogs VR?
Isle of Dogs: Behind the Scenes (in Virtual Reality) is the first project to result from the pact between Félix & Paul Studios and the Fox Innovation Lab at Twentieth Century Fox to develop VR experiences for Fox properties. Since Headspace Studio is the exclusive sound partner of Félix & Paul Studios, it was only natural that we got the opportunity—or shall I say privilege, since I’m such a fan of Wes Anderson’s body of work—to work on this project.

What did the studio request regarding immersive audio or did you bring your expertise and make suggestions from the top?
I’m glad that you ask this question because audio in VR can be so much more than just ‘immersive’ or ‘spatial’ and being involved early in a project is a prerequisite for that. Not just in terms of the creative per se but, as importantly, to plan the technology required to execute that creative vision. That’s one of the benefits of working with true VR storytelling visionaries such as Félix & Paul and FoxNext VR Studio. They not only understand that audio has the power to not merely support a story but to elevate it, but also that that requires careful planning.

Isle of Dogs

A key enabler here was the fact that we knew from the start that the premium platform for the project would be Google Spotlight Stories—a storytelling driven and feature-rich VR platform. Thanks to Scott Stafford and the rest of the team there, this allowed us to elaborate a creative approach that relies on temporal metadata, a technology normally found in game engines that goes beyond what all other 360 video platforms currently support in terms of spatial audio. In comparison, the stripped-down soundtrack on the YouTube version speaks for itself.

In your experience, was there anything unusual about this project?
Because of the colossal amount of time and resources required for doing puppet animation, the behind-the-scenes had to be filmed not only for the VR experience but also for traditional screen format. The latter was directed by Wes, and the former by Félix & Paul in collaboration with the movie’s production team. Our approach sort of embraced that dualism. First, we wanted to pay homage to Wes’ most celebrated visual traits as a filmmaker: masterful framing and immaculate composition—something which can be considered ‘meta’ in 360 video given that it is a frameless visual medium. Secondly, it is derived from Félix & Paul’s overall concept of the VR experience itself where the set layout is divided in two zones: the world of the movie with the puppets that’s happening in real-time and the behind-the-scenes with the animators, that’s a timelapse.

Isle of Dogs

Explain the ‘Magic Window’ concept. That was fabulous.
When the viewer is facing the world of the puppets—which is very similar to Wes’ framing composition—we wanted them to be in a kind ‘magic window’ where they are immersed in that POV, and that POV only, so you hear only the dog dialogue as well as the ambience of the Isle of Dogs world. In contrast, when they start turning around and ‘break’ that visual composition, they realize that they are now in the time-lapse world of the animators, and so they start hearing all the action going on as well as music from the film that’s different for each scene. I like the fact that it’s quite a simple and playful mechanic. Yet, it elevates the storytelling of the piece.

Do you have any good stories about interacting with Wes Anderson? Was he in Montreal for the production?
I wish! The project was actually shot at London’s 3 Mills Studios and he was not present for the post production in Montreal. We got to interact with his closest collaborators but not directly with him, which is my only small regret working on this project. I mean, being that close!

Anything you learned on this assignment that will change how you do spatial audio on the next one?
One anecdote that I can share—and which would later turn into a lesson learned—is that the audio quality of the production dialogues that we had received initially before the actual filming took place was in some cases terrible. I would even say unusable in the case of spatial audio since every source is so exposed—and that’s even after using state-of-the-art tools to repair them. Bill Murray recorded his bit on a mobile phone whilst in a taxi (probably driving through Manhattan!) and Edward Norton’s had all these unsalvageable ugly mic pops in it. Wes, instead of asking his actors to redo their lines in better conditions, used those flaws at his advantage and conceived those scenes with that in mind. So King ended up in this noisy cable car and Rex speaking (and popping) into a microphone!

The man’s a genius.
Right.

Isle of Dogs Behind the Scenes (in Virtual Reality) is available on multiple platforms for VR, 360, and 2D via YouTube VR, Fox Searchlight YouTube channel, Sony PlayStation VR, but the Google Spotlight Stories one has the most advanced interactive audio mix.



Source link

LEAVE A REPLY

Please enter your comment!
Please enter your name here