Talking to The Jungle Book‘s VFX Supervisor Robert Legato
Based on Rudyard Kipling’s classic stories, Disney’s sweeping, epic, live action version of The Jungle Book is an immersive delight. We talk to Oscar-winning visual effects supervisor Robert Legato about creating realistic worlds using computers, why it’s an exciting time to be working in visual effects and how they can make a The Revenant sequel minus the grueling shoot.
Congratulations on The Jungle Book. I was just blown away by it!
Oh, thank you. Well, that was the intention. I'm so delighted that's your reaction. We wanted to do something that hasn't really been quite done on this level before. Hopefully, we achieved it.
I think you did. I watched the Ted Talk that you gave about your work on Apollo 13, Titanic and Hugo. In it, you mention that your friend said that a particular scene in Hugo was so successful because no one knows you had anything to do with it. I feel like this film really captures that feeling. Was it important to you in creating this world that people engage with it and sort of forget that special effects are even involved?
Yes. That's my feeling anyway, that what we're really doing is just telling stories on film and then, you want to transcend the technique that you're using. A lot of movies generally do anyway. You forget that you're watching Leonardo DiCaprio or Marlon Brando. You forget he's wearing the costumes. You forget that he's in a set that only has one wall. You forget all that stuff.
You just get sucked into the movie. When you overdo it, overdo any of those things, you're sort of doing a disservice to the movie because it takes you out of it. In this particular one, why I was so excited to do this film, was using a computer not to look like a computer was used. To use it as a camera that is capturing, I don't want to say real life because you never capture real life. We capture movie life. We're used to cinema movie life, the way it's lit, the way the camera moves, the lens choices, the blocking choices, the lighting. When we see a beautifully photographed film, every shot is lit to feel like it could be real, but it's not. It's meant to be filled with drama.
To be beautiful and a fantasy that you're buying into?
Yeah, you want it to hit both marks. You want it to be something that your brain says, Oh it must be real, I would buy into it, and then you still want it to look great. If you do a documentary, one shot looks great, the other shot looks crappy. It takes you out of it because you're kind of bumping against the technique of doing it. That was in the forefront of my mind, and it's never really been done before to this level is, to disappear into the fabric of the movie making.
For the most part, what you're seeing just looks like a real movie. You would never know a computer was involved in it and we're not taking advantage of the luxury that you have where you could do anything you want, and just because you can do anything you want, you shouldn't. You should show restraint. That holds true even for a conventional film. You can make everything look a certain way, or have people act a certain way that's not real and things like that, but you shouldn't: you should be a mirror to real life or your version of real life.
When I was a second unit director and a camera man and designed shots and supervised it all and stuff, but you should never even know I'm there in a regular film, so you shouldn't really know I'm there in this film either. You just should be watching the film. You shouldn't even know who the director is, you shouldn't even know that the actors are actors, you should just fall into it.
Was that the main challenge for you?
That was the challenge. The fun part of doing this is to use these techniques that we've learned how to use to our advantage, make these sort of analog meets virtual camera systems that mimic real life cinematography. Now, we've been able to mimic real life lighting. If you were to put a lamp in a particular scene, a computer now could replicate it so it looks like something that was shot in The Godfather, say. Using that and sustaining it over the entire film to tell the drama parts, as well as the effects, the mudslide and the bigger, larger than life things that you know can't be real.
If you see everything done to the same level, you can even buy those a little more, because it kind of looks like an action scene that was conventionally shot as opposed to a miracle computer generated super hero movie thing, where the physics can't be real.
To break it down, you used a combination of photo-real animation and motion capture techniques and live action shooting. Can you briefly give me an overview of how those three things related to each other?
Sure, besides the preliminary stuff of coming up with what the jungle was going to look like and various things like that, assuming that that's all done, we created this sort of pre-vis shoot of the entire movie with Neel [Seethi, who plays Mowgli] in a motion capture suit and a virtual camera. We would block out the scenes, scene by scene, shot by shot and shoot it in a conventional manner which is, we'd shoot multiple coverage and things, over-the-shoulder shots, wide shots, close-ups, crane shots. Then, we would edit them together like a conventional movie where later, as you're designing or finessing what the story is you're trying to tell, you cut to the close-ups for this emphasis, you cut to the wide shots for that one. That again, is now sort of conventional and the same art form we're used to seeing, so there's that.
That becomes the template for the movie. Then we take the scene that was shot with Neel in the main version of the jungle and say, ‘We now want to replicate that’. So, we blueprint it out and say, ‘Okay, Neel will be a hundred feet away from us. He would have these kind of trees around him’. We'll put up fake cucoloris and things that block light or create shadows. It gets more higher tech than that, but I won't go into details. Then we shoot them as an element to then be put into the photo-realistic version of that scene that we pre-vis.
We walk into the shoot with an edit that … we've already started to hone in our choices to tell the best story we can, and we take the backgrounds that we did our pre-vis on, and we attach it to our camera. We call it simulcam. So now as I move my real camera around, I'm seeing the computer version camera background move, so we can frame things with the authority that we know what's in the frame. The first step informs the second step. The third step is now marrying those disparate elements together to look believable.
Then the biggest, final, and probably the most convincing part of it is the rendering of that and also the rendering of the animation of the animals, making the animals behave and do things that they could really do and not what we would love them to do, but what they really do. The same thing with the digital doubles we have. We have a lot of them there and are very proud of them because you can't really tell that it was done because it feels real. We did real physics. A real kid does that. We just put them on a computer or did a computerized version of him to look like it's the real kid and not do something super hero-like, that takes you out of it.
Neel was the only human in the process, or human actor performing live-action. I understand that you used puppeteers to help him engage during his performance. Can you tell me a little bit about that?
Yeah, that was one of director Jon [Favreau]'s ideas that was a very good one. He got Henson puppeteers to create the improvisational part of life that you get on a set with actors interacting with each other. The puppeteers were able, because they engage with children a lot, to keep his interest alive. Sometimes they would mimic the voice, so you'd hear the voice, like a music video where you're playing the track and they're miming to it. Sometimes we'd turn that off and they would ad lib a line or two to get a more life like or real expression from Neel, something that he wasn't expecting and you get a little sense of surprise. It's those little tricks like that.
They were essentially engaging to look at and engaging to interact with, as opposed to a tennis ball on a stick or something else. It's amazing how much life, just the chemistry of talking to another person, gives you that can't be underestimated. Jon was able to get, especially for a novice actor, some good reactions and interactions with them based on that. We knew where they were standing because we had the pre-vis and we also had the simulcam—we'd see a composite on the stage and see that the eye lines are correct and all the various things. We put the puppeteers where he's supposed to be looking and all these various things.
Jon would very liberally use takes that were in between takes. Somebody said something funny on set and Neel would laugh, or he would smile, and he would use every trick one can, to make sure that he was engaging all the time.
One thing that is particularly impressive in this film is the realistic nature of the animals. I know that skin and fur are notoriously difficult to create realistically. Do you feel like this is the first time something like this was actually able to be created because of the technology available?
I think so. I think the sensibility and the technology met and was used in a way, I would like to say, the way it should be used, or could be used, to replicate real life and be no different than the normal artifice of a movie, where there's an actor in a role and the more you believe him, the more you believe the character. The same holds true for these creatures, they are actors in the movie. They're voiced by real people who have emotions, who have timing and all that stuff.
If you marry that with something that looks like it's behaving like a mammal behaves, and render it in such a way that you believe every hair on its back, every interaction with shrubs and things, that your brain assumes are real, you go into another avenue of filmmaking that is very exciting to do. Part of it is the will to do it. Everything we've ever done from every movie we've ever made hasn't really been done before.
You go to the movies to see something you've never seen before and experience something that you can't experience in your real life. This was the gauntlet that was laid down. Let's try to tell this story, not in an animated way. It almost gives it a bad name, animation. It feels like it's cartoony or whatever. Forget animation. It's a real animal.
Do you feel excited about the possibilities of what you can do next?
Oh yeah, that's one of the reasons I was so interested in doing it. It opens the door for all kinds of movies that you would never use the connotation that we would do that on a computer, or we just couldn't do it at all, it was so expensive to do. In this particular case, forget the talking and all the rest of the stuff, you want to do a movie in a jungle, we are not allowed to photograph real, exotic animals. They come down pretty hard on that. This is the only way you could do it, but you still want it to look like you really did. It opens the doors for that.
Forget the animals, the other part that was exciting was the backgrounds. The ease of shooting makes you be able to do a Revenant without having to go through the pain that The Revenant caused you. Those guys, after they're finished, it's a fabulous film and it looks great and it's everything that you want, except it was so arduous they don't want to make another one. It's like, I will never do that again. We as an audience want to see that.
Leo will be like, ‘Now you tell me!’
Right. But we want to see another one. And what they did with the bear was amazing but now you can create that whole background, that whole exotic thing. You don't have to search for hundreds of miles to find snow. You could realistically depict it, and tell the story you want to tell without the connotation of people going to them saying, Oh well you did it on a computer, so it's fake. How would you know how we did it? How would you know?
