“Kingdom of the Planet of the Apes” VFX Team on Bringing a New Generation of Primates to Life

The events in Wes Ball’s Kingdom of the Planet of the Apes take place 300 years after the death of Caesar (Andy Serkis), the focus of the previous Apes trilogy kickstarted by director Rupert Wyatt in 2011 and then led by director Matt Reeves for the final two films. Apes have now evolved into a fully talking species while humans tread backward. It would seem like a win for the apes, but it’s still not enough for the powerful junky tyrant Proximus Caesar (Kevin Durand)—he wants to wipe out mankind for good, and the key to doing that are the secrets a young girl named Mae (Freya Alan) might hold. While the goons of the self-proclaimed king track her down, they ransack the peaceful village of the Eagle Clan. The devastation marks the start of a long journey for the franchise’s new protagonist, Noa (Owen Teague), who wants to bring his family back home.

Weta FX was behind the demanding visuals, delivering more than a dozen new characters, creating new environments, and teaching apes to talk. More than 1,500 visual effects shots were required, and 33 minutes of the film were completely digital. Heading the way was visual effects supervisor Erik Winquist, who worked on the three latest films, starting with Rise of the Planet of the Apes in 2011. Winquist and animation supervisor Paul Story explains how they took the franchise’s sensational effects to new heights.

Performance Capture

To capture performance, actors wear a performance capture suit that is equipped with active markers that allow motion capture cameras to record the action. The technology has evolved dramatically, back to Weta’s work on Peter Jackson’s The Lord of the Rings trilogy. For the Apes franchise, the challenge was moving from a studio setting to a physical location. “Mocap is done in a controlled space where an array of mocap cameras emit infrared light that bounces off reflective markers on the suits and back to the cameras. That doesn’t work in sunlight,” Winquist tells The Credits. “So instead of reflective markers, we have infrared LED lights on the suit. Those are the markers that allow us to take the process outside.” Kingdom has actors using third-generation suits. The upgrade has markers embedded in the suits so nothing can get caught on them when they come in contact with something.

The other key capture ingredient is the camera that records the actor’s face. “In the previous film, there was a single small camera on the end of a boom. In the march of technology moving forward and miniaturization, we now have a pair of stacked stereo cameras that are viewing their facial performance at 48fps,” says Winquist. “So we can use those perspective views of the two cameras and derive a three-dimensional mesh of their face at 48fps. This gives us a much more nuanced view of every little tick in their face.”

All the information then feeds into a Deep Learning Facial Solver, which Winquist says has never been used on the Apes movies before. “What the solver enables us to do is get the mechanics of the performance and frees up the animator to really focus on the actual emotional content of the performance,” Winquist explains.

Facial Animation

Even with the latest technology, the emotional soul of the characters starts with the actors. The trick for visual effects is taking the human performance and transposing that onto the ape’s face and body. “What we are putting on screen is entirely the performance the actors give on the day. The translation is the secret, but we are not inventing those moments,” Winquist says. Bringing it all together is a challenging task for the animators.

“With our setup, the Deep Learning Facial Solver translates their performance and gives us a base motion. Then each animator can go through with the reference side by side so we can match the ape performance exactly to the human performers are doing,” explains Paul Story.

For performance moments that needed to be pushed further, visual effects could augment and enhance them with the facial solver or keyframe them to better match a performance. The team didn’t rely solely on the solver, as many of the characters are fully keyframe facial animated.

“Not having to set keyframes for the entire facial performance now, we can build upon that base the solver has established and let the animator use their time on every shot, every character to make sure they can match the emotional beats of a particular performance,” Winquist continues. “The animator is part of the recipe and translates what the actor did to make sure what you see on the screen [as apes] gives you the same emotional response you would get from a human actor.”

Talking Apes

With Kingdom, apes have more dialog than in the previous three films combined. The challenge for visual effects was translating how much an ape’s mouth should move since the muzzles are much larger than a human mouth. “We had to rely on a solver to get the consistency between the shots, but it was definitely tough up front to find that balance between how a human and ape would talk,” notes Story. “There’s all these sort of ratios you need to find, good talking ratios that change per character.”

For Proximus Caesar, the team tied his physicality to his speech. “Part of the design is making sure we have a similar eye shape, and when we are animating, we are making sure the chimp brows read the same,” says Story. “The human brow is different; we can read more about the hairy shape of our brows, whereas chimps tend not to have as much hair up there. It’s all about the underlying side of the brow as it’s related to the eye that conveys that emotion.” The team of animators would reference the captured human performance to create the speech of the ape and, at times, push it further to help sell it.

Soaring Eagles

Noa belongs to the Eagle Clan, and the golden eagle plays a crucial role in their culture. Each ape must climb to a nest to secure an egg as a rite of passage. From birth, they bond with the eagle, forming a lifelong kinship where the bird of prey becomes their eye-in-the-sky protector. Ball introduces us to the adventurous ritual in the film’s opening sequence. Noa and close friends Anaya (Travis Jeffery) and Soona (Lydia Peckham) bring us to the peak of their world, a lush, overgrown forest as far as the eye can see. Still without an egg, Noa climbs to the top nest, and it nearly costs him his life, a feat he tries to impress upon his father Koro (Neil Sandilands), who is the tribe’s master of birds.

Visual effects looked to next-gen technology Weta calls Apteryx, which is the scientific name for the kiwi, to create a village of digital eagles. The computer-based toolset allows artists to generate, hand sculpt, and groom feathers more efficiently and realistically. “Birds are very difficult; every feather is not a single piece of geometry but a complex organization of what is fundamentally hair coming off a central stem. And every bird has thousands of these,” explains Winquist. “It very quickly becomes a computational nightmare not only to simulate but also to groom and sculpt the layout of the feathers.” 

Winquist points out Weta initially used their existing plumage tools for grooming because Apteryx “wasn’t quite production ready” when they started building the eagle characters. “By the time we got through to the end of the show, the toolset had become rich enough that we were able to go back and re-groom our eagles. There are some shots of the before and after that go from a really nice-looking CG bird to a bird that looks real.”

Water World  

War of the Planet of the Apes (2017) gave Weta an early crack at water interaction as Caesar and his tribe take refuge behind a waterfall. Some of the shots included rain dripping down Caesar’s face and walking through the waterfall. At the time, Winquist admits even those limited frames “were a struggle.” Fast-forward to post Avatar: The Way of Water (2022), the team was able to springboard techniques from the James Cameron film to now have the ape characters interact, swim, or even fight in the water with believable results. One of the more challenging sequences happens during a raging flood at the oceanside encampment of Proximus Caesar. It’s a death-defying moment for every ape on screen.

“It was a big collaboration between our effects team, which does all the water simulation, and our creature team, which takes care of the hair simulation, whether that is interacting with wind, water, or other characters,” Winquist says of the climactic scene.  

Creating the imagery required some baton passing in post. “With a lot of the water stuff, we had to previs and make the shots ourselves, with the direction of Wes,” notes Story. “That was a fun task, but keeping that in line with the water, there was a lot of back and forth.” To start, the effects team would make a low-resolution simulation of the water, for example, a crashing wave, and then pass that on to the creature team for them to animate how the apes would interact with the water. After the initial low-res simulations, the effects team create a targeted high resolution simulation to add additional water not part of the original pass. For instance, water running down an ape’s face or water draining off its fur as it emerges from the water. “With water, it needs to feel like a thin film draining off their bodies,” notes Winquist. “It was important to make sure the volume of water coming off seemed justified for how much they got splashed. If they come out of the water, we also have to show how long the water needs to continue to drain out of their hair.”

Kingdom of the Planet of the Apes is in theaters now.

Featured image: Raka (played by Peter Macon) in 20th Century Studios’ KINGDOM OF THE PLANET OF THE APES. Photo courtesy of 20th Century Studios. © 2024 20th Century Studios. All Rights Reserved.