Two Enormous Sequences Multiply the Challenges in Revenge of the Fallen

There’s a sequence in DreamWorks/Paramounts’ Transformers: Revenge of the Fallen in which the Devastator, a gigantic Decepticon robot, scales an Egyptian pyramid as easily as a gorilla climbing a tree and rips off the top. With over 52,632 parts and nearly 12 million polygons, the robot is the biggest model Industrial Light & Magic has built in its 30 years of model-making. And they built the bot for IMAX shots.
“What’s happened in this picture is that the scale and density of the characters, and the number of characters, have gone way up,” says Industrial Light & Magic’s Scott Farrar, who was visual effects supervisor for director Michael Bay’s 2007 Transformers and his 2009 sequel.

Two effects houses created the robots. There were 16 robots in the first film, and 59 in this one. Digital Domain handled 13 robots and ILM created 46, including the returning Autobots Optimus Prime and Bumblebee, the revived and somewhat altered Decepticon leader Megatron, an old robot named Jetfire, and the huge Devastator, among many others. Devastator and Optimus Prime, two of the most complicated robots, star in the two IMAX sequences, both created by ILM and both among the most complex in the film.

The first IMAX sequence takes place in a forest where Optimus fights two Decepticon robots while crashing through trees, many of which are CG models with branches broken through rigid-body simulations. During the forest fight, the 28-foot-tall Optimus Prime and his 10,000 parts appear actual-size on IMAX screens.

The second IMAX sequence begins with the Devastator forming itself by smashing into other Decepticons that transform from giant mining construction vehicles. It starts with a mining excavator, crunches a dump truck into its torso, piles on a cement mixer to create a mouth, smashes into another dump truck and a bulldozer to make legs, and forces a scoop loader and a crane into arms.

Once built, the voracious creature sucks everything nearby into its huge cement-mixer mouth, creating a vortex of sand and dust that ILM formed using fluid simulation. Then it climbs the Great Pyramid. When it slashes the top of the pyramid with its mighty arms, 120,000 bricks shoot off and tumble down the sides, an effect powered by the largest rigid-body simulation ILM has ever run.

“We put a lot of hard work into the sequence,” says Jeff White, associate visual effects supervisor. “Getting our pipeline to handle IMAX resolution has been quite a challenge.”

Transformers: Revenge of the Fallen

Digital production supervisor Jason Smith was in charge of that pipeline. After deciding to scan the IMAX frames at 6K resolution, ILM ran tests at 4K and 5K. “At the beginning of the show, when we were deciding what resolution we needed to work at, we ran shots all the way down the pipeline, including color timing at Company 3,” Smith says. They also talked with Paul Franklin, who supervised the effects at Double Negative on The Dark Knight. For that film. Double Negative worked primarily at 5.6K (See Film &  Video‘s original coverage), but ILM settled on 4K resolution.

“Their visual effects covered some portions of the frame,” Smith says of Dark Knight. “But we were dealing with robots everywhere covering up everything. When we compared the resolution we were scanning and putting onscreen to 35mm film at 2K, we decided 4K looked great. Plus we knew painting clean frames and color grading would be more difficult above 4K. When you hear 2K to 4K it might sound like double, but the render times are six times bigger and the memory requirements are six times bigger in IMAX. Across the board, it turned out to be six times bigger.”

During the height of production, ILM dedicated 80 percent of its total rendering capacity to Transformers 2, one time even hitting 83 percent. “We broke all the ILM records,” Smith says. “Everyone else squeezed into 17 percent.” How much is that? ILM’s render farm has 5700 core processors, the newest of which are dual processor and quad cores (eight cores per blade), with up to 32 GB of memory per blade. In addition, the render farm can access the 2000 core processors in the artists’ workstations, which ups the total core processors to 7700. As for data storage, the studio’s data center currently has 500 TB online. Transformers 2 sucked up 154 TB, more than seven times the 20 TB needed for 2007’s Transformers.

The switch to 4K resolution for the IMAX sequences had an impact beyond rendering. “Everything is bigger with IMAX,” Smith says. “When we were rolling out the IMAX sequences, we had more model resolution and detail, and we had a huge wave of machine upgrades all the way through paint and compositing. We switched to [The Foundry’s] Nuke to make handling the comps easier.”

The IMAX sequences also meant ILM had to create multiple image pipelines – Bay filmed these sequences with three IMAX cameras and three VistaVision cameras. “Because the VistaVision film runs sideways, it gives you that wide screen, and that extra surface area made it a great bridge format between IMAX and 35mm,” Smith says. “When you don’t want the audience to feel the loss of resolution, VistaVision can act as a buffer.”

Transformers: Revenge of the Fallen IMAX shot

Unlike IMAX sequences in Dark Knight, the IMAX sequences in Transformers 2 happen in full daylight. “We watched Dark Knight to see whether we’d notice the letterboxing when they cut between 35mm and IMAX, and we didn’t, but those were night shots. So we did tests and found that even for daytime, you didn’t really read those black bars top and bottom. What you did notice was the field of vision opening up when you go into the IMAX shots.”

All told, ILM needed to create IMAX plates as IMAX, IMAX as 35mm, IMAX as VistaVision, anamorphic as IMAX, anamorphic as 35mm, VistaVision as 35mm, and VistaVision as IMAX. Anamorphic working resolution was about 1.5 million pixels (1828 x 778), VistaVision was about 6.5 million pixels (3144 x 2078), and 4K IMAX about 12.5 million pixels (4096 x 3072). “2K to 4K isn’t double,” Smith reminds. “It’s about eight times the surface area when you look at the number of pixels.”

To create the 35mm frames from IMAX composites, Bay had ILM crop out windows. “You’ll definitely get much more image in IMAX,” White says. “I think that people who see it in IMAX will have almost a completely different experience. The IMAX plates come out almost square. We did our work to that IMAX square and at the end, we cropped out [image on the top and bottom to achieve] 2.35. Michael [Bay] could push the crop up and down to pick the most important parts of the frame.”

White pulls up a shot of Devastator forming on a computer screen to show what he means. Blue lines cut across the top of the enormous robot and the bottom of the frame. “With 2.35, you can get the top or the bottom,” he says. “But with IMAX you get the whole robot in there. When people see this on a six-story screen, I think it’s going to be incredible.”

To pull out the 2.35, the crew used a separate camera. “It wasn’t an up-and-down pan and scan,” Smith says. “We wanted actual camera moves. So a nodal camera pulled the 2.35 crop out, but we put these animated crop cameras together with Michael’s approval all the way. And, it wasn’t just a matter of 2K and 4K. It was what the crop looked like for [going from] VistaVision to anamorphic, too. We spent a lot of time getting the most for every pixel so it was as seamless as possible for every audience.”

Bay made sure that viewers watching the 35mm film saw plenty of action, too. “Michael really got on us,” White says. “He told us we had to pay attention to what the 2.35 version looked like and that went all the way from animation to how we composited the elements. He didn’t want an explosion that was halfway out of the 2.35.”

Adding to ILM’s IMAX work was the need to film new elements for the effects-heavy shots. “We have an extensive element library for fire, dust, smoke, and so forth,” White says. “But we scanned a lot of those elements at 2K resolution and they wouldn’t hold up on IMAX. So we had to go out and shoot new elements.”

In addition, although ILM had IMAX plates to start with for many of the shots, Bay had designed camera moves that the studio couldn’t accomplish with the plates they had. “We sent out a virtual background team on location and they took photographs of the entire environments,” White says.

“They shot spheres of images with a nodal camera up on a crane or tripod,” White says. “They had to be flexible and really fast because often the only time we could shoot spheres was over lunch.” Back at ILM, the digimatte team led by Richard Bluff built IMAX-resolution background plates for the effects crew to use with their virtual cameras.

“It was a challenge that the bigger frames made bigger,” Smith says. “It’s just slower and a lot more work for the artists. And, pixels aside, we looked at the model resolution in a more detailed way. We pushed beyond anything we had done before.”

The Devastator is a great example. “Michael [Bay] said, ‘OK, you guys. This is going to be on a screen five or six stories tall. It can’t look dead anywhere.’ So we went back through and added a layer of detail. Tiny hoses, wires, ladders, little details that show you how big the thing is. The challenge wasn’t only when the robot is far away. When you look at his hand or his arm in IMAX, he’s not just a creature. He’s individual small parts at 4K resolution.”

Modelers built Devastator specifically for these high-resolution shots because the megaton menace spends most of its time in IMAX res. Optimus, however, spends as much time in 35mm as in IMAX, so the key for that robot was in painting high-res textures. “We knew the damage to Optimus would be in high res, so we added that detail as we went,” Smith says.

Dave Fogler supervised the team of around 25 modelers and viewpainters (texture mapping) who worked on the film. “More often than not, we worked with 2K maps,” he says. “But we had lots and lots and lots of them for individual parts. The reason is that things start to slow down once you go to 4K. So we had a collection of parts with 2K maps and if they didn’t work in IMAX we’d go to 4K.”

“Lots and lots and lots” is an understatement. The viewpainters created 6,467 texture maps for Devastator alone – 32 GB of textures. “On an average show, we keep one viewpainter and one modeler on to the end to keep an eye on things,” Fogler says. “On this film, we had a crew of six or seven adding details and damage and every time we had an IMAX shot, we’d cross our fingers, see what we could see, then go in and add nuts and bolts.”

Despite the extra work, the crew is understandably amped about the film, about the IMAX version of the film, and about working with Michael Bay. “What can possibly compare to seeing Optimus Prime at actual size fighting Megatron in a forest?” Smith asks. “It’s not like [Michael Bay] picked boring sequences. He picked the most astounding sequences in the movie. Imagine Devastator taking down one of the pyramids in full IMAX.”

Adds White: “You always have a couple signature shots on movies that everyone wants to do because they’re so cool. What’s incredible about this film is that you look through every sequence of the film and they all have signature shots. There’s really no shot that isn’t pushed to an incredible level.”