Rendering Technology, Digital Costumes, and CG That Doesn't Suck

The visual effects Oscar nomination for Iron Man 2 is Ben Snow’s fourth. He previously received nominations for Iron Man, Star Wars Episode II – Attack of the Clones, and Pearl Harbor. A member of the Academy since 2007, Snow has worked as a visual effects supervisor at Industrial Light & Magic on seven feature films and in various roles on 17. The indefatigable supe sailed straight onto Pirates of the Caribbean: On Stranger Tides after Iron Man 2, which he jumped onto after Terminator Salvation, which followed the first Iron Man. Snow shares the Iron Man 2 Oscar nomination with overall visual effects supervisor Janek Sirrs, Double Negative vfx supervisor Ged Wright, and special effects coordinator Daniel Sudick. For this film, ILM created Iron Man and War Machine’s suits in part and as full CG characters, the always digital drones, and several virtual environments.
F&V: Having worked on the first Iron Man film, the sequel must have been much easier, right?

[laughs] Oh, no. It was harder. Jon [Favreau, director] asked more of us because he was comfortable. We had more complexity. We had to push digital costuming further. We had larger environments with the Stark expo, and more set extensions. And, you always want to up the ante on a sequel, so we pushed destruction.

F&V: How and why did you push digital costuming further?

On the first Iron Man, we tried to use the Legacy and Stan Winston suits as much as we could. For the second one, Jon [Favreau] was confident we could create the CG suits, and the action dictated using them. So, Legacy created what we called the “football suits” from the torso up with a chest plate and helmet. We’d usually put in some arm pieces, but not the whole arm.

In the house fight sequence, where [Iron Man] Robert Downey Jr. staggers around tipsy, we used some of the practical suit and extended it digitally. Same thing in the Randy’s Donuts scene. But in the rest of the film, we used the CG suit entirely. And Double Negative did an all-digital suit for the Monaco chase. At the end, for our Mark VI suit and War Machine, we had some practical reference, but we replaced all that with CG. Really, every shot in the second half was with the CG suit, even when we shot plates with the real suit. One reason was that the crew had to make War Machine’s costume with fairly light-weight materials and we felt we could get a heavier, stronger, darker, more metallic suit with digital material. It was still fantastic to have the Legacy suits for lighting reference, though. On set, the DP would light as well as he could, which gave us great lighting design.

F&V: Did the fact that Iron Man is a human and doesn’t have super powers himself make the task of creating the “super” suited Iron Man easier or harder?

We liked that aspect of him. We based his thrusters on footage of real thrusters. His repulse technology is not something we can do, but we looked at ways to anchor it in science. I have a policy of starting with science and reality as much as I can. Favreau also favors that approach, and it’s one that Janek Sirrs favors as well.

One way that evidenced itself was in the expo chase. Even though that was CG, we filmed it with cable cams to give us real photography as an anchor. One of the things Jon [Favreau] said when he saw our first renders of the Stark expo was that we had to shitty it up to have it look like real photography. So we looked at the real photography we had, and saw how it had flaring from street lamps, lens aberrations, glaring and grain. And we tried to reproduce that.

F&V: It seems, looking back through your career, that there is a thread of rendering innovation in many of the films you’ve worked on.

I’ve had a fascination with rendering for 21 years now. I hate it when people say stuff looks CG. And, that’s been a charge levied on and off through the years. Now, I think we aren’t getting some of those charges. Starting with Pearl Harbor, where we first used ambient occlusion to make the planes more realistic, the technology has been advancing. We’re getting to the point where the first render from a technical director, presuming the materials are all in line, looks quite real out of the box. That’s been a big thing. Getting rid of the CG curves.

The benefit is that we can spend more time on creative lighting, doing what the DP is doing on set to make things more realistic. That’s been the big desire. It still feels like we have a ways to go. It’s still a pain with rendering technology to do indirect lighting, and it’s still a pain to do full ray tracing. But we’re gradually pushing all those aspects to make them more controlled. You can definitely get into a situation where, as we did with Jon Favreau, a director says something like, “I don’t like CG. How do we make it not suck?” After the first film, I think Favreau changed to “CG isn’t bad, but there is bad CG.” He came onto Iron Man 2 trusting what we can do with the suits.

F&V: How have the improved rendering techniques affected the kinds of shots you do?

During the first film when Favreau added material later, after filming, we did a lot of synthetic environments and set reconstructions. But, trusting we can do that became a bit of a dangerous thing.

In Iron Man 2, they decided they wanted to add a new sequence in which Mickey Rourke [War Machine] comes in with the reverse-engineered suit he made for a big confrontation at the end. We couldn’t shoot new plates, but Jon knew we could create plates because we had documented all the sets. So the confrontation between Iron Man, War Machine and the drones happened in re-purposed backgrounds. Jon was confident we could do that in a limited time frame. It was rough. We were under a lot of high pressure during the last six weeks to make that happen.

F&V: Were you confident you could pull it off?

We were freaking out. We’re thinking, “Boy, we’re going to have to lay down the rules for what we can and can’t do here.” We were cutting it as fine as I’d like to cut it to get good quality work that I’m proud of. It was round the clock. It was a lot of pressure and a lot of work to get through. But, everyone was stoked as we worked on it. We were proud of how it was turning out and that helped. Everyone was happy.

F&V: Was that the hardest shot?

I would say that the hardest shot… there was one shot that got split into two. Iron Man and War Machine are back-to-back fighting an onslaught of drones. There was a cut, so we split the teams. We based the shot on the previs; we planned it in great detail from the start. It has every trick in the book, from the synthetic suits to the gratuitously enjoyable drone destruction. It was really cool. The beauty of the drones was that we could treat them like ’70s samurai with hydraulic fluid, not blood. It was hard, but incredibly fun to do.

F&V: Are you noticing any trends in visual effects?

Definitely. We’ve been pushing for realism for years, but I think we’ve made inroads recently. The new lighting tools we’ve developed here hit their maturity on Iron Man 2, and now we’re using them on all the shows at ILM. They’re better and faster. And we have new compositing techniques. On Iron Man 2, we had a crossover between digital matte painting and compositing because of [Foundry’s] Nuke, which we haven’t had before. The main set for the end battle is an ornamental Japanese garden inside the dome in the Stark expo. This was a set they created on stage, surrounded by a big blue screen. We added extra vegetation. Rather than rendering everything in 3D, we were able to make cycloramas with projections on geometry and take those into Nuke so the artists and compositors could get the shots running. Then, we could go in and say, “On this one, we need real 3D.”

And in the sequence in the start with the dancers. when they introduce the droids, the first seven or 10 rows of people are real; the rest were CG crowds and set extensions. The increased power of our 2D and 3D compositing tools have improved these things and this is a big trend that we will leverage in the future.

F&V: Do these faster, better tools ease the scheduling pain?

The time pressures aren’t going away. They’ve gotten worse and worse on every successive project over the last four or five years in terms of the time crunch. But we’re starting to engage in the process earlier.

I think the filmmakers realize that we can do what they want in the time allowed and obviously can do a good job, but it’s better to have more breathing room. The tools give us a bit more time; we can achieve better realism faster. But, it’s still good to have extra padding. If we can be involved from the get-go, we can start building assets earlier. If something is going to be fully synthetic, there’s no reason to wait.

F&V: You’re deep into Pirates right now. What can you tell us about that film?

It’s more of a return to the scale of the first film. It’s a lot of fun. Certainly the mermaids. They’re a challenge, but I really like them. I’ve always liked mermaids.