Creating seamless, realistic digital effects for director Chris Nolan is always a tall order, but doing so in IMAX resolution was an extra added attraction for the post-production studios who worked on Warner Bros.’ box office hit The Dark Knight. “Chris [Nolan] has a very astute eye,” says Nick Davis, the film's visual effects supervisor. “It was important for the visual effects to look as though they were shot on set. He didn’t want the film to take on a gimmicky visual-effects look.” In keeping with that philosophy, Nolan shot footage for the major visual-effects sequences with IMAX cameras, making The Dark Knight the first feature film shown in IMAX that wasn't originally shot in 35mm.
“By shooting in IMAX format we could fill the whole frame, and the images are sharper and have more detail,” Davis says. “But the vendors had to work in much larger resolution and, logistically, it was a difficult movie to finish.”
Visual-effects post-production houses Double Negative, Framestore CFC, BUF, and Cinesite worked on the film's roughly 700 visual-effects shots. Framestore CFC’s 200-plus shots centered on a major IMAX sequence set amidst the tall buildings in Hong Kong, and on digital makeup for Harvey Dent, who appeared in 112 shots as Batman's nemesis Two-Face, none of which were IMAX shots. BUF created the opener in IMAX resolution and Batman’s “Sonar Vision,” which appears primarily during the Prewitt Building sequence. Cinesite helped with rig removal, wire removal and 2D cleanup for the complicated mechanical effects.
The lion’s share of the effects, including all the sequences in Gotham City, went to Double Negative. Of their 370 shots, 170 take place during three major IMAX sequences: The opening bank heist, the extravagant armored-car chase, and a hostage rescue in the Prewitt Building. In addition, Double Negative worked on a few IMAX shots during the Gotham City evacuation. “The visual effects go into the cracks, forming backgrounds and foregrounds,” says Paul Franklin, visual effects supervisor at Double Negative. “They’re always there to support the images in a scene. Never the reason to do the scene.”
To support the IMAX scenes, the studios could not work in full IMAX resolution, which is theoretically 18K; instead, the target resolution was approximately 8K, the maximum resolution for scanned film. Even that was difficult. “A single 8K frame requires 200 MB of data,” Franklin says. “So we had to upgrade our whole infrastructure. We needed faster network speeds to move data around, massively beefed up servers, and — the most important thing — a new compositing solution.”
Shake Up
The studio had upgraded its machines to 64-bit operating systems about a year ago, but Double Negative’s compositing software of choice, Apple’s Shake, still operated in 32-bit memory space. “Shake falls over with images much more than 3K resolution,” Franklin says. “Fortunately, we have a strong R&D team.”
Because the studio had Shake-experienced artists on the crew and existing Shake-based software and workflows, the R&D department created an additional application that worked in conjunction with Shake to get around the memory limitations. “It’s a separate, standalone set of tools, not a plug-in,” Franklin says. “A kind of overlay. Users could transition to the new software easily.”
New blade servers increased data-storage space and improved data transfer rates. “In 1999, when we worked on Pitch Black [released in 2000], we needed to access 2 TB of data,” Franklin says. “This show used over 100 TB of data.”
Additional acreage in the render farm amped up the processing power. “For Batman Begins, we had 200 RenderMan licenses, each of which ran on dual-processor machines,” Franklin says. “For this film, we beefed up to 900 licenses, each running on four processors — 3600 processors. The limiting factor was how much power we could get into the building.” The new processors’ lower power consumption requirements released enough of the energy budget to accommodate the additional servers. However, a larger crew and more artist workstations still tipped the studio toward a power wall. “We tripped the fuses twice, but there were no other outages.”
With the new infrastructure, the 64-bit compositing solution, and efficient load distribution techniques for the 3D pipeline in place, the crew wrangled a small number of shots for the opening bank-heist sequence, which they delivered in October 2007, six months ahead of the rest of film. Warner Bros. released that sequence as a prologue in front of IMAX screenings of I Am Legend.
“It was our test sequence for the pipeline,” Franklin says. “We did those shots at 8K. After that, we went down to 5.6K except for high-contrast shots where the camera lingers on vehicles and architecture.” The lower resolution reduced the data requirements to approximately 160 MB per frame.
As for the content in the shots, to support high-res IMAX close-ups of Gotham City, Double Negative artists overhauled all the buildings that they picked up from Batman Begins, adding geometric detail and repainting all the textures. In addition, the studio modeled the Prewitt Building from scratch for IMAX shots using reference filmed on location in Trump International Hotel and Tower, which was under construction in Chicago.
“We had to create more environments from scratch and had more matte paintings than in Batman Begins,” Franklin says. “We also built the digital Batpod and Batmobile, and all those assets had to hold up in full-frame, harsh lighting conditions. Things we could get away with at 2K wouldn’t cut the mustard on Dark Knight.” In addition to the higher-resolution format pushing the need for detail, the camera moved closer to the digital architecture, the digital vehicles and the digital stunt doubles.
Ground Work
“I think Chris [Nolan] had greater confidence this time,” Franklin says. “For example, they attempted to do the shot of the Batmobile turning into the Batpod as a physical effect with rigs. But Chris wasn’t getting the speed and energy he wanted, so we did a digital version.” In shots with the Batpod racing straight ahead, the bike and rider are real; when Batman turns the bike, lays it down flat, or climbs a wall with it, both are digital, as is ( sometimes) the road beneath the bike’s wheels.
“We tracked the Batpod into live-action plates filmed in Chicago, but in some cases we needed a digital road for new camera moves,” Franklin says. “The only way was to photograph the road in excruciating detail. We recreated a quarter mile of LaSalle Street and 250 yards of Lower Wacker Drive to one-centimeter resolution.”
For tracking, the studio uses boujou, PF Track, 3D Equalizer and a proprietary photogrammetry toolset. “We can give it point tracks from any source and it will give us a solve for the camera,” Franklin says of the proprietary software. To match the IMAX camera lenses, Double Negative primarily characterized Zeiss lenses from Hassleblad cameras. “The difference is that the IMAX camera uses proprietary mounts to fit the lens to the camera body,” Franklin says. “We shot lens-specific distortion grids, but did all the other characterizations with the still lenses.”
To create the road, they photographed the surface at 5K resolution with Canon EOS-1DS Mark II still cameras and LIDAR-scanned the surface to produce a terrain map. “With the 1DS Mark II, we could shoot many times faster than before,” Franklin says. “We’d wait for a gap in traffic or have the police stop traffic for 10 minutes. For Batman Begins, we had to wait for nighttime. It was the same for photographing the buildings. We could wait for the light to be flat rather than wait for the buffer to empty.”
Matte painters worked in 8K resolution, and the artists painted texture maps in either 8K or 16K resolution, depending on the view. “That was a bottleneck,” Franklin says. “Photoshop doesn’t handle images above 4K very efficiently and it’s a closed tool, so we couldn’t get in there and add stuff to it. Working with Photoshop was possible, but slow. It took three or four times longer than usual to paint the textures.”
Rendering was less of a problem. To optimize render times, the studio rewrote shaders and created a new tool called Spangle to preview RenderMan renders in realtime. “It renders on NVIDIA cards,” Franklin says. “It doesn’t impact the render farm.” In addition, a 3D relighting system allowed compositors to change the direction of lights, if needed, without going back into the 3D pipeline.
What You See (Not)
A second big bottleneck, though, was in viewing the images. “Our biggest monitors are 2K,” Franklin says. “You can’t realistically buy a 5.6 x 3.6K monitor, and the highest-resolution digital projector is about 4K.” So the studio wrote a set of tools that extracted 2K tiles from the images for the artists to view, but for dailies, they sent files from their London-based studio to DKP 70mm, the IMAX-subsidiary post facility in Los Angeles, for recording onto film stock.
“It was a minimum of 10 days before we saw the shots back in the UK,” Franklin says. What’s more, to view the shots, they had to book time at London’s only IMAX theater. “And you can’t rock and roll on IMAX projectors,” he adds. “You have to rewind and go again, so we had only a couple chances to view the output.”
As a result, instead of doing IMAX dailies, they did tests to establish the end result and made assumptions based on those tests.
“It was like 10 or 15 years ago, when it was difficult to view 2K frames on digital monitors,” Franklin says. “But this is what 3D artists in particular have always done. We’re used to looking at low res images during the process.”
Framestore CFC VFX supervisor Tim Webber echoes Franklin’s analysis. “Most of our problems came from the process,” he says. “We expected it to be a little harder, but it was way harder. These days, we work with people all over the world, send images digitally and get feedback. But with physical film, there’s a big delay. And matching the color space was harder because they didn’t do a digital grade, so we had to match the IMAX output recorders. They still use CRT recorders, which have a different color response than the laser output recorders most people use today, so we had to put work into that.”
Rather than DI, director Nolan and DP Wally Pfister used their preferred photochemical process. “Wally would go into the lab and call the lights and they’d print it,” says Franklin. “There was no real-time, hands-on grading.”
Scoping it Out
In addition to the IMAX shots, the studios had to produce versions of their shots for anamorphic 35mm prints, as well. [See“Marrying IMAX and 35mm in The Dark Knight”]. For the all-digital sonar-vision shots, BUF created separate IMAX and scope images. “Chris wanted to different shot values,” says Alain Lalanne, visual effects producer. “They tell the same story, but because of the size of the images in IMAX, we had to do things differently. So we had to find technical ways to make all the versions and compute them. It took a lot of organization.”
Double Negative and Framestore CFC, however, could lift scope images from the IMAX footage. “We worked at IMAX resolution all the way through and then did a 2:40 extraction,” Franklin says, “taking the cinemascope slug out of the larger picture, and then downresing to 4K. We initially started working in 2K for cinemascope, but decided that since we had the resources to do IMAX at 5.6K, we’d do the scope at 4K.”
Even though editorial provided extraction guides, cutting the correct shot out of the IMAX scans was tricky. “The telecine operators set up the racking for the color cine transfers to give editorial a safe image for projection, but we got a scan of the whole negative, so there was a discrepancy,” Franklin says. “Tiny things like that wouldn’t normally make much difference in 2K resolution, but everything in IMAX gets magnified.”
Franklin believes the lessons learned on this film will help the studio in the future — even if it never again works in 5.6 or 8K resolution. “Filmmakers increasingly want us to work in 4K, and that used to be a big deal,” he says, and then laughs. “After working on 5.6 and 8 resolution, 4K is proxy resolution.”
Comments (16) for "Delivering 8K VFX Shots for The Dark Knight"
1.
My calculations for the 8k scans came to 187 megs & 5.6k @ 88megs. I wonder how they are getting 200meg to 160 meg for the downrezzes.
Posted by celboy on Thursday, July 24, 2008 @ 09:23 AM
2.
Just when we thought we had blisteringly fast machines to do 2k, some jumped up director comes along and denotes that '4k was too small' oh please! Just give it up. From about 3k onwards the difference in sharpness between the resolutions starts to level off considerably. There is little perceptible difference between a 4,5,6 or even 7k scan in terms of sharpness and resolution, hence why the Digital Cinema Initiative denotes that 4k digital projectors will be sufficient to display correctly all the nuances that film has to offer.
Really at this
Posted by majik on Thursday, July 24, 2008 @ 07:02 PM
3.
Give me a break... there's a huge "perceptible difference" between 4K and 7K scan in res. Buy some new glasses nerd!
Posted by Dan Bailey on Friday, July 25, 2008 @ 10:03 AM
4.
Thanks for the comments.
Celboy, good question. Barbara went back to Paul Franklin at Double Negative and asked him to run us through the exact numbers. Turns out the 5.6K files took up a little less space than he had remembered. Here are his notes:
###
5.6K: 5616x4096;
A full 5.6K was actually about 100 meg for the exr and 122 meg for the
cineon/dpx;
8K: 8192x6144;
approximately 150 meg for the exr and 200 for the cineon/dpx;
exr files are run length encoded whereas cineon/dpx is not, so as well as
being more efficient they also vary in size depending on how much detail
(difference between pixels) there is on a frame to frame basis;
My mistake on the downrez figure for the 8K to 5.6
Posted by Bryant Frazer on Friday, July 25, 2008 @ 10:57 AM
5.
Thank you for explanation about 5.6 and 8k, is fantastic get this quality.
Posted by Oswaldo on Friday, July 25, 2008 @ 11:31 AM
6.
I'd love to see a full resolution lossless 8K frame from the film. It makes me curious to see how much practical detail is actually there.
Posted by atomisok on Friday, July 25, 2008 @ 05:01 PM
7.
Clerk: "What should we do with all these Gates 4000s?"
Manager: "Throw those pieces of crap in the trash!"
Posted by Dan on Friday, July 25, 2008 @ 05:38 PM
8.
Was there any rotoscoping in the picture? It isn't mentioned even though it's a big part of post, isn't it?
Posted by George H on Friday, July 25, 2008 @ 06:07 PM
9.
To Mr Bailey - have you actually done a comparison between 4K and 7K? Perhaps you should. And you would discover that the previous poster is correct - you get diminishing returns the higher the resolution - due to several factors. The only people who claim outrageous benefits of big resolutions sell film scanners.
Posted by Ian on Saturday, July 26, 2008 @ 02:25 AM
10.
I wonder what is the purpose of all these extreme resolutions when, for example, Fuji gives a maximun (decent) resolving power of 60 cycles for its new Eterna-RDI.
(In case nobody knows 60 cycles could be thought as 120 pixels per millimeter, which gives us for a 20 mm area on a 35mm negative around 2400 pixels horizontally.)
Not to mention that Contact printers, on their manuals, specify a maximum resolution of 50 line pairs, which can be thought as 100 pixels per millimeter por a projection copy.
One can also take into account that a color film dye cloud is around 10-25 microns diameter, and you get an idea of the resolution limitations.
Posted by J. M. M. Fiebelkorn on Saturday, July 26, 2008 @ 02:32 AM
11.
Thats odd with all the 8K processing and stuff the movie should have looked really sharp but it was no better than any other 35mm film - in fact i think the camera was so out of focus they probably could have done it in 2K and it would have looked the same !
Posted by stephen on Saturday, July 26, 2008 @ 04:42 AM
12.
I'm just a normal movie-goer, but I can tell you that the IMAX portions of the movie were enormously effective. The audience I was in gave a collective "whoa!" when the opening IMAX shot appeared. It was like I've been looking at the world through a slit in a box, and suddenly someone took the box off my head and I could see EVERYTHING. A profoundly moving experience.
Posted by Dave on Saturday, July 26, 2008 @ 09:56 AM
13.
Fiebelkorn:
Using your number of 100pixels per mm.
IMAX uses 70mm film (turned sideways)
so 7k pixels in height and at 2.4 aspect ratio =16800
So the IMAX film has a theoretical pixel limit of 16k x 7k image size.
They did release this in IMAX so they did need to keep the image quality up to similar standards. The "enhancement" to the 35mm print? See the linked article for comments on the tests they did in deciding if it was worth it.
Posted by Dale on Saturday, July 26, 2008 @ 02:31 PM
14.
stephen, clearly the projectionist at your theater had the focus off.
Posted by ben on Tuesday, July 29, 2008 @ 03:34 PM
wiki and others indicate a 1.43 aspect ratio, or 70mm x 49mm.
Posted by mo_joe on Friday, August 1, 2008 @ 12:22 AM
16.
What's the MTF of the film recorder the print stock and the neg as a concatenated tranfer function?
Is there a film recorder out there that can even shoot 8K?
Sorry, I'm with the guys who are saying "WTF" on this one. The screen value of the quadrupled file size is not justified.
It's not the size that matters, it's how you use it.
Posted by Huh What? on Tuesday, January 20, 2009 @ 06:23 PM
Post your comments below
