Lights, Camera, iPod

The director and assistant editor of the upcoming film Together Again for the First Time explain how they used an iPod and Final Cut Pro for video playback during production of the indie feature. While not achieving TRUE dailies, their new-found continuity tool Did change the entire creative process.

Continuity in film and video production has always been a complicated task for filmmakers. Apart from continuity stills and the relatively new use of video playback, there has been little improvement in this area.

During pre-production for the feature film, Together Again for the First Time, directed by Jeff and being shot in 35mm with the Moviecam Compact camera system, we decided to look in another direction to address this challenge. Instead of recording our footage to a video tape recorder, we captured it to an iPod. The iPod’s size and its ability to randomly access multimedia like video, pictures and music helped us in more areas than continuity. It gave us more freedom to focus on all of our creative choices.

The Problem

Last spring, as we began pre-production on the film, we discussed our approach for video playback. Because films are created one shot at a time, scenes are most often shot over several days or weeks. Having a reliable system for playing back the previous shots is critical for continuity in performance, art direction, camera movement and angles, and so on. It’s also a given if we want to create the illusion of a seamless reality and give the story a chance to engage the audience on its own terms.

On previous projects we’ve recorded shots from the camera’s video tap to VHS, Hi-8 or MiniDV cassettes. However, with each shooting day, the number of cassettes grows. On Together Again, our shooting ratio was roughly 15:1, which would have amounted to approximately 25 one-hour tapes. Also, like most productions, we shot Together Again out of sequence. It took precious production time, which we came to view as lost production time, to find the appropriate tape and specific scene whenever we needed a shot from a previous shooting day. On top of this, the video playback area, known as "video village," was located away from the set; we knew that running between the actors and video village would have affected shooting times and performance spontaneity. There had to be a better way.

As we considered our options for playback, we listed the qualities of our dream playback system: We wanted something that would not only let us maintain continuity by storing all the shot footage, but would also be portable, have random access to all shots and maintain all of this footage with a well-organized system. Later, we added even more to our wish list: This system would need to store previsualization materials, such as animatics, storyboards, reference photographs, location photos and music. Jeff and Associate Producer Tom Morrill agreed that the Sony Play Station Portable (PSP) Jeff’s son used so frequently could be a terrific tool for capturing and viewing footage. But the realities of making the PSP do what we needed seemed too time consuming, and we eventually abandoned the idea.

The Solution

Four months before principle photography of Together Again, David, also assistant editor, came up with another way to solve our video playback dilemma: the Apple video iPod. We both work primarily on the Mac, so any QuickTime-based workflow seemed less daunting to us from the start. David’s original proposal was to capture the transferred film dailies to the iPod so they would be available on set. However, because we were shooting 35mm film in Utah, processing and transferring in Los Angeles, editing and mixing in Utah and then cutting negatives in Los Angeles, the turnaround time of our dailies would have made them more like "weeklies." In other words, using the iPod to view dailies wouldn’t take advantage of all of iPod’s potential. Yet Jeff was intrigued— what if we were able to make the footage we had just shot immediately available on the iPod?

There were three major problems with this: How to capture the video and compress it for the iPod, how to do this quickly enough to keep up with production and how to maintain all of the required equipment in a compact, mobile setup.

David returned with a new proposal: Use an analog-to-digital converter box to import the video tap signal into a PowerBook, then import the captured files into iTunes and upload them to the iPod. The iPod could be updated between setups, or more frequently if needed. David also suggested we use the captured files for some initial editing— both on set and off.

Testing A QuickTime Pro Workflow

To accomplish this, David decided to use a simple digital-to-analog box so we could convert the SD video to DV and input it into his PowerBook (1.67 GHz, 1 GB of RAM, 64 MB of VRAM) via FireWire using QuickTime Pro’s capture tool. Using QuickTime Pro would also keep down software costs— an important consideration for a low-budget show. David began running tests with the different capture settings, video quality and compression time to test the feasibility of this approach. [Note: QuickTime Pro conversion has also been used successfully on bigger budget projects as well. The QT-centric remote collaborative editing workflow pioneered by San Francisco’s PIX System, for example, was used by director Sam Mendes and editor Walter Murch during production and post on Jarhead and by director David Fincher on Zodiac. To read more about PIX System’s workflow, go to www.studiomonthly.com.]

Capture Settings

QuickTime Pro offers three capture codecs: MPEG-4, H.264 and device native. Image brightness, David discovered, was a major variable in each of these codecs. Though this usually isn’t a problem, a video tap system can significantly affect the amount of light delivered to the external monitor. Once the film camera’s shutter begins to spin, depending on the optics, the light hitting the video tap’s chip can decrease by as much as 80 percent. In new film cameras and tap systems, this light loss has been reduced substantially. Each codec has its own list of advantages and disadvantages, and David evaluated each codec based on its picture quality and ability to upload to the iPod.

If we were to capture with MPEG-4, we thought, we could then upload the clip immediately to the iPod. However, MPEG-4 doesn’t handle low-light situations well, and much of Together Again was shot at night. We quickly ruled out MPEG-4.

H.264 is an excellent codec and we were excited to see how well it performed. We especially liked the fact that H.264 compression saved hard disk space while retaining quality. David captured several files with H.264, then attempted to upload them to the iPod. However, the iPod wasn’t so eager. The error message read: ".mov was not copied to the iPod because it cannot be played on this iPod." David then compressed the H.264 files to MPEG-4. But this took nearly four times longer than capturing in real time. (The iPod could, however, use these files and they looked great. In fact, it was hard to tell the difference in picture quality between clips imported at DV quality and at H.264 quality.) There was another problem with H.264— at least on David’s PowerBook— it was unable to capture at a constant frame rate.

Next, David tested the device’s native setting. On Together Again, this was DV due to the digital-to-analog converter box. (While it’s possible to use other input devices for a higher-resolution input, it’s also important to consider the resolution of the video tap output. Our video tap’s output was analog, so there was no reason to go to a higher-resolution format.) The image looked good and we knew it would give us the ability to edit. And compression to MPEG-4 only took twice as long as real time.

Keeping up with Production

Once the footage was captured, we needed a procedure for compressing the footage and uploading it to the iPod while simultaneously capturing the next take being filmed. Tests showed that QuickTime Pro was capable of capturing at DV quality and compressing to MPEG-4 at the same time, with no apparent problems. But we learned otherwise once we attempted to edit on set. Because we knew DV could be imported into Final Cut Pro, we made the assumption that there would be no problems editing that footage. We all know what happens when we assume— more on that later.

Assembling The Equipment

David presented his tests and the provisional workflow to Jeff, who was excited to move forward on it. The next step was to decide what specific equipment to use and how to integrate it with the demands of production. Our final equipment list included: a PowerBook G4 (1.67 GHz, 1 GB RAM, 64 MB VRAM, 80 GB hard drive), a Datavideo DAC-200 Media Format Converter with FireWire input/output, analog input/output with external power supply and a FireWire repeater, a Maxtor 160 GB hard drive, a LaCie Big Disk 300 GB hard drive, both with external power supplies, and one 30 GB iPod.

We initially planned to capture each day’s files directly to the PowerBook’s internal drive and then back them up to DVDs. However, these files consumed an average of 20 GB of storage per day, amounting to roughly 500 gigs for the entire shoot. We quickly calculated that this would require over 100 DVDs and would create similar problems if using video tapes. It also would have created the additional step of transferring the files back from the DVDs to the Final Cut Pro system when we wanted to edit. Ultimately, we decided to capture directly to a hard drive, then simply back up the day’s files to another hard drive.

A week into production, we found that we frequently lost power to the drives and the D/A converter box whenever we changed camera setups. We decided to add an APC PS 500 battery backup/surge protector, which gave us time to power down the hard drives without corrupting the data (the PowerBook would default to its battery).

The last piece we needed was a setup to contain this equipment, one that would be compact, mobile and require few cables. Taking a cue from the sound crew, we assembled everything on a media cart purchased from an office supply store. David coined the term "capture camp" to refer to this setup.

Setup and Workflow

The initial consideration for our workflow was acquiring the video signal from the video tap. We also needed the video signal to go to three different areas: The director’s monitor, the sound cart and capture camp. Since the production couldn’t afford a wireless video tap system to send the signal to all three places at the same time, we needed to deliver the signal via cable without tangling up set operations. We decided to run a single cable from the camera’s video tap box to the director’s monitor. We would then split the signal to the sound cart and capture camp, where it could be input into the RCA jack of the DA converter box.

We also decided to feed the audio into the DA converter box via a wireless COMTEK system, and feed the mono audio signal from the COMTEK box into the RCA audio input of the DA converter box.

We would connect the DA converter box to the PowerBook via FireWire 400. From there, we planned to capture the signal using QuickTime Pro, as we had done in our tests. After capturing, we would begin compressing the clip in QuickTime Pro using the "Movie to iPod" setting. While one clip was compressing, we would begin capturing the next take. We would then drop the compressed clip— named by scene, shot number, take and description— into an iTunes playlist, and name it by scene number with a brief description. If all went according to plan, we would update Jeff’s iPod between setups, or as needed. After all our testing, we thought we had the ideal system. We soon learned what we had overlooked.

In Production

Once production started, capturing and compressing the clips went smoothly. However, because compression takes longer than capturing, we soon had multiple clips compressing while we were capturing new clips. As you might expect, this maxed out the processing power of the PowerBook and caused dropped frames in the captured clips. It all seems obvious now, but we didn’t see at the time how quickly the number of clips being compressed during capturing could add up.

We discovered an additional snag when David tried importing clips captured with QuickTime Pro into Final Cut Pro. This time, the error read: "Final Cut Pro does not support this frame rate or the way it is represented." QuickTime was unable to maintain a constant frame rate while capturing, which also explained why we weren’t able to import the other codecs, MPEG-4 and H.264, into Final Cut Pro. Further testing after production showed that this was the case no matter how fast the machine was. (David tested with a Mac Pro and received the same error.)

Being unable to simultaneously capture and compress multiple clips at an accurate frame rate nullified our entire system. After some quick tests, however, we switched from capturing in QuickTime Pro to capturing in Final Cut Pro. Separating the capture program from the compression program allowed the computer to do both.

A week into production we’d ironed out all the major wrinkles and were able to operate smoothly. In fact, during production, we needed someone else to run capture camp when David left the set to capture dailies as assistant editor. The workflow was easy enough that we were able to use three other operators on set without any problems.

There are countless examples of how using the iPod for playback made us more efficient. In one case, an actor left the production early and had to be replaced. This meant that several scenes featuring this actor needed to be reshot. But because this put us significantly behind schedule, we couldn’t afford to reshoot entire scenes. We decided to reshoot with the new actor only those shots where the original actor appeared. This meant matching the action and the blocking of the original master shot so that it would intercut with the previously shot scene. Not only did Jeff frequently refer to the iPod, but so did DP Brandon Christensen, the art department, the script supervisor and the actors. The iPod’s portability, random access and ease of use made shooting these scenes far less complicated and time consuming than expected. The new shots cut into the existing scenes without a problem.

Next Time

Based on what we learned making Together Again, we’ll make some modifications for our next project. For example, we’ll use a more powerful computer, build a larger cart, hardwire the audio into the computer and use more than one iPod. We’re also considering using Apple’s newly released AppleTV. Although the PowerBook was able to handle all that we threw at it, as always, a more powerful computer will provide quicker turnaround from capture to compression to the iPod import.

Throughout production, the size of our cart was problematic: it barely accommodated all the equipment. This caused the powerstrip cables, the cables connecting the drives to the computer and the cable connecting the converter box to the computer to hang off the cart. Despite zip-tying these cables to the cart, we were always worried that the actors or crew might trip on them or that the cables would accidentally get caught by a passing c-stand. Our cart had small wheels and wasn’t designed to be all-terrain, so transporting it through a field or along a dirt road became a serious problem. Our ideal cart design will have a rack to secure the computer, the input box, a monitor and additional hard drives. It should also give us a better way to organize all our cables.

On our next project, we’ll hardwire audio into the computer instead of the Comtek boxes. At times during production, capture camp and the audio cart were so far away from each other that the audio often broke up or was distorted. Even when the carts were close together, the audio signal was sporadic at best, making it difficult to review performances. To maximize the iPod’s efficiency, clean audio is a must.

The camera and art department heads frequently visited capture camp, but this took them away from set. Providing them with their own iPods would let them remain on set without disruption.

In writing this article, we tested other possible workflows to see if we could leapfrog compression and import the clips directly onto the iPod from Final Cut Pro. However, even though we could capture at MPEG-4 and H.264 without a problem, the iPod would still not import those files without being compressed in QuickTime Pro.

In most instances, the iPod was sufficient for the Jeff’s use. Sometimes, however, it was still necessary for him to visit capture camp to see the picture on a larger screen. But with Apple’s announcement of the AppleTV, we see a possibility that could let the director remain on set more often. When necessary, we could use AppleTV to wirelessly send a clip to the director’s monitor and immediately update his screen after each take.

We also see potential enhancements to our workflow with Apple’s announcement of the iPhone. In the current process, David and the others sometimes had to track Jeff down just to update his iPod. Ideally, we would love to be able to wirelessly update the iPod with iTunes. The current iPhone specs seem to indicate that it’s only possible to beam files to iPhone through Bluetooth exchange, but not to iTunes. Since Zune already has the ability to send songs wirelessly, we hope that something similar will be offered in a future iPod or iPhone release.

The freedom we experienced using the iPod was invaluable. The combination of random access, user-friendly software, media management and portability created a much more efficient approach to video playback. Not only was the director able to remain on set more often and maintain continuity, but using the iPod was, and is, simply more fun.