The director and assistant editor of the upcoming
film
Together Again for the First Time explain how
they used an iPod and Final Cut Pro for video playback during
production of the indie feature. While not achieving TRUE dailies,
their new-found continuity tool Did change the entire creative
process.
Continuity in film and video production has always been a complicated
task for filmmakers. Apart from continuity stills and the relatively
new use of video playback, there has been little improvement in this
area.
During pre-production for the feature film, Together Again for
the First Time,
directed by Jeff and being shot in 35mm with
the Moviecam Compact camera system, we decided to look in another
direction to address this challenge. Instead of recording our footage
to a video tape recorder, we captured it to an iPod. The iPod’s size
and its ability to randomly access multimedia like video, pictures and
music helped us in more areas than continuity. It gave us more freedom
to focus on all of our creative choices.
The Problem
Last spring, as we began pre-production on the film, we discussed our
approach for video playback. Because films are created one shot at a
time, scenes are most often shot over several days or weeks. Having a
reliable system for playing back the previous shots is critical for
continuity in performance, art direction, camera movement and angles,
and so on. It’s also a given if we want to create the illusion of a
seamless reality and give the story a chance to engage the audience on
its own terms.
On previous projects we’ve recorded shots from the camera’s video tap
to VHS, Hi-8 or MiniDV cassettes. However, with each shooting day, the
number of cassettes grows. On Together Again, our
shooting ratio was roughly 15:1, which would have amounted to
approximately 25 one-hour tapes. Also, like most productions, we shot
Together Again out of sequence. It took precious production time, which
we came to view as lost production time, to find the appropriate tape
and specific scene whenever we needed a shot from a previous shooting
day. On top of this, the video playback area, known as "video village,"
was located away from the set; we knew that running between the actors
and video village would have affected shooting times and performance
spontaneity. There had to be a better way.
As we considered our options for playback, we listed the qualities of
our dream playback system: We wanted something that would not only let
us maintain continuity by storing all the shot footage, but would also
be portable, have random access to all shots and maintain all of this
footage with a well-organized system. Later, we added even more to our
wish list: This system would need to store previsualization materials,
such as animatics, storyboards, reference photographs, location photos
and music. Jeff and Associate Producer Tom Morrill agreed that the Sony
Play Station Portable (PSP) Jeff’s son used so frequently could be a
terrific tool for capturing and viewing footage. But the realities of
making the PSP do what we needed seemed too time consuming, and we
eventually abandoned the idea.
The Solution
Four months before principle photography of Together
Again,
David, also assistant editor, came up with another way
to solve our video playback dilemma: the Apple video iPod. We both work
primarily on the Mac, so any QuickTime-based workflow seemed less
daunting to us from the start. David’s original proposal was to capture
the transferred film dailies to the iPod so they would be available on
set. However, because we were shooting 35mm film in Utah, processing
and transferring in Los Angeles, editing and mixing in Utah and then
cutting negatives in Los Angeles, the turnaround time of our dailies
would have made them more like "weeklies." In other words, using the
iPod to view dailies wouldn’t take advantage of all of iPod’s
potential. Yet Jeff was intrigued- what if we were able to make the
footage we had just shot immediately available on the iPod?
There were three major problems with this: How to capture the video and
compress it for the iPod, how to do this quickly enough to keep up with
production and how to maintain all of the required equipment in a
compact, mobile setup.
David returned with a new proposal: Use an analog-to-digital converter
box to import the video tap signal into a PowerBook, then import the
captured files into iTunes and upload them to the iPod. The iPod could
be updated between setups, or more frequently if needed. David also
suggested we use the captured files for some initial editing- both on
set and off.
Testing A QuickTime Pro Workflow
To accomplish this, David decided to use a simple digital-to-analog box
so we could convert the SD video to DV and input it into his PowerBook
(1.67 GHz, 1 GB of RAM, 64 MB of VRAM) via FireWire using QuickTime
Pro’s capture tool. Using QuickTime Pro would also keep down software
costs- an important consideration for a low-budget show. David began
running tests with the different capture settings, video quality and
compression time to test the feasibility of this approach. [Note:
QuickTime Pro conversion has also been used successfully on bigger
budget projects as well. The QT-centric remote collaborative editing
workflow pioneered by San Francisco’s PIX System, for example, was used
by director Sam Mendes and editor Walter Murch during production and
post on Jarhead and by director David Fincher on
Zodiac. To read more about PIX System’s workflow, go to
www.studiomonthly.com.]
Capture Settings
QuickTime Pro offers three capture codecs: MPEG-4, H.264 and device
native. Image brightness, David discovered, was a major variable in
each of these codecs. Though this usually isn’t a problem, a video tap
system can significantly affect the amount of light delivered to the
external monitor. Once the film camera’s shutter begins to spin,
depending on the optics, the light hitting the video tap’s chip can
decrease by as much as 80 percent. In new film cameras and tap systems,
this light loss has been reduced substantially. Each codec has its own
list of advantages and disadvantages, and David evaluated each codec
based on its picture quality and ability to upload to the iPod.
If we were to capture with MPEG-4, we thought, we could then upload the
clip immediately to the iPod. However, MPEG-4 doesn’t handle low-light
situations well, and much of Together Again was shot at
night.
We quickly ruled out MPEG-4.
H.264 is an excellent codec and we were excited to see how well it
performed. We especially liked the fact that H.264 compression saved
hard disk space while retaining quality. David captured several files
with H.264, then attempted to upload them to the iPod. However, the
iPod wasn’t so eager. The error message read: ".mov was not copied to
the iPod because it cannot be played on this iPod." David then
compressed the H.264 files to MPEG-4. But this took nearly four times
longer than capturing in real time. (The iPod could, however, use these
files and they looked great. In fact, it was hard to tell the
difference in picture quality between clips imported at DV quality and
at H.264 quality.) There was another problem with H.264- at least on
David’s PowerBook- it was unable to capture at a constant frame rate.
Next, David tested the device’s native setting. On Together Again, this
was DV due to the digital-to-analog converter box. (While it’s possible
to use other input devices for a higher-resolution input, it’s also
important to consider the resolution of the video tap output. Our video
tap’s output was analog, so there was no reason to go to a
higher-resolution format.) The image looked good and we knew it would
give us the ability to edit. And compression to MPEG-4 only took twice
as long as real time.
Keeping up with Production
Once the footage was captured, we needed a procedure for compressing
the footage and uploading it to the iPod while simultaneously capturing
the next take being filmed. Tests showed that QuickTime Pro was capable
of capturing at DV quality and compressing to MPEG-4 at the same time,
with no apparent problems. But we learned otherwise once we attempted
to edit on set. Because we knew DV could be imported into Final Cut
Pro, we made the assumption that there would be no problems editing
that footage. We all know what happens when we assume- more on that
later.
Assembling The Equipment
David presented his tests and the provisional workflow to Jeff, who was
excited to move forward on it. The next step was to decide what
specific equipment to use and how to integrate it with the demands of
production. Our final equipment list included: a PowerBook G4 (1.67
GHz, 1 GB RAM, 64 MB VRAM, 80 GB hard drive), a Datavideo DAC-200 Media
Format Converter with FireWire input/output, analog input/output with
external power supply and a FireWire repeater, a Maxtor 160 GB hard
drive, a LaCie Big Disk 300 GB hard drive, both with external power
supplies, and one 30 GB iPod.
We initially planned to capture each day’s files directly to the
PowerBook’s internal drive and then back them up to DVDs. However,
these files consumed an average of 20 GB of storage per day, amounting
to roughly 500 gigs for the entire shoot. We quickly calculated that
this would require over 100 DVDs and would create similar problems if
using video tapes. It also would have created the additional step of
transferring the files back from the DVDs to the Final Cut Pro system
when we wanted to edit. Ultimately, we decided to capture directly to a
hard drive, then simply back up the day’s files to another hard drive.
A week into production, we found that we frequently lost power to the
drives and the D/A converter box whenever we changed camera setups. We
decided to add an APC PS 500 battery backup/surge protector, which gave
us time to power down the hard drives without corrupting the data (the
PowerBook would default to its battery).
The last piece we needed was a setup to contain this equipment, one
that would be compact, mobile and require few cables. Taking a cue from
the sound crew, we assembled everything on a media cart purchased from
an office supply store. David coined the term "capture camp" to refer
to this setup.
Setup and Workflow
The initial consideration for our workflow was acquiring the video
signal from the video tap. We also needed the video signal to go to
three different areas: The director’s monitor, the sound cart and
capture camp. Since the production couldn’t afford a wireless video tap
system to send the signal to all three places at the same time, we
needed to deliver the signal via cable without tangling up set
operations. We decided to run a single cable from the camera’s video
tap box to the director’s monitor. We would then split the signal to
the sound cart and capture camp, where it could be input into the RCA
jack of the DA converter box.
We also decided to feed the audio into the DA converter box via a
wireless COMTEK system, and feed the mono audio signal from the COMTEK
box into the RCA audio input of the DA converter box.
We would connect the DA converter box to the PowerBook via FireWire
400. From there, we planned to capture the signal using QuickTime Pro,
as we had done in our tests. After capturing, we would begin
compressing the clip in QuickTime Pro using the "Movie to iPod"
setting. While one clip was compressing, we would begin capturing the
next take. We would then drop the compressed clip- named by scene, shot
number, take and description- into an iTunes playlist, and name it by
scene number with a brief description. If all went according to plan,
we would update Jeff’s iPod between setups, or as needed. After all our
testing, we thought we had the ideal system. We soon learned what we
had overlooked.
In Production
Once production started, capturing and compressing the clips went
smoothly. However, because compression takes longer than capturing, we
soon had multiple clips compressing while we were capturing new clips.
As you might expect, this maxed out the processing power of the
PowerBook and caused dropped frames in the captured clips. It all seems
obvious now, but we didn’t see at the time how quickly the number of
clips being compressed during capturing could add up.
We discovered an additional snag when David tried importing clips
captured with QuickTime Pro into Final Cut Pro. This time, the error
read: "Final Cut Pro does not support this frame rate or the way it is
represented." QuickTime was unable to maintain a constant frame rate
while capturing, which also explained why we weren’t able to import the
other codecs, MPEG-4 and H.264, into Final Cut Pro. Further testing
after production showed that this was the case no matter how fast the
machine was. (David tested with a Mac Pro and received the same error.)
Being unable to simultaneously capture and compress multiple clips at
an accurate frame rate nullified our entire system. After some quick
tests, however, we switched from capturing in QuickTime Pro to
capturing in Final Cut Pro. Separating the capture program from the
compression program allowed the computer to do both.
A week into production we’d ironed out all the major wrinkles and were
able to operate smoothly. In fact, during production, we needed someone
else to run capture camp when David left the set to capture dailies as
assistant editor. The workflow was easy enough that we were able to use
three other operators on set without any problems.
There are countless examples of how using the iPod for playback made us
more efficient. In one case, an actor left the production early and had
to be replaced. This meant that several scenes featuring this actor
needed to be reshot. But because this put us significantly behind
schedule, we couldn’t afford to reshoot entire scenes. We decided to
reshoot with the new actor only those shots where the original actor
appeared. This meant matching the action and the blocking of the
original master shot so that it would intercut with the previously shot
scene. Not only did Jeff frequently refer to the iPod, but so did DP
Brandon Christensen, the art department, the script supervisor and the
actors. The iPod’s portability, random access and ease of use made
shooting these scenes far less complicated and time consuming than
expected. The new shots cut into the existing scenes without a problem.
Next Time
Based on what we learned making Together Again,
we’ll make some modifications for our next project. For example, we’ll
use a more powerful computer, build a larger cart, hardwire the audio
into the computer and use more than one iPod. We’re also considering
using Apple’s newly released AppleTV. Although the PowerBook was able
to handle all that we threw at it, as always, a more powerful computer
will provide quicker turnaround from capture to compression to the iPod
import.
Throughout production, the size of our cart was problematic: it barely
accommodated all the equipment. This caused the powerstrip cables, the
cables connecting the drives to the computer and the cable connecting
the converter box to the computer to hang off the cart. Despite
zip-tying these cables to the cart, we were always worried that the
actors or crew might trip on them or that the cables would accidentally
get caught by a passing c-stand. Our cart had small wheels and wasn’t
designed to be all-terrain, so transporting it through a field or along
a dirt road became a serious problem. Our ideal cart design will have a
rack to secure the computer, the input box, a monitor and additional
hard drives. It should also give us a better way to organize all our
cables.
On our next project, we’ll hardwire audio into the computer instead of
the Comtek boxes. At times during production, capture camp and the
audio cart were so far away from each other that the audio often broke
up or was distorted. Even when the carts were close together, the audio
signal was sporadic at best, making it difficult to review
performances. To maximize the iPod’s efficiency, clean audio is a must.
The camera and art department heads frequently visited capture camp,
but this took them away from set. Providing them with their own iPods
would let them remain on set without disruption.
In writing this article, we tested other possible workflows to see if
we could leapfrog compression and import the clips directly onto the
iPod from Final Cut Pro. However, even though we could capture at
MPEG-4 and H.264 without a problem, the iPod would still not import
those files without being compressed in QuickTime Pro.
In most instances, the iPod was sufficient for the Jeff’s use.
Sometimes, however, it was still necessary for him to visit capture
camp to see the picture on a larger screen. But with Apple’s
announcement of the AppleTV, we see a possibility that could let the
director remain on set more often. When necessary, we could use AppleTV
to wirelessly send a clip to the director’s monitor and immediately
update his screen after each take.
We also see potential enhancements to our workflow with Apple’s
announcement of the iPhone. In the current process, David and the
others sometimes had to track Jeff down just to update his iPod.
Ideally, we would love to be able to wirelessly update the iPod with
iTunes. The current iPhone specs seem to indicate that it’s only
possible to beam files to iPhone through Bluetooth exchange, but not to
iTunes. Since Zune already has the ability to send songs wirelessly, we
hope that something similar will be offered in a future iPod or iPhone
release.
The freedom we experienced using the iPod was invaluable. The
combination of random access, user-friendly software, media management
and portability created a much more efficient approach to video
playback. Not only was the director able to remain on set more often
and maintain continuity, but using the iPod was, and is, simply more
fun.