Creating a Destructive Tsunami to Open a Quiet, Character-Driven Drama

The Oscar nomination for Hereafter is visual effects supervisor and Scanline VFX president Stephan Trojansky’s first, and the first for the studio. However, it isn’t Trojansky or Scanline’s first award. Earlier this year, the studio’s Danielle Plantec, Joel Mendias, Bryan Grill, and Hereafter‘s overall visual effects supervisor Michael Owens received a VES award for outstanding supporting visual effects in a feature. And, in 2008, Trojansky, Thomas Ganshorn and Oliver Pilarski from Scanline received a scientific and technical achievement award from the Academy for the developing the studio’s Flowline fluid effects system. That same year, Trojansky opened the Los Angeles branch of the well-established Munich studio. For Hereafter, Scanline VFX created a sequence in which a main character is trapped inside the 2004 tsunami. Also nominated for an Oscar for Hereafter‘s visual effects are overall supervisor Michael Owens and Scanline’s visual effects supervisor Bryan Grill and compositing supervisor Joe Farrell.
F&V: Were you surprised by the Oscar nomination?

Actually, yes. Hereafter is a great movie, but it isn’t a visual-effects movie with a mega budget. We created a complex sequence in the beginning. When we got long-listed, that was a surprise. And then onto the short list, so we thought, ‘Someone thinks this work should have a place.’ And then the bake-off, where we were fighting for our life to sell the work we did. We thought, ‘2800 people worked on the movies on the short list, and we had 50 people working on our movie.’ But you can’t always work on movies with thousands of people and 90 minutes of visual effects. Maybe that’s why they opened the list to five nominations – to bring more variety into the selections. When we got in, we were so surprised, thankful and excited. I’m learning so much new stuff. The nominees lunch was so much fun. We could see all the other nominees and see that they are all normal people who one time had a dream, followed their career and goal, and here they are.

F&V: When did you first start working on fluid simulations?

I think it was around 2003-2004. I saw the first Navier Stokes simulations at SIGGRAPH and got hooked. It wa so amazing, so much more realistic than particle simulations. I was a visual effects artist at Scanline. I didn’t have any programming experience, so I taught myself C++ and on weekends and at nights, I started to work on my first solver. After a while, I brought a little example to work and showed everybody. They got excited and said that if I put more work into it, they could use if for their next project, which was a shark movie. Later we began thinking about moving this weekend project into the production pipeline. I was working as an artist, so we created a team of developers and it became an in-house R&D project. At that point, we didn’t realize the monster we had released. We were thinking it would be great if we could work on big American movies, and then we became involved in Poseidon, and that gave us huge momentum. Now the studio in Los Angeles has 60 seats and in Germany, we have 80.

F&V: Did the effects in Hereafter differ from your previous work?

Instead of creating big-bang visual effects from a production design, we re-created an historical event, the 2004 tsunami. Before, we made water do something it could not do ‘ crash into the Himalayas [in 2012] or make a water spirit [The Chronicles of Narnia: Prince Caspian]. Hereafter‘s story was about Marie Lelay’s [Cà©cile De France] near death experience as she washed through the water. We watched horrible pictures from mobile cameras, house roofs and balconies. We created a whole street and city environment, with a lot of detail in the buildings that we would destroy with cars crashing into them and telephone poles going into the water.

F&V: How much time did you have for the sequences?

We had three months of preparation in pre-vis and post-vis, with the shoot in between. From the moment the sequence locked to final it was four-and-a-half months. We worked closely with Michael Owens, the overall visual effects supervisor. He was on site and we did the edit together to fine-tune each little shot to be exactly what it was supposed to be.

F&V: What was most difficult about this work?

The incredible technical challenge of matching the live-action plates of her in water, partly filmed in a tank and partly in Lahaina, Hawaii. We had said, “Let’s take the actress into the water, into the ocean, instead of having a green-screen tank with divers underneath and a safety belt one foot away.” So we filmed her in open ocean floating in the water. We had to integrate that footage and that motion into the CG water that rushes down the street at 30 miles per hour, so it was like creating a CG double. The first foot of water between her and the camera is live action, the next foot transitions into CG, and farther in the distance the water is CG, doing what it’s supposed to do. We had to teach the water.

F&V: How did you teach the water? What did you teach the water?

We captured the movement of the live-action water by manually tracking and by using optical flow techniques. We analyzed the motion of the pixels in the image, converted them to 3D space, and had a 3D velocity field in the 3D scene file so we could tell the water to follow exactly where she was. We constrained the simulation to these live-action motion vectors near her and then loosened them farther away so the water could do its own thing as it flowed around the corners of buildings.

When she gets knocked out by an obstacle and transitions into the near-death experience, we went from live-action shots of Maria in the tank to CG. We had a motion-capture shoot using travel rigs that could rotate in all axes that we used to build libraries of moves for digital characters in the water. We used that to slam “her” against an object and then applied that data to a CG double that interacted with CG water.

To define where she would hit the piece of wood, we had to determine where to drop her in the CG water. So, we took 10,000 CG tennis balls and dropped them into the CG water in the first frame of the shot to see which tennis ball would hit the wood at the end of the shot. Then we dropped Marie in the water, knowing she’d end up at the correct spot.

F&V: In addition to creating the CG water, CG doubles, and CG buildings, what work did you do on the show?

All the buildings, the cars. In the main sequence when she’s rushing down the street under water, above water nothing is real except a couple feet around her. In some shots she’s a CG double. We created complete CG environments, did facial performance enhancements ‘ changed an actor’s face to look more emotional, added digital makeup to Maria to make her more bruised as the shot continued. All the CG techniques came together here. Water is our specialty, but the shots always require everything else, too.

F&V: Are you planning to sell Flowline, your fluid simulation software?

Maybe in the future. Our motivation now is to create cool visual effects.