Translating Chinese to English at 25 frames per second

Imagine a product commercial shot in Taiwan with a beautiful celebrity holding up the packaging… all in Chinese. Now, imagine having to air this commercial on an English channel, with no way to reshoot the footage using the English packaging. Oops.

Fortunately, with great faith, nothing is impossible!

And it was with great faith that the good people at Muse Post handed us 3 shots to solve. 2 of these shots involved tracking and replacing the Chinese packaging with new English artwork, while the third shot required us to create the ‘hero shot’ of the English packaging in 3d animation.

We first needed to rotoscope the new English pack design into position. Here’s what we had to work with:

One of the key challenges was the heavy motion blur in the source footage, which made it hard to motion track, and also meant that we would have to replicate the motion blur in order for the shot to look believable. We also needed to match the lighting, texture and even grain on the replaced element, so that the entire effect would look as seamless and invisible as possible. The comparison below shows just how much work was done to make it look ‘natural’.

Left: The new pack design tracked in. Right: Reflections and metallic shimmers were added during compositing, as well as fixing the overexposed side of the box.

With great care and attention to detail, we replaced both the front and side of the packaging and even matched the harsh highlights reflecting off the side of the box.

With the two roto shots wrapped up, we moved on to the 3d animation for the ‘hero shot’. We did the modeling, texturing and lighting, as well as the camera move, and Muse comped it onto their background and graphics to complete the shot. We also provided the wispy light effect that wipes across the box:

Left: The final 3d element. Right: The smoke effect element

Be sure to check out our other full 3d animation project for Darlie!