Motion Capture, Mo-Cap for short, is a way of recording the movement of people and objects, which can be used in filmmaking. In computer animation and visual effects studios the technique is used to combine actions in the real world with digitally-created computer objects and actions. The more advanced form of mo-cap used to recreate facial expressions is called 'Performance Capture'. Perhaps the technique's biggest successes to date include the character of Gollum, who was played by Andy Serkis in The Lord of the Rings: The Two Towers (2002). James Cameron's Avatar (2009) is another milestone. The technique has also been used to make animated films, particularly those by Robert Zemeckis' studio ImageMovers.

Mo-Cap

The basic level of the technique is called Mo-Cap. For this, performers wear special mo-cap suits. These have markers, which traditionally look like blue ping-pong balls, attached to key parts of their body. As the performer moves around, cameras record the markers' motion and the information is fed into a computer. By seeing how these markers move, the computer is able to replicate the movement on a computer-generated character, moving the character in an identical fashion to the original performer. So if a performer waves their arm, the computer sees these markers moving. Each marker is linked to the relevant part of a computer-created character and so that movement is copied.

Where To Film

As with all filming techniques, mo-cap can be utilised within location filming to have genuine backgrounds, or, for the greatest level of control, within special soundstages called a Volume.

Volume

The Mo-Cap process can involve actors giving realistic performances in a three-dimensional cube-shaped stage area known as the Volume. Instead of ordinary cameras, the Volume is surrounded by numerous Receptors on each of the surrounding four walls and possibly even the ceiling, with Infrared spotlights flooding the Volume with infrared light. The receptors detect any movement made within the volume.

All the actors and props inside the Volume have markers on their bodies and faces, typically 250 markers on each actor. The receptors pick up the movement of the reference markers that capture the full range of body movement to map face and body movement. This information is combined with an animated 3D digital character linked to the actor so that any move the actor makes is precisely replicated by the animated character.

In order for the character to interact with the environment, full-scale props are built, which are wire-framed hollow models to allow the infrared light to pass through and not block the receptors. Each prop has a barcode to allow the computer to realise which prop is in use, and large-scale props and rigs are used to make everything as realistic as possible. So for example, door props are used to capture the movement of opening a door. For the flying scenes in A Christmas Carol (2009), Jim Carrey was hoisted up in the air. In Beowulf (2007) to recreate the animation of a ship, a boat-shaped rig was built that actually swayed side-to-side. For the horse-riding scenes in the same film, real horses covered in 95 markers were brought into the Volume to capture a horse's movements, and were even ridden up a replica spiral staircase.

Computer software combines all the cameras' data, triangulating to make a virtual reconstruction viewable from any conceivable angle.

On Location

Mo-Cap can also be used on location and is often combined with green-screen technology. For example, The Lion, the Witch and the Wardrobe (2005) is a typical fantasy film that utilises special effects to bring a range of unearthly characters and creatures to life. These include creatures who are part-human, part-animal such as goat-like fauns, centaurs who are half-human and half horse, minotaurs and so forth. In order for these creatures to be brought realistically to life the actors wear costumes that include sections made of a tight green fabric covered with orange tracking marker dots in the areas to be edited out. In the case of a faun, which resembles a human with goat-like legs, this would mean the actor wearing green tights with orange dots. The dots help keep a sense of perspective, distance and depth of field to ensure the characters remain in focus. They also ensure that when the actors move during their scenes, the computer records their movement and weight distribution information. The actor's legs can then be easily edited out of the scene and realistically replaced by their computer-generated goat equivalent.

T-Pose

A T-Pose is nothing to do with standing around waiting for the kettle to boil so you can make a nice cuppa. Before actors begin the Mo-Cap process, they stand in what is called a T-Pose. Legs are together and arms out to the sides like a letter T, as this exposes all the markers on body and face allowing the computers to pick up their exact location. They may also have to make a series of movements called ROM (Range of Motion) movements to ensure their every action is being picked up.

As mo-cap records performances in a 3-D environment, it excels at creating realistic 3D images that are more impressive than films filmed in 2D and converted into 3D. For example, all ImageMovers' films have been released in 3D.

Dead Eyes and Uncanny Valley

In animation, Mo-Cap has been criticised for leading to eerily realistic-looking creepy characters in animated films who appear to have strangely plastic skin and dead eyes, in a phenomenon known as 'uncanny valley'. This postulates that the more humanlike something appears, the more positively humans respond to it up until a point; following this point people feel increasing revulsion and creepiness when confronted by artificial replicas. The more the replica appears almost, but not quite, entirely like a real person, the greater the revulsion.

Performance Capture

When mo-cap is used to replicate far more subtle movement, particularly facial expressions, it is called 'Performance Capture'. As The Polar Express was heavily criticised for the characters' dead eyes, for Beowulf ImageMovers used Electrooculograph electrodes to record eye-muscle movement to create more realistic eyes and convert what the actor's eyes were doing into the animated character. Later, high definition video camera helmets were developed to film every movement of the actors' faces and allow its digital replication.

Technique Advantages and Disadvantages

Advantages

• Mo-cap and performance capture can create digital characters who move naturally in exactly the same way people do in real life.
• It creates animated characters with stunning degrees of realism, particularly facial expressions.
• As mo-cap is filmed in a 3D environment, it translates to 3D films particularly well.
• An actor is a completely blank slate and can become anyone or anything. An actor in a mo-cap suit can convincingly play a giant gorilla, a young Cleopatra, an elderly Gandhi, Nelson Mandela or anyone or anything all in the same day. Their age and appearance is irrelevant, only their body and facial muscle movements matter.
• It can be quick to film. A film traditionally needs a plan of the camera angles from which a scene is captured in advance and cameras moved into position, possibly requiring the scene to be re-enacted several times to ensure it is captured in long shot and close-up from all the required perspectives. With mo-cap the scene only needs to be filmed once and the computer combines the data from all the receptors so that the digital characters can be seen from any conceivable angle.

Disadvantages

• Uncanny Valley.
• Although filming itself is quick, processing and turning the filmed actions into realistic images is extensively time consuming.
• Cost. Not only of computer equipment and all the Receptors, but also the process of filming, which essentially combines the costs of a live-action film with the costs of an animated film.

Role in Animation

In 2006 two of the three films nominated for the Best Animated Feature Oscar had been created using mo-cap; ImageMovers' Monster House and winner Happy Feet¹, with Pixar's Cars the non-mo-capped nominee. This led to animators claiming mo-cap was puppetry rather than true animation, with the end credits to Pixar's 2007 film Ratatouille bearing the message '100% Pure Animation – No Motion Capture or any other performance shortcuts were used in the production of this film'. Despite being a proponent of Performance Capture, Zemeckis agrees that the two are very different techniques, stating,

To call performance capture 'animation' is a disservice to the great animators.

The debate was settled in 2010 when the rules for qualifying for the Best Animated Feature Oscar were amended. Under this change it was stated that motion capture

by itself is not an animation technique' and that 'a significant number of the major characters must be animated, and animation must figure in no less than 75% of the picture's running time… movement and characters' performances are created using a frame-by-frame technique.

Studios:

ImageMovers

Since 2000 ImageMovers, an independent film company that was created in 1997 by acclaimed filmmaker Robert Zemeckis², has made a number of live action films as well as pioneering performance-capture animation techniques in five animated films. Despite some early success, its last animated film, Mars Needs Moms (2011), was a huge box office disaster.

ImageMovers animation style combined not only mo-capped animation but also the effective use of 3D. Their first three films were co-produced by various different studios, while the final two were made by ImageMovers Digital, a joint-venture with the Walt Disney Company.

• ImageMovers:
1. The Polar Express (2004)
2. Monster House (2006)
3. Beowulf (2007)
• ImageMovers Digital:
1. A Christmas Carol (2009)
2. Mars Needs Moms (2011)

At the time of release The Polar Express was incredibly successful despite being the most expensive animated film made to date, however the film was criticised for the 'plastic' appearance of its characters as well as the expressionless eyes. ImageMovers would work hard to improve this in later films. Monster House was even nominated for a Best Animated Feature at the 79th Academy Awards. Like Who Framed Roger Rabbit it was executively produced by Robert Zemeckis and Steven Spielberg. Beowulf was the most-complained about film of 2007 on both sides of the Atlantic, attracting criticism about the amount of horror, violence, nudity and sex for a film rated 12.

ImageMovers Digital announced plans to make six animated films yet following the disastrous performance of Mars Needs Moms, Disney ended the joint venture and the four remaining planned films were cancelled. These had included a mo-cap remake of Yellow Submarine³, having secured approval to use 16 Beatles songs. It had been scheduled for release during the 2012 London Olympics, as this was felt would be the ideal time to promote a British-themed film. This would have been followed by a mo-cap remake or sequel of Zemeckis' own Who Framed Roger Rabbit. Following Mars Needs Moms, Disney immediately severed all ties with ImageMovers and these projects were abandoned. Since 2011 ImageMovers still exists as Zemeckis' own production company but it has only made live-action films.

Other Studios

Although ImageMovers is the animation studio that to date has most strongly embraced mo-cap, it is not the only one to do so, although on the whole results have been variable. India's low-budget Sinbad: Beyond the Veil of Mists (2000) dipped its toe in the water to be the first feature-length mo-cap film. 2001's Final Fantasy: The Spirits Within was the first photorealistic mo-cap film and the first to enjoy a widespread release, however it was a box office bomb, recouping only half its budget. Although Happy Feet (2006) was a success, animated by Animal Logic, Happy Feet Two (2011), this time animated by Dr D Studios, flopped and lost over $40 million. The Adventures of Tintin: The Secret of the Unicorn (2011) was a successful film directed by Steven Spielberg, who had previously executively produced ImageMovers' Monster House.

Andy Serkis has so embraced the technology that he has founded a Performance Capture studio based in London called The Imaginarium. He uses Performance Capture techniques to create special effects characters, rather than to make animated films. Films that have benefited from his experience include the recent Planet of the Apes trilogy (2011-17), Avengers: Age of Ultron (2015) and Star Wars: Episode VII – The Force Awakens (2015). In 2012 he began an adaptation of The Jungle Book, only to be forced to postpone the release when Disney also made a The Jungle Book (2016) live action remake. In December 2017 he announced the film would be renamed Mowgli to avoid confusion and would be released in October 2018.