In this guide, you will find the workflow to use Move Pro Multi-Camera Markerless Motion Capture technology, as well as quick links to the full docs for each step!
For any other guidance, reach out to [email protected]
Contents
1. Shoot Prep
1.1. Check you have the necessary equipment
Cameras
Mounting methods
Floor markers
1.2. Consider your environment and actor’s clothing
Lighting
Reflections
Clothing
Footwear
Space
1.3. Configure your setup based on your capture volume
How many cameras do you need?
How big is your volume?
What resolution/framerate should you capture at?
1.4. Camera Setup
Setup your cameras to make sure you can get the best mocap data from your footage. Consider things such as the resolution and framerate, usually the higher the better! For specific set up guides, check out these docs:
For GoPro’s, use the GoPro Setup Guide
For Sony RX0II’s use the Sony Setup Guide
For unsupported cameras, you will need to use Experimental Mode
1.5. Frame Up The Cameras:
Position the cameras evenly around your space, to create your capture volume.
Walk towards one of your cameras and raise your arms in a Y-pose, standing as close as possible whilst still fully framed. If there is extra space above or below your body, you may be able to adjust the orientation of the camera to enable you to stand closer to the camera (in turn, increasing the size of your capture volume). You can use the App to see the live preview.
At a minimum, all cameras must see your full body when you're standing in the centre of the volume (as this is part of the calibration process), but ensuring they can also see you fully framed at the edges of the volume will ensure you achieve good quality mocap everywhere in your capture volume.
It is important to consider if the movements you'll be capturing may require more height, or if your actor may be taller than you.
Actors should not be less than 250 pixels in height within the image.
It is recommended to use an object, marker or tape to identify the limits of your capture volume so that you don't exceed the space during your capture.
Repeat this step for each camera until you are happy that you have good visibility around the entire volume. As a bare minimum, two cameras must be able to see your entire body at any time during the capture, but more cameras will improve the quality.
Once you have framed them all up, you're ready to calibrate!
2. Capture Your Motion
The calibration provides the system with the position and orientation of the cameras, so this only needs to be done once, as long as the cameras aren't touched/moved. The system will automatically determine the bone lengths of each actor at the beginning of each action take, so there's no need for an actor calibration.
2.1. Calibration:
Hit record.
Stand in the centre of the capture area and:
Clap 3 times above your head clearly, ensuring it is sufficiently louder than any background noise. (Note - this isn’t necessary if your cameras are synchronized)
Stand in a Y pose for 2 seconds, then walk from the middle of the capture area to the edge of the volume in front of each camera, facing the camera at all times. Move in a natural motion, pause at each camera and return to the centre while still facing the camera.
Repeat this motion walking to each camera, then backwards to the centre, before moving on to the next.
If using less than 4 cameras, walk around the perimeter of the volume to ensure the system has an accurate understanding of the shape of the capture volume. (The perimeter will be used to ensure only the desired actors are tracked during action takes)
Now finish the recording.
At this stage, we recommend measuring the height of your actor and noting it down. You'll need it later when you process the calibration! Their height should not include the footwear they are wearing.
Additional notes:
Make sure only the actor (i.e no other people) are in the capture volume, or holding a Y-pose during the calibration otherwise the system may track the wrong actor. If the cameras are moved (even slightly) at any point during the shoot, a new calibration will be required for optimal results.
2.2. Action Takes:
Once you have your calibration, anyone can be the actor for your action takes without the need for an actor calibration, just make sure each take begins with 3 claps (unless using synchronized cameras) and all actors holding a T-pose for 2 seconds.
Press record.
Stand in the centre of the volume.
Clap 3 times above your head. (Note - this isn’t necessary if your cameras are synchronized)
Hold a T-Pose for 2 seconds.
Now, perform!
Make sure the actor remains inside the capture volume and that the time from the T-pose to the end of your movements does not exceed 5 minutes.
When you are finished, stop your recording.
We always recommend doing an extra calibration at the end of your session, just in case a camera was moved during the session.
Additional Note:
To capture multiple actors, make sure they stand side by side and perform a T-Pose at the same time at the start of the take, though only one person needs to clap.
The Multi-Cam system can track soccer balls. To do this, ensure it begins on the floor, not moving and remains within the capture volume at all times. Alternative balls may be used, but they ball must be spherical, bounce like a soccer ball, be of similar size and be of good contrast to the environment.
2.3. Actor Profiles: (Optional for Enterprise Only)
Actor profiles can help reduce the impact of clothing that occludes many key points of the body, or if you're wanting to ensure each actor has a consistent size skeleton across multiple takes/shoots. This enables you to apply a pre-determined skeleton to a chosen actor, to enable the system to skip the bone length estimation process.
Record a separate take of the actor doing the specified movement sequence as shown in the videos to create an actor profile. You can apply then this skeleton to future action takes when processing them.
3. Prepare Your Footage
3.1. Renaming your files:
The files must be named consistently according to the following convention:
camX_takename
Navigate to app.move.ai to create your project & session.
3.3. Uploading:
You can batch upload the videos from your PC to the session you’ve created.
4. Process Your Footage
Select the '+ Calibration' icon
Note down the start & end times for your claps and for your calibration sequence (beginning in the Y-pose)
Click Next
Title: Give your calibration a title
Calibration Actor Height: You must do this in order for the system to accurately extract a person's bone lengths - this information is required in metres (i.e. 1.88m) and will be saved after the first time entering their details. This should not include footwear.
Time sync clapping starts: This is the time in the calibration when the clapping starts.
Time sync clapping ends: This is the end time of the clapping.
Time calibration sequence starts: This is the start time of the calibration (first moment in a Y-pose)
Time calibration sequence ends: This is the end time of the calibration sequence.
Hit 'run' and you'll be notified when your calibration is done.
While this is processing, you can queue up your action takes, and Enterprise users can queue up their actor profiles to be created.
It's always worth checking your calibration.
'Calib autosync' feature - This should be enabled if you need the system to sychronize your footage by providing the clap times, so it should only be disabled if you're using timecode sync'd cameras such as the Sony RX0II.
'Process entire video' feature - This should only be enabled if your actor is stood in the Y-pose at the beginning of the video, otherwise it should be disabled so you can provide the Y-pose time for the system to use.
Within your session, click the '+ Take' icon and select the video of your desired action take.
Note down the start & end times for your claps and for your action sequence (beginning in the T-pose)
Click Next
Scene: Create a New Scene or select the Scene in which you wish to add your take.
Title: Give your Take a Title.
Actor Number: You must tell the system how many actors are in the take, to ensure it tracks all actors.
Prop: Here you can select if you have any props in the scene you would like to track. Currently, the platform supports the tracking of a single football.
Time sync clapping Start/End: Input the clap times, this should be the window of time at which the actor is clapping.
Time take action starts: This is the start time of the action in your take. Always start your take when the Actor(s) in the video are clearly visible, holding the t-pose and not occluding each other.
Time take action ends: This is the end time of the action in your take, or when you want your animation output data to finish.
Click ‘Save’.
At this stage, Enterprise users have the option to apply an actor profile. If you haven’t created one yet, take a pause on processing your action take and do that now.
Select the calibration you want to use. This should be for the same camera positions as the Take you are about to process
Fingers: ON or OFF. Finger tracking will increase processing time.
Rigs: Select the Rig you want to retarget to. (If you plan to retarget elsewhere, we recommend choosing the Move_Mo or Move_Ve rig, as these will maintain the bone lengths of your actor.
Click 'Run' and your video will now start processing.
It's always worth checking your action takes.
'Calib autosync' feature - This should be enabled if you need the system to sychronize your footage by providing the clap times, so it should only be disabled if you're using timecode sync'd cameras such as the Sony RX0II.
'Process entire video' feature - This should only be enabled if your actor is stood in the Y-pose at the beginning of the video, otherwise it should be disabled so you can provide the Y-pose time for the system to use.
4.31. Creating an Actor Profile
Once you've recorded your videos and uploaded them to a session.
Process the calibration as usual
Process the actor profile according to the below steps:
Hit the + Actor button within your session
Select the correct take containing your actor profile movement sequence
Note down the clap window (the time you begin and end the claps) and the movement sequence time window (the times you begin and end the movements).
Click Next
Enter the times you have noted down, alongside the name of the actor
Click Next
Choose the respective calibration to be used for this take
Click Run
4.3.2. Using an Actor Profile
Begin processing your action take as usual, then, when you reach Step 3 of creating an action take:
Draw a bounding box around the desired actor(s) in one camera (whichever has a clear view of the actor), ensuring no other actors are visible within the bounding box.
Select the actor profile you'd like to use for each of the actor's you've drawn bounding boxes around.
Click Next and continue processing the action take as usual.
5. Export Your MoCap
Download FBX Pre-Retarget: This is the FBX skeleton with motion capture data applied to it - straight from the platform
Download FBX Retargeted: This is an FBX with motion capture retargeted to the skeleton of your rig
Download Blender Mocap: This downloads both the pre- and the retargeted motion to your rig, inside it's mesh alongside the camera positions too. This file can be automatically opened in Blender
Download MAYA HIK Pre-Retargeted: This downloads the Maya HIK file which can be opened directly in Maya