I’ve used depth and world position passes from CG renders for years. On Shot 4 of Leica Geosystems’ MANTIS showcase, I got to use these passes for the live-action plate itself. Sitting in Nuke, I could pull a matte based on how far things were from the camera. And when I needed tracking points for roto, I had a world-position pass I could use to help find them. Those were the two things that made me excited to play with the data on Leica's behalf for its launch at Infinity Festival 2026.
Leica Geosystems’ MANTIS launched on November 12, 2026, and was demonstrated at Infinity Festival, where I had a booth area to show what was possible with the data once it was captured. It’s a camera-mounted system that records scene geometry and camera movement while the cinema camera rolls. MANTIS Studio then processes that data into a point cloud, a camera track, a LiDAR-corrected depth pass, lens distortion maps, and a world-position pass aligned to the footage. For a compositor, the interesting part is what those four pieces let you do together.
Leica calls Shot 4“Tracking & Set Extension.” It was shot on a Sony VENICE, handheld with an EZ Rig. The focus was shallow; there were no tracking markers, and the actors moved across features you might otherwise use for tracking. The workflow brought the MANTIS camera and point cloud into Nuke and Unreal Engine, with camera track refinement in SynthEyes.
The point cloud and camera track gave 3D and comp a common starting point. The camera described the move, while the cloud gave us a spatial reference for the set. When both departments start from that same reference, you can get to work sooner.
A depth pass tells you how far a visible surface is from the camera. A world-position pass gives you its XYZ coordinates in the scene. That distinction mattered on this shot. With the position pass and the camera in Nuke, I could identify points in 3D and project them back into the image to get 2D tracking references for my roto.
On a handheld shot like this, trying to follow a small feature by eye can get tedious, especially when the actors cross in front of it. A stable point on the set gave me a reference for how the camera move carried that part of the image across the frame. I still shaped and adjusted the roto where it needed it, but I had a much better place to start.
In CG, I’m used to having a world position pass and using it to isolate parts of a render. Having depth for the photographed scene meant I could use that same logic on the plate. I could choose a band of depth and turn it into a starting matte, then refine the edges where the shot needed more control.
That changes what I can try early in a comp. If I’m testing where atmosphere should sit or how an element should pass behind part of the set, I can use the plate’s depth to get a first pass in context. It was a familiar Nuke workflow applied to footage I’d usually approach with hand-built masks.
I’d love to see future workflows to build Gaussian splats out of a set. Further out, I’m curious about time-varying Gaussian splats, often called 4D Gaussian splatting, where the representation could preserve changes in a scene over time. Imagine being able to inspect a moving scene from another viewpoint while keeping the timing of the original take. I think this could add new possibilities for what could be achieved, even if I'm not aware of all the possibilities yet. At the very least, it could make training data more efficient to create and individualized.
For now, what stays with me from my shot is more immediate. The 3D team can begin with the same camera and point cloud as comp. I can open the plate in Nuke and use depth and world position to make decisions I’d otherwise have to work out by hand. When the data comes from the take itself, there’s more of the set to work with after everyone has gone home. That time savings certainly saves money in production over time.
If you’re developing hardware or software for film, VFX, or virtual production and want to see how it fits into real production workflows, CG Pro can help with hands-on R&D. Get in touch with CG Pro.