Research Scientist, Hand Tracking & Manipulation - ON-SITE

Research Scientist, Hand Tracking & Manipulation

Location: New York, Toronto or Vancouver (on-site) 

Type: Full-time · Openings: 1 

Salary: US$170K – $210K + equity

The role

Architect and train proprietary foundation models from scratch for 3D hand tracking and articulated pose estimation. This role targets a critical bottleneck in embodied AI: dexterous manipulation and hand-object interaction. You'll build in-house equivalents to leading 3D hand and mesh recovery architectures, plus robust interaction models for heavily occluded egocentric manipulation. You'll train on a large, continuous stream of proprietary ground-truth manipulation data.

What you'll do

• Design, implement and train state-of-the-art networks for 3D hand pose estimation, dense mesh recovery and kinematic tracking


• Scale multi-view and temporal architectures across multi-GPU clusters on massive datasets of hands in action


• Develop novel loss functions that enforce biomechanical constraints, temporal smoothness and physical plausibility


• Build models robust to motion blur, severe self-occlusion and rapid rotation in first-person manipulation


• Track objects through complex grasps, segment tools from hands, and map contact points and forces for downstream robotics models


• Rapidly prototype models for tactile-visual fusion, fine-grained action segmentation and new hardware sensors


• Feed tracking outputs into pipelines that reason about contact surfaces and object affordances, bridging human video and robot control policies

What you bring

• Deep expertise in deep learning, 3D computer vision and articulated tracking / hand pose estimation


• Proven experience training large-scale vision models from scratch, beyond inference or fine-tuning existing checkpoints


• Strong grasp of parametric hand models (e.g., MANO), inverse kinematics and dense mesh estimation


• Mastery of PyTorch and deep learning scaling frameworks


• Experience curating multi-terabyte image and video training datasets


• Comfort in a fast-paced environment where priorities shift with new research or hardware

Strong signals

• First-author publications at CVPR, ICCV, ECCV or NeurIPS on 3D hand tracking, hand-object interaction, dexterous manipulation or human mesh recovery


• Work with egocentric manipulation datasets (e.g., Ego4D, Ego-Exo4D, DexYCB, EPIC-KITCHENS)


• Custom CUDA kernels for 3D operations, differentiable mesh rendering or collision/contact computation

Please note: Research Scientist roles require highly specific expertise. Apply to only one research role; you may also apply to non-research roles.