Data Algorithms and Pipelines

Background

Data quality is a core determinant of the capabilities of embodied foundation models. We need to build an end-to-end data system covering internet video, real-robot manipulation data, and simulation-generated synthetic data, continuously supplying the pre-training and post-training teams with high-quality and diverse training data.

Responsibilities

• Build multimodal data pipelines covering the complete lifecycle of collection, cleaning, annotation, and management for internet videos, embodied-manipulation videos, and trajectory data.
• Design an automated quality-control system for embodied-manipulation videos, including anomalous-frame detection, task-segment splitting, and temporal alignment across multiple views.
• Standardise heterogeneous trajectory-data formats, including RLDS, HDF5, and ROS bag, and establish standards for trajectory-quality evaluation and filtering.
• Build a simulation-data generation factory with procedural randomisation of scenes, objects, and lighting, supporting the production of millions of trajectories.
• Build an automated video and trajectory annotation pipeline based on vision-language models (VLMs), reducing the cost of manual annotation.
• Build an agent framework that automates the workflow for simulation assets and simulation scenes.
• Improve simulation quality so that both visual fidelity and physical behaviour more closely match real-world data.

Qualifications

• Bachelor’s or Master’s degree in a relevant field, with hands-on engineering experience building large-scale data pipelines.
• Familiarity with embodied-data standards such as RLDS, LeRobot, or Open X-Embodiment is preferred.
• Experience with game engines or simulation platforms such as Unreal Engine, Isaac, or MuJoCo is preferred.