Robot training data templates
Specify, collect, label and evaluate robot training data with working templates. Every field says why it is there, so you can copy one into your project and fill it in, not start from a blank page.
Robot training data is the synchronized record of what a robot (or a person demonstrating) saw, felt and did during a task: camera frames, joint states, actions, touch and force signals, and the labels that say what happened. Most of the work is deciding what to record and keeping it consistent. These templates cover that.
task: statement: "pick one pouch, place it flat in the tote" success_criterion: "pouch inside tote, gripper clear" streams: - id: wrist_rgb - id: fingertip_tactile - id: ft_wrist timing: sync_tolerance_ms: contact_streams_to_proprioception: … # set from the shortest event you label volume: episodes_per_condition: … # sized from a pilot, not a guess
Pick a template by task
Each template has its own page with a full preview, when to use it, and buttons to copy it or download it as a file. Filter by the stage you are at.
- specify
Data requirements spec
The one document that says what data a policy needs: task, embodiment, streams and rates, sync tolerance, environments and how many episodes.
- specifycollect
Sensing modality selection
A worksheet to decide which of video, depth, tactile, force and EMG a manipulation task actually needs, scored on the same criteria for each.
- collect
Demonstration collection protocol
A step-by-step protocol for operators and sessions: consent, task scripts, calibration, sync checks, failure logging and close-out.
- label
Episode annotation schema
A schema for episodes, segments, contact events, success labels and sensor metadata, with copyable JSON, JSON Schema and spreadsheet versions.
- evaluate
Policy evaluation plan
Define trials, success criteria, ground-truth references and real-hardware test logs before you run a trained policy, so the result means something.
- evaluate
Dataset datasheet
Describe a finished dataset's contents, collection method, limits and intended use, so someone else can decide whether to train on it.
What each template is for
The templates follow the order a dataset is built in. The answers in one feed the next: the modality worksheet fills the spec's stream list, the protocol carries out the spec, and the same failure taxonomy runs from collection through evaluation.
Specify
Decide what data the policy needs before anyone records it.
Collect
Run demonstration sessions that produce usable, synchronized episodes.
Label
Give every episode the same structure, segments and metadata.
Evaluate
Judge a trained policy on hardware and document what the dataset is.
Built to be used, not read once
The reasoning behind every field
Each field has a note on why it is there and what goes wrong without it, so a reviewer can disagree with a choice instead of guessing at it.
Formats you can commit
Copy any template as Markdown. The spec also comes as YAML, the annotation schema as JSON, JSON Schema and CSV sheets, and the logs as CSV.
No made-up numbers
Where a value depends on your task or hardware (a sampling rate, a dataset size) the template says how to decide it and where to check it, instead of quoting a figure.
Choosing sensors? Read the sensing stack reference
A short reference to the layers of a robot sensing stack, from video and depth to tactile, force and EMG, and the ways to capture touch and force from human demonstrations, compared on the same criteria.
Vision
RGB video, egocentric, wrist or fixed
Depth
Geometry for grasping and clearance
Proprioception
Joint states and actions
Force / torque
How hard, along which axis
Tactile
Where contact is, and when it slips
EMG
Grip effort from the demonstrator's muscles