6 templates · Markdown, JSON, CSV, YAML

Robot training data templates

Specify, collect, label and evaluate robot training data with working templates. Every field says why it is there, so you can copy one into your project and fill it in, not start from a blank page.

Robot training data is the synchronized record of what a robot (or a person demonstrating) saw, felt and did during a task: camera frames, joint states, actions, touch and force signals, and the labels that say what happened. Most of the work is deciding what to record and keeping it consistent. These templates cover that.

data-requirements-spec.yaml
task:
  statement: "pick one pouch, place it flat in the tote"
  success_criterion: "pouch inside tote, gripper clear"
streams:
  - id: wrist_rgb
  - id: fingertip_tactile
  - id: ft_wrist
timing:
  sync_tolerance_ms:
    contact_streams_to_proprioception: …
    # set from the shortest event you label
volume:
  episodes_per_condition: …
  # sized from a pilot, not a guess

Pick a template by task

Each template has its own page with a full preview, when to use it, and buttons to copy it or download it as a file. Filter by the stage you are at.

  • specify

    Data requirements spec

    The one document that says what data a policy needs: task, embodiment, streams and rates, sync tolerance, environments and how many episodes.

    32 fields · Markdown, YAML

  • specifycollect

    Sensing modality selection

    A worksheet to decide which of video, depth, tactile, force and EMG a manipulation task actually needs, scored on the same criteria for each.

    21 fields · Markdown, CSV

  • collect

    Demonstration collection protocol

    A step-by-step protocol for operators and sessions: consent, task scripts, calibration, sync checks, failure logging and close-out.

    29 fields · Markdown, CSV

  • label

    Episode annotation schema

    A schema for episodes, segments, contact events, success labels and sensor metadata, with copyable JSON, JSON Schema and spreadsheet versions.

    21 fields · Markdown, JSON, CSV

  • evaluate

    Policy evaluation plan

    Define trials, success criteria, ground-truth references and real-hardware test logs before you run a trained policy, so the result means something.

    23 fields · Markdown, CSV

  • evaluate

    Dataset datasheet

    Describe a finished dataset's contents, collection method, limits and intended use, so someone else can decide whether to train on it.

    24 fields · Markdown

What each template is for

The templates follow the order a dataset is built in. The answers in one feed the next: the modality worksheet fills the spec's stream list, the protocol carries out the spec, and the same failure taxonomy runs from collection through evaluation.

  1. Specify

    Decide what data the policy needs before anyone records it.

  2. Collect

    Run demonstration sessions that produce usable, synchronized episodes.

  3. Label

    Give every episode the same structure, segments and metadata.

  4. Evaluate

    Judge a trained policy on hardware and document what the dataset is.

Built to be used, not read once

The reasoning behind every field

Each field has a note on why it is there and what goes wrong without it, so a reviewer can disagree with a choice instead of guessing at it.

Formats you can commit

Copy any template as Markdown. The spec also comes as YAML, the annotation schema as JSON, JSON Schema and CSV sheets, and the logs as CSV.

No made-up numbers

Where a value depends on your task or hardware (a sampling rate, a dataset size) the template says how to decide it and where to check it, instead of quoting a figure.

Choosing sensors? Read the sensing stack reference

A short reference to the layers of a robot sensing stack, from video and depth to tactile, force and EMG, and the ways to capture touch and force from human demonstrations, compared on the same criteria.

Vision

RGB video, egocentric, wrist or fixed

Depth

Geometry for grasping and clearance

Proprioception

Joint states and actions

Force / torque

How hard, along which axis

Tactile

Where contact is, and when it slips

EMG

Grip effort from the demonstrator's muscles