Readiness and certificate
Readiness is Euler’s judgement about whether a recording is fit to train on. It is a weighted blend of deterministic checks, renormalised over the checks that actually apply to your data, and every check below its threshold raises a named flag that travels with the episode’s readiness receipt. The certificate is the same measurements stated as rules for the whole project, with a verdict.
Nothing about it is a black box. Each check has a plain name, a reason it matters for training, the failure it causes downstream, a weight, a threshold, and the data kinds it can never apply to. The review screen shows each check’s measured value against its threshold.
The 55 checks
| Family | How many | Applies to | What it looks at |
|---|---|---|---|
| General | 16 | every kind, minus what the data cannot carry | timeline, control, sync, length, gripper, language, cameras, image quality, labels |
| Capture | 12 | first-person (egocentric) data | how the recording was made: framing, camera placement, idle time, lens, exposure, head motion, frames |
| Output | 27 | first-person data | what Euler produced: the camera’s telemetry, its sync with the video, and the annotations; each one is a certificate rule |
General checks
| Check | What it asks |
|---|---|
timestamp_gap | Is there missing time between frames? |
timeline_jitter | Do frames arrive at a steady rate? |
action_smoothness | Is the commanded motion physically plausible? |
idle_segments | Is there wasted time at the head and tail? |
trajectory_completeness | Does each episode start settled and finish the task? |
stream_sync | Do the cameras and telemetry agree on when things happened? |
episode_length_z | Are there wildly odd-length episodes? |
gripper_event_sanity | Do gripper open and close events look real? |
language_action_align | Does the written task match what happens? |
camera_dropout | Did every declared camera record? |
image_integrity | Are frames sharp, exposed and readable? |
image_exposure | Are images crushed to black or blown to white? |
image_sharpness | Is any image markedly softer than the rest of the set? |
image_contrast | Is any image markedly flatter than the rest? |
label_readiness | Do the objects found survive their quality floors? |
annotation_coverage | Did the layers you asked for actually land? |
camera_dropout passes only when every camera recorded: a declared camera that
did not record is a defect, not a degree. For loose images, where no timeline
exists, the image checks decide the score.
Capture checks (first-person data)
| Check | What it asks |
|---|---|
| Hands in shot | Is the work in frame, not at its edge? |
| Camera on the head | Are there stretches recorded from a table or a shelf? |
| Working pace | How much dead time is there, and is the pace like the rest of the set? |
| Clear lens | Is a strap, hair, fingertip or smudge parked in the frame? |
| Exposure stability | Are there blown-out or crushed stretches, or pumping exposure? |
| Head motion | Are there head turns too fast for the sensor? |
| Frame integrity | Are frames dropped or duplicated? |
| Clip length consistency | Is this clip an odd length against the set? |
| Uninvolved objects | Are there tracked objects always present and never touched? |
| Near-duplicate clips | Is this clip a copy of one you already have? |
| Collection diversity | How much new is there per recorded hour? (report only) |
| Personal data present | Are faces, location or speech present? (a handling notice, never scored) |
Each one stores where it happened: the review screen shows the stretches and the worst frames.
Output checks (first-person data)
| Group | Checks |
|---|---|
| Timing and sync | video frame rate; video timestamps in order; IMU sampling rate; IMU timestamps in order; longest IMU gap; IMU covering the whole episode; each video frame having an IMU sample beside it; the lag between motion in the picture and motion the gyro felt, and how strongly the two agree |
| Motion | gyro and accelerometer noise above normal head motion; head turn speed and jolts (advisory); the camera’s gravity estimate agreeing with its accelerometer; orientation without jumps |
| Task and language | the clip broken into named steps; steps covering the active timeline; actions covering it; actions short enough to be atomic; every span inside the clip and running forwards; a one-line caption present |
| Annotation coverage | hands posed in most frames; no long stretch without a hand; objects outlined, not just boxed; tracks followed, not sampled; depth maps filled; the camera path covering the whole clip |
Each output check measures one number with a unit (a rate in Hz, a gap in seconds, a coverage ratio) and is held to a pass bound and a review bound. The receipt shows the number, the bounds, and where most of your project sits. The task, language and coverage checks are fixable: a failure means re-running an annotation, not re-recording. Where a measurement can be placed in time (an IMU gap, an orientation jump, the longest stretch without a hand) the review screen shows the moment and a thumbnail.
What the general checks mean for training
The point of a check is the consequence of ignoring it.
Timeline continuity
A policy learns the rhythm of a task from the gaps between frames. Missing time teaches it that the world jumps. Downstream: jumpy rollouts and mistimed grasps at inference. Needs: per-frame timestamps.
Clock stability
Unstable frame timing quietly changes the effective control rate, so a model trained on it learns the wrong speed. Downstream: actions that fire early or late once deployed. Needs: per-frame timestamps.
Control smoothness
Erratic teleoperation is copied faithfully by behaviour cloning. Clean demonstrations are the cheapest quality win available. Downstream: shaky, unsafe motion in the learned policy. Needs: an action or state timeseries.
Dead time
Long idle stretches teach a model to wait, and they inflate training cost for frames that carry no task signal. Downstream: policies that hesitate before acting. Needs: an action or state timeseries.
Complete attempts
A truncated demonstration teaches the ending it never showed. Complete attempts are what a policy needs to imitate. Downstream: policies that abandon tasks near the end. Needs: an episodic commanded trajectory, which is why this check does not exist for egocentric, drive-log, aerial or vision-asset data.
Sensor alignment
If the camera and the arm disagree by even a few frames, the model learns to act on what it has not seen yet. Downstream: confident actions taken on stale observations. Needs: two or more timed streams.
Length consistency
Outlier-length episodes are usually a capture bug, not a rare skill, and they dominate batch statistics. Downstream: skewed training batches and unstable loss. Needs: more than one episode.
Gripper signal
The gripper channel is the highest-signal, lowest-dimension part of a manipulation demonstration. Noise here poisons grasping. Downstream: policies that open the gripper at the wrong moment. Needs: a gripper channel.
Instruction match
Vision-language-action models ground on the instruction. A mismatched task string teaches the wrong association directly. Downstream: models that follow instructions incorrectly. Needs: a task description.
Camera coverage
A missing view at training time becomes a missing view at inference time, and multi-view policies fail closed. Downstream: views the model expects but never receives. Needs: at least one camera.
Image quality
Blur, blowout and compression artefacts are learned as features. Vision encoders happily fit noise nobody wanted. Downstream: brittle perception that degrades on clean data. Needs: image or video frames.
Annotation coverage
A dataset is only as trainable as its labels are complete. This check reports what each enabled annotation layer actually produced. Downstream: silent gaps that surface as label noise in training. Needs: at least one annotation layer enabled.
Applicability
A check that a class of data cannot physically satisfy is removed from the blend before scoring, not scored as zero. An egocentric human clip has no commanded trajectory to complete and no gripper to be sane about; a drive log never settles the way a demonstration does; a state-only log has no cameras to drop; a loose image has no timeline at all.
The data kind is derived automatically from what the episode contains and decides which checks exist. The remaining weights are renormalised, so an episode is never penalised for a contract its class of data cannot have.
Two registries agree on this gate: the data-kind registry is the authority the scorer uses, and the check catalog carries the same information so the settings screen can explain the reason in plain language.
Reading and configuring readiness
curl -fsS -H "Authorization: Bearer $EULER_TOKEN" \
"$EULER_BASE_URL/v1/projects/$PROJECT/readiness/settings"For every check the response gives the catalog row plus what is in force for this project: enabled, the effective weight and threshold, whether it applies to this project’s episodes and the reason when it does not, and the applicable episode count. It also reports max_threshold_offset, the furthest a threshold may be moved.
Changing configuration needs the engineer role and is audited:
curl -fsS -X PUT -H "Authorization: Bearer $EULER_TOKEN" \
-H "Content-Type: application/json" \
"$EULER_BASE_URL/v1/projects/$PROJECT/readiness/settings" \
-d '{
"disabled": ["idle_segments"],
"threshold_offsets": {"action_smoothness": -0.1},
"weight_overrides": {"camera_dropout": 0.2},
"certificate": true
}'Every field is optional; omitting one leaves it untouched and an empty value clears it back to the default. certificate: true seals a certificate with each run. For an output check the Readiness metrics tab shows the physical bar a threshold change produces (for example, the longest acceptable stretch without a hand, in seconds). Only differences from the defaults are stored, so a project that never opens the screen scores exactly as it did before the screen existed.
Thresholds are bounded on purpose. A threshold offset is clamped to ±0.3 and a weight override to the range 0 to 1. A team can tune a check when their domain legitimately disagrees with the default. A team cannot silently define a failure away, because a certificate that can be tuned to always pass would not mean anything.
Unknown check ids are rejected outright. Out-of-range values are clamped rather than rejected, so a stale stored setting can never break scoring.
The certificate
GET /v1/projects/{id}/certification/report (and report.html, the same thing
as a page) is the project’s readiness certificate, also on the Readiness
certificate tab in Datasets & export. For first-person data the policy is the
27 output checks as rules plus a few image-statistics rules. For every episode
and rule it states the observed value, the unit, how it was measured, where the
value and the bounds came from, and the status: pass, review, fail or
not_assessable. Per rule it also summarises the project: the best, typical and
worst episode and the band most episodes fall in.
The project verdict is certified, review_required, not_ready or
not_assessable. Missing modalities reduce evidence coverage instead of
becoming a zero or a pass. Your readiness settings flow into it: a moved
threshold moves the rule’s review bound, and a check you turned off becomes an
advisory row rather than vanishing, so the certificate always says where the bar
was set. Certification policies are immutable and content-hashed: saving a
changed policy creates a new version and earlier reports are kept.
How review routing follows the certificate
Each episode is routed to auto accept, needs review or reject from its score and its flags. Then the certificate gets a say: an automatic accept is held for review if any required certificate rule on the episode is in the review band or fails. So the review queue and the certificate never disagree. The certificate never rejects anything on its own, and human decisions are never changed.
Re-measuring readiness (Re-measure from stored data on the Readiness
metrics tab, or POST /v1/projects/{id}/readiness/remeasure) re-routes in the
same pass; Re-route review in Pipeline settings
(POST /v1/projects/{id}/review/reroute) applies today’s rules without
re-measuring. Neither uses a GPU or spends.
Next
- Split this score into integrity, semantic quality and utility, and see what has not been measured: Data utility.
- Select a training subset from what scored well: Curation.
- See what the annotation layers actually produced: Annotation layers.