Docs menu

// Command reference

simforge evaluate

Generated from the CLI's own help (CLI 0.2.0-rc.9). Global flags and exit codes are on the reference index.

All commands

simforge evaluate

The reject filters (and an optional intent rubric) over a trace; exit 2 when it rejects. With --cloud: test a model by letting it drive your saved scenarios: simforge evaluate --cloud --model MODEL --dataset NAME [--scenarios A,B] [--yes] [--wait]. It is priced first and starts only once you confirm the total (--yes when nothing can prompt); no TRACE is read

simforge evaluate [OPTIONS] <TRACE>

Runs on: local runtime, or hosted with --cloud

Argument / flagNotesWhat
<trace>requiredThe trace (.trace.json or .trace.json.gz, any released format)
--filter MODEReject filters: critical | negative-control | all
--trivial-ttc SThe trivially-safe TTC threshold, seconds
--reject-collisionsReject any collision
--motion-audit-version MOTION_AUDIT_VERSIONone of declared-speed-v1, pose-motion-v2The motion-plausibility audit to run. Default: declared-speed-v1
--rubric FILEAn intent rubric (IntentRubric JSON) evaluated alongside criticality
--blind-review-out FILEWrite the context-blind review packet here (needs --rubric)
--traffic-law MAP_DIRMeasure red-light and stop-bar compliance on this installed map directory (map.xodr, derived/odr/locations-v2.json.gz); reported as trafficLaw
--traffic-law-actor ACTORrepeatableAn actor to measure (repeatable; default: the trace's metric subject)
--model MODELThe model that drives, by name (case-insensitive) or id: one of the models the host offers now (run without it to list them). Required
--dataset DATASETYour dataset of saved scenarios, by name (case-insensitive) or id. Required
--scenarios SCENARIOSrepeatableOnly these scenarios of the dataset, by name or id (comma-separated). Default: all of them
--param NAME=VALUErepeatableOne of the model's parameters, as NAME=VALUE (repeatable): the options' params of the model, typed as listed (e.g. --param seed=7). Absent ones take their defaults
--seed SEEDSimulation seed for every scenario, 0 to 2^53-1. Default: 0
--yesone of true, falseStart at the quoted price without asking; required when no one can answer the prompt
--waitone of true, falseFollow the run until it finishes