// Command reference
simforge evaluate
Generated from the CLI's own help (CLI 0.2.0-rc.9). Global flags and exit codes are on the reference index.
simforge evaluate
The reject filters (and an optional intent rubric) over a trace; exit 2 when it rejects. With --cloud: test a model by letting it drive your saved scenarios: simforge evaluate --cloud --model MODEL --dataset NAME [--scenarios A,B] [--yes] [--wait]. It is priced first and starts only once you confirm the total (--yes when nothing can prompt); no TRACE is read
simforge evaluate [OPTIONS] <TRACE>
Runs on: local runtime, or hosted with --cloud
| Argument / flag | Notes | What |
|---|---|---|
| <trace> | required | The trace (.trace.json or .trace.json.gz, any released format) |
| --filter MODE | Reject filters: critical | negative-control | all | |
| --trivial-ttc S | The trivially-safe TTC threshold, seconds | |
| --reject-collisions | Reject any collision | |
| --motion-audit-version MOTION_AUDIT_VERSION | one of declared-speed-v1, pose-motion-v2 | The motion-plausibility audit to run. Default: declared-speed-v1 |
| --rubric FILE | An intent rubric (IntentRubric JSON) evaluated alongside criticality | |
| --blind-review-out FILE | Write the context-blind review packet here (needs --rubric) | |
| --traffic-law MAP_DIR | Measure red-light and stop-bar compliance on this installed map directory (map.xodr, derived/odr/locations-v2.json.gz); reported as trafficLaw | |
| --traffic-law-actor ACTOR | repeatable | An actor to measure (repeatable; default: the trace's metric subject) |
| --model MODEL | The model that drives, by name (case-insensitive) or id: one of the models the host offers now (run without it to list them). Required | |
| --dataset DATASET | Your dataset of saved scenarios, by name (case-insensitive) or id. Required | |
| --scenarios SCENARIOS | repeatable | Only these scenarios of the dataset, by name or id (comma-separated). Default: all of them |
| --param NAME=VALUE | repeatable | One of the model's parameters, as NAME=VALUE (repeatable): the options' params of the model, typed as listed (e.g. --param seed=7). Absent ones take their defaults |
| --seed SEED | Simulation seed for every scenario, 0 to 2^53-1. Default: 0 | |
| --yes | one of true, false | Start at the quoted price without asking; required when no one can answer the prompt |
| --wait | one of true, false | Follow the run until it finishes |