Docs menu

// Command reference

simforge bench

Generated from the CLI's own help (CLI 0.2.0-rc.9). Global flags and exit codes are on the reference index.

All commands

simforge bench

Benchmark the renderer on this machine: the Helios Score, per-factor sub-scores, frame correctness

simforge bench [OPTIONS]
       simforge bench <COMMAND>

Runs on: local runtime

Argument / flagNotesWhat
--suite SUITEone of quick, full; default quickWhich scenes to run
--scenes IDrepeatableOnly these scene ids (comma-separated), from the suite's pack
--set KEY=VALUErepeatableDiagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu NThe GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU)
--out FILEWrite the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIRA local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIRKeep every sampled frame and the session logs in this directory
--download-onlyDownload and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busyRun even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDSdefault 300Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quietNo progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIRMap cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIRActor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets

simforge bench compare

Compare two results: per-scene time deltas against the noise of both runs, and frame identity

simforge bench compare [OPTIONS] <A.json> <B.json>

Runs on: local runtime

Argument / flagNotesWhat
<a.json>requiredThe baseline result
<b.json>requiredThe candidate result
--suite SUITEone of quick, full; default quickWhich scenes to run
--scenes IDrepeatableOnly these scene ids (comma-separated), from the suite's pack
--set KEY=VALUErepeatableDiagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu NThe GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU)
--out FILEWrite the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIRA local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIRKeep every sampled frame and the session logs in this directory
--download-onlyDownload and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busyRun even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDSdefault 300Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quietNo progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIRMap cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIRActor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets

simforge bench run

Run the benchmark (the default when no subcommand is given)

simforge bench run [OPTIONS]

Runs on: local runtime

Argument / flagNotesWhat
--suite SUITEone of quick, full; default quickWhich scenes to run
--scenes IDrepeatableOnly these scene ids (comma-separated), from the suite's pack
--set KEY=VALUErepeatableDiagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu NThe GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU)
--out FILEWrite the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIRA local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIRKeep every sampled frame and the session logs in this directory
--download-onlyDownload and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busyRun even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDSdefault 300Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quietNo progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIRMap cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIRActor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets
--suite SUITEone of quick, full; default quickWhich scenes to run
--scenes IDrepeatableOnly these scene ids (comma-separated), from the suite's pack
--set KEY=VALUErepeatableDiagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified)
--gpu NThe GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU)
--out FILEWrite the result JSON here too (default: simforge-bench-<time>.json in the current directory)
--pack DIRA local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK)
--keep DIRKeep every sampled frame and the session logs in this directory
--download-onlyDownload and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering
--allow-busyRun even when another process is using the GPU (the result is flagged gpu-shared)
--stall-timeout-s SECONDSdefault 300Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session
--quietNo progress or table on stderr (the JSON result on stdout is unchanged)
--cache-root DIRMap cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps
--assets-root DIRActor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets