# SimForge documentation > SimForge is a driving simulator for autonomous-vehicle research: scenarios in SimForge Worlds (real-world HD maps with their digital twins), closed-loop simulation with reactive traffic, 2D, schematic and photoreal 3D sensor rendering with the Helios Engine, Recast (a recorded drive turned into an editable scenario), and SimForge Gym, where you train driving policies (reinforcement learning today, supervised fine-tuning recipes coming). One command line, `simforge`, runs everything on your machine, or on SimForge's hosted cloud with `--cloud`. # SimForge docs > Install the simforge CLI, download the runtime and a map, render a scenario in 2D, schematic or 3D, train a policy with reinforcement learning, and run the same commands in the cloud. Source: https://simforge.ai/docs SimForge simulates driving scenarios in SimForge Worlds: real-world HD maps with their digital twins. Traffic reacts to your vehicle, the renderer produces camera images, and SimForge Gym, a Gymnasium environment, lets a learning policy drive. One command line, `simforge`, does all of it. ## How the pieces fit | Piece | What it is | Licence | |---|---|---| | `simforge` CLI | The one command line. It prints JSON, so people, scripts and AI agents use it the same way. | Apache-2.0 | | `simforge-gym` | SimForge Gym: the Python package for training driving policies (reinforcement learning today). | Apache-2.0 | | Runtime | The simulation engine and the renderer the CLI drives. A separate download after you sign in. | Closed binary under PolyForm Noncommercial 1.0.0, downloaded under the SimForge account and download terms | | SimForge Worlds | Each World is an HD map of a real-world road network and its digital twin, the 3D environment the renderer draws. Downloaded after you sign in. | SimForge account and download terms: non-commercial, no redistribution | | Cloud | SimForge Cloud: the same commands with `--cloud`, run on SimForge's machines. New accounts start with $25 in free credits. | Paid, with a cost cap on every run | Commercial licences and questions about the license and terms: [contact@simforge.ai](mailto:contact@simforge.ai). Commands run on your machine by default. Add `--cloud` and the same command runs hosted. Nothing ever switches between the two on its own: a missing runtime or a refused cloud run is an error that names the fix. ## From install to first render The [quickstart](/docs/quickstart) does all of it in four steps. In more detail: 1. [Install the CLI](/docs/install). 2. [Sign in and install the runtime](/docs/sign-in). 3. [Download a map](/docs/maps). 4. [Author a scenario](/docs/scenarios) with typed operations. 5. [Render it](/docs/render) in 2D, schematic or 3D. Then [train a policy](/docs/gym) locally, or [run renders in the cloud](/docs/cloud). ## For AI agents Agents use the same CLI. Start at [For AI agents](/docs/agents): it lists the machine-readable command surface, the [skills pack](/docs/skills) for Claude Code, Codex and AGENTS.md agents, and [/llms.txt](/llms.txt). ## Get help `simforge doctor` checks your machine and prints the fix for each problem it finds. When a command fails unexpectedly or a page is wrong, `simforge feedback --attach-last-error ""` sends us a ticket with the failure attached; see [Send feedback](/docs/feedback) for exactly what it sends. For anything else, ask on the [SimForge Discord](https://discord.gg/Whmn9brNRd) or write to [contact@simforge.ai](mailto:contact@simforge.ai). --- # Quickstart > From nothing to a rendered scenario - install the CLI, sign in, render on SimForge cloud with no GPU, then install the runtime and a map to render locally in 3D and 2D. Source: https://simforge.ai/docs/quickstart This takes you from an empty machine to a rendered scenario. Step 3 runs on SimForge cloud and needs no GPU, no runtime and no download: **no GPU? start there**. Step 4 adds the local runtime and a map to author and render on your own machine; check [what the machine needs](/docs/install#what-the-machine-needs) first. ## 1. Install the CLI ```sh curl -fsSL https://dev.simforge.ai/install.sh | sh simforge doctor --offline # this machine: every check, with the fix for anything missing ``` ## 2. Sign in ```sh simforge login --device --no-wait # prints a URL and a code: approve it in your browser simforge login poll # after the approval: stores the login simforge whoami ``` `simforge login` alone opens a browser on this machine instead. [Sign in](/docs/sign-in) covers API keys for machines nobody can approve a code for. ## 3. No GPU? Start here: author and render on SimForge cloud ```sh simforge scenario new --cloud --map richmond-field-station --out s.json simforge authoring discover --map richmond-field-station --kind lanes --limit 20 simforge scenario apply s.json --cloud add-actor --id ego --catalog-id vehicle.sedan --lane-rsl 0:0:2 --s 5 \ -- set-ego --role-id ego -- set-clip --clip-seconds 8 simforge render s.json --mode 3d --rig rig.json --cloud --max-cost --wait --download --out out-3d ``` The scenario is a hosted draft with a local copy; `authoring discover` lists the map's real lanes (`rsl`, `lengthM`) to place actors on. Cloud renders are 3D-only. For a 2D visualization, run `simforge render s.json --mode 2d --out out-2d` locally or use Studio's browser player; it creates no cloud job. ## 4. Locally: the runtime and a map ```sh simforge runtime status --available # what you can install, with sizes simforge runtime license # read the agreement; nothing is accepted yet simforge runtime install --for 3d --accept-license simforge maps pull easterbrook-discovery-school --for 3d --accept-license simforge doctor # now with the runtime's own checks ``` Accepting the agreement is what entitles your account to the runtime and the maps. Every download is signed by SimForge: the CLI checks the signature and the sha256 of each archive before it installs anything, and refuses one that cannot fit on the disk before it starts. For authoring and 2D renders only, leave out `--for 3d`: the 2D runtime and the 2D map are megabytes instead of gigabytes. ```sh simforge scenario new --map easterbrook-discovery-school --out s.json ``` Then add actors with [typed authoring operations](/docs/scenarios). A scenario from the SIMLOG-25k dataset works the same way: download its whole folder and pull the map it names. The SIMLOG-25k dataset is on Hugging Face: `huggingface.co/datasets/SIMFORGE-AI/simlog-25k`; with the CLI, `simforge datasets pull simlog-25k --out ./simlog-25k` (signed in). ```sh simforge render s.json --mode 3d --out frames # photoreal, one front camera simforge render s.json --mode 2d --out topdown # top-down video, no GPU needed ``` The 3D render needs an NVIDIA GPU with at least 8 GiB or Apple M3 or newer, and uses one front camera on the ego: 1280x720 at 24 fps, the `showcase` preset, locally and with `--cloud`. In Docker, start the container with the NVIDIA runtime's graphics capability ([Running in Docker](/docs/install#running-in-docker)); plain `--gpus all` cannot start Vulkan. [Preview and render](/docs/render) covers other rigs, presets, texture tiers and the [output format](/docs/render#output-format). --- # Install the CLI > Install the simforge command line on Linux, macOS or Windows and check the machine with simforge doctor. Source: https://simforge.ai/docs/install `simforge` is a single binary for Linux, macOS and Windows. It contains no engine or renderer: those come with the runtime, a separate download after you [sign in](/docs/sign-in). Linux and macOS: ```sh curl -fsSL https://dev.simforge.ai/install.sh | sh ``` Windows (PowerShell): ```powershell irm https://dev.simforge.ai/install.ps1 | iex ``` Use these terminal installation commands. New macOS releases use Developer ID signing and notarization; older Mac releases and Windows binaries may still be unsigned. macOS checks a bare executable's notarization ticket online, so its first launch needs internet access. Do not disable operating-system security or change PowerShell execution policy to run a downloaded installer file. The installer picks the build for your machine, downloads it, checks the archive's and the binary's sha256, and installs `simforge`: into `~/.local/bin` on Linux and macOS, into `%LOCALAPPDATA%\SimForge\bin` on Windows (set `SIMFORGE_INSTALL_DIR` to change either). It reads no credentials. `install.sh` edits no shell files: when `~/.local/bin` is not on your `PATH` it prints the line to add (for zsh, the macOS default, to `~/.zshrc`). `install.ps1` adds its folder to your user `PATH`. When it finishes it prints the next steps: [sign in](/docs/sign-in), accept the agreement, install the runtime, and install `ffmpeg`, which renders need to write their videos (`sudo apt-get install ffmpeg` on Ubuntu, `brew install ffmpeg` on macOS, `winget install Gyan.FFmpeg` on Windows). On Windows, keep the default short `%LOCALAPPDATA%\SimForge\bin` prefix for shell use. The installer can write longer paths, but PowerShell 5.1 and some .NET launchers can reject executable paths over 260 characters even when installation succeeded; choose a shorter `SIMFORGE_INSTALL_DIR` rather than changing system policy. ## Check the install ```sh simforge --version ``` Then [sign in](/docs/sign-in). From there, two paths: - **Cloud only.** Add `--cloud` to commands and they run on SimForge's machines; this machine needs no runtime, GPU or ffmpeg. `simforge auth status` checks the sign-in. See [Run in the cloud](/docs/cloud). - **On this machine.** [Install the runtime](/docs/sign-in#install-the-runtime), then check the machine with `simforge doctor`. To pin a published runtime release, use `simforge runtime install --version VERSION --accept-license`. This selects the runtime version; `simforge --version` reports the CLI version. ## Check the machine ```sh simforge doctor ``` `simforge doctor` runs right after the install, with or without the runtime. It checks the sign-in, the runtime (and every place it looked when there is none), free disk for the runtime and the maps, whether the GPU supports local 3D and is free right now, ffmpeg, the map cache and the host's map catalogue; with the runtime installed it adds the runtime's own checks (the GPU adapter it would render on, sky plates, the open-file limit, the actor models). Each check reports `ok`, `warn`, `fail` or `skipped`, and every `warn` or `fail` comes with its fix. A machine without a runtime or a GPU gets warnings, not failures: `--cloud` needs neither, and simulation and 2D renders run on the CPU. `--offline` makes no network request; the checks that need one are reported as skipped, never as ok. A failing check you cannot fix? `simforge feedback --include-doctor ""` sends us the checks ([what it sends](/docs/feedback)). ## What the machine needs | | Linux | macOS | Windows | |---|---|---|---| | CLI and `--cloud` | x86_64, glibc 2.39 or newer (Ubuntu 24.04 or later) | macOS 13 or later, Apple silicon or Intel | Windows 10 (1803) or 11, x64 or Arm64 | | Local runtime: simulation, 2D | x86_64 | Apple silicon or Intel | x64 | | Local 3D and schematic | NVIDIA, Vulkan, 8 GiB, driver 570 or newer | Apple M3 or newer with 16 GB of memory or more | NVIDIA, Vulkan, 8 GiB, driver 570 or newer | - On Windows on Arm the CLI runs every command with `--cloud`; a local command stops with `runtime_platform_unsupported` and says so. An Intel Mac runs simulation and 2D renders locally and 3D with `--cloud`. - Outside the 3D column, render commands stop with `gpu_unsupported`, `gpu_memory_unsupported` or `gpu_driver_unsupported` before any work. An Apple silicon Mac gives three quarters of its memory to the GPU, so a 16 GB Mac renders with 512 px textures and 24 GB or more with full textures. 2D rendering and simulation run on the CPU everywhere. - Video encodes with `ffmpeg` (libx264) on macOS and Windows; the NVIDIA hardware encoder is used on Linux. GPU lidar needs Vulkan ray queries, so on macOS lidar uses the CPU reference. - `ffmpeg` with libx264 on the `PATH` to encode videos: `sudo apt-get install ffmpeg` (Ubuntu), `brew install ffmpeg` (macOS), `winget install Gyan.FFmpeg` (Windows), or point `SIMFORGE_FFMPEG_BINARY` at one. `simforge doctor` checks it and prints the command; without it renders still write their frames and warn `video_encoder_missing`. - Disk for the runtime and the maps you download. For authoring, simulation and 2D renders: the 2D runtime (about 140 MB installed) and each map's 2D install (tens of MB); `--cloud` needs neither. A local 3D render needs the 3D runtime (about 4.3 GB to download, 5.2 GB installed) and the map with its 3D environment (about 5 to 22 GB to download). `simforge runtime status --available` and `simforge maps show ` give the exact sizes; an install needs its download and its unpacked size free at once, and is refused before downloading when they do not fit (`disk_insufficient`). - A shared or busy GPU: `SIMFORGE_GPU=none` keeps local work off it. 3D and schematic renders are then refused (`gpu_disabled`; render them with `--cloud`), and 2D renders and simulation run on the CPU as always. ## Running in Docker Run the container with the NVIDIA container runtime and its graphics capability, and install the EGL and OpenGL libraries in the image: ```sh docker run --runtime nvidia -e NVIDIA_DRIVER_CAPABILITIES=graphics,compute,utility ... apt-get install -y libegl1 libgl1 # in the image ``` Plain `--gpus all` exposes the GPU for compute only: Vulkan cannot start, and 3D and schematic renders fail with `render_failed`. 2D renders and simulation need no GPU in the container. ## Remove it Run `simforge logout` first if you also want to sign this machine out and delete its stored tokens. Linux and macOS: ```sh curl -fsSL https://dev.simforge.ai/install.sh | sh -s -- --uninstall ``` Windows (PowerShell): ```powershell & ([scriptblock]::Create((irm https://dev.simforge.ai/install.ps1))) -Uninstall ``` Use the same `SIMFORGE_INSTALL_DIR` if you customized it. The uninstall removes only the CLI and its previous binary, not the runtime, maps, credentials or unrelated files. Windows removes its user PATH entry only if the installer originally added it; pre-existing PATH entries and the registry value's type remain unchanged. Reinstall before uninstalling a Windows installation that predates the install receipt. The PowerShell commands run the script in memory; they do not require changing execution policy. On macOS and Linux, remove any PATH line you added yourself if you no longer need it. --- # Sign in and install the runtime > Sign in with simforge login, accept the runtime's PolyForm Noncommercial license and SimForge's account and download terms, and install the closed runtime that simulates and renders. Source: https://simforge.ai/docs/sign-in Downloads need a SimForge account: the runtime and the maps are tied to the account that accepted their agreement. Create one at [simforge.ai](https://simforge.ai) with your email, or continue with Google or GitHub, then sign in from the CLI. Downloads and cloud runs need a verified email: open the link SimForge emails you after sign-up (Google and GitHub accounts are already verified). To get a new link, sign in at [simforge.ai/auth/verify-email](https://simforge.ai/auth/verify-email) and resend it. ## Sign in ```sh simforge login # opens a browser to approve this machine simforge login --device # no browser here (SSH): prints a URL and a short code, then waits simforge whoami # account, organization, workspace and scopes ``` No browser and no terminal that can stay open while you approve (an agent, a flaky SSH session)? Start the sign-in, approve it whenever you like, then finish it from any shell on the same machine: ```sh simforge login --device --no-wait # prints the URL and the code, and exits simforge login poll # after you approved it: stores the login ``` `login poll` answers `login_pending` while the code is not approved yet (run it again) and finishes a `login --device` that stopped waiting too. ### Headless machines: API keys A machine nobody can approve a code for (CI, a remote agent) uses an API key. Mint one from a signed-in machine; the secret is printed once: ```sh simforge keys create --name ci --expires-in-days 30 ``` On the headless machine set `SIMFORGE_API_KEY` to the `sfk_…` secret (and `SIMFORGE_API_URL` to the host's `/api/v1` for a host other than simforge.ai). A key acts in the workspace it was created in with at most the scopes of the login that made it, and runs `--cloud` and hosted commands; runtime and map downloads need `simforge login` on the machine that installs them. `simforge keys list` and `simforge keys revoke ` manage your keys. `login` waits until you approve the request in a browser. The CLI keeps its tokens in the operating system's keychain, or in a file only you can read when there is no keychain; the output says which and why. `simforge auth status` checks the login with the host without printing any token: `tokenValid: true` means the host accepted it, and `emailVerification` says separately whether your email is verified (with the page that resends the link when it is not). A login acts in the workspace of the organization you chose when approving it. Sign in again to switch. `simforge logout` revokes this machine's session and deletes its tokens. ## Read and accept the license and terms The runtime is licensed under the [PolyForm Noncommercial License 1.0.0](https://polyformproject.org/licenses/noncommercial/1.0.0). Your SimForge account, downloads and the maps are covered by the SimForge account and download terms. You accept both together. Read the current version first: ```sh simforge runtime license ``` In short: PolyForm Noncommercial lets you use, change and pass on the runtime for noncommercial purposes as it defines them (personal use, and use by noncommercial organizations such as universities, charities and public research bodies), keeping its license and the `Required Notice:` line (both ship in the runtime's `share/licenses/`). The account and download terms require signing in to download from SimForge and make [usage telemetry](/docs/telemetry) a condition of using SimForge's services and downloads. The maps are free for non-commercial use under the SimForge account and download terms and may not be redistributed. Commercial use needs a licence from SimForge. For a commercial licence or questions about the license and terms, write to [contact@simforge.ai](mailto:contact@simforge.ai). The text `runtime license` prints is what you accept; this summary is not. ## Install the runtime ```sh simforge runtime status --available # what you can install, with download and installed sizes simforge runtime install --accept-license simforge runtime status ``` `--accept-license` accepts exactly the agreement version and digest the host serves now; without it the install stops with `agreement_acceptance_required`. The download is checked against its signature and sha256 and only then installed. `runtime status` reports the installed runtime's version, protocol, capabilities (for example `capabilities.renderModes`) and `installed`: what it is installed for. `--for` says what the runtime is for, like [`maps pull --for`](/docs/maps#download-a-map): | `--for` | Installs | Enough for | |---|---|---| | `2d` (default for a new install) | the runtime without the 3D actor models and sky: about 50 MB to download, 140 MB installed | `scenario new` and `scenario apply`, `locations`, `simulate`, `render --mode 2d` | | `3d` | the whole runtime with the actor models and sky: about 4.3 GB to download | all of the above plus a local `render --mode 3d` | Without `--for`, `runtime install` keeps what is installed (an update of a `3d` runtime stays `3d`). A local 3D render or preview on the 2D runtime stops with `runtime_3d_not_installed` and names `simforge runtime install --for 3d --accept-license`; `--cloud` renders 3D without either. The same agreement covers both. Without a runtime, local commands fail with `runtime_not_installed`, listing every place they looked. Commands with `--cloud` do not need it. `--root DIR` installs the runtime somewhere else. The CLI records the root, so `runtime status`, `doctor` and every local command find it there; `SIMFORGE_RUNTIME_ROOT` overrides it. The download and its unpacking stay inside the root, an install that cannot fit is refused before it starts (`disk_insufficient`), and the install writes `distribution.download.progress` and `distribution.extract.progress` events on stderr while it runs. ## Errors you may see | Code | Meaning | Do | |---|---|---| | `not_logged_in` | No login for this host | `simforge login` | | `session_invalid` | The login expired or was revoked | `simforge login` again | | `email_unverified` | The account's email is not verified yet | Open the link in your email, or resend it from the page the hint names, then retry | | `terms_acceptance_required` | The SimForge Terms of Service changed and this account has not accepted them yet | Open the page the hint names, sign in, accept, then retry | | `agreement_unavailable` | This host publishes no agreement for downloads yet | Check the host (`simforge.ai` by default) | | `agreement_acceptance_required` | The install was not told to accept the agreement | Read `simforge runtime license`, then add `--accept-license` | | `entitlement_required` | This download is not available to your account | Pick another map or version; do not retry | --- # SimForge Worlds and assets > Download SimForge Worlds (the public maps) with simforge maps pull, fetch the actor models, and manage the local cache. Source: https://simforge.ai/docs/maps SimForge Worlds are where scenarios happen. Each World is an HD map of a real-world road network (lanes, signals, the traffic network) with its digital twin, the 3D environment the renderer draws. The CLI calls a World a map. Scenarios are built on an installed map, and the installed version is the one they pin. ## Download a map ```sh simforge maps pull easterbrook-discovery-school@v9 --accept-license ``` `maps pull` downloads a map your signed-in account is entitled to, checks the download against its sha256, and installs it into the local maps cache. Pin a version with `name@version`; without one you get the current release. `--accept-license` accepts the current agreement, as for the runtime. `--cache-root DIR` puts the cache somewhere else. An interrupted download resumes when you run the command again. `--for` says what the map is for: | `--for` | Installs | Enough for | |---|---|---| | `2d` (default) | the road network (OpenDRIVE, lanes, signals, typed locations), its 2D drawing, the ground and the static colliders: a 3–25 MB download | `scenario new` and `scenario apply`, `locations`, `simulate`, `render --mode 2d` | | `3d` | the whole map, adding the meshes and textures the renderer draws: a 5–22 GB download | all of the above plus a local `render --mode 3d` | Without `--for`, `maps pull` installs `2d`, unless the map is already installed for 3D: an update of a 3D install stays `3d`. A render with `--cloud` needs no local map at all: the cloud resolves the map the scenario names. So the usual local setup is the default 2D install for authoring and 2D renders, and `--cloud` for 3D. A local 3D render of a map installed for 2D only fails with `map_3d_not_installed`, which names the command that adds the rest: `simforge maps pull @ --for 3d`. Accepting the agreement is all a public map needs: the first download entitles your workspace. The public maps are `richmond-field-station`, `easterbrook-discovery-school`, `el-camino-road`, `belmont-research-center`, `yale-street`, `san-ramon-phase-1` and `di-rosa-sf`. A map that is not public fails with `entitlement_required` (pick another map; retrying does not help), and one an operator withdrew from your workspace with `entitlement_revoked`. Every map download is signed, and the CLI refuses an unsigned or untrusted one (`signature_*`). `simforge maps list` and `simforge maps show ` give each map's exact sizes before you download: its `packs` list every variant with its `version`, `downloadBytes` and `installedBytes`. A `--for 3d` download is large. A pull that cannot fit is refused before it downloads anything (`disk_insufficient`, naming the free and required bytes); the download and its unpacking stay inside the cache folder. While it runs, the pull writes `distribution.download.progress` and then `distribution.extract.progress` events (JSON lines on stderr) at each tenth and at least every 30 seconds. Pulling a map again downloads it again, so pull each version once. After the download, commands take the map's name (`easterbrook-discovery-school`), not `name@version`. ```sh simforge maps status ``` `maps status` lists the maps in the local cache and what each is installed for (`installed`: `2d` or `3d`) without going online. ## Maps in the cloud ```sh simforge maps list simforge maps show ``` These list the hosted maps your workspace can use with `--cloud`, with their `packs` (sizes per variant); each row's `installed` says what of it is in the local cache (`2d`, `3d` or `null`). Signed out they list the public maps (`"catalogue": "public"`), so you can see what there is before you create an account. ## Actor models ```sh simforge assets pull ``` `assets pull` downloads the actor models the renderer draws (cars, people, props) and their `ATTRIBUTION.json`, checking every file's sha256. The public actor set is the CARLA asset library under CC BY 4.0: keep the attribution file with anything you publish that shows them. ## When a map is missing A local command on a map that is not installed fails with `map_not_installed` and names the map. Pull it, then run the command again. `simforge doctor` shows the cache folders and their free space. --- # Author a scenario > Create a scenario on an installed map and change it with typed, atomic authoring operations instead of editing JSON by hand. Source: https://simforge.ai/docs/scenarios A scenario is a small JSON file: the map, the actors and their routes, what triggers what, the weather and the clip length. It holds no video or sensor data. Never write it by hand: create it on an installed map and change it with authoring operations, which check every change against the map. Scenario files use UTF-8, with or without a byte-order mark. Windows CRLF line endings are supported; save UTF-16 files as UTF-8 before running them. ## Find places on the map ```sh simforge locations find --map easterbrook-discovery-school --limit 20 simforge scenario actors ``` `locations find` returns real lanes and junctions as `{records, total, nextCursor}`, each record with its lane address (`anchor.road.rsl`, as `road:section:lane`) and distance along the lane in its direction of travel (`anchor.road.s`, which `scenario apply add-actor --lane-rsl --s` takes as it is); `--cursor ` reads the next page. Hosted `authoring discover --kind locations` prints the same records. `scenario actors` lists the actors you can place, such as `vehicle.sedan`. ## Create and edit ```sh simforge scenario new --map easterbrook-discovery-school --out s.json simforge scenario apply s.json add-actor --id ego --catalog-id vehicle.sedan --lane-rsl --s 20 \ -- set-ego --role-id ego -- set-clip --clip-seconds 20 simforge scenario check s.json --strictness warnings --preview ``` - Operations are chained with `--`; the whole chain applies atomically or not at all. - `--expect-version` and `--expect-digest`, given right after the file and before the first operation, refuse a change when the file changed since you last read it. - `scenario check` validates the scenario on its map and exits 2 on findings; `--preview` also drives every actor on the engine. - `simforge scenario ops list` lists every operation: actors, ego, speed, lane changes, junction turns, routes, event timing, weather, clock and clip length. ## Packages A scenario package (`.scenario.zip`) carries a scenario with what it pins, for moving it between machines. ```sh simforge package inspect scenario.zip simforge package verify scenario.zip ``` `package verify` checks the manifest and every member's digest and exits 2 on a mismatch. Both need the runtime. > **Note.** `simforge package import` resolves a package's map from SimForge's internal registry; it does not read the maps you pulled. ## For agents The `simforge-authoring` skill in the [skills pack](/docs/skills) covers every operation, its arguments and the refusals, with worked examples. --- # Preview and render > Render a scenario as a 2D top-down video, a schematic view or photoreal 3D sensor data locally; hosted rendering is schematic/3D only. Source: https://simforge.ai/docs/render One renderer, the Helios Engine, in three modes: | Mode | What you get | Runs on | |---|---|---| | `2d` | Top-down map video of the scenario | CPU | | `schematic` | A clean schematic view through the rig's cameras | GPU | | `3d` | Photoreal camera images and sensor passes | GPU | ## Pick a sensor rig Schematic and 3D renders look through a rig: the cameras on the vehicle. Without `--rig` they use the `basic-dash-camera` rig on the ego, a front camera at the preset's capture (1280x720 at 24 fps for the default `showcase`); write your own to choose others. ```sh simforge rig init --preset basic-dash-camera --out rig.json simforge rig show rig.json ``` `rig init` writes the cameras at the `--profile` capture: `showcase` (1280x720 at 24 fps) unless you pass `--profile training` (512x384 at 20 fps). The basic dash camera's camera is `dashcam-front`. Other presets include `alpamayo-2cam`, `alpamayo-6cam`, `tesla-hw3` and `waymo-5th-gen`; `simforge rig init --help` lists them all, and the vehicle classes. ## Preview a frame ```sh simforge preview frame s.json --mode 2d --at 5 --out plan.png simforge preview frame s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --out frame.png simforge preview clip s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --seconds 3 --out clip.mp4 ``` ## Render ```sh simforge render s.json --mode 2d --out out-2d simforge render s.json --mode schematic --rig rig.json --out out-schematic simforge render s.json --mode 3d --rig rig.json --out out-3d ``` `--out` is a folder that must not exist or be empty. 3D renders take `--preset showcase` or `--preset training` (the dataset look at a fixed cost per frame); without `--preset`, a 3D render uses `showcase`, locally and with `--cloud`. They also take `--passes` to choose among `rgb`, `id`, `depth` and `semantic`. The 2D mode is local or browser-only and does not accept `--cloud`. ## GPU memory and texture tiers GPU memory depends on the source texture tier, the material texture limit and the camera rig. The local CLI still selects a full-chain source (`uastc-full`) on GPUs with at least 16 GiB, or a preprocessed 512 px source (`bc7-512`) below that, and records the choice in `localDefaults`. `--textures` overrides the source tier. On Apple silicon the budget is three quarters of system memory. Material textures now have a default maximum dimension of **1024 px**, including map, vehicle, pedestrian and prop textures. This is an intentional fine-detail softening to reduce GPU memory; it does not change camera resolution, geometry or sensor cadence. The limit is an upper bound, not an upscale: a 512 px source stays 512 px, and normal mip planning may choose smaller textures. Base-color textures whose alpha controls cutouts or transparency keep their normal mip selection instead of the size cap, preserving fence openings and foliage coverage. The exception uses actual source alpha; an opaque texture is not exempt just because its material is labelled MASK or BLEND. Environment plates keep their authored resolution: reducing the star map to 1024 px removed fine night-sky detail. The material cap does not resize sky/moon plates, render targets or sensor buffers. Set `render.set.textureMaxSize` to `1024`, `2048`, `4096` or `"full"` in an authored scenario, or pass `--set textureMaxSize=...` to the CLI. `"full"` removes this limit while preserving normal mip/residency planning. To recover full source detail on a machine that otherwise selects the 512 px tier, set both: ```sh simforge render s.json --mode 3d --textures uastc-full --set textureMaxSize=full --out out-full ``` The historical tier measurements below have this material limit disabled (`textureMaxSize=full`); they are not measurements of the new 1024 px default. | GPU memory | What fits (measured peak) | |---|---| | 8 GiB | 512 px textures on every public map, up to six 1920x1080 cameras (4.5 GiB) | | 12 GiB | 512 px textures everywhere; full textures on every public map except san-ramon-phase-1, one 1280x720 camera (8.1 GiB) | | 16 GiB | Full textures on every public map, up to six 1920x1080 cameras (11.9 GiB) | Peak GPU memory of one 3D render (showcase preset, one 1280x720 camera unless stated): | Map | Full textures | 512 px textures | |---|---|---| | san-ramon-phase-1 | 11.3 GiB | 3.8 GiB | | san-ramon-phase-1, six 1920x1080 cameras | 11.9 GiB | 4.5 GiB | | di-rosa-sf | 8.1 GiB | 2.8 GiB | | richmond-field-station | 7.3 GiB | 2.1 GiB | | belmont-research-center | 7.2 GiB | 2.3 GiB | | yale-street | 6.5 GiB | 2.3 GiB | | el-camino-road | 3.5 GiB | 1.9 GiB | | easterbrook-discovery-school | 2.0 GiB | 1.8 GiB | The training preset needs about the same; a schematic render needs 0.2 GiB. Measured on an RTX 5080 (driver 595.84). The machine-readable table is `sdk/contracts/runtime-support.v1.json` in the toolkit. ## Output format A 3D render writes: | Path | Format | |---|---| | `render.json` | The render's summary, also printed by the command: preset, rig, texture tier, map and actor-pack identities, gates, timings, videos, warnings | | `results.json` | Every captured file with its sensor, tick, pass, byte count and sha256 | | `/.rgb.png` | The camera image, RGBA8 PNG, sRGB, with `--video off` or without `ffmpeg` (a video render encodes the frames into `video/.mp4` as it draws them) | | `/.id.png` | Instance ids, RGBA8 PNG: id = R + 256·G + 65536·B, 0 = sky or background | | `legend.json` | Instance id to the static map mesh it is | | `/.semantic.png` | Semantic class in the B channel of an RGBA8 PNG, CARLA CityScapes ids: 0 unlabeled, 1 building, 2 fence, 4 pedestrian, 5 pole, 7 road, 9 vegetation, 10 vehicle, 12 traffic sign, 18 traffic light | | `/.depth.f32.bin` | Raw little-endian float32, width x height, row-major, reverse-Z: distance along the camera axis in metres = `nearM / value` (`nearM` is the camera's near plane, 0.05 m by default); 0 = nothing hit | | `observed-frames.jsonl` | One line per frame: every actor's position, rotation and visibility as rendered | | `parity.json` | How closely the rendered poses match the simulated trace | | `video/.mp4` | The RGB frames as H.264 video (needs `ffmpeg`; `--video off` skips it) | | `job/` | The exact inputs the renderer ran with (timeline, scene state, texture plan) | Ticks count from the clip start at the rig's frame rate. Lidar sources write `/.ply` and radar sources `.csv`. A 2D render writes `render.json` and `video/topdown.mp4`. Traces are identical on every machine; pixels are identical run to run on the same GPU model and driver, and differ slightly between GPUs. Without ffmpeg, a 2D render that requests video preserves its images under `frames/topdown/.png` and reports that directory in `frame.directory`; it warns that no video was encoded. `--video off` remains a render-and-measure operation without retained frames. ## Before you render Check the GPU with `simforge doctor` and the runtime's render modes with `simforge runtime status`. A render that cannot run on this machine fails with a named error; it never falls back to another device or to the cloud. A scene that does not fit in GPU memory fails with `native_gpu_out_of_memory`: use `--textures bc7-512`, fewer or smaller cameras, or `--cloud` for schematic/3D. A 3D render whose GPU memory cannot be measured warns `gpu_memory_unmeasured`; `--vram-budget` sets the budget explicitly. ## Render in the cloud Hosted rendering runs on SimForge's machines with `--cloud` and a cost cap in cents; it is limited to schematic/3D. 2D stays in Studio's browser player or the local CLI: ```sh simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 200 --wait --download ``` Cloud video downloads include only the encoded videos and final metadata (`render.json`, `results.json`, `manifest.json`, `observed-frames.jsonl`, `parity.json` when present, and `runtime-info.json`). A cloud render draws only the RGB camera pass; its captured per-frame RGB, lidar and radar files are intermediate outputs and are not uploaded. A requested still preview retains its final RGB images instead of videos. The cloud `results.json` inventory lists only published files, so every listed capture is downloadable. The local renderer's diagnostic `job/` directory is not exported in full; its authoritative `job/timeline.json` is included. See [Run in the cloud](/docs/cloud) for pricing, caps and following a run. --- # Custom sensor rigs > Bring your own cameras, lidar and radar: the rig file, the vehicle frame and rotation conventions, render profiles, and how to use one rig locally, in the cloud and across scenarios. Source: https://simforge.ai/docs/sensors A sensor rig is a JSON file that lists what a render captures: the cameras, lidars and radars, where each one sits on the vehicle, which way it looks and what it records. The format is `simforge.render-rig/v1`. A rig is not part of a scenario: you pass it to each render, so one rig works with every scenario, and one scenario can be rendered through many rigs. Schematic and 3D renders look through a rig. Without `--rig` they use the built-in `basic-dash-camera` on the scenario's ego. 2D renders are a top-down map view and take no rig. ## The quick path ```sh simforge rig init --preset basic-dash-camera --class car --out rig.json simforge rig validate rig.json simforge rig show rig.json simforge render s.json --mode 3d --rig rig.json --out out-3d ``` `rig init` writes a built-in rig fitted to a vehicle class's default box, which is the easiest starting point to edit. `rig validate` checks the file exactly as a render reads it and exits 2 with the problem. `rig show` prints what the rig captures: each sensor's mount, the direction it faces, its field of view, and how much of the horizon the cameras cover together, with any gaps. Every field is in the [rig file reference](/docs/sensor-reference), and three complete, rendered rigs are in [example rigs](/docs/sensor-examples). ## How a rig attaches to the vehicle Every scenario names an ego: the vehicle that carries the sensors (a car, truck, bus, van, motorcycle or drone). A render mounts every source of the rig on that ego, whatever `actorId` the file names, and says so with the warning `render_rig_rehosted` when it moves one. So `actorId: "ego"` in a rig works for any scenario. A scenario without an ego is refused (`render_ego_missing`). Mounts are rigid: a sensor moves with the vehicle body, including its pitch and roll over bumps and on slopes, and it never sees the vehicle that carries it. The one exception is a drone camera with `mountFrame: "gimbal"`, whose rotation follows the drone's stabilized gimbal while its position rides the body. ## The vehicle frame Positions are metres in the vehicle's own frame. The origin is the centre of the vehicle's box at ground level. | Axis | Points | Notes | |---|---|---| | +X | forward, the direction of travel | the front bumper of a 4.8 m car is at x = 2.4 | | +Y | up | the roof of a 1.5 m tall car is at y = 1.5 | | +Z | left | the left side of a 1.9 m wide car is at z = 0.95 | ```text Top view (looking down) Side view (from the left) +X forward +Y up ^ ^ | | o-> camera, looking +X +------+------+ +-----+----------+ | | | | | | +Z <----+------o | | | | left | vehicle | ---+-----o----------+---> +X forward | box | ground (y = 0); o = origin +-------------+ o = origin (box centre, at ground level) ``` X forward, Y up and Z left make a left-handed frame. Keep that in mind when you bring mounts from a right-handed convention: see [converting calibrations](#converting-a-calibration-you-already-have) below. > **Note.** Lidar point clouds (PLY) use a different frame: +Z is to the right (CARLA parity). See [sensor outputs](/docs/sensor-outputs#lidar-files). Default boxes, used to fit the built-in rigs: car 4.8 x 1.9 x 1.5 m, van 5.5 x 2.0 x 2.2 m, truck 9.5 x 2.5 x 3.5 m, drone 1.0 x 1.0 x 0.45 m (length x width x height). A scenario actor with its own dimensions keeps its own box, and your rig's positions are used as written. ## Rotations A sensor's rotation is three angles in radians, all optional and 0 by default. At zero rotation a sensor looks straight ahead along +X, level, with its image upright. | Field | Turns the sensor | Positive means | Range | |---|---|---|---| | `yawRad` | about the vehicle's up axis | to the left (counter-clockwise seen from above) | -pi to pi | | `pitchRad` | about the sensor's sideways axis | nose up | -pi/2 to pi/2 | | `rollRad` | about the sensor's forward axis | right side down | -pi to pi | They apply in that order: yaw first, then pitch about the turned sensor, then roll about its own line of sight. There is no quaternion or matrix input. Worked examples: | Sensor | yawRad | pitchRad | rollRad | |---|---|---|---| | Forward, level | 0 | 0 | 0 | | Forward, tilted 5 degrees down | 0 | -0.0873 | 0 | | Left-facing (+90 degrees) | 1.5708 | 0 | 0 | | Right-facing (-90 degrees) | -1.5708 | 0 | 0 | | Front-left at 60 degrees | 1.0472 | 0 | 0 | | Rear-facing | 3.14159 | 0 | 0 | | Straight down (a mapping camera) | 0 | -1.5708 | 0 | A rear camera's yaw must stay inside the range: write 3.14159 or -3.14159, not 3.1416, which `rig validate` refuses (`transform.rotation is outside yaw/roll [-pi, pi], pitch [-pi/2, pi/2]`). ## Converting a calibration you already have Many calibrations use a right-handed vehicle frame with x forward, y left and z up (FLU), or x forward, y right and z down (FRD). To move a mount into a SimForge rig: | Your frame | SimForge x | SimForge y | SimForge z | yaw | pitch | roll | |---|---|---|---|---|---|---| | FLU (x fwd, y left, z up), origin at ground under the box centre | x | z | y | yaw | -pitch (FLU pitch is nose down) | roll | | FRD (x fwd, y right, z down), origin at ground under the box centre | x | -z | -y | -yaw | pitch | roll | If your origin is somewhere else, such as the rear axle, subtract that point's position in the SimForge frame first: for a rear axle 1.4 m behind the box centre, x = x_axle - 1.4. Check every converted rig with `simforge rig show`: its `facing` column names the side each sensor looks at (front, front-left, left, rear-left and so on), which catches a flipped sign at once. ## Render profiles The `--preset` flag picks how a 3D render looks; the rig picks what it captures. | Profile | Look | Built-in rigs record | |---|---|---| | `showcase` | photoreal, the default for a 3D render (local and `--cloud`) | 1280x720 at 24 fps | | `training` | the dataset look at a fixed cost per frame | 512x384 at 20 fps | | `schematic` | a clean schematic view (`--mode schematic`) | 1920x1080 at 30 fps | Without `--rig`, the default dash camera gets the profile's capture size and rate. `rig init` writes its cameras at a profile's capture (`--profile`, default `showcase`): fit a rig to the profile you render with, such as `--profile training` for a training render. A rig file keeps its own: each camera renders at its own `width`, `height` and `fps` whatever the profile. Give every camera of a rig the same `fps`; cameras at different rates are scheduled on the union of their frame times. ## Use a rig Locally, with the [runtime installed](/docs/sign-in): ```sh simforge rig validate rig.json simforge preview frame s.json --mode 3d --rig rig.json --camera cam-front --at 5 --out frame.png simforge render s.json --mode 3d --rig rig.json --out out-3d simforge render s.json --mode 3d --rig rig.json --preset training --passes rgb,depth --out out-train ``` `preview frame` renders one camera at one moment, which is the fastest way to check a mount; `--camera` takes a camera's `outputName`. `--passes` chooses the camera passes (`rgb`, `id`, `depth`, `semantic`; all four by default). In the cloud, the same file goes with the request: ```sh simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --dry-run simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 100 --wait --download ``` A cloud render sends the rig as part of the run (the `rig` of a `render` run in the [Cloud API](/docs/cloud)) and renders it with the same runtime. It returns each sensor's video plus `render.json` and `results.json`, not the per-frame images, depth or point clouds: render locally when you need those. See [sensor outputs](/docs/sensor-outputs). In Studio, the Render panel's sensor step offers the built-in rigs, a single front camera and the rig this scenario last rendered with. Studio has no rig upload yet: to render your own rig file, use the CLI, locally or with `--cloud`. ## Reuse a rig across scenarios A rig names no map, scenario or actor of its own, so keep one file per sensor setup and pass it to every render: ```sh for s in scenarios/*.json; do simforge render "$s" --mode 3d --rig rigs/surround.rig.json --out "out/$(basename "$s" .json)" done ``` Positions are absolute metres on the vehicle, so a rig written for a car sits in the same place on a truck: the cameras are not moved to the truck's roof. Write one rig per vehicle class you render, or start each from `simforge rig init --preset --class `, which fits a built-in rig to the class's box. One built-in rig is calibrated rather than fitted: `qwen-drive-3cam`, Qwen-Drive's three native 120° f-theta cameras as measured on one car (1920x1080 at 10 fps). `simforge rig init --preset qwen-drive-3cam --out rig.json` writes it as it is, for `--class car` only and without `--profile`. Render it with `--set lens.distortion=0`: the f-theta lens is the image, so the renderer's own lens post-warp must be off. A cloud render with it records the cameras' native 10 fps unless you pass `--fps`. ## What a rig cannot describe yet These are not supported today. A field for any of them is refused as unknown: - Pinhole intrinsics as fx, fy, cx, cy, or a shifted principal point: a pinhole camera is `horizontalFovDeg` with square pixels and the principal point at the image centre. - Brown-Conrady (OpenCV radial-tangential), Kannala-Brandt or equidistant fisheye distortion: the only lens model is the f-theta polynomial (`cameraModel`). - Exposure, shutter speed, rolling shutter, motion blur per camera, noise or lens flare settings. - IMU, GNSS, ultrasonic, thermal or event cameras. The vehicle's pose at every frame is in `observed-frames.jsonl`. - Separate depth, semantic or instance camera sources: those are passes of an RGB camera (`--passes`); a source with `modality` `depth`, `semantic` or `instance` validates but the renderer refuses it (`native_modality_unsupported`). - A lidar whose vertical field is not symmetric about its mount (for example +15 to -25 degrees), lidar noise or dropout, and radar noise or clutter. --- # Rig file reference > Every field of a simforge.render-rig/v1 sensor rig, with its type, range, default and units, and what the renderer supports on top. Source: https://simforge.ai/docs/sensor-reference A rig is one JSON object. Unknown fields are refused everywhere, so a typo fails `simforge rig validate` instead of being ignored. The JSON Schema is [/docs/render-rig.schema.json](/docs/render-rig.schema.json), and the CLI checks exactly the same rules. ## The file | Field | Type | Required | What | |---|---|---|---| | `schema` | string | yes | Always `simforge.render-rig/v1` | | `sources` | array of 1 to 64 sources | yes | The cameras, lidars and radars | | `clip` | object | no | `{startSeconds, endSeconds}`: render only this part of the scenario, `[start, end)` in seconds from the scenario's start. Default: the whole scenario (render.json `rig.clip.source: "timeline"`) | | `video` | object | only with a radar | `{width, height, fps, container, codec, quality}`. The renderer uses `fps` as the radar sample rate; `container` must be `mp4`, `codec` `h264`, and `quality` one of `draft`, `standard`, `high` (`lossless` is refused). Each video takes its size from the cameras | | `sensorHosts` | array | no | Which vehicle carries each source: `{sourceId, actorId, vehicleAsset: {catalogAssetId}}`. Leave it out: every source is then carried by the scenario's ego | ## Every source | Field | Type | Required | What | |---|---|---|---| | `actorId` | string | yes | The vehicle that carries it. Any value works: a render mounts every source on the scenario's ego. Use `ego` | | `sensorId` | string | yes | Your name for the sensor | | `sensorLabel` | string, 1 to 200 characters | no | A human-readable name, shown by `rig show` and in Studio | | `outputName` | string | yes | The sensor's folder and file prefix in the output: a letter or digit, then up to 63 letters, digits, `-` or `_`. Unique in the rig | | `modality` | string | yes | `rgb` for a camera, `lidar`, or `radar` | | `transform.position` | `{x, y, z}`, metres | yes | The mount in the [vehicle frame](/docs/sensors#the-vehicle-frame): x forward, y up, z left, from the box centre at ground level | | `transform.rotation` | `{yawRad, pitchRad, rollRad}`, radians | no | Each 0 by default. [Rotations](/docs/sensors#rotations): yaw left-positive in -pi to pi, pitch nose-up in -pi/2 to pi/2, roll right-side-down in -pi to pi | | `mountFrame` | `body` or `gimbal` | no, cameras only | `gimbal` follows a drone's stabilized gimbal; default `body` | | `attributes` | object | yes | By modality, below | Two sources may not share an `outputName`, or the same `actorId`, `sensorId` and `modality`. > **Note.** Lidar point clouds (PLY) use a different frame: +Z is to the right (CARLA parity). See [sensor outputs](/docs/sensor-outputs#lidar-files). ## Camera (`modality: rgb`) | Attribute | Type | Range | What | |---|---|---|---| | `width` | integer, pixels | 64 to 8192 (the renderer draws at most 4096) | Image width | | `height` | integer, pixels | 64 to 8192 (the renderer draws at most 4096) | Image height | | `fps` | number, frames per second | above 0, up to 240 | Frame rate | | `horizontalFovDeg` | number, degrees | above 0, up to 180 | Pinhole camera: the horizontal field of view. The vertical field follows from the aspect ratio | | `cameraModel` | object | | Lens camera: an f-theta lens (below) | | `nearM` | number, metres | above 0 | Near clip plane; also the depth pass's scale. Typical: 0.05 | | `farM` | number, metres | above `nearM` | Far clip plane. Typical: 1000 | A camera has exactly one of `horizontalFovDeg` and `cameraModel`. A pinhole camera's intrinsics follow from these fields: the principal point is the image centre, pixels are square, and the focal length in pixels is fx = fy = (width / 2) / tan(horizontalFovDeg / 2). A 1280x720 camera with a 90 degree field of view has fx = fy = 640, cx = 640 and cy = 360, and a vertical field of 58.7 degrees; `rig show` prints each camera's vertical field. Every camera writes the passes you ask for with `--passes`: `rgb` (the image), `depth`, `id` (instances) and `semantic`. See [sensor outputs](/docs/sensor-outputs). ## Lens camera (`cameraModel`) For wide-angle and fisheye cameras, an f-theta lens maps the distance of a pixel from the principal point to the angle of its ray from the optical axis. | Field | Type | What | |---|---|---| | `type` | `ftheta` | The only lens model | | `referenceResolution` | `[width, height]`, pixels | The calibration image's size. The camera's `width` and `height` must keep its aspect ratio (within 1%); the image is a centre-aligned resize of it | | `principalPoint` | `[x, y]`, reference pixels | The optical centre; integer values are pixel centres | | `pixeldistToAnglePoly` | 2 to 8 numbers | theta(r) = c0 + c1 r + c2 r^2 + ..., r in reference pixels from the principal point, theta in radians. It must rise steadily out to the image corners and stay below 90 degrees there | | `maxAngleRad` | number, radians, above 0 | The calibration's valid half-angle; informational | | `angleToPixeldistPoly` | numbers | The inverse fit (radians to pixels); informational, for your own projections | | `linearCde` | `[c, d, e]` | Linear skew: pixel = principal point + `[[c, d], [e, 1]]` x the polynomial's offset. Default `[1, 0, 0]` | ```json { "schema": "simforge.render-rig/v1", "sources": [ { "actorId": "ego", "sensorId": "cam-front-fisheye", "outputName": "cam-front-fisheye", "transform": { "position": { "x": 2.3, "y": 0.8, "z": 0 }, "rotation": { "pitchRad": -0.2 } }, "modality": "rgb", "attributes": { "width": 960, "height": 600, "fps": 20, "nearM": 0.05, "farM": 1000, "cameraModel": { "type": "ftheta", "referenceResolution": [1920, 1200], "principalPoint": [959.5, 599.5], "pixeldistToAnglePoly": [0, 0.00105, 0, 0], "maxAngleRad": 1.2 } } } ] } ``` ```sh simforge preview frame s.json --mode 3d --rig fisheye.rig.json --camera cam-front-fisheye --at 5 --out fisheye.png ``` ![The f-theta camera above, previewed at 5 s on easterbrook-discovery-school](/docs/images/sensors-ftheta-preview.jpg) The open CLI checks a lens's shape; the installed runtime plans it, so `rig validate` and `rig show` of a rig with a lens need the runtime. Only the Helios Engine draws f-theta lenses. ## Lidar (`modality: lidar`) | Attribute | Type | Range | What | |---|---|---|---| | `channels` | integer | 1 to 256 | Laser rings, spread evenly over the vertical field | | `rangeM` | number, metres | above 0, up to 1000 | Maximum range | | `pointsPerSecond` | integer | above 0 | Points across all channels per second | | `rotationFrequencyHz` | number, Hz | above 0, up to 240 | Revolutions per second; sets how dense a scan is. A full scan is written at every rendered frame | | `upperFovDeg` | number, degrees | -180 to 180, above `lowerFovDeg` | Highest ring's elevation | | `lowerFovDeg` | number, degrees | -180 to 180 | Lowest ring's elevation. The renderer needs `upperFovDeg = -lowerFovDeg` (`native_lidar_asymmetric_fov_unsupported`) | | `horizontalFovDeg` | number, degrees | 5 to 360, default 360 | Azimuth coverage, centred on the sensor's forward direction | Each ring takes pointsPerSecond / (channels x rotationFrequencyHz) samples per revolution, rounded and kept between 64 and 4096. A beam that hits nothing within `rangeM` returns no point. ## Radar (`modality: radar`) | Attribute | Type | Range | What | |---|---|---|---| | `horizontalFovDeg` | number, degrees | above 0, up to 180 | Horizontal field, centred on the sensor's forward direction | | `verticalFovDeg` | number, degrees | above 0, up to 180 | Vertical field | | `rangeM` | number, metres | above 0, up to 1000 | Maximum range | | `pointsPerSecond` | integer | above 0 | Rays cast per second; at the rig's `video.fps` this must give at least one ray per frame (`native_radar_budget_invalid`) | A rig with a radar needs `video` (its `fps` is the radar's sample rate). Each frame lists the detections with their range, angles and radial velocity. ## What the renderer adds on top `rig validate` checks the format. A render also checks what the Helios Engine can draw, and fails with a named error before any work: | Rule | Error | |---|---| | At least one `rgb` camera | `native_render_camera_missing` | | Cameras at most 4096x4096 | `native_camera_size_unsupported` | | Only `rgb` cameras, `lidar` and `radar` sources | `native_modality_unsupported` | | A symmetric lidar field (`upperFovDeg = -lowerFovDeg`) | `native_lidar_asymmetric_fov_unsupported` | | A rig with a radar names `video` | `render_schedule_invalid` | | Enough radar rays per frame | `native_radar_budget_invalid` | | `video` is mp4, h264, not lossless | `native_video_profile_unsupported`, `native_video_quality_unsupported` | --- # Example rigs > Three complete sensor rigs, each rendered end to end on a public map - a front camera, a seven-camera surround rig, and a camera with lidar and radar. Source: https://simforge.ai/docs/sensor-examples Each rig below passes `simforge rig validate` and was rendered with the runtime on easterbrook-discovery-school, exactly as shown, on an RTX 5080. Copy one into a file, change what you need, and check it with `simforge rig show`. All three use the [vehicle frame](/docs/sensors#the-vehicle-frame) of a car: x forward, y up, z left, in metres from the centre of the car's box at ground level. ## Front camera One forward camera on the windscreen, 1280x720 at 20 fps with a 90 degree field of view, tilted 3 degrees down (`pitchRad` -0.0524). ```json { "schema": "simforge.render-rig/v1", "sources": [ { "actorId": "ego", "sensorId": "cam-front", "sensorLabel": "Front camera", "outputName": "cam-front", "modality": "rgb", "transform": {"position": {"x": 1.6, "y": 1.45, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": -0.0524, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000} } ] } ``` ```sh simforge render s.json --mode 3d --rig front-camera.rig.json --out out-front ``` Every camera writes four passes per frame unless you narrow them with `--passes`: the image (into the camera's video; `--video off` writes PNGs instead), metric depth, semantic classes and instance ids (here colourised for display; the files hold the raw values, see [sensor outputs](/docs/sensor-outputs)). ![Front camera at 5 s: the RGB image, depth, semantic classes and instance ids](/docs/images/sensors-front-camera-passes.jpg) ## Seven-camera surround rig A layout like the camera arrays of common driving datasets: a wide and a telephoto camera looking forward, two front-corner cameras at 60 degrees, two rear-corner cameras at 120 degrees, and one camera looking back. Together they cover the whole horizon; `simforge rig show` reports `"coveredDeg": 360.0` and no gaps. ```json { "schema": "simforge.render-rig/v1", "sources": [ { "actorId": "ego", "sensorId": "cam-front", "sensorLabel": "Front wide", "outputName": "cam-front", "modality": "rgb", "transform": {"position": {"x": 1.7, "y": 1.5, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": 0, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 120, "nearM": 0.05, "farM": 1000} }, { "actorId": "ego", "sensorId": "cam-front-tele", "sensorLabel": "Front telephoto", "outputName": "cam-front-tele", "modality": "rgb", "transform": {"position": {"x": 1.72, "y": 1.52, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": 0, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 30, "nearM": 0.05, "farM": 1000} }, { "actorId": "ego", "sensorId": "cam-front-left", "sensorLabel": "Front left", "outputName": "cam-front-left", "modality": "rgb", "transform": {"position": {"x": 1.5, "y": 1.46, "z": 0.45}, "rotation": {"yawRad": 1.0472, "pitchRad": 0, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 120, "nearM": 0.05, "farM": 1000} }, { "actorId": "ego", "sensorId": "cam-front-right", "sensorLabel": "Front right", "outputName": "cam-front-right", "modality": "rgb", "transform": {"position": {"x": 1.5, "y": 1.46, "z": -0.45}, "rotation": {"yawRad": -1.0472, "pitchRad": 0, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 120, "nearM": 0.05, "farM": 1000} }, { "actorId": "ego", "sensorId": "cam-rear-left", "sensorLabel": "Rear left", "outputName": "cam-rear-left", "modality": "rgb", "transform": {"position": {"x": -0.6, "y": 1.44, "z": 0.45}, "rotation": {"yawRad": 2.0944, "pitchRad": 0, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000} }, { "actorId": "ego", "sensorId": "cam-rear-right", "sensorLabel": "Rear right", "outputName": "cam-rear-right", "modality": "rgb", "transform": {"position": {"x": -0.6, "y": 1.44, "z": -0.45}, "rotation": {"yawRad": -2.0944, "pitchRad": 0, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000} }, { "actorId": "ego", "sensorId": "cam-rear", "sensorLabel": "Rear", "outputName": "cam-rear", "modality": "rgb", "transform": {"position": {"x": -2.2, "y": 1.2, "z": 0}, "rotation": {"yawRad": 3.14159, "pitchRad": 0, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000} } ] } ``` | Camera | Faces | yawRad | Field of view | |---|---|---|---| | `cam-front` | forward | 0 | 120 degrees | | `cam-front-tele` | forward | 0 | 30 degrees | | `cam-front-left` | front left | 1.0472 (60 degrees) | 120 degrees | | `cam-front-right` | front right | -1.0472 (-60 degrees) | 120 degrees | | `cam-rear-left` | rear left | 2.0944 (120 degrees) | 90 degrees | | `cam-rear-right` | rear right | -2.0944 (-120 degrees) | 90 degrees | | `cam-rear` | back | 3.14159 (180 degrees) | 90 degrees | The rear camera's yaw is 3.14159, inside the allowed range of -pi to pi; 3.1416 is just outside it and fails validation. ```sh simforge render s.json --mode 3d --rig surround-7cam.rig.json --preset training --out out-surround ``` ![The seven cameras at 5 s, training preset: front left, front, front right, rear left (top row); front telephoto, rear right, rear (bottom row)](/docs/images/sensors-surround-7cam.jpg) ## Camera, lidar and radar The front camera, a 64-channel roof lidar and a forward radar 0.2 m ahead of the front of the box. The lidar spins at 10 Hz with a symmetric vertical field of +15 to -15 degrees and 1.2 million points per second, so each scan has up to 64 x 1875 points. The radar casts 1500 rays per second over a 90 x 10 degree field. A rig with a radar needs `video`: its `fps` (20 here) is the radar's sample rate. ```json { "schema": "simforge.render-rig/v1", "sources": [ { "actorId": "ego", "sensorId": "cam-front", "sensorLabel": "Front camera", "outputName": "cam-front", "modality": "rgb", "transform": {"position": {"x": 1.6, "y": 1.45, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": -0.0524, "rollRad": 0}}, "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000} }, { "actorId": "ego", "sensorId": "lidar-roof", "sensorLabel": "Roof lidar", "outputName": "lidar-roof", "modality": "lidar", "transform": {"position": {"x": 0, "y": 1.9, "z": 0}}, "attributes": {"channels": 64, "rangeM": 120, "pointsPerSecond": 1200000, "rotationFrequencyHz": 10, "upperFovDeg": 15, "lowerFovDeg": -15, "horizontalFovDeg": 360} }, { "actorId": "ego", "sensorId": "radar-front", "sensorLabel": "Front radar", "outputName": "radar-front", "modality": "radar", "transform": {"position": {"x": 2.6, "y": 0.5, "z": 0}}, "attributes": {"horizontalFovDeg": 90, "verticalFovDeg": 10, "rangeM": 150, "pointsPerSecond": 1500} } ], "video": {"width": 1280, "height": 720, "fps": 20, "container": "mp4", "codec": "h264", "quality": "standard"} } ``` ```sh simforge render s.json --mode 3d --rig camera-lidar-radar.rig.json --out out-lidar ``` The render writes the camera's passes, one `.ply` point cloud for the lidar and one `.csv` of detections for the radar at every frame, and a review video for each sensor. ![The camera, and the lidar and radar review videos at 5 s](/docs/images/sensors-camera-lidar-radar.jpg) Mount a radar outside the vehicle's body. A radar inside it, for example at the box's front edge where the body's nose is, sees only the body: every detection is a few centimetres away. ## The scenario used ```sh simforge maps pull easterbrook-discovery-school --accept-license simforge scenario new --map easterbrook-discovery-school --clip-seconds 10 --out s.json simforge scenario apply s.json add-actor --id ego --catalog-id vehicle.sedan --lane-rsl 10:0:-1 --s 6 \ -- set-ego --role-id ego -- add-actor --id lead --catalog-id vehicle.sedan --lane-rsl 10:0:-1 --s 30 -- set-clock --minutes 780 ``` A sedan ego following a second sedan down road 10 at about 13 m/s, at 13:00, for 10 seconds: 200 frames at 20 fps. --- # Sensor outputs > What a render writes for each camera, lidar and radar - the folder layout, file formats, timestamps, the camera and vehicle poses - and how to load it in Python. Source: https://simforge.ai/docs/sensor-outputs A local 3D render writes everything into its `--out` folder, which must not exist yet or be empty. This is the layout for the [camera, lidar and radar example](/docs/sensor-examples#camera-lidar-and-radar): ```text out-lidar/ render.json the render's summary: rig, preset, render settings, timings, videos, warnings results.json every captured file: sensor, tick, pass, bytes, sha256, and each camera frame's projection and time observed-frames.jsonl one line per frame: every actor's position, rotation and visibility as rendered legend.json instance id -> the static map mesh it is parity.json how closely the rendered poses match the simulated trace render.log cam-front/00000000.rgb.png one file per frame and pass (RGB only with --video off) cam-front/00000000.depth.f32.bin cam-front/00000000.semantic.png cam-front/00000000.id.png lidar-roof/00000000.ply one point cloud per frame radar-front/00000000.csv one detection list per frame video/cam-front.mp4 one video per sensor (lidar and radar as review videos) job/render-job.json the exact inputs, including every camera's pose at every frame ``` Folder and file names come from each source's `outputName`. Ticks are zero-padded to eight digits. A lidar or radar review video shows the sensor's right on the right of the picture. Review videos rendered before 13 October 2026 are mirrored left to right; the point clouds and radar files of those renders are correct. ## Camera files | File | Format | |---|---| | `.rgb.png` | The image, RGBA8 PNG, sRGB. Written with `--video off` or without ffmpeg; otherwise the frames go straight into `video/.mp4` | | `.depth.f32.bin` | Little-endian float32, width x height, row-major, reverse-Z: distance along the camera axis in metres = `nearM / value`; 0 means nothing was hit (sky) | | `.semantic.png` | The class in the blue channel of an RGBA8 PNG, CARLA CityScapes ids: 0 unlabeled, 1 building, 2 fence, 4 pedestrian, 5 pole, 7 road, 9 vegetation, 10 vehicle, 12 traffic sign, 18 traffic light | | `.id.png` | The instance id, RGBA8 PNG: id = R + 256 x G + 65536 x B, 0 = sky or background. `legend.json` names the static map meshes; ids missing from it are actors | `--passes rgb,depth` (any of `rgb`, `id`, `depth`, `semantic`) writes only the passes you need. A 1280x720 frame takes about 1 MB as RGB and 3.7 MB as depth, so 7 cameras x 200 frames with all four passes come to 6.3 GB. The RGB image carries the preset's camera look: exposure, tone mapping, bloom, a vignette and a slight lens distortion (0.008). Depth, id and semantic are drawn without them, so their pixels line up with an undistorted pinhole. For pixel-exact geometry in the RGB image too, turn the distortion off: `--set lens.distortion=0`. `render.json` records every setting in `renderConfig`. ## Lidar files `.ply` is an ASCII PLY point cloud with one row per point: ```text ply format ascii 1.0 element vertex 91524 property float x property float y property float z property float intensity property uint instance_id end_header ``` `x`, `y` and `z` are metres in the lidar's own frame: x forward along the sensor's line of sight, y up, and **z to the right** (unlike the rig's vehicle frame, whose z points left). `intensity` is a reflectance proxy in 0 to 1, and `instance_id` uses the same ids as the cameras' id pass. The renderer writes one full scan at every rendered frame, the camera rate: the example's 10 Hz lidar gives 200 point clouds in 10 seconds at 20 fps, not 100. Every point of a scan is taken at the frame's instant (no motion during the sweep), and `rotationFrequencyHz` sets how dense a scan is: each ring has pointsPerSecond / (channels x rotationFrequencyHz) samples, kept between 64 and 4096. Beams that hit nothing within `rangeM` return no point, so the count varies from frame to frame. ## Radar files `.csv` has one detection per row: | Column | Unit | Meaning | |---|---|---| | `depth_m` | metres | Range along the beam | | `azimuth_rad` | radians | Beam direction; 0 is the radar's forward axis, positive to the right | | `altitude_rad` | radians | Beam elevation; positive up | | `velocity_mps` | m/s | Radial velocity relative to the radar; negative when the target closes in. Static objects ahead of a vehicle driving at 13 m/s read about -13 | ## Time and synchronization Every sensor is captured at the same instants. Frame `n` is at time clipStart + n / fps, in seconds from the scenario's start: `results.json` gives each camera frame's `sampleTime.seconds`, and `observed-frames.jsonl` each frame's `time`. The rig's `clip` sets clipStart (0 by default). A frame is an instantaneous snapshot: rolling shutter, exposure time and sensor clock drift are not modelled (`results.json` says so per frame under `camera.sampling`). ## Poses and calibration There is no separate calibration file: the rig you rendered with is the calibration. - **Intrinsics.** A pinhole camera has fx = fy = (width / 2) / tan(horizontalFovDeg / 2) and its principal point at the image centre. `results.json` gives each frame's `projection`: `kind`, `fovYDeg`, `nearM`, `farM` and the depth convention. - **Extrinsics.** Each source's `transform` in the rig: its position and rotation on the vehicle. - **The vehicle in the world.** `observed-frames.jsonl` gives every actor's pose at every frame: `position` `[x, y, z]` in metres and `rotation` as a quaternion `[x, y, z, w]`, in the map's scene frame (x east, y up, z south). - **Each camera in the world.** `job/render-job.json` lists every camera at every frame (`rig.cameraSchedule`): its mount as the renderer applied it (`attach`), and its position (`eye`) and a point 50 m along its line of sight (`target`) in the scene frame. ## Load it in Python The toolkit has no reader for these files; NumPy and Pillow are enough: ```python import json from pathlib import Path import numpy as np from PIL import Image out = Path("out-lidar") results = json.loads((out / "results.json").read_text()) # Every captured file: its sensor, tick, pass and, for cameras, the projection and sample time. for artifact in results["artifacts"][:3]: print(artifact["sensorId"], artifact["tick"], artifact["pass"], artifact["path"]) # One camera frame: the image (a PNG with `--video off`), instance ids, semantic classes and metric depth. cam, tick = "cam-front", 40 rgb = np.asarray(Image.open(out / cam / f"{tick:08d}.rgb.png"))[..., :3] idx = np.asarray(Image.open(out / cam / f"{tick:08d}.id.png")).astype(np.uint32) instance = idx[..., 0] + 256 * idx[..., 1] + 65536 * idx[..., 2] semantic = np.asarray(Image.open(out / cam / f"{tick:08d}.semantic.png"))[..., 2] height, width = rgb.shape[:2] raw = np.fromfile(out / cam / f"{tick:08d}.depth.f32.bin", dtype=" 0, near_m / np.maximum(raw, 1e-12), np.inf) # inf: nothing hit # Pinhole intrinsics from the rig: horizontalFovDeg 90 at 1280x720. fx = fy = (width / 2) / np.tan(np.radians(90) / 2) cx, cy = width / 2, height / 2 # One lidar scan: x forward, y up, z right of the sensor, metres. def read_ply(path): lines = path.read_text().splitlines() start = lines.index("end_header") + 1 return np.loadtxt(lines[start:], ndmin=2) # columns: x y z intensity instance_id points = read_ply(out / "lidar-roof" / "00000004.ply") # One radar frame: range, azimuth, elevation, radial velocity. radar = np.genfromtxt(out / "radar-front" / "00000040.csv", delimiter=",", names=True) ``` ## In the cloud A cloud 3D render (`--cloud`) returns each sensor's video (`video/.mp4`, lidar and radar as review videos) with `render.json` and `results.json`. It does not return the per-frame images, depth, id or semantic passes, point clouds or radar files: render locally when you need them. `simforge preview frame --cloud` returns the RGB frame as a PNG. --- # Sensor performance and troubleshooting > How sensor count, resolution and passes change render time, GPU memory and cloud price; the limits; and the rig errors you can hit, with their fixes. Source: https://simforge.ai/docs/sensor-troubleshooting ## Render time A 3D render draws every camera at every one of its frames, so its time grows with the number of cameras x pixels x frames, plus a fixed cost to load the map. Lidar and radar add ray casts and a review video each. Measured on an RTX 5080 with the three [example rigs](/docs/sensor-examples) on easterbrook-discovery-school, a 10 second scenario: | Rig | Preset | Frames written | Render time | Of which drawing | Output on disk | |---|---|---|---|---|---| | Front camera, 1280x720 | showcase | 200 x 4 passes | 15.3 s | 9.1 s | 0.9 GB | | Seven cameras, 1280x720 | training | 1400 x 4 passes | 36.0 s | 23.8 s | 6.3 GB | | Camera, lidar and radar | showcase | 200 x 4 passes, 200 scans, 200 radar frames | 25.6 s | 14.8 s | 1.8 GB | Peak GPU memory stayed under 2.5 GiB for all three, with full textures. Most of the rest of the time is encoding the videos (10.9 s for the seven cameras) and loading the map and actors (about 3 s). A single `simforge preview frame` took 9 s. To render faster: use `--preset training`, lower `width` and `height`, render a part of the scenario with the rig's `clip`, ask only for the passes you need (`--passes rgb`), and skip videos you do not need (`--video off`). Check a mount with `simforge preview frame` before a full render. ## GPU memory GPU memory depends far more on the map and the texture tier than on the cameras. On san-ramon-phase-1 with full textures, one 1280x720 camera peaks at 11.3 GiB and six 1920x1080 cameras at 11.9 GiB; with 512 px textures (`--textures bc7-512`), 3.8 and 4.5 GiB. The full table per map is in [Preview and render](/docs/render#gpu-memory-and-texture-tiers). A render that does not fit fails with `native_gpu_out_of_memory`: use `--textures bc7-512`, fewer or smaller cameras, or `--cloud`. ## Cloud price A cloud render's [price](/docs/cloud#prices) does not depend on the rig: more cameras, lidar or radar do not change it. 2D visualization runs in Studio's browser player or locally, not in the cloud. `--dry-run` shows the price before anything runs, and `--max-cost` caps it ([Run in the cloud](/docs/cloud)). A cloud render returns each sensor's video, `render.json` and `results.json`; render locally for the per-frame images, depth and point clouds. ## Limits | What | Limit | |---|---| | Sources in a rig | 1 to 64 | | Camera size | 64 to 8192 pixels in the file; the renderer draws at most 4096x4096 | | Camera frame rate | above 0, up to 240 fps | | Pinhole field of view | above 0, up to 180 degrees | | Lidar | 1 to 256 channels, range up to 1000 m, up to 240 revolutions per second, symmetric vertical field | | Radar | fields up to 180 degrees, range up to 1000 m, at least one ray per frame | | Rotation | yaw and roll -pi to pi, pitch -pi/2 to pi/2, in radians | ## Rig errors and fixes `simforge rig validate` prints one JSON error and exits 2. These are its real messages: | Message (`render_rig_invalid`) | Fix | |---|---| | `sources[0].attributes: unknown field fx, expected one of width, height, fps, horizontalFovDeg, nearM, farM, cameraModel` | The rig has no fx/fy/cx/cy: give `horizontalFovDeg` = 2 x atan(width / (2 x fx)) in degrees | | `sources[0]: a camera declares exactly one of horizontalFovDeg and cameraModel` | Keep the field of view or the lens, not both and not neither | | `sources[0]: camera size 10000x720 is outside 64..8192` | Lower `width` or `height` (the renderer draws at most 4096) | | `sources[0]: fps 0 is outside (0, 240]` | Give a frame rate above 0 | | `sources[0]: nearM must be positive and farM greater than nearM` | Use `nearM` 0.05 and `farM` 1000 unless you need others | | `sources[6]: transform.rotation is outside yaw/roll [-pi, pi], pitch [-pi/2, pi/2]` | A rear camera is yaw 3.14159, not 3.1416; a downward camera is pitch -1.5708 | | `duplicate source outputName "cam-front"` | Give every source its own `outputName` | | `sources[0]: outputName "front cam" must match ^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$` | Use letters, digits, `-` and `_`, no spaces | | `sources[0].modality "thermal" (rgb\|depth\|semantic\|instance\|lidar\|radar)` | Thermal is not supported; use `rgb`, `lidar` or `radar` | | `rig schema "simforge.render-rig/v2" (expected simforge.render-rig/v1)` | Set `schema` to `simforge.render-rig/v1` | A render then checks what the renderer supports; its errors are listed in the [rig file reference](/docs/sensor-reference#what-the-renderer-adds-on-top). The most common: | Error | Fix | |---|---| | `native_modality_unsupported` | A `depth`, `semantic` or `instance` source: make it `rgb` and choose passes with `--passes` | | `native_lidar_asymmetric_fov_unsupported` | Make the lidar's field symmetric: `upperFovDeg = -lowerFovDeg` | | `render_schedule_invalid` (a radar) | Add `video` to the rig; its `fps` is the radar's sample rate | | `native_camera_size_unsupported` | Keep cameras at 4096x4096 or smaller | | `render_ego_missing` | The scenario has no ego: `simforge scenario apply s.json set-ego --role-id ` | | `preview_camera_unknown` | `--camera` takes a camera's `outputName`; the error lists the rig's cameras | | `out_not_empty` | `--out` must be a new or empty folder; a failed render can leave `.input` behind: remove it too | ## FAQ **My images are mirrored left to right, or a side camera shows the wrong side.** The frame is x forward, y up, z left, and yaw is positive to the left. A calibration in a right-handed frame needs [converting](/docs/sensors#converting-a-calibration-you-already-have). `simforge rig show` names each sensor's `facing`. **Can I give fx, fy, cx and cy?** Not yet. A pinhole camera is square-pixelled with the principal point at the centre: set `horizontalFovDeg` = 2 x atan(width / (2 x fx)). For a lens with distortion, fit an f-theta polynomial. **Which distortion models are supported?** The f-theta polynomial only. Brown-Conrady (OpenCV), Kannala-Brandt and equidistant fisheye are not supported yet. **Do I need one rig per scenario?** No. A rig names no scenario: pass the same file to every render. Every source is mounted on the scenario's ego. **Do I need one rig per vehicle?** Positions are absolute metres, so a car rig on a truck keeps the car's mount heights. Write one rig per vehicle class, or start from `simforge rig init --preset --class `. **Can Studio use my rig file?** Not yet: Studio offers the built-in rigs and the rig a scenario last rendered with. Render your rig with the CLI, locally or with `--cloud`. **Why does my cloud render have no PNGs or point clouds?** A cloud 3D render returns the videos, `render.json` and `results.json`. Render locally for the per-frame files. **Every radar detection is a few centimetres away.** The radar is inside the vehicle's body and sees only the body. Move it just outside: the example's front radar sits 0.2 m ahead of the box's front edge. **My lidar wrote more scans than it made revolutions.** It writes one full scan per rendered frame, at the camera rate, not one per revolution. See [lidar files](/docs/sensor-outputs#lidar-files). **Are IMU or GNSS supported?** Not as sensors. The ego's exact pose at every frame is in `observed-frames.jsonl`. **Still stuck?** Send us a ticket right after the failing command: `simforge feedback --attach-last-error ""` ([what it sends](/docs/feedback)). Or ask on the [SimForge Discord](https://discord.gg/Whmn9brNRd) with the rig file and the command you ran. --- # Train with SimForge Gym > Train driving models with SimForge Gym (the open simforge-gym package) on your own machine. Three recipes, one command each: a starter that fine-tunes Qwen Drive on four scenarios, supervised fine-tuning of Qwen Drive and Alpamayo 1.5, and reinforcement learning with PPO and closed-loop GRPO. Source: https://simforge.ai/docs/gym `simforge env serve` runs episodes of a scenario on the local runtime, with reactive traffic and optional rendered cameras. SimForge Gym is the one place to train driving models on SimForge: supervised fine-tuning, reinforcement learning and closed-loop evaluation. Its open `simforge-gym` Python package (formerly `simforge-rl`) is the Gymnasium client: it never contains a simulator of its own, so without a running server there is no episode. Training runs on your machine; the cloud does not run it. Install it from PyPI: ```sh python -m pip install 'simforge-gym[ppo]' ``` ## Recipes Each recipe is one command on your own machine. Every GPU need and time below was measured; each recipe's section says how. | Recipe | GPU | Time | | --- | --- | --- | | [Starter](#starter-recipe-fine-tune-qwen-drive-on-your-scenarios): four scenarios to a fine-tuned Qwen Drive 1 4B and a before/after report card | measured on 96 GB (training fits 16 GB) | 59.1 min on an RTX PRO 6000 | | [Qwen Drive SFT](#fine-tune-qwen-drive), one GPU | 16 GB (24 GB to prefill online) | 5,000 windows on an RTX 5080: about 50 min of prefill, then 10 min per epoch | | [Qwen Drive SFT](#fine-tune-qwen-drive), several GPUs | 16 GB per GPU | not measured yet | | [Alpamayo 1.5 SFT](#fine-tune-alpamayo-1-5), stage 2 | 40 GB or more | 2.3 s per step on an RTX PRO 6000 | | [PPO](#train-with-ppo) on state observations | none | 37.5 min on a 24-core CPU | | [GRPO](#closed-loop-rl-with-grpo) in closed loop, from an SFT checkpoint | 48 GB or more | about 20 min per update of 48 drives on an RTX PRO 6000 | Nothing runs in the cloud or is billed. Training on SimForge cloud comes later. ## Before you start ```sh simforge login simforge runtime install --accept-license pip install simforge-gym ``` Starter, Qwen Drive fine-tuning and GRPO also need the training extras and the Qwen Drive weights. `qwen_drive` (Apache-2.0, not on PyPI) declares flash-attn and causal-conv1d, which do not build before torch is installed and which its `sdpa` path never imports, so it installs without its own dependencies: ```sh pip install 'simforge-gym[train]' 'flash-linear-attention>=0.5.1' 'opencv-python-headless>=4.10' 'matplotlib>=3.8' 'tqdm>=4.66' pip install --no-deps "qwen_drive @ git+https://github.com/QwenLM/Qwen-Drive-1.0@28091c1532e869bc7aee91fc0aef6b3e6fd0b2e0" huggingface-cli download Qwen/Qwen-Drive-1.0-4B --local-dir Qwen-Drive-1.0-4B ``` Qwen Drive 1 4B (`Qwen/Qwen-Drive-1.0-4B`) is Apache-2.0. Local closed-loop evaluation (Starter, GRPO) needs simforge-gym 0.3.0 or later and the next runtime release. ## Serve episodes ```sh simforge env serve s.json --socket /tmp/simforge-env.sock --no-sensors --max-sessions 4 ``` The server prints a ready document and runs until you stop it. `--no-sensors` serves state only and needs no GPU. For camera observations, serve with `--rig rig.json` instead (see [Preview and render](/docs/render) for rigs). `--max-sessions` must cover every environment you open, including the evaluation one. On Windows use a runtime advertising `capabilities.envSocketTransport: "named-pipe"`. The same `--socket` path selects a secure local named pipe with a bounded hashed name, so long profile paths do not hit the Unix socket path limit. Older Windows runtimes fail with an upgrade instruction. macOS and Linux use Unix sockets. ## Drive it from Python ```python from simforge_gym import SimForgeEnv, SocketTransport, WeightedTerms with SimForgeEnv( SocketTransport("/tmp/simforge-env.sock"), action_mode="setpoint", observation="state", reward_fn=WeightedTerms({"progress": 1.0, "proximity": 2.0}), ) as env: observation, info = env.reset(seed=7) observation, reward, terminated, truncated, info = env.step([5.0, 1.0]) ``` - A `setpoint` action is `[target_speed_mps, target_accel_mps2]`; `action_mode="control"` takes `[throttle, brake, steer]`; `None` keeps the scenario's scripted control. - `observation="state"` is privileged engine state. `"sensors"` returns only rendered camera passes keyed `/`, with no engine state in `info`; `"state+sensors"` returns both. - `reward_fn` replaces the reward; `info["native_reward"]` keeps the runtime's own. - Reset after an episode terminates or truncates; nothing resets on its own. ## Train with PPO ```sh python -m simforge_gym.examples.ppo --tasks richmond-field-station --runtime --num-envs 8 --total-timesteps 200000 --out run/ ``` State observations need no GPU. `--tasks` picks a bundled task set: one per released map (`belmont-research-center`, `easterbrook-discovery-school`, `el-camino-road`, `richmond-field-station`, `yale-street`), each 24 to 29 weighted, seeded scenario templates (car following, cut-ins, lead braking, crossing pedestrians, unprotected turns, red-light runners) at sites verified on that map. A set is pinned to the map version it was verified on: pull that version. `--runtime` runs each worker's episodes in its own runtime process; without it the example starts `simforge env serve` processes instead, with the same observations and rewards. `run/` then holds `curve.csv`, `curve.svg`, `metadata.json` and `policy.pt`, and each evaluation also scores the scenarios' scripted driving as a baseline. `python -m simforge_gym.examples.ppo --help` lists every flag. The recorded baseline ran 1,228,800 environment steps in 37.5 minutes on a 24-core CPU host with no GPU: ```sh python -m simforge_gym.examples.ppo --tasks richmond-field-station --runtime --num-envs 32 \ --total-timesteps 1228800 --eval-every 10 --seed 0 --out run/ ``` Mean held-out return rose from 0.79 to 2.74, against 0.71 for the scenarios' authored driving. The trained policy beats that driving on 12 of the 24 tasks and still crashes on some. ## Train on a pulled dataset ```sh simforge datasets pull --include scenarios --out data/my-dataset python -m simforge_gym.examples.ppo --dataset data/my-dataset --split train --runtime --max-open 16 --eval-limit 8 --out run/ ``` `--dataset` takes a pulled folder or a hosted dataset id, which it pulls first (see [Getting data out](/docs/datasets)); every file is checked against its digest, and each episode runs exactly the input its scenario's simulation ran, on the installed map the scenario names. `--max-open` bounds the sessions each worker keeps open, so a split of thousands of scenarios trains with a bounded number of sessions. In Python, `ScenarioSet.from_dataset(...)` and `ScenarioSet.bundled(...)` give the same task sets. ## Evaluate a driving model in closed loop `simforge_gym.closed_loop.episode` drives one episode of an imported scenario with a camera-based driving model, rendering its cameras live on your GPU with `simforge env serve`. It is the runner SimForge cloud evaluation uses, so the episode folder is scored the same way: ```sh python -m simforge_gym.closed_loop.episode --plan plan.json --out episode/ python -m simforge_gym.eval.score episode/ ``` The plan names the scenario workspace, its camera ego, the rig and the model: a model host over HTTP (`modelHost`) or a checkpoint loaded in this process (`localPolicy`, which needs `simforge-gym[train]`). The score reports driving score, route completion, collisions, infractions and comfort. The plan fields are in the package README. To check that your own model server speaks the host wire before a run, see the [model host contract](/docs/model-host) and its one-command conformance check. ## Starter recipe: fine-tune Qwen-Drive on your scenarios One command turns a few scenarios into a fine-tuned Qwen-Drive-1.0 4B planner, then compares it with the base planner in closed loop. It all runs on your GPU; nothing runs in the cloud or is billed: ```sh python -m simforge_gym.recipes.starter --scenarios cut-in-brake,lead-hard-brake,queue-tail,opposing-turn-encroachment \ --base Qwen-Drive-1.0-4B/ --out starter/ ``` The recipe runs these stages: 1. Each scenario (a bundled template or your own v2 template) is placed at the sites of your installed maps that fit it, with several parameter draws per site. One whole map is held out for evaluation. 2. The training drives (48 by default) are rendered through `simforge env serve` with the model's three cameras. These are the same renderer settings the closed-loop evaluation uses. 3. The planner is fine-tuned with the vision-language model frozen. Part of each scenario's drives is held out as a validation set, and the checkpoint with the lowest validation loss is kept. 4. The base and the fine-tuned planner each drive the held-out variants with the same seed, and every drive is scored as SimForge cloud evaluation scores it. 5. `starter/report/report.html` shows driving score, route completion, collisions and off-road per scenario and overall, the paired difference per drive, videos, the GPU, and each stage's time and GPU memory peak. Re-running the command resumes where it stopped. `--baseline-only` reports the base planner alone, which shows where it has room to improve. `python -m simforge_gym.recipes.starter --help` lists every flag. You need: - the CLI signed in and the runtime installed; - two or more maps with their 3D packs (`simforge maps pull NAME --for 3d --accept-license`, 5 to 14 GB each) and `simforge assets pull`; - an NVIDIA GPU: training fits 16 GB, and a whole run has been measured on a 96 GB one; - the training extras and the Qwen Drive weights, and the closed-loop releases (see [Before you start](#before-you-start)). On an RTX PRO 6000 with these four scenarios, a run with 48 training drives took 59.1 minutes: 10.4 to render the 48 drives, 12.6 to train and 34.4 to drive the 12 held-out variants with both planners. Training with the prefill cache fits a 16 GB GPU (9.5 GiB peak); a full run on a smaller card has not been measured. In that run, the fine-tuned planner collided less (4 collisions down to 2) and left the road less (6 off-road down to 1) than the base planner. It also completed less of its route, so its driving score was about the same: 0.694 against 0.710, better on 4 of 12 held-out drives. Fine-tuning teaches the planner how your scenarios are driven. The closed-loop score is what reinforcement learning (GRPO) then optimizes, starting from the recipe's best checkpoint. ## Fine-tune Qwen Drive `python -m simforge_gym.train.sft` fine-tunes the Qwen Drive 1 4B planner expert with the vision-language model frozen, on renders exported with `python -m simforge_gym.export --format qwen-drive` (the package README covers rendering and export): ```sh python -m simforge_gym.train.sft --data qwen-drive-data/ --base Qwen-Drive-1.0-4B/ --out runs/sft --steps 500 ``` `--prefill cache` (the default) prefills the frozen model once per window to disk, about 140 MB per window, and trains in 9.5 GiB, so a 16 GB GPU fits. `--prefill online` needs 24 GB and no disk. For 5,000 windows on an RTX 5080, the cache takes about 50 minutes of prefill, then 10 minutes per epoch. ### On several GPUs ```sh torchrun --nproc-per-node 4 -m simforge_gym.train.sft --data qwen-drive-data/ --base Qwen-Drive-1.0-4B/ --out runs/sft --steps 500 ``` The same flags train with DDP on one machine. `--batch-size` counts over all GPUs. With the cache, each GPU peaks at 10.8 GiB (measured with 2 ranks), so 16 GB per GPU fits. Throughput on several GPUs has not been measured. ## Fine-tune Alpamayo 1.5 `python -m simforge_gym.train.alpamayo` runs NVIDIA's Alpamayo 1.5 SFT recipe on renders exported with `python -m simforge_gym.export --format pai`. Stage 2 is the default: the vision-language model stays frozen and the trajectory expert trains. It runs in NVIDIA's recipe environment, with simforge-gym installed into it: ```sh git clone https://github.com/NVlabs/alpamayo-recipes && git -C alpamayo-recipes checkout 6172113260522fbf290db85e21989b0cdb5302b6 cd alpamayo-recipes/recipes/alpamayo1_5_sft && uv venv a1_5_sft && source a1_5_sft/bin/activate && uv sync --active && cd - uv pip install simforge-gym gymnasium pyarrow huggingface-cli download nvidia/Alpamayo-1.5-10B --revision 7aba8293c09993f2e125c6819df05d7fa3e873ea --local-dir Alpamayo-1.5-10B (cd alpamayo-recipes && python scripts/convert_checkpoint.py to-a1 --input ../Alpamayo-1.5-10B --output ../Alpamayo-1.5-10B-A1) ``` ```sh python -m simforge_gym.train.alpamayo simforge.data=alpamayo-data/ simforge.val_data=alpamayo-val/ \ model.pretrained_model_name_or_path=Alpamayo-1.5-10B-A1/ paths.output_dir=runs/alpamayo-stage2 ``` On one RTX PRO 6000, stage 2 peaked at 36.5 GiB and took 2.3 s per step at batch 1 (30 steps on a synthetic export), so it needs a 40 GB or larger GPU; a 24 GB card does not hold it. Each checkpoint is about 30 GB on disk. A full training run has not been measured. The model builds its VLM from the configuration and tokenizer of `nvidia/Cosmos-Reason2-8B`, a gated Hugging Face repository under the NVIDIA Open Model License: request access to it and sign in (`huggingface-cli login`) before the first run. simforge-gym ships only Apache-2.0 code, never the weights: you download `nvidia/Alpamayo-1.5-10B` yourself at revision `7aba8293c09993f2e125c6819df05d7fa3e873ea`. The `LICENSE` file at that revision is OpenMDW-1.1, and the model card at that revision says: "This model is ready for non-commercial use. Commercial licensing available upon request." Review both, and the NVIDIA Open Model License of `nvidia/Cosmos-Reason2-8B`, before you train or use the model. ## Closed-loop RL with GRPO `python -m simforge_gym.train.grpo` continues from a Qwen Drive planner, such as an SFT checkpoint, with reinforcement learning in closed loop. Every rollout is a closed-loop episode scored the way the report card scores it, and the planner learns from the drives that scored better than their group: ```sh simforge rig init --preset qwen-drive-3cam --out rig.json python -m simforge_gym.train.grpo --base Qwen-Drive-1.0-4B/ --planner runs/sft/planner-000500 \ --train train.json --heldout heldout.json --rig rig.json --out runs/grpo --updates 20 \ --actor-closure "$(basename ~/.local/share/simforge/runtime/share/actors/closures/*.json .json)" ``` `--train` and `--heldout` are scenario sets; put the held-out set on another map. Each update drives `--scenarios-per-update` scenarios `--group` times each (default 4 x 8), `--parallel` drives at a time against the one loaded model, then trains the planner on what those drives scored. The held-out set is driven as an evaluation drives it at update 0 and every `--eval-every` updates; `runs/grpo/train.jsonl` records both, and `--resume` continues a stopped run. Measured on one RTX PRO 6000 (96 GB) from the starter recipe's checkpoint, 6 scenarios x 8 drives per update: - GPU memory: the trainer peaks at 32 to 35 GiB (the frozen model, a copy of the starting planner, float32 master weights and AdamW states). The whole GPU peaked at 39.9 GB with two drives rendering at once (a one-update run) and 48.8 GB with four (a 6-update run), so plan for a 48 GB or larger GPU; a 40 GB GPU is at its limit with `--parallel 2`. - Time: about 20 min per update of 48 drives of 10 to 13 s: 13 to 15 min driving with `--parallel 4` (each model decision takes about 0.8 s, and decisions of drives running at once take turns on the one model), then about 4.5 min training. `torchrun --standalone --nproc-per-node N` runs one trainer per GPU, each driving its share of the episodes; its memory per GPU and speed have not been measured. Whether GRPO raises the held-out driving score over the SFT checkpoint is still being measured. `python -m simforge_gym.train.grpo --help` lists every flag. ## Persistent worlds `simforge world serve` hosts one shared, realtime world that several clients join and drive, rather than isolated episodes. It authenticates clients with a token from a file you create first and keep private (for example `umask 077; head -c 32 /dev/urandom | xxd -p > world.token`), passed as `--token-file`; see `simforge world serve --help`. --- # Run in the cloud > Run the same simforge commands on SimForge's machines with --cloud, price them with --dry-run, cap them with --max-cost, and download the results. Source: https://simforge.ai/docs/cloud Add `--cloud` to a command and it runs on SimForge's hosted API instead of your runtime: the same command, the same scenario file, the same output. Use it for what your machine cannot do, such as photoreal renders without a large GPU, or many jobs at once. > **Beta.** SimForge Cloud is open in beta. Paid compute (3D renders, evaluations, transfers, Recast and asset generation) runs on SimForge's machines; 2D is a browser/local visualization, not a cloud job. [Sign up](https://simforge.ai/?auth=sign-up) and start with $25 in free credits. Work the cloud does not run yet is refused with `cloud_not_open`: run the command without `--cloud`. ## Prices | What you run | Price | |---|---| | 3D render | $1.00 per render | | Closed-loop driving evaluation | $3.00 per scenario | | Open-loop driving evaluation | $3.00 per scenario | | Transfer to another map | $0.50 per scenario | | Recast (log to scenario) | $1.00 per log | | 3D asset generation | $1.00 per asset | Each price is fixed before a run starts, and `--dry-run` shows it before anything runs. Prices are in cents on the command line (`--max-cost 100` is one dollar). Authoring, simulation, templates, OpenSCENARIO import and export, and validation are free. - A new account starts with $25 in free credits, and a workspace spends what its credits cover: there is no daily limit. A paid run the workspace's credits do not cover is refused before anything starts (`insufficient_credits`), with a link to buy credits on the workspace's credits page. Credits are checked first, so this is the answer even for work the cloud does not run yet. - 2D visualizations run in Studio's browser player or locally with `simforge render --mode 2d`; they do not use cloud credits or a cloud worker. - A run that fails costs nothing. A partial or cancelled run is charged only for the jobs that finished, and Recast only when the log became a scenario. ## Price, cap, run, download ```sh simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --dry-run simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 200 --wait --download simforge simulate s.json --out sim --cloud --max-cost 50 --wait --download ``` - `--dry-run` returns the estimate (`costEstimateCents`) and starts nothing. On a scenario file it first saves the file to your workspace as an immutable revision (free), and the run that follows reuses exactly that revision. With `--max-cost` it still prices the run and adds `maxCost: {limitCents, exceeded}`: `exceeded: true` means the same command without `--dry-run` would be refused. - `--max-cost` (in cents) is required with `--cloud`. A run whose estimate is above it is refused before anything starts (`budget_exceeded`); one the workspace's credits do not cover, with `insufficient_credits`. `--max-cost 0` runs only free work. - `--wait` follows the run to `succeeded`, `failed`, `partial` or `cancelled`; `--download` then writes every artifact into `--out`, sha256-checked. Giving `--out` also waits and downloads; without it the command returns as soon as the run is submitted. While it waits, stderr gets a `run.progress` line with the run id straight away, then the run's status and its jobs on each change and at least every 30 seconds; the result still comes on stdout at the end. - `--idempotency-key` makes resending a lost submission safe: the host answers with the original run and charges once. The cloud runs `render`, `simulate`, `preview frame` and `preview clip`, Recast (log ingestion), transfer (below), and downloads hosted datasets as files ([Getting data out](/docs/datasets)). Reinforcement learning stays local. Test a model by letting it drive your saved scenarios: **Run an evaluation** on Evaluations or Model Comparison. A closed-loop evaluation by Alpamayo 1.5 Nano, SimForge Pilot-4B v1.5 or Qwen Drive 1 4B costs **$3 per scenario**, including preparation. A model shows as temporarily unavailable while no machine is serving it. Pick a dataset or a few scenarios, review the total and the balance after it, then run. ```sh simforge evaluate --cloud --model "Alpamayo 1.5 Nano" --dataset "My scenarios" --wait ``` The CLI prints the quote and asks before starting (`--yes` skips the prompt; `--dry-run` only quotes). `--scenarios "Crossing,Turning"` picks a subset. It prints the run URL and `--wait` follows progress. Only scenarios and maps your workspace can already use are offered. An open-loop evaluation scores a driving model's predictions along one of your finished scenario renders and costs **$2 per scenario**, charged only when it succeeds. `--model` names the model that predicts (without it, the CLI lists the models you can run): ```sh simforge open-loop run RENDER_RUN_ID --model MODEL --cloud --max-cost 200 --idempotency-key my-eval-1 --dry-run ``` `simforge open-loop list --model MODEL` lists one model's evaluations. `--model 'SimForge Pilot-4B v1.5'` and `--model 'Qwen Drive 1 4B'` both see through their own three cameras, so render the scenario with that rig first (`simforge rig init --preset qwen-drive-3cam --out rig.json`, then render with `--rig rig.json`); a render with another rig is refused, naming the preset. That render records the rig's native 10 fps unless you pass `--fps`. A render at another rate is still scored: each input is the nearest rendered frame, and the dry run and the run both say so in a note with the largest time offset used. Scripted `closed-loop`, your own registered models, other endpoints, AlpaSim, comparisons and finished-run requeues stay locked with `403 evaluation_launch_disabled`. Use **Request evaluation** or **Use your own model** for an email request instead; requests start no work and charge nothing. ## Recast: bring your own logs ```sh simforge import log drive.recording.json --map auto --name "left turn" --cloud --max-cost 100 --out ./drive simforge import log drive.mcap --map auto --cloud --dry-run ``` Recast turns a recorded drive into an editable scenario: a recording (`simforge.recording/v1`) or a ROS 2 MCAP lidar log becomes a scenario on one of SimForge's maps. `--map auto` takes the recording's own map, or the map its GNSS path lies in. An MCAP log is perceived on a GPU worker first; its detector's weights are CC BY-NC-SA 4.0, so scenarios made from MCAP logs are for non-commercial use. ## Bring your own OpenSCENARIO file ```sh simforge import openscenario cut-in.xosc --cloud --stage simforge import openscenario cut-in.xosc --cloud --max-cost 100 --wait ``` The file is imported onto the SimForge map whose OpenDRIVE it names, and arrives as an editable scenario. `--stage` uploads the file and prices the import without starting it; a file whose road network is not a SimForge map is refused `osc_map_unknown` there, for free. ## The same event on another map ```sh simforge transfer s.json --map yale-street --cloud --max-cost 50 --out ./yale ``` A transfer that lands creates a new scenario in the workspace and costs $0.50; one no site on the map accepts creates nothing, is charged nothing and exits 2 with its verdict. Each target map is its own transfer, priced on its own. > Run transfer with `--cloud` for now. A local transfer needs the target map's `odr` derivative, which `simforge maps pull` does not install yet, so it stops with `map_facts_missing`. ## Follow a run ```sh simforge runs list --limit 20 simforge runs watch simforge runs cancel simforge results download --out ./results simforge usage ``` A partial or cancelled run keeps the artifacts it produced. ## Hosted scenarios A scenario stored in your workspace runs at an immutable revision, which pins the exact map version, actors and recordings: ```sh simforge scenarios list --limit 20 simforge render --revision --mode 3d --rig rig.json --out ./render --cloud --max-cost 100 --wait --download ``` To change a scenario and keep the original, copy it. The copy is a new scenario with the same map, actors, timeline and settings, named ` (copy)`; the original's renders and runs stay with the original. It lands in the original's dataset, or in your default dataset when you cannot change that one (`--dataset` picks another): ```sh simforge scenarios copy --name "Left turn, wet" ``` For a scenario shared from another workspace, the copy keeps its pinned recordings and provenance. If that original already has a copy in your target dataset, the existing copy opens instead of making another; the shared original stays read-only. ## Sign-in, scopes and API keys `simforge login` asks for scopes that cover downloads and cloud runs. A CI job can use an API key instead: set `SIMFORGE_API_KEY` (`sfk_…`) and `SIMFORGE_API_URL`. The key acts in the workspace it was created in, and your workspace role still applies: a viewer only reads, runs need a member. Managing keys needs the `keys:admin` scope, which a plain login does not ask for. `--scope` adds a scope to the default ones, so sign in again with it, then create the key: ```sh simforge login --scope keys:admin # the default scopes plus keys:admin simforge keys create --name ci --expires-in-days 90 simforge keys list simforge keys revoke ``` A new key's secret (`sfk_…`) is printed once. You can also create and revoke keys in the browser, on your workspace's [API keys page](https://simforge.ai/dashboard/workspace/settings?view=api-keys). If `simforge auth status` then lists only `keys:admin` under `scopes`, the CLI is an older one that replaced the defaults: reinstall it (`curl -fsSL https://dev.simforge.ai/install.sh | sh`), or name every scope (`simforge login --scope keys:admin --scope runs:write …`) as its `insufficient_scope` hint prints. An AI agent that calls the API without the CLI signs in through [auth.md](/auth.md): you approve it on simforge.ai with a code it shows you, for one organization and the scopes it asks for. ## Errors | Code (HTTP) | Meaning | Do | |---|---|---| | `cloud_not_open` (403) | SimForge Cloud is not open on this host yet (beta), does not run this work yet ("3D renders aren't available yet on SimForge Cloud."), or its workers for it are down for now (`detail.temporary: true`, "3D renders are temporarily unavailable on SimForge Cloud: no worker is running them right now."); `detail.code` is `run_kind_unavailable` and `detail.kinds` its job kinds | Try again in a few minutes when `detail.temporary` is true; otherwise run the command without `--cloud` | | `email_unverified` (403) | Runs that submit cloud work need a verified email; `detail.resendUrl` resends the link | Open the link in your email, or resend it from `detail.resendUrl` | | `terms_acceptance_required` (403) | The Terms of Service changed and the account has not accepted them; `detail.acceptUrl` is the page that does | Open it, accept, then retry | | `insufficient_credits` (402) | The workspace's credits do not cover the paid run; `detail.creditsUrl` is its credits page | Ask the person to buy credits, or use local/browser 2D visualization | | `max_cost_required` (the CLI's) | `--cloud` without a cost cap; nothing was sent | Pass `--max-cost` deliberately | | `budget_exceeded` (402) | Estimate above `--max-cost` (`detail.limit` is `max_cost`). A `--dry-run` is never refused for `--max-cost`: it answers the price with `maxCost.exceeded` | Ask the person whose budget it is; an agent never raises the cap itself | | `insufficient_scope` (403) | A scope (`detail.required`) or role (`detail.requiredRole`) is missing, or `--workspace` names another workspace than the login's (`detail.code` `workspace_mismatch`) | Run the login the hint names (`simforge login --scope keys:admin` for API keys), sign in for that workspace's organization, or ask an admin | | `not_found` (404) | The object does not exist or is not visible to you | Check the id and the workspace | | `conflict` (409) | A stale version or digest, or the host's `detail.code` reason | Read the object again, then decide | The hosted API is described by its [OpenAPI document](/docs/openapi.json). --- # Run Sim-Auto in the cloud > Make generated camera clips of your scenarios, or run your own Sim-Auto requests on SimForge's GPUs, with simforge factory; every generator option, its flag and its API field. Source: https://simforge.ai/docs/run-sim-auto-in-the-cloud SimForge Factory runs the Cosmos3-Nano-Sim-Auto video generator on SimForge's GPUs. It does two kinds of work: - **Scenario clips.** You name scenarios and a look (or your own prompt). SimForge renders each scenario, draws the layout videos the generator follows, generates the clip, checks it against the render and labels it. You get checked, labeled camera clips. Every generated clip is billed. Our automatic check labels each clip accepted, flagged or rejected, with its reasons, so you can see what the model got wrong; every clip is delivered with its label, a rejected one too, and nothing is made again unless you ask. The check looks for the scenario's key actors and the look, not for image quality: an accepted clip can still show artifacts such as a smeared road surface. - **Raw generations.** You bring the generator's inputs yourself: a control (layout) video per camera, a prompt per camera, and in simulator mode a first frame per camera. SimForge runs exactly the request a local Sim-Auto server would get, and hands back what it made. No scenario, no labels, no check. | | Scenario clips | Raw generations | |---|---|---| | You give | scenarios, looks or a prompt | control videos, prompts, first frames | | Length | always 20 s: 600 frames at 30 fps | what you ask for (frames, fps, aspect) | | Checked and labeled | yes | no | | Charged | every generated clip, whatever its label; a clip we fail to make is free | $1.00 per generation, whatever it shows | Every scenario clip is 20 seconds, 600 frames at 30 fps, at 832x480: the generator makes 601 frames and the last is dropped. A scenario shorter than 20 s is simulated on to 20 s, and a longer one gives its first 20 s; the quote says so. ## Make scenario clips ```sh simforge factory looks simforge factory clip create --scenario SCENARIO_ID --look fog-dawn --look snow --dry-run simforge factory clip create --scenario SCENARIO_ID --look fog-dawn --seed 7 --seed 8 --wait simforge factory clip create --dataset DATASET_ID --prompt "Heavy rain at night, wet asphalt reflecting street lights." --yes ``` `simforge factory looks` lists the looks; an experimental one may not apply well. Name scenarios with `--scenario` (repeatable) or every scenario of a dataset with `--dataset`, and looks with `--look` (repeatable) or your own conditions with `--prompt`. `--camera-prompt CAMERA=TEXT` replaces one camera's whole prompt, and `--camera` (repeatable) makes fewer cameras of the 4-camera rig, priced on those cameras. `--seed` (repeatable) picks the seeds, or `--seeds-per-look N` makes N clips per scenario and look. `--mode simulator` also gives the generator each camera's first rendered frame, so the clip keeps the render's lighting (its look is then always Rendered). The CLI prices the request first and shows the clips and the most it can cost, then asks before starting: type `y`. `--yes` starts without asking (needed when nothing can prompt, such as in a script), `--dry-run` only prices, and `--max-cost-cents` sets the cap yourself (by default the quote). A clip made with your prompt is labeled custom, and no check verifies its conditions. ## Run your own Sim-Auto request ```sh simforge factory params simforge factory clip create --raw \ --camera camera_front_wide_120fov --control camera_front_wide_120fov=front.mp4 \ --camera-prompt camera_front_wide_120fov="A wet city street at dusk, seen ahead from the windscreen." \ --frames 601 --fps 30 --seed 1234 --wait simforge factory clip create --raw --mode simulator \ --control camera_front_wide_120fov=front.mp4 --vision camera_front_wide_120fov=front.png \ --camera-prompt camera_front_wide_120fov="Golden hour, long shadows." ``` Give each camera its control video with `--control CAMERA=FILE.mp4` and its prompt with `--camera-prompt CAMERA=TEXT`; the cameras are generated in the order you name them (`--camera`, else the order of `--control`). In simulator mode give each camera its first frame with `--vision CAMERA=FILE.png` (or `.jpg`). `--prompt` is the request's overall prompt. Without `--frames` and `--aspect` the request names neither, exactly as a local call: the server makes its default 201 frames, at the aspect bucket closest to your first control video. The CLI uploads each file and checks it against what the GPU worker accepts (format, size, frame rate and frame count) before anything starts; a file that would be refused stops the request, with the reason, and nothing is charged. It then shows the price and asks: ``` 1 raw generation: $1.00 (charged per generation, whatever it shows; no check). Start it? Type y and press Enter to start; anything else cancels. ``` A raw generation costs $1.00, charged when it delivers, whatever the video shows: there is no check. One that never delivers (no GPU came up, the worker failed) is free. `simforge factory params` shows the price this deployment charges. ## Every generator option `simforge factory params` lists every option the generator has, what a scenario clip and a raw generation may set, what is fixed and why, and whether this deployment offers it. The raw generation sends the Sim-Auto option itself (the first column); a scenario clip request (`POST /api/v1/factory/runs`) names it by its scenario clip field. A fixed option may be given with its fixed value only, so a request you used locally carries over unchanged; any other value is refused with the reason below. | Sim-Auto option | CLI flag | Scenario clip field | Scenario clip | Raw generation | |---|---|---|---|---| | `seed` | `--seed` | `seeds`, `seedsPerLook` | settable; by default derived from the clip's id, so the same request makes the same clip | settable; by default derived from the request | | `extra_params.multiview.views[].vision_reference_index` | `--mode` | `mode` | `positions` (default) or `simulator` | `positions` (default) or `simulator`; simulator needs one first frame per camera | | none: SimForge's own | `--look` | `lookIds` | a look from `simforge factory looks`, or your own prompt | not used: a raw generation's prompts are yours, sent as given | | `prompt` | `--prompt` | `prompt` | your conditions in place of a look, at most 2000 characters | sent as given; by default "A photorealistic driving scene." | | `extra_params.multiview.views[].prompt` | `--camera-prompt` | `cameraPrompts` | replaces a camera's look sentence and description | one for every camera, at most 6000 characters | | `extra_params.multiview.views[].camera_key` | `--camera` | `cameras` | any of the 4-camera rig, in rig order (default all four) | any of the model's 11 cameras, in your order | | `input_references (extra_params.multiview.views[].control_reference_index)` | `--control` | none | fixed: drawn from the scenario's render | one MP4 per camera (H.264 or HEVC, the output size and fps, at least frames - 1 frames) | | `input_references (extra_params.multiview.views[].vision_reference_index)` | `--vision` | none | fixed: the render's first frame | one PNG or JPEG per camera, in simulator mode only | | `num_frames` | `--frames` | `numFrames` | fixed at 601 (600 kept): every scenario clip is 20 s | 5 to 2404, on the 4k+1 grid; at most 2404 over all cameras. By default the server's own (201), as a local call | | `fps` | `--fps` | `fps` | fixed at 30: every scenario clip is 20 s | 1 to 60 (default 30; 10 to 30 recommended) | | `extra_params.resolution` | `--resolution` | `resolution` | fixed at 480: the checkpoint is trained at 480 | fixed at 480: the checkpoint is trained at 480 | | `extra_params.aspect_ratio (and size)` | `--aspect` | `aspectRatio` | fixed at 16,9 (832x480): the clip's labels are drawn at that size | 16,9 (832x480), 9,16, 4,3, 3,4 or 1,1. By default the bucket closest to the first control video's shape, as the server picks it | | `num_inference_steps` | `--steps` | `numInferenceSteps` | fixed at 4: the checkpoint is distilled to 4 steps | fixed at 4: the checkpoint is distilled to 4 steps | | `guidance_scale` | `--guidance` | `guidanceScale` | fixed at 1: the distilled model runs guidance 1 only | fixed at 1: the distilled model runs guidance 1 only | | `extra_params.multiview.condition_video_as_image` | `--condition-video-as-image` | `conditionVideoAsImage` | fixed at true: the model conditions on each camera's first frame only | fixed at true, sent only with first frames | | `negative_prompt` | `--negative-prompt` | `negativePrompt` | fixed at none: the model takes no negative prompt | fixed at none: the model takes no negative prompt | | `model` | `--model` | `model` | fixed: one variant is deployed, nvidia/Cosmos3-Nano-Sim-Auto | fixed: one variant is deployed, nvidia/Cosmos3-Nano-Sim-Auto | ## Follow, retry, cancel ```sh simforge factory clip list simforge factory clip get RUN_ID simforge factory clip wait RUN_ID simforge factory clip retry RUN_ID --flagged simforge factory clip cancel RUN_ID ``` `clip get` shows each clip's status, verdict, whether its prompt came from a look or from you, what it is waiting for and its cost. `clip wait` follows a run until every clip is done, printing what the clips wait for whenever it changes ("Waiting for a GPU, 3 clips ahead"); it exits 0 when every clip is done and non-zero when any failed. `clip retry` makes a run's failed and cancelled clips again (or `--failed`, `--cancelled`, `--flagged` for new seeds), or a failed raw generation, after the same quote and confirmation as `clip create`. When you cancel, clips already checked stay charged and the rest are free. A clip SimForge fails to make or check (no GPU came up, the worker failed) is never charged; `--flagged` retries are new clips, charged like any other. ## Many clips from one file ```sh simforge factory batch clips.yaml --dry-run simforge factory batch clips.yaml --yes --wait ``` ```yaml - scenario: SCENARIO_ID look: fog-dawn seed: 7 - scenario: SCENARIO_ID prompt: Dense fog at dawn, low sun behind the haze. cameras: [camera_front_wide_120fov] mode: positions - raw: true fromRequest: run_0123/request.json ``` Each row is its own run: a scenario row takes `scenario`, `look` or `prompt`, and optionally `seed`, `cameraPrompts`, `cameras` and `mode`; a raw row takes `fromRequest` (a request.json, below) or `cameras`, `controls`, `visions`, `cameraPrompts`, `prompt`, `frames`, `fps`, `aspect`, `seed` and `mode`. Paths are relative to the file. Every row is priced first and the total is confirmed once. Each row is submitted under a key made from what it asks, so running the same file again submits nothing new: the CLI prints the runs it already made. JSON works too (a list of the same rows). ## Download ```sh simforge factory clip download RUN_ID --out ./clips simforge factory clip download RUN_ID --out ./clips --links-only ``` The CLI writes `./clips/RUN_ID/manifest.json`, every file with its signed link, sha256 and size, then fetches each file to its place and checks its sha256; running it again fetches only what is missing. `--links-only` writes just the manifest, for your own downloader. Nothing is zipped. A scenario clip's files are in `SCENARIO_ID/generated/LOOK-SEED/`: every file its `manifest.json` lists, among them `video/CAMERA.mp4`, `layout/CAMERA.mp4`, `inputs/CAMERA.mp4` and `inputs/CAMERA.png` (what the generator was given), `labels/boxes_2d.parquet`, `labels/actors_3d.parquet`, `labels/ego.parquet` and `quality.json`. A raw generation's are `request.json`, `response.mp4` (the server's own output), `video/CAMERA.mp4`, `timing.json` and `inputs/` (the files it was sent). Every output is a standard format: MP4 video, JSON, and Parquet labels. ## Your request, here or there A raw generation is exactly a local Sim-Auto call, in both directions. **From the cloud to your own server.** A delivered `request.json` holds the request as it was sent: the form fields under `request.fields`, and the files under `request.files` in order, each read from `inputs/` beside it. Start Sim-Auto with the model card's server: ```sh vllm serve nvidia/Cosmos3-Nano-Sim-Auto --omni --model-class-name Cosmos3NanoSimAutoPipeline \ --deploy-config /vllm_omni/deploy/cosmos3_nano_sim_auto.yaml --port 8000 ``` and send it the delivered request unchanged, from the folder `clip download` wrote: ```sh cd ./clips/RUN_ID eval "curl -sS --fail -o local.mp4 http://127.0.0.1:8000/v1/videos/sync $(jq -r ' [ (.request.fields | to_entries[] | "--form-string", "\(.key)=\(.value | if type == "string" then . else tojson end)"), (.request.files[] | "-F", "input_references=@\(.path);type=\(.mediaType)") ] | map(@sh) | join(" ")' request.json)" ``` Each field becomes a `--form-string` part (taken literally) and each file an `input_references` part, in the request's order. **From your server to the cloud.** A request you used locally runs on SimForge's GPU as it is: ```sh simforge factory clip create --raw --from-request request.json ``` It reads a delivered `request.json`, or your own `{"endpoint": "/v1/videos/sync", "fields": {...}, "files": [{"filename": "front.mp4", "path": "front.mp4"}]}` with `fields` exactly as you send them to the server (values as text, `extra_params` as its JSON text or the object) and `files` in `input_references` order, each `path` relative to the request.json's folder. The CLI uploads the files in order, checks them, and sends `fields` byte for byte. ## Over the API Every command is an `/api/v1` call ([the command reference](/docs/cli/factory) lists every flag): `GET /api/v1/factory/params` and `/api/v1/factory/looks`; `POST /api/v1/factory/estimate`, then `POST /api/v1/factory/runs` with `maxCostCents` and an `Idempotency-Key`; for a raw generation, `POST /api/v1/uploads` for each file, `POST /api/v1/factory/inputs/check`, then `POST /api/v1/factory/generations` with the request's `fields`, its `files` and `maxCostCents`. Retrying a create with the same `Idempotency-Key` returns the run it already made. --- # Getting data out > Pull a hosted dataset, a filter of it or chosen scenarios as files with the CLI, an agent, a link list or the API; resumable, checked and verifiable offline. Source: https://simforge.ai/docs/datasets A dataset is a collection of scenarios in your hosted workspace, or one shared with it. Getting it out means downloading the files SimForge already stores for each scenario (its document, OpenSCENARIO export, trace and simulation record, and, if you ask for them, renders, labels, ground truth and recordings) straight from storage, through signed links. Nothing is packed or zipped for you, so a pull starts at once and an interrupted one picks up where it stopped. The public SimForge scenario dataset, SIMLOG-25k, is on Hugging Face: `huggingface.co/datasets/SIMFORGE-AI/simlog-25k`. With the CLI it is `simlog-25k`: `simforge datasets pull simlog-25k --out ./simlog-25k` (signed in with `simforge login`). Route recordings contain trajectory data, not original camera or sensor video. Renders are generated by SimForge. Public SIMLOG files omit source dataset names, log IDs and collection provenance. ## With the CLI ```sh simforge datasets list simforge datasets pull --dry-run simforge datasets pull --out ./data simforge datasets verify ./data ``` `--dry-run` prints how many scenarios and files the pull has and their size, and downloads nothing. Narrow the pull with `--filter 'grade:A tag:cut-in'` (the library's filters), `--q `, `--source-log ` or `--ids a,b,c`. `--include` chooses what comes with each scenario: `renders[:training|showcase|schematic]`, `labels`, `gt`, `recordings`, or `scenarios` for nothing else (default `renders:training`). `--passes rgb,depth` keeps only some render modalities. `--jobs` sets the parallel downloads (default 8). Every file is checked against its sha256 before it is put in place. Run the same command again to resume: it fetches only what is missing or changed. A file SimForge is still preparing (an OpenSCENARIO export, ground truth) is waited for, up to `--wait-pending` seconds (default 1800); if some are still not ready, the pull lists them and you run it again later. When everything is in place the CLI writes `manifest.json`, `scenarios.jsonl` and `SHA256SUMS`; `simforge datasets verify DIR` checks the folder against them, offline. The folder: ``` manifest.json, scenarios.jsonl, SHA256SUMS recordings/.rec.json.zst scenarios//// document.json, scenario.xosc, trace.json.gz, resolution.json.gz labels/labels.json, gt/ego.parquet, gt/actors.parquet renders/-/... ``` Scenarios or files that cannot be had (a scenario never saved, an OpenSCENARIO export that failed) are listed with their reason in `manifest.json` under `excluded`, never dropped silently. ## With an agent Give your coding agent the setup and the task, for example: ```text Install the SimForge CLI (curl -fsSL https://dev.simforge.ai/install.sh | sh), sign in with `simforge login` and install the SimForge skills for yourself (`simforge skills install --claude` or `--codex`). Then pull dataset , only grade A scenarios with their training renders and labels: first `simforge datasets pull --filter 'grade:A' --include renders,labels --dry-run` to tell me the size, then the same command with `--out ./data`, then `simforge datasets verify ./data`. ``` Studio's Export panel shows this prompt, and the exact command, for the selection you are looking at, after checking that your CLI is installed and signed in. ## A link list, without the CLI ```sh curl -H "Authorization: Bearer $SIMFORGE_API_KEY" \ "https://simforge.ai/api/v1/datasets//files?format=urls&include=renders,labels" -o links.txt aria2c -i links.txt -d ./data -j 8 ``` The list holds a link per file, valid for 24 hours, with the file's place in the folder and its sha256, which aria2c checks. With wget instead (it does not check digests): `grep '^http' links.txt | wget -x -nH --cut-dirs=5 -i - -P ./data`. Files still being prepared are listed as comments at the end of the list; download the list again later for them. A link list has no `manifest.json`: use the CLI when you want the folder verified. ## From CI, with an API key Create a key with `datasets:read` and `artifacts:read` (`simforge keys create --name ci-pull --scope datasets:read --scope artifacts:read`, which needs a login with `keys:admin`, or the API keys page in workspace settings) and set it as `SIMFORGE_API_KEY`; the CLI and the API then need no login: ```sh SIMFORGE_API_KEY=sfk_... simforge datasets pull --include scenarios --out ./data ``` The API itself: `GET /api/v1/datasets//files` pages the selection's files (signed links valid for an hour; ask for the page again for fresh ones), `dryRun=true` counts them, and `POST` takes the same fields as a JSON body for long id lists. See the API reference. ## Copy SIMLOG-25k scenarios into your workspace Every workspace lists SIMLOG-25k read-only. Browse it, open a scenario and watch it in 2D; to edit or run scenarios, copy them into one of your datasets. ```sh simforge datasets copy simlog-25k --filter 'map:yale-street split:test' --new 'SIMLOG-25k copy' simforge datasets copy simlog-25k --ids --into ``` In Studio, **Copy to dataset…** copies the selected scenarios, current filter or whole dataset into an existing or new dataset. Large copies run in the background. A scenario already copied into that dataset is skipped, and each copy identifies its SIMLOG-25k source. --- # Model host contract > The versioned wire a SimForge closed-loop evaluation speaks to a driving model served over HTTP (simforge.model-host/v1), and a conformance command that checks your own server against it in one step. Source: https://simforge.ai/docs/model-host A closed-loop evaluation renders cameras live and asks a driving model, over HTTP, what to do next. The model sits behind four calls: `GET /healthz`, `POST /v1/sessions`, `POST /v1/sessions/{id}/invoke` and `DELETE /v1/sessions/{id}`. This page is that contract, version `simforge.model-host/v1`. The same wire serves SimForge's own hosted models, so a server that passes the check below speaks what they speak. Serving your own model on SimForge is not available yet. This is the first step: prove your server speaks the contract, on your own machine, with no account and no cost. ## Check your server in one command ```sh pip install simforge-gym export SIMFORGE_MODEL_HOST_TOKEN=... # any value if your server ignores the bearer python -m simforge_gym.closed_loop.conformance --url http://127.0.0.1:8765 ``` The command acts as a closed-loop episode against your server, with the client and the readers of your answers that the episode itself uses. It reads `/healthz`, opens a session, sends decisions with the frame window, ego history and route your declaration asks for, times every answer, probes how you refuse a malformed decision, and closes the session. The frames are synthetic gray JPEGs and the ego drives straight at 5 m/s, so it needs no scenario, rig or account. ```text PASS health.reachable: GET /healthz answered with a JSON object FAIL health.observation: capabilities.observation.frameHistoryOffsetsS must end at 0, not -0.1: the newest frame is the decision instant SKIP session.open: the declaration failed; the episode would refuse the run before opening a session FAIL: simforge.model-host/v1, 1 failed, 0 warnings, 3 passed ``` Each check ends `pass`, `warn`, `fail` or `skipped`. A `fail` is something a SimForge episode would refuse, or one of the v1 rules listed under [Conformance requirements and loop tolerances](#conformance-requirements-and-loop-tolerances). A `warn` is something the loop tolerates today and a hosted run should not depend on (a missing `reasoning`, an unknown field accepted). `skipped` means an earlier failure made it moot. The exit status is 0 without a failure, 1 with one (also on a warning under `--strict`) and 2 when the command was misused. | Flag | Meaning | |---|---| | `--url URL` | Your server's base URL, `http` or `https`. | | `--token-env NAME`, `--token-file FILE` | Where the bearer token comes from (default `SIMFORGE_MODEL_HOST_TOKEN`). It is never printed and never a command-line argument. | | `--deadline-ms N` | The longest an invoke may take. Default 600000, the limit the closed loop itself applies to one request. | | `--decisions N` | Decisions to send (default 3). | | `--replan-every N` | Steps of 0.1 s between decisions (default 5, the episode default). | | `--camera-ids 0,1,2` | Camera indices to open the session with, when your declaration does not fix them. | | `--frame-size WxH` | Size of the synthetic frames (default 1920x1080, the native rig). | | `--rig FILE` | A fitted rig. Checked equal to `capabilities.rig.definition` when you declare one. | | `--healthz-file FILE` | Check only a saved `/healthz` document; no server needed. | | `--json`, `--strict` | One `simforge.model-host-conformance/v1` document on stdout; fail on warnings. | `python -m simforge_gym.closed_loop.reference_host --port 8765` starts the smallest server that passes: a fake model that answers every decision with the same straight-ahead trajectory. Start from it when you wire your own model in. ## Versioning and the schema The contract is `simforge.model-host/v1`. A change that makes a valid v1 server invalid is a new version; new optional fields are additions to v1. The messages are published as JSON Schema (draft 2020-12), `model-host.v1.schema.json`, packaged with `simforge-gym` beside the conformance command. It has one definition per message: `Health`, `OpenSessionRequest`, `OpenSessionResponse`, `InvokeRequest`, `InvokeResponse` and `Refusal`. Camera JPEGs are `bytes` on the MessagePack wire; the schema describes them as base64 strings only to stay JSON. ## Transport | Rule | Detail | |---|---| | Authentication | Every request carries `Authorization: Bearer `. A host verifies it and answers a wrong or missing one with HTTP 401. | | Encodings | `/healthz`, session creation and the refusals are JSON. The invoke request is MessagePack (`application/msgpack`). The invoke answer must be `application/msgpack`. | | One host | A run contacts only the host it was configured with. A failure is never retried and never answered by another model. | | Unknown fields | A host refuses unknown fields in a session or invoke request, so the names on this page are exact. | ## `GET /healthz`: the declaration The loop sends the model exactly what the host declares, renders with the settings it declares and adopts its plans after the delays it declares. There is no guessing: v1 requires the declaration, and hosted evaluations require `capabilities.observation` before rendering. Local `simforge-gym` may use a registered family's observation contract and record `observation_contract_undeclared`; an undeclared unknown family is refused with `host_observation_undeclared`. | Field | Required | Meaning | |---|---|---| | `ok` | yes | `true`. | | `family` | yes | Nonempty name of the model family. | | `camera_profile` | yes | The rig preset SimForge renders for the model: `alpamayo-4cam` (cameras 0, 1, 2, 6), `alpamayo-6cam` (0, 1, 2, 3, 5, 6) or `qwen-drive-3cam` (0, 1, 2). | | `horizon_s` | no | Seconds a plan covers (default 6.4). Positive and finite. | | `samples.session` | no | Trajectories returned per decision (default 1). The loop drives the first. | | `samples.openLoop` | no | Native samples of open-loop evaluation, the `numTrajSamples` of the model registry (wire parameter `params.num_traj_samples`). | | `capabilities.observation` | yes | What the loop feeds the model; see below. | | `capabilities.cameras.required` | no | Camera indices; must equal the profile's. | | `capabilities.rig.definition` | no | The calibrated rig the model was trained on. When present, the rendered rig must equal it or the run is refused (`model_rig_mismatch`). | `capabilities.observation`: | Field | Default | Rule | |---|---|---| | `frameHistoryOffsetsS` | none, required | Nonempty list of seconds before the decision, oldest first: each 0 or less, on the 0.1 s grid, strictly increasing, last exactly 0. `[-0.3, -0.2, -0.1, 0]` is four frames per camera at 10 Hz; `[-1.5, -1.0, -0.5, 0]` four frames over 1.5 s. Every camera sends the same instants. | | `frameEncoding` | `jpeg` | Only `jpeg`. | | `requiresRoute` | `false` | `true` adds `route_xyz` to every decision. | | `egoHistoryS` | `1.5` | Seconds of ego history, a multiple of 0.1. The history holds `egoHistoryS / 0.1 + 1` poses (16 by default). | | `planAdoptDelayS` | `{first: 0, later: 0}` | Both keys, seconds on the 0.1 s grid. `first` must be 0: the first plan steers at once. `later` must be shorter than the replanning interval (0.5 s by default). | | `renderOverrides` | `{}` | Only `{"lens.distortion": 0}`: render without lens distortion. | ## `POST /v1/sessions` Request (JSON): `{"camera_ids": [0, 1, 2, 6], "seed": 0}`, the cameras of the declared profile and an integer seed. A host that does not serve those cameras refuses with `camera_set_invalid`. Answer (JSON): `{"ok": true, "sessionId": "...", "model": {...}, "gpu": {...}}`. `model` carries four nonempty strings, `family`, `revision`, `checkpointDigest` and `quant`: a run can pin them (`model_revision_mismatch` otherwise) and records them as the model's identity. `gpu` is any nonempty object naming the accelerator; SimForge treats it as opaque provenance. `framesPerCamera` and `frameHistoryOffsetsS` are optional and, when present, must agree with `/healthz`. ## `POST /v1/sessions/{id}/invoke`: one decision The episode asks for a decision every 0.5 s of simulated time, once enough real history exists: the declared frame history and ego history are never padded. Request (MessagePack): | Field | Meaning | |---|---| | `cameras` | `[{camera_id, jpeg, t_s}]`: JPEG `bytes`. Only frames the host has not been sent yet, oldest first. The host keeps each camera's window of the declared offsets. The first decision sends the whole window; each later one sends the frames new since. | | `ego_history_xyz`, `ego_history_rot`, `ego_history_t_s` | The last poses in the ego frame at `t_s` (x forward, y left, z up, origin at the footprint centre): `[x, y, z]` points, 3x3 rotations and times relative to `t_s`, 0.1 s apart, ending at 0 with the identity pose. | | `t_s` | The decision instant, equal to the newest frame's `t_s`. Increases between decisions. | | `seed`, `params` | The session seed and the run's parameters (`{}` by default). | | `route_xyz` | With `requiresRoute`: at least 2 `[x, y, z]` points of the remaining route in the ego frame at `t_s`. | Answer (MessagePack, `application/msgpack`): | Field | Required | Meaning | |---|---|---| | `ok` | yes | `true`. | | `trajectories` | yes | One or more plans, each a list of at least 2 finite `[x, y]` or `[x, y, z]` waypoints in the ego frame at `t_s`, `dt_s` apart, the first `dt_s` after `t_s`. Every plan covers the same number of waypoints. | | `dt_s` | no | Must be `0.1`; assumed when absent. | | `reasoning` | no | A list whose first entry is the model's text; without it the run records `reasoning_missing`. | | `timings`, `tokens`, `rng_provenance` | no | Kept in the run's record. | A plan shorter than `horizon_s` is allowed (the check warns): the vehicle follows the plan it is given. ## Timing - **Deadline.** A request that has not been answered within the deadline fails the run with `model_host_unreachable`. The closed loop's limit is 600 s per request; set `--deadline-ms` to the budget you want to hold yourself to. - **Simulated time stands still while the model thinks.** An episode runs in offline simulated time: the wall time of an invoke costs the run's duration, not the vehicle's progress. The only simulated delay is the one you declare in `planAdoptDelayS`: a plan decided at `t` steers from `t + later`, and the first plan from `t`. - **Order.** One decision at a time per session. `t_s` increases; a host may refuse a decision that does not (`input_error`). ## Errors A refusal is `{"ok": false, "error": {"code": "...", "message": "..."}}`, answered with HTTP 200 on a session call or with a 4xx status; `error.fields` may name the offending request fields. A host answers its own faults with a 5xx and the same body. The loop never retries. | What happened | Run fails with | |---|---| | The host cannot be reached, closes the connection or misses the deadline | `model_host_unreachable` | | HTTP 401 or 403 | `model_host_unauthorized` | | An undecodable body, or a body that is not an object | `model_host_protocol_error` | | `ok: false` or another HTTP error | the host's `error.code` (or `model_host_error`) | | An invalid declaration | `observation_contract_invalid` | | A declared rig that differs from the rendered one | `model_rig_mismatch` | | No trajectories, fewer than 2 waypoints, non-finite values or a `dt_s` other than 0.1 | `model_output_invalid` | Use stable, specific codes for your own refusals, such as `missing_fields`, `input_error`, `camera_set_invalid`, `frame_window_incomplete` and `session_not_found`: they appear in your run's failure reason. ## Conformance requirements and loop tolerances The conformance check reports each violation below as `fail`. Some requirements are stricter than the local gym or the loop's decoding tolerance; hosted evaluations already require an observation declaration. - v1 requires `capabilities.observation` and `camera_profile`. Hosted evaluations require the observation declaration; only local `simforge-gym` may use a registered family's observation contract with a warning. An undeclared unknown family is refused. - The invoke answer is `application/msgpack`. The loop also decodes a JSON body. - `trajectories` holds equally long plans, and as many as `samples.session` when declared. The loop reads only the first plan. - The opened session's `model` has nonempty `family`, `revision`, `checkpointDigest` and `quant`. The loop requires the object and compares only the keys a run pins. `capabilities.cameras.required` disagreeing with `camera_profile` is a `warn`: the loop never reads it. ## The checks | Check | What it verifies | |---|---| | `health.reachable`, `health.auth` | `/healthz` answers with the bearer; it is refused without one (warning if not). | | `health.shape`, `health.observation`, `health.horizon`, `health.cameras`, `health.samples`, `health.rig` | The declaration, field by field, with the offending path and value in the message. | | `session.open`, `session.unknown_fields` | A session opens with the declared cameras and identifies the model; an unknown field is refused. | | `invoke.accepted`, `invoke.content_type` | Valid decisions are answered `ok`, as `application/msgpack`. | | `invoke.trajectories`, `invoke.cadence`, `invoke.horizon`, `invoke.samples`, `invoke.reasoning` | The shape of the answer, element by element. | | `invoke.deadline` | Every answer arrives within `--deadline-ms`. | | `invoke.refusal`, `invoke.unknown_fields` | A decision without an ego history is refused with a code, never a 5xx or an `ok` answer; an unknown field is refused. | | `session.close` | `DELETE /v1/sessions/{id}` succeeds. | --- # JSON output, errors and exit codes > The contract every simforge command keeps - one JSON document on stdout, a structured error on stderr, and exit codes 0, 1 and 2 - so scripts and agents can rely on it. Source: https://simforge.ai/docs/output Every `simforge` command keeps the same contract, so a script or an agent can drive it without parsing prose. ## stdout: one JSON document A command prints exactly one JSON document on stdout. JSON is the default; `--json` asks for it explicitly, and `--pretty` indents the same document for people. Nothing else is ever written to stdout. ## stderr: a structured error When a command fails it prints one JSON line on stderr: ```json {"code": "max_cost_required", "reason": "…", "hint": "Pass --max-cost CENTS to explicitly cap this cloud job."} ``` - `code` is stable: key your handling on it. Never parse `reason`, which is for people. - `hint` says what to do next, often the exact command. - `path` names the input at fault, and `detail` carries structured context, when there are any. - `httpStatus` and `requestId` are there when the SimForge host refused a request: quote the `requestId` to support, which traces the request by it. `login` also writes event lines (`{"event": …}`, never a `code`) on stderr while it waits: the URL and code a person must approve. `runtime install` and `maps pull` write `{"event": "distribution.download.progress", "bytes": {"done": …, "total": …}, "percent": …}` lines as they download, at each tenth of the archive and at least every 30 seconds. A command that waits on a cloud run (`--cloud --wait`, `runs watch`) writes `{"event": "run.progress", "runId": …, "status": …, "jobs": [{"kind", "status", "progress", "queue"}], "elapsedSeconds": …, "message": …}` when it first reads the run, whenever the run or a job changes, and at least every 30 seconds. Saving a scenario file as a hosted revision (the first `--cloud` run or dry run of its content) writes `{"event": "revision.freeze", "draftId": …, "simulation": …, "elapsedSeconds": …, "message": …}` when it starts and each time the host answers that its authoritative simulation (`queued` or `running`) is still going. ## Exit codes | Exit | Meaning | What to do | |---|---|---| | 0 | OK | Read stdout | | 1 | The command could not run: bad flags, a missing file, the runtime or a map not installed, not signed in | Fix the invocation, or install what the error names | | 2 | The command ran and found something wrong with its input: a failed check, a refused operation, a digest mismatch | Report or repair the input; retrying the same input gives the same answer | ## The command surface as data ```sh simforge --help simforge render --help simforge help --json ``` `simforge --help` prints a compact index: every command with a one-line summary. `simforge --help` prints one command: its arguments and flags with types, allowed values, ranges, defaults, which ones are required or exclude each other, and the global flags it takes. `simforge help --json [command]` prints the full schema of every command, or of one subtree. An unknown flag is an error (`unknown_flag`), except after the scenario of `render`, `simulate` and `preview`, whose trailing arguments are runtime options: check those against the command's help. The same document is published at [/docs/cli.json](/docs/cli.json) and rendered as the [command reference](/docs/cli). A failure on stderr is one `simforge.error/v1` line: `{code, path?, reason, hint, detail?, httpStatus?, requestId?}`. `hint` says what to do next; for an invalid value it quotes the allowed values or range and names the command's `--help`. `detail` carries structured extras only. A request the hosted API refuses (an HTTP 4xx answer, such as `budget_exceeded` or `insufficient_scope`) exits 2 with the API's own code; a server failure exits 1. ## Common error codes | Code | Meaning | |---|---| | `unknown_command`, `unknown_flag` | Not a command or flag of this CLI; read `--help` | | `missing_argument`, `bad_value` | A required argument is missing, or a value is invalid | | `not_logged_in`, `session_invalid` | Sign in with `simforge login` | | `runtime_not_installed` | Install the runtime: `simforge runtime install --accept-license` | | `runtime_platform_unsupported` | This platform has no local runtime (Windows on Arm): add `--cloud` | | `map_not_installed`, `unknown_map` | Pull the map: `simforge maps pull @ --for 2d --accept-license` (`--for 3d` for a local 3D render) | | `map_3d_not_installed` | The map is installed for 2D only: `simforge maps pull @ --for 3d`, or render with `--cloud` | | `runtime_3d_not_installed` | The runtime is installed for 2D only: `simforge runtime install --for 3d --accept-license`, or render with `--cloud` | | `agreement_acceptance_required` | Read `simforge runtime license`, then pass `--accept-license` | | `entitlement_required` | This download is not available to your account | | `entitlement_revoked` | SimForge revoked this account's right to the download; it is not re-granted by accepting again | | `gpu_unsupported`, `gpu_memory_unsupported`, `gpu_driver_unsupported` | The GPU is outside the [supported range](/docs/install#what-the-machine-needs) (NVIDIA with at least 8 GiB and driver 570 or newer, or Apple M3 or newer); 2D renders, simulation and `env serve --no-sensors` still work | | `native_gpu_out_of_memory` | The scene did not fit in GPU memory: use `--textures bc7-512`, fewer or smaller cameras, or `--cloud` | | `signature_missing`, `signature_invalid`, `signature_untrusted`, `signature_wrong_host`, `signature_mismatch` | The download is not one SimForge signed for this host; nothing was installed | | `runtime_install_invalid` | The runtime found is not an installed SimForge runtime release (no `runtime-manifest.json` beside it): run `simforge runtime install --accept-license` | | `recording_missing` | A scenario that replays a recording cannot find `recordings/.rec.json.gz` beside its `document.json` | | `plan2d_map_missing` | The map's 2D derivative is not installed: pull the map again | | `max_cost_required`, `budget_exceeded` | A cloud run needs a cost cap, or its estimate is over it | | `insufficient_credits` | The workspace's credits do not cover a cloud run's [price](/docs/cloud#prices) | | `cloud_not_open` | SimForge Cloud is not open on this host yet, or does not run that work yet (`detail.code` `run_kind_unavailable`): run without `--cloud` | | `email_unverified` | Downloads and cloud runs need a verified email: open the link in your email, or resend it from the page the hint names | | `file_exists` | The output exists; pass `--force` to overwrite it | Codes for a specific area are on its page: [sign-in and runtime](/docs/sign-in), [cloud](/docs/cloud). --- # Usage telemetry > What the SimForge runtime reports about how it is used, what it never collects, and how to see the data before it is sent. Source: https://simforge.ai/docs/telemetry Usage telemetry is a condition of the SimForge account and download terms, which cover SimForge's services and downloads; the runtime's own license is PolyForm Noncommercial 1.0.0. The runtime reports how it is used, tied to the account you signed in with, so we can see which GPUs people use, how many jobs they run and where the local stack breaks. There is no off switch; commercial licences can agree otherwise. ## Collected - Your account id (from sign-in) and an id for this installation. - The CLI and runtime versions, the operating system and architecture. - The GPU model, its memory, the driver version and the GPU count. - For each job: its kind (simulate, render 2D, schematic or 3D, preview, RL episodes, evaluate), the render profile, the sensor count, frames, wall-clock seconds, GPU-busy seconds, and the outcome (ok, or the error code). - For reinforcement learning: episodes and environment steps. - The name of a public map, when one is used. ## Never collected - Your email, name, username or host name. The account id is enough; your email stays in the account system. - Your IP address: the server drops it. - File paths, scenario content or names, environment variables. - Anything about a private or customer map. ## See what is sent ```sh simforge telemetry show ``` prints the batches waiting to be sent, exactly as they will be sent. When you are signed out it shows the batches sealing would make, with no account id. The runtime records each finished job (and, for reinforcement learning, a session's episodes and steps) in `~/.simforge/usage/`. The runtime, not the open CLI, sends the records in the background, at most once a day after a job and also after `simforge login`, `maps pull` and `runtime install`. A job never waits on the network, and records that cannot be sent yet stay on the machine until the next attempt. `DO_NOT_TRACK` does not apply. --- # Send feedback > Report a bug, an idea or a question to SimForge from the CLI or Studio, see exactly what is sent before it goes, and what is never collected. Source: https://simforge.ai/docs/feedback When a SimForge command fails in a way its `hint` does not explain, or a page or skill is wrong, send us a ticket. It reaches the SimForge team directly; you do not need to be signed in. ```sh simforge feedback --attach-last-error --dry-run "maps pull failed right after login" simforge feedback --attach-last-error "maps pull failed right after login" ``` The first line sends nothing: it prints the exact document the second one posts (`wouldSend`). Without a message, a terminal asks for one, shows the same preview and sends only after you type `y`. | Flag | What it does | |---|---| | `--kind bug\|idea\|question` | What this is. Default: `bug` | | `--attach-last-error` | Attach the last command's failure on this machine. On by default when the previous command failed in the last 24 hours | | `--no-attach-last-error` | Attach no failure | | `--include-doctor` | Attach `simforge doctor`'s checks | | `--email EMAIL` | Signed out: where we may answer. Signed in, we answer your account | | `--dry-run` | Print what would be sent and send nothing | The result is one JSON document with the ticket's `id` and `status`, like every other command. In Studio, open **Help** in the top bar and choose **Send feedback**. ## For coding agents Every SimForge skill tells agents to report an unexpected failure or a wrong doc this way: run the `--dry-run` first, show you the preview, then send it. The [skills pack](/docs/skills) and the CLI's root `--help` say the same. ## What is sent Only what the preview shows: - **Your message**, after redaction. - **The machine:** CLI version and commit, runtime version (when installed), operating system, CPU architecture, GPU model, and which coding agent ran the command (`claude-code`, `codex`, `cursor`, `other-agent` or `none`). - **The last failure** (with `--attach-last-error`): the command and the names of the options it was given (never their values), the error code and reason, the exit code, the HTTP status and request id when a SimForge host answered, the map slug it was about, and when it failed. Every command records this on your machine only (`last-run.json` in the SimForge config directory), except `feedback` itself and a `--dry-run`, which writes nothing. - **Doctor's checks** (with `--include-doctor`): each check's id, status, summary and fix. Never a check's detail. - **Who you are:** your account and workspace when you are signed in or use an API key; otherwise the email you give, if any. ## What is never sent File contents, environment variables, tokens and API keys, and paths outside the directory you run the command in. Before the preview is built, the CLI replaces anything that looks like a credential (`sfk_…` API keys, access tokens, `Bearer …` values, JWTs, cloud provider keys, the value of any `token=`, `password=` or `apiKey:` field, long random strings) with `[redacted]`, and every path outside the working directory with `[path]`; a path inside it is shown relative (`./scenario.json`). The SimForge host runs the same credential check again before it stores a ticket. ## Limits A machine sends at most 5 tickets an hour (`feedback_rate_limited`); the host also limits each account and, signed out, each network address (`rate_limited`). For a conversation instead of a ticket, ask on the [SimForge Discord](https://discord.gg/Whmn9brNRd). --- # For AI agents > Where an AI agent finds the SimForge docs, the machine-readable command surface and API, and the skills pack, and how to drive the CLI safely. Source: https://simforge.ai/docs/agents SimForge is built to be driven by agents. The CLI an agent uses is the one people use: every command prints JSON and fails with a stable error code. ## Where things are | What | Where | |---|---| | Docs index for agents | [/llms.txt](/llms.txt) | | Every page and the command reference in one file | [/llms-full.txt](/llms-full.txt) | | Any page as Markdown | add `.md`: [/docs/install.md](/docs/install.md) | | The command surface as JSON (`simforge.cli-surface/v1`) | [/docs/cli.json](/docs/cli.json), or `simforge help --json` | | The hosted API (`--cloud`) as OpenAPI | [/docs/openapi.json](/docs/openapi.json) | | Agent skills for Claude Code, Codex and AGENTS.md agents | [Skills pack](/docs/skills), or `simforge skills install` | | How an agent signs in to the hosted API without the CLI (auth.md) | [/auth.md](/auth.md) | ## Driving the CLI 1. Read the surface first: `simforge --help` lists the commands, `simforge --help` gives one command's flags. Never guess a flag. 2. Parse stdout as one JSON document. On failure, key on `code` in the stderr JSON and follow its `hint`. 3. Exit 1 means fix the invocation or install what is missing. Exit 2 means the input itself is wrong: report or repair it, do not retry it unchanged. 4. Sign-in needs a person: `simforge login --device` prints a URL and code. Relay them to the user and wait. Never print or store tokens. 5. Cloud runs cost money. Always price with `--dry-run` first and pass a `--max-cost` no higher than the budget you were given. 6. Long-running servers (`simforge env serve`, `simforge world serve`, `simforge mcp`) run until stopped: start them in their own terminal, not inside a call that waits for them to finish. ## Signing in without the CLI An agent that calls the hosted API (`/api/v1`) directly signs in through [auth.md](/auth.md), the recipe for agent sign-in that every `/api/v1` 401 points to (`WWW-Authenticate: Bearer resource_metadata=…`). The agent registers with the email of the person's SimForge account and shows the person a link and a code. The person signs in on simforge.ai, enters the code, chooses the organization and approves. The agent then gets a one-hour access token and an identity assertion that renews it. Only that account can approve, and only once its email is verified. The person sees the agent under Account → CLI sessions and can revoke it there. A backend with no person at hand uses an API key instead. ## MCP ```sh simforge mcp ``` `simforge mcp` serves the CLI over the Model Context Protocol (stdio). By default it lists a small core set of tools (under 40 KB of schemas): the authoring, render and run commands, plus a `simforge` tool that runs any other command from its argv. `--profile full` lists every command as its own tool; `--only render --only scenario` lists the commands under those paths; `--cwd` sets the working folder. Inputs are typed with each flag's allowed values, range and default; annotations mark read-only, destructive and idempotent tools. Every call runs the same binary and returns the command's JSON result, exit code and error once, as structured content, with a one-line text summary. Local runs are the default; a tool call with `cloud: true` and a `max-cost` runs hosted. ## Skills The [skills pack](/docs/skills) gives an agent task-level instructions: the quickstart, authoring, rendering, the cloud, SimForge Gym training, maps, scenario packages and troubleshooting. It ships inside the CLI: ```sh simforge skills install --claude ``` --- # Agent skills pack > Install the SimForge agent skills into Claude Code, Codex or any agent that reads AGENTS.md, from the CLI or as a zip download. Source: https://simforge.ai/docs/skills The skills teach a coding agent to use SimForge: which command comes next, what each error means, and what never to do (such as spending past a cost cap). Each skill is a folder with a `SKILL.md`, in the Agent Skills format. ## Install from the CLI The skills ship inside the `simforge` binary, so they always match its commands: ```sh simforge skills list simforge skills install --claude # ~/.claude/skills simforge skills install --codex # $CODEX_HOME/skills, else ~/.codex/skills simforge skills install --dir ./skills # any folder ``` `skills install` records what it wrote. Installing again updates those files; a file you changed by hand is refused (exit 2, `skills_modified`) unless you pass `--force`. ## Download the pack [simforge-skills.zip](/docs/simforge-skills.zip) holds the same skills, an `AGENTS.md` index and a README: ```sh curl -fsSLO https://simforge.ai/docs/simforge-skills.zip unzip simforge-skills.zip ``` | Agent | Install | |---|---| | Claude Code | `cp -R simforge-skills/skills/* ~/.claude/skills/`, or a project's `.claude/skills/` | | Codex | `cp -R simforge-skills/skills/* "${CODEX_HOME:-$HOME/.codex}/skills/"` | | Agents that read AGENTS.md | Copy `simforge-skills/skills/` into the repository and add `simforge-skills/AGENTS.md` to its AGENTS.md | | Any other agent | Give it [/llms.txt](/llms.txt), or the `SKILL.md` files as instructions | ## The skills | Skill | Use it for | |---|---| | `simforge-quickstart` | A first task: install, sign in, download a map, author, render, RL | | `simforge-cli` | The JSON contract, sign-in, cost caps and the MCP server | | `simforge-authoring` | Building and editing scenarios with typed operations | | `simforge-render` | Previews and renders in 2D, schematic and 3D | | `simforge-cloud` | Hosted runs with `--cloud`, caps, runs and results | | `simforge-gym` | Local training with SimForge Gym (`simforge-gym`) | | `simforge-maps-assets` | Map and actor downloads, caches and licences | | `simforge-scenario-package` | Verifying and importing `.scenario.zip` packages | | `simforge-troubleshoot` | Reading `simforge doctor` and error codes, and reporting a problem with `simforge feedback` | --- # simforge command reference CLI version 0.2.0-rc.9. Generated from `simforge --help --json`; the same document is at /docs/cli.json. Output: stdout: one JSON document (indented with --pretty); stderr: one simforge.error/v1 line {code, path?, reason, hint, detail?, httpStatus?, requestId?} on failure. ## Global flags | Flag | Notes | What | |---|---|---| | `--pretty` | | Indent the JSON result for humans (the same document, never a different one) | | `--json` | | Explicit machine-readable JSON output (also the default) | | `--cloud` | | Execute locally by default; submit the same input as an authenticated cloud job | | `--workspace cloud_workspace` | env SIMFORGE_WORKSPACE_ID | With --cloud: the workspace you expect the login to act in; a different one is refused (workspace_mismatch) | | `--idempotency-key IDEMPOTENCY_KEY` | | With --cloud: a key that makes resending a lost submission safe (the host answers with the original run and charges once) | | `--dry-run` | | With --cloud: price the job (costEstimateCents) and start nothing | | `--api-url API_URL` | | Hosted API base URL for API-key use. Default: SIMFORGE_API_URL, then the signed-in host's /api/v1 | | `--map-version MAP_VERSION` | | Exact hosted map version for a scenario file. Default: the installed release's, else the one hosted version carrying the file's OpenDRIVE digest | | `--help` | | Print machine-readable command help. | ## Exit codes | Exit | Meaning | |---|---| | 0 | ok | | 1 | command error: the command could not run | | 2 | findings: the command ran and found something wrong with its input | ## assets ### simforge assets pull Pull the actor-asset closure and its ATTRIBUTION.json, verifying every blob's sha256 Usage: `simforge assets pull [OPTIONS]` Runs on: local runtime. | Argument / flag | Notes | What | |---|---|---| | `--only ONLY` | one of actors, sky | Pull only this closure. Default: both the actor closure and the sky plates | | `--closure SHA256` | | The actor closure digest (sha256). Default: the closure this build is pinned to | | `--base-url URL` | | Asset store base URL (https:// or file://). Default: SIMFORGE_ACTOR_ASSETS_BASE_URL, then the public store | | `--root DIR` | | Local asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets | | `--timeout SECONDS` | default 60 | Seconds to wait for a connection or for a response to start | ## auth ### simforge auth status The account, organization, scopes, expiry, host and credential store of this machine's login, checked with the host (tokenValid), and the account's email verification; with SIMFORGE_API_KEY set, the key's account, workspace and scopes Usage: `simforge auth status [OPTIONS]` | Argument / flag | Notes | What | |---|---|---| | `--host HOST` | | SimForge host. Default: SIMFORGE_HOST, then the host of the last login, then simforge.ai | ## authoring ### simforge authoring capabilities What this credential may do on this host: operation, host-operation, feature and worker status Usage: `simforge authoring capabilities [OPTIONS]` ### simforge authoring catalogue The host's operation catalogue (simforge.authoring-operations/v1) Usage: `simforge authoring catalogue [OPTIONS]` ### simforge authoring discover One page of map discovery without the map package: locations, lanes, movements or the actor catalogue Usage: `simforge authoring discover [OPTIONS] --map --kind ` | Argument / flag | Notes | What | |---|---|---| | `--map MAP_VERSION_ID` | required | The exact map version to look up (`maps list` names them) | | `--kind KIND` | required; one of locations, lanes, movements, catalogue | lanes, movements (junction connectors), locations (crosswalks, junctions, ...) or catalogue (built-in actors) | | `--limit N` | default 20 | Records per page (1 to 100) | | `--near X,Z` | | Nearest first from this scene point, metres (`x` east, `z` minus north); not with --kind catalogue | | `--within-m M` | | Only records within this many metres of --near (> 0) | | `--road ROAD_ID` | | Only records on this OpenDRIVE road: its lanes, movements entering, leaving or on it, locations anchored on it; not with --kind catalogue | | `--type TYPE` | one of junction, junction_movement, driving_corridor, bike_corridor, walking_corridor, midblock_segment, merge_zone, lane_drop, parking_lane, parking_space, parking_area, parking_access_point, driveway, loading_zone, bus_stop, crosswalk, sidewalk, curb, median, refuge_island, building_entrance, school_zone, work_zone_suitable, occlusion_zone, conflict_zone, poi_frontage, address | Only locations of this type (with --kind locations) | | `--cursor CURSOR` | | The previous page's `nextCursor`, with the same --map, --kind and filters | ### simforge authoring operations Every operation with its `scenario apply` flags, result schema and the host's status for this credential Usage: `simforge authoring operations [OPTIONS] [OP_ID]` | Argument / flag | Notes | What | |---|---|---| | `` | | One operation id (camelCase or its kebab-case command); absent, every operation | ### simforge authoring spawn Whether an actor can be placed at a lane position or scene point, optionally against a draft's actors Usage: `simforge authoring spawn [OPTIONS] --map --catalog-id ` | Argument / flag | Notes | What | |---|---|---| | `--map MAP_VERSION_ID` | required | The exact map version to place on | | `--catalog-id ID` | required | The actor's catalog id (`authoring discover --kind catalogue`) | | `--lane-rsl RSL` | | A lane handle from `authoring discover --kind lanes` (with --s) | | `--s S` | | Metres along the lane from its start (with --lane-rsl) | | `--x X` | | A scene point (with --z) | | `--z Z` | | The scene point's z in metres (with --x) | | `--heading-rad HEADING_RAD` | | The heading in radians in the scene frame. Default: the lane's direction | | `--lateral-m LATERAL_M` | | Sideways offset from the lane centre in metres (positive left) | | `--opposing` | | Face against the lane's direction | | `--static` | | Place a parked (static) actor | | `--draft DRAFT_ID` | | Check overlap against this draft's actors (needs --expected-version and --expected-digest) | | `--expected-version N` | | The draftVersion of the receipt the check is based on (with --draft) | | `--expected-digest SHA256` | | The contentDigest of the receipt the check is based on (with --draft) | ## bench ### simforge bench Benchmark the renderer on this machine: the Helios Score, per-factor sub-scores, frame correctness Usage: `simforge bench [OPTIONS] simforge bench ` Runs on: local runtime. | Argument / flag | Notes | What | |---|---|---| | `--suite SUITE` | one of quick, full; default quick | Which scenes to run | | `--scenes ID` | repeatable | Only these scene ids (comma-separated), from the suite's pack | | `--set KEY=VALUE` | repeatable | Diagnostics: a render-config override (`simforge render --set` keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status `modified`) | | `--gpu N` | | The GPU to benchmark: its index in `nvidia-smi` (NVIDIA) or in the adapter list `simforge doctor` prints. Default: the renderer's choice (the first discrete GPU) | | `--out FILE` | | Write the result JSON here too (default: `simforge-bench-