// Guides
Preview and render
Render a scenario as a 2D top-down video, a schematic view or photoreal 3D sensor data locally; hosted rendering is schematic/3D only.
One renderer, the Helios Engine, in three modes:
| Mode | What you get | Runs on |
|---|---|---|
2d | Top-down map video of the scenario | CPU |
schematic | A clean schematic view through the rig's cameras | GPU |
3d | Photoreal camera images and sensor passes | GPU |
Pick a sensor rig#
Schematic and 3D renders look through a rig: the cameras on the vehicle. Without --rig they use the basic-dash-camera rig on the ego, a front camera at the preset's capture (1280x720 at 24 fps for the default showcase); write your own to choose others.
simforge rig init --preset basic-dash-camera --out rig.json
simforge rig show rig.jsonrig init writes the cameras at the --profile capture: showcase (1280x720 at 24 fps) unless you pass --profile training (512x384 at 20 fps). The basic dash camera's camera is dashcam-front. Other presets include alpamayo-2cam, alpamayo-6cam, tesla-hw3 and waymo-5th-gen; simforge rig init --help lists them all, and the vehicle classes.
Preview a frame#
simforge preview frame s.json --mode 2d --at 5 --out plan.png
simforge preview frame s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --out frame.png
simforge preview clip s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --seconds 3 --out clip.mp4Render#
simforge render s.json --mode 2d --out out-2d
simforge render s.json --mode schematic --rig rig.json --out out-schematic
simforge render s.json --mode 3d --rig rig.json --out out-3d--out is a folder that must not exist or be empty. 3D renders take --preset showcase or --preset training (the dataset look at a fixed cost per frame); without --preset, a 3D render uses showcase, locally and with --cloud. They also take --passes to choose among rgb, id, depth and semantic. The 2D mode is local or browser-only and does not accept --cloud.
GPU memory and texture tiers#
GPU memory depends on the source texture tier, the material texture limit and the camera rig. The local CLI still selects a full-chain source (uastc-full) on GPUs with at least 16 GiB, or a preprocessed 512 px source (bc7-512) below that, and records the choice in localDefaults. --textures overrides the source tier. On Apple silicon the budget is three quarters of system memory.
Material textures now have a default maximum dimension of 1024 px, including map, vehicle, pedestrian and prop textures. This is an intentional fine-detail softening to reduce GPU memory; it does not change camera resolution, geometry or sensor cadence. The limit is an upper bound, not an upscale: a 512 px source stays 512 px, and normal mip planning may choose smaller textures.
Base-color textures whose alpha controls cutouts or transparency keep their normal mip selection instead of the size cap, preserving fence openings and foliage coverage. The exception uses actual source alpha; an opaque texture is not exempt just because its material is labelled MASK or BLEND.
Environment plates keep their authored resolution: reducing the star map to 1024 px removed fine night-sky detail. The material cap does not resize sky/moon plates, render targets or sensor buffers.
Set render.set.textureMaxSize to 1024, 2048, 4096 or "full" in an authored scenario, or pass --set textureMaxSize=... to the CLI. "full" removes this limit while preserving normal mip/residency planning. To recover full source detail on a machine that otherwise selects the 512 px tier, set both:
simforge render s.json --mode 3d --textures uastc-full --set textureMaxSize=full --out out-fullThe historical tier measurements below have this material limit disabled (textureMaxSize=full); they are not measurements of the new 1024 px default.
| GPU memory | What fits (measured peak) |
|---|---|
| 8 GiB | 512 px textures on every public map, up to six 1920x1080 cameras (4.5 GiB) |
| 12 GiB | 512 px textures everywhere; full textures on every public map except san-ramon-phase-1, one 1280x720 camera (8.1 GiB) |
| 16 GiB | Full textures on every public map, up to six 1920x1080 cameras (11.9 GiB) |
Peak GPU memory of one 3D render (showcase preset, one 1280x720 camera unless stated):
| Map | Full textures | 512 px textures |
|---|---|---|
| san-ramon-phase-1 | 11.3 GiB | 3.8 GiB |
| san-ramon-phase-1, six 1920x1080 cameras | 11.9 GiB | 4.5 GiB |
| di-rosa-sf | 8.1 GiB | 2.8 GiB |
| richmond-field-station | 7.3 GiB | 2.1 GiB |
| belmont-research-center | 7.2 GiB | 2.3 GiB |
| yale-street | 6.5 GiB | 2.3 GiB |
| el-camino-road | 3.5 GiB | 1.9 GiB |
| easterbrook-discovery-school | 2.0 GiB | 1.8 GiB |
The training preset needs about the same; a schematic render needs 0.2 GiB. Measured on an RTX 5080 (driver 595.84). The machine-readable table is sdk/contracts/runtime-support.v1.json in the toolkit.
Output format#
A 3D render writes:
| Path | Format |
|---|---|
render.json | The render's summary, also printed by the command: preset, rig, texture tier, map and actor-pack identities, gates, timings, videos, warnings |
results.json | Every captured file with its sensor, tick, pass, byte count and sha256 |
<sensor>/<tick:08>.rgb.png | The camera image, RGBA8 PNG, sRGB, with --video off or without ffmpeg (a video render encodes the frames into video/<sensor>.mp4 as it draws them) |
<sensor>/<tick:08>.id.png | Instance ids, RGBA8 PNG: id = R + 256·G + 65536·B, 0 = sky or background |
legend.json | Instance id to the static map mesh it is |
<sensor>/<tick:08>.semantic.png | Semantic class in the B channel of an RGBA8 PNG, CARLA CityScapes ids: 0 unlabeled, 1 building, 2 fence, 4 pedestrian, 5 pole, 7 road, 9 vegetation, 10 vehicle, 12 traffic sign, 18 traffic light |
<sensor>/<tick:08>.depth.f32.bin | Raw little-endian float32, width x height, row-major, reverse-Z: distance along the camera axis in metres = nearM / value (nearM is the camera's near plane, 0.05 m by default); 0 = nothing hit |
observed-frames.jsonl | One line per frame: every actor's position, rotation and visibility as rendered |
parity.json | How closely the rendered poses match the simulated trace |
video/<sensor>.mp4 | The RGB frames as H.264 video (needs ffmpeg; --video off skips it) |
job/ | The exact inputs the renderer ran with (timeline, scene state, texture plan) |
Ticks count from the clip start at the rig's frame rate. Lidar sources write <sensor>/<tick:08>.ply and radar sources .csv. A 2D render writes render.json and video/topdown.mp4. Traces are identical on every machine; pixels are identical run to run on the same GPU model and driver, and differ slightly between GPUs.
Without ffmpeg, a 2D render that requests video preserves its images under frames/topdown/<index:08>.png and reports that directory in frame.directory; it warns that no video was encoded. --video off remains a render-and-measure operation without retained frames.
Before you render#
Check the GPU with simforge doctor and the runtime's render modes with simforge runtime status. A render that cannot run on this machine fails with a named error; it never falls back to another device or to the cloud. A scene that does not fit in GPU memory fails with native_gpu_out_of_memory: use --textures bc7-512, fewer or smaller cameras, or --cloud for schematic/3D. A 3D render whose GPU memory cannot be measured warns gpu_memory_unmeasured; --vram-budget sets the budget explicitly.
Render in the cloud#
Hosted rendering runs on SimForge's machines with --cloud and a cost cap in cents; it is limited to schematic/3D. 2D stays in Studio's browser player or the local CLI:
simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 200 --wait --downloadCloud video downloads include only the encoded videos and final metadata (render.json, results.json, manifest.json, observed-frames.jsonl, parity.json when present, and runtime-info.json). A cloud render draws only the RGB camera pass; its captured per-frame RGB, lidar and radar files are intermediate outputs and are not uploaded. A requested still preview retains its final RGB images instead of videos. The cloud results.json inventory lists only published files, so every listed capture is downloadable. The local renderer's diagnostic job/ directory is not exported in full; its authoritative job/timeline.json is included.
See Run in the cloud for pricing, caps and following a run.