Skip to content

Strands Robots Sim

GPU-accelerated NVIDIA Isaac Sim backend for strands-robots.

strands-robots-sim is the GPU-accelerated Isaac Sim companion to strands-robots. It ships an IsaacSimulation that plugs into the same SimEngine ABC the upstream MuJoCo backend implements, so a Strands Agent that drives a MuJoCo world today can switch to Isaac Sim by swapping the backend it constructs:

from strands_robots.simulation import create_simulation

sim = create_simulation("mujoco")
sim.create_world()
sim.add_robot("so100")
sim.step(100)

from strands_robots.simulation import create_simulation

sim = create_simulation("isaac", render_mode="rtx_pathtracing", headless=True)
sim.create_world()
sim.add_robot("so100")                         # procedural; no asset files needed
sim.step(100)
frame = sim.render(camera_name="default")

The agent code, the policy interface, and the Simulation AgentTool surface are all identical across backends — once the IsaacSimulation instance exists, every downstream call is SimEngine-shaped regardless of how it was constructed.

create_simulation("isaac", ...) resolves to this repo's IsaacSimulation via the strands_robots.backends entry point (see How it works), exactly like create_simulation("mujoco") resolves to the built-in MuJoCo backend. The kwargs flow straight into IsaacConfig. You can still construct IsaacSimulation(IsaacConfig(...)) directly if you want the config object in hand, but the factory is the recommended path.

When you want this repo

Pick this if you need... ...because
RTX path-traced rendering Photoreal observations / paper-grade frames / sim2real visuals
USD-native scenes Real CAD / Nucleus assets, IsaacLab compatibility
Replicator synthetic data Domain randomization at scale, ground-truth depth / segmentation
Fleet RL with PhysX GPU IsaacLab-style training loops, 1024+ parallel envs
Real-asset robots Bring your own URDF / MJCF / USD; load Franka, Panda, custom CAD

If none of those fit, install the lightweight default at strands-labs/robots instead — it runs everywhere (including Apple Silicon), boots in seconds, and the agent contract is the same.

How it works

strands-robots-sim registers IsaacSimulation as a strands_robots.backends entry point. create_simulation("isaac") resolves to it without strands-robots ever needing a hard dependency on Isaac Sim:

graph LR
    A[Strands Agent] --> B[Simulation<br/>AgentTool]
    B --> C[create_simulation 'isaac'<br/>walks entry points]
    C --> D[Entry-point lookup<br/>strands_robots.backends]
    D --> E[IsaacSimulation<br/>this repo]
    E --> F[Isaac Sim Kit<br/>SimulationApp]
    F --> G[PhysX + RTX]

    classDef agent fill:#0969da,stroke:#044289,color:#fff
    classDef glue fill:#8250df,stroke:#5a32a3,color:#fff
    classDef plugin fill:#bf8700,stroke:#875e00,color:#fff

    class A,B agent
    class C,D glue
    class E,F,G plugin

Entry-point discovery

The entry point above is declared and discoverable (importlib.metadata.entry_points(group="strands_robots.backends") lists isaac). strands-robots>=0.4.1 walks that group from its create_simulation factory (shipped via strands-labs/robots#131), so create_simulation("isaac") resolves to this repo's IsaacSimulation with nothing more than pip install strands-robots-sim.

The same plugin shape is what makes the mujoco backend in strands-robots and isaac here interchangeable: both are SimEngine subclasses; the user-facing API is the Simulation AgentTool.

See Architecture for the full plugin contract.

Install

System requirements: NVIDIA RTX GPU, Ubuntu 22.04+, CUDA 12+, Isaac Sim 6.0 (Python 3.12). macOS / Apple Silicon contributors should install strands-robots directly and skip this repo.

pip install 'strands-robots-sim[isaac]'

Isaac Sim itself is not on PyPI — it is an Omniverse Kit application that must be installed separately via the Omniverse Launcher, Isaac Lab, or the NGC Docker image. The [isaac] extra above installs only the pip-installable helper (usd-core) plus strands-robots; it does not pull in Isaac Sim. Full instructions, including a "verify your install boots" snippet, in Getting Started → Installation.

Quickstart

from strands_robots_sim.isaac import IsaacSimulation, IsaacConfig

sim = IsaacSimulation(IsaacConfig(render_mode="rtx_realtime", headless=True))
sim.create_world()
sim.add_robot("so100")                         # procedural builder, no asset files
sim.add_object(name="cube", shape="cuboid", position=[0.4, 0.0, 0.05])
sim.add_camera(name="front", position=[1.2, 0.0, 0.6], target=[0.0, 0.0, 0.1])
sim.step(120)
frame = sim.render(camera_name="front")        # RTX RGBA + depth dict
sim.destroy()

(Once an upstream strands-robots release walks the strands_robots.backends entry-point group, the first two lines collapse to create_simulation("isaac", render_mode="rtx_realtime", headless=True) — same kwargs, forwarded into IsaacConfig.)

For URDF / MJCF / USD ingestion, use the loader module:

from strands_robots_sim.isaac.loaders import load_urdf, load_mjcf, load_usd

panda = load_urdf("/path/to/panda.urdf")        # stdlib XML, no extra deps
print(panda.num_joints, panda.joint_names)

See Getting Started → Quickstart for the end-to-end LIBERO demo on RTX.

Backend matrix

Capability MuJoCo
(strands-robots)		 Isaac Sim
(this repo)
Native GPU — ✅ PhysX
Apple Silicon ✅ ❌
Photoreal rendering — ✅ RTX path-traced
num_envs per GPU 1–8 1024+
USD scene format — ✅ native
Synthetic data (Replicator) — ✅
Download size ~50 MB ~30 GB
Setup friction low high
  • Pick MuJoCo for fast iteration, debugging, macOS / Apple Silicon, CI.
  • Pick Isaac Sim for RTX photoreal eval, USD-native scenes, IsaacLab fleet RL, or Replicator synth-data.

The full per-backend table with install hints lives in Simulation → Overview.

Where to next