Tools¶

The @tool functions re-exported from strands_robots, with their parameters as the agent sees them.

The @tool functions re-exported from strands_robots; hand any of them to a Strands Agent and the agent can call it. Below: each tool's parameters as the agent sees them. The catalog with every action value, including tools not re-exported at the top level, is Tools.

Hardware and simulation¶

strands_robots.tools.use_lerobot.use_lerobot ¶

use_lerobot(module: str = '__discovery__', method: str = 'list_modules', parameters: dict[str, Any] | None = None, label: str = '') -> dict[str, Any]

Universal LeRobot access - call any lerobot module, class, or function dynamically.

Like use_aws wraps boto3.client[service].operation(params), this wraps lerobot[module].method(params). The agent discovers available APIs and registered configs (robots/teleoperators/cameras/policies) directly from lerobot's own draccus registries - nothing is hardcoded.

Results are serialized at full fidelity (no lossy truncation), and any image frames (numpy HxW / HxWxC arrays) returned by a call are emitted as proper Strands image content blocks so the model can actually see them.

Calls under restricted namespaces (lerobot.scripts, lerobot.common.datasets.push) and side-effecting methods (push_to_hub, upload_folder, save_to_disk, ...) are refused before invocation - whether named via method or reached through the module path - so a prompt-injected call cannot push to the Hub or spawn a training subprocess.

Parameters:

Name	Type	Description	Default
`module`	`str`	Dotted path into lerobot (e.g. "cameras.opencv.OpenCVCamera", "datasets.lerobot_dataset.LeRobotDataset", "policies.factory"). Special values: "discovery" - explore modules + all registered configs. "registry" - list a single registry; pass its name as `method` (robots\|teleoperators\|cameras\|policies).	`'__discovery__'`
`method`	`str`	Method/function/attribute name to call or read. Special values: "list_modules" - discovery output (with module="discovery"). "describe" - inspect the object without calling it.	`'list_modules'`
`parameters`	`dict[str, Any] \| None`	Dict of kwargs to pass to the method. Omit for no-arg calls.	`None`
`label`	`str`	Human-readable description of what this call does.	`''`

Returns:

Type	Description
`dict[str, Any]`	Dict with status and content; content may include text + image blocks.

Examples:

Discover everything (modules + registered choices)¶

use_lerobot(module="discovery", method="list_modules")

List just the robot choices (dynamic, from registry)¶

use_lerobot(module="registry", method="robots")

Describe a class¶

use_lerobot(module="cameras.opencv.OpenCVCamera", method="describe")

Find cameras¶

use_lerobot(module="cameras.opencv.OpenCVCamera", method="find_cameras")

Get a policy class¶

use_lerobot(module="policies.factory", method="get_policy_class", parameters={"name": "act"})

Read a calibration path constant¶

use_lerobot(module="utils.constants", method="HF_LEROBOT_CALIBRATION")

strands_robots.tools.lerobot_camera.lerobot_camera ¶

lerobot_camera(action: str = 'list', camera_type: str = 'opencv', camera_id: int | str | None = None, save_path: str = './lerobot_captures', filename: str | None = None, camera_ids: list[int | str] | None = None, width: int = 640, height: int = 480, fps: int = 30, color_mode: str = 'RGB', rotation: str = 'NO_ROTATION', format: str = 'jpg', capture_duration: float = 5.0, preview_duration: float = 10.0, async_mode: bool = False, timeout_ms: float = 1000, warmup: bool = True, save_config: bool = False) -> dict[str, Any]

Advanced LeRobot-based camera tool for professional camera management.

Parameters:

Name	Type	Description	Default
`action`	`str`	Action to perform - "discover": Discover all available cameras (OpenCV + RealSense) - "list": List camera details and configurations - "capture": Capture single image from camera - "capture_batch": Capture from multiple cameras simultaneously - "record": Record video sequence from camera - "preview": Show live preview from camera - "test": Test camera functionality and performance - "configure": Configure camera settings and save	`'list'`
`camera_type`	`str`	Camera type ("opencv" or "realsense"). "realsense" needs the Intel SDK installed on top of lerobot; without it the action is refused naming that install, rather than reported as unsupported.	`'opencv'`
`camera_id`	`int \| str \| None`	Camera device ID (int for index, str for path like "/dev/video0")	`None`
`save_path`	`str`	Directory to save captured images/videos	`'./lerobot_captures'`
`filename`	`str \| None`	Custom filename (without extension). Resolved inside save_path; a value naming a location outside it is refused rather than written there.	`None`
`camera_ids`	`list[int \| str] \| None`	Cameras to capture from in one capture_batch call - a list of distinct ids, each an int index or a device path string. Omit it for the default robot cameras. An empty list selects no camera and is refused rather than widened to those defaults; a single id passed as a bare string is refused rather than read one camera per character.	`None`
`width`	`int`	Frame width in pixels (a positive whole number)	`640`
`height`	`int`	Frame height in pixels (a positive whole number)	`480`
`fps`	`int`	Frames per second (a positive whole number)	`30`
`color_mode`	`str`	Color mode; one of "RGB" or "BGR", compared case-insensitively. Any other value is refused rather than silently read as "BGR".	`'RGB'`
`rotation`	`str`	Image rotation; one of "NO_ROTATION", "ROTATE_90", "ROTATE_180" or "ROTATE_270", compared case-insensitively. Any other value is refused rather than silently read as "NO_ROTATION".	`'NO_ROTATION'`
`format`	`str`	Image format ("jpg", "png", "bmp"). Becomes the saved file's extension, so like filename it is resolved inside save_path and refused if it names a location outside it. The image returned alongside the file is encoded as JPEG only for a JPEG request; any other format is carried back losslessly as PNG, so the inline copy has the frame's own pixels.	`'jpg'`
`capture_duration`	`float`	Duration for video recording (positive seconds)	`5.0`
`preview_duration`	`float`	Duration for preview display (positive seconds)	`10.0`
`async_mode`	`bool`	Use async reading for better performance. A boolean; it selects a read path rather than scaling one, so any other value is refused rather than read as its opposite.	`False`
`timeout_ms`	`float`	Timeout for async operations (positive milliseconds; read only when async_mode is on)	`1000`
`warmup`	`bool`	Enable camera warmup on connection. A boolean, refused rather than read by truthiness - it is also recorded in the saved configuration, so a non-boolean would persist there.	`True`
`save_config`	`bool`	Save camera configuration to file. A boolean, refused rather than read by truthiness: it writes a file, so a truthy spelling of off would leave one behind.	`False`

Returns:

Type	Description
`dict[str, Any]`	Dict containing status and detailed camera operation results

strands_robots.tools.lerobot_teleoperate.lerobot_teleoperate ¶

lerobot_teleoperate(action: str = 'start', session_name: str | None = None, background: bool = True, robot_type: str = 'so101_follower', robot_port: str | None = '/dev/ttyACM0', robot_id: str | None = None, robot_cameras: dict[str, Any] | None = None, robot_left_arm_port: str | None = None, robot_right_arm_port: str | None = None, teleop_type: str | None = 'so101_leader', teleop_port: str | None = '/dev/ttyACM1', teleop_id: str | None = None, teleop_left_arm_port: str | None = None, teleop_right_arm_port: str | None = None, dataset_repo_id: str | None = None, dataset_single_task: str | None = None, dataset_num_episodes: int = 50, dataset_fps: int = 30, dataset_episode_time_s: int = 60, dataset_reset_time_s: int = 60, dataset_root: str | None = None, dataset_video: bool = True, dataset_push_to_hub: bool = False, record_resume: bool = False, replay_episode: int = 0, display_data: bool = False, fps: int = 60, teleop_time_s: float | None = None, play_sounds: bool = True, auto_accept_calibration: bool = True, policy_path: str | None = None, dagger_record_autonomous: bool = False, dagger_input_device: str = 'keyboard', dagger_num_episodes: int | None = None) -> dict[str, Any]

Advanced LeRobot teleoperation tool with recording capabilities for robot training data collection.

This tool integrates teleoperation and recording functionality from lerobot, allowing users to: - Control robots through teleoperation devices - Record demonstrations for training machine learning models - Replay recorded episodes - Manage multiple teleoperation sessions

Features: - Session Management: Start, stop, list, and monitor teleoperation sessions - Background Execution: Run teleoperation in background with logging - Recording Mode: Automatically record demonstrations when dataset configuration is provided - Multi-Robot Support: Support for single-arm and bimanual robots - Camera Integration: Multi-camera support with configurable settings - Replay Capability: Replay recorded episodes on physical robots - Safety Features: Graceful shutdown and process management

Actions

start: Start a new teleoperation session - Simple teleoperation (just teleop_type specified) - Recording mode (dataset_repo_id specified) - Background or foreground execution

stop: Stop a running session by name (SIGTERM, then SIGKILL), reported successful only once the process has left the process table

list: List all active teleoperation sessions

status: Get detailed status of a specific session - Process information, uptime, logs

replay: Replay a recorded episode on the robot - Requires dataset_repo_id and replay_episode

dagger: Human-in-the-loop correction (DAgger / teleop takeover) - A policy drives the follower; the leader can pre-empt to record corrections, appended to the dataset as new episodes. - Requires policy_path (policy to roll out) and dataset_repo_id. - Drives lerobot-rollout with --strategy.type=dagger. Toggle the correction window with the keyboard/pedal (dagger_input_device); set dagger_record_autonomous=True to also record the autonomous phase, dagger_num_episodes to cap collected corrections.

Robot Types

so101_follower: Single-arm SO-101 robot
bi_so100_follower: Dual-arm SO-100 robot
koch_follower: Koch robot
hope_jr: HOPE Jr robot

Teleoperator Types

so101_leader: SO-101 leader device
bi_so100_leader: Dual SO-100 leader devices
koch_leader: Koch leader device
gamepad: Gamepad controller
homunculus: Homunculus teleoperator

Camera Configuration Format

{ "camera_name": { "type": "opencv", # a lerobot camera backend; "intelrealsense" for a RealSense "index_or_path": 0, # camera index or device path (opencv) "width": 640, "height": 480, "fps": 30 } }

"type" selects the backend from lerobot's camera registry, and the other options an entry may name are the fields that backend's config declares - an opencv camera is identified by "index_or_path", a RealSense by "serial_number_or_name" - the same vocabulary Robot(cameras=...) reads. An unstated "index_or_path" / "width" / "height" / "fps" is rendered as the default shown above. So a misspelled option is refused rather than dropped: "index" instead of "index_or_path" would otherwise record the default device under this camera's name. The name itself must be a bare token (letters, digits, "_", "-"); "width" / "height" / "fps" must be positive whole numbers, the same domain lerobot_camera reads them with; and "index_or_path" must be a non-negative index or a device path. A string value is quoted in the rendered argv, so a device path or a serial is read back exactly as given ("0123" stays "0123"). The map is rendered into the argv of a detached subprocess, so a value that would not be carried as given is reported here instead of in a session log.

Examples:

Simple teleoperation¶

lerobot_teleoperate( action="start", robot_type="so101_follower", robot_port="/dev/ttyACM0", teleop_type="so101_leader", teleop_port="/dev/ttyACM1" )

Recording demonstrations¶

lerobot_teleoperate( action="start", robot_type="so101_follower", robot_port="/dev/ttyACM0", teleop_type="so101_leader", teleop_port="/dev/ttyACM1", dataset_repo_id="my_user/cube_picking", dataset_single_task="Pick up the red cube and place it in the box", dataset_num_episodes=25, robot_cameras={ "front": {"type": "opencv", "index_or_path": 0, "width": 1920, "height": 1080, "fps": 30} } )

Bimanual robot teleoperation¶

lerobot_teleoperate( action="start", robot_type="bi_so100_follower", robot_left_arm_port="/dev/ttyACM0", robot_right_arm_port="/dev/ttyACM1", teleop_type="bi_so100_leader", teleop_left_arm_port="/dev/ttyACM2", teleop_right_arm_port="/dev/ttyACM3" )

List sessions¶

lerobot_teleoperate(action="list")

Stop session¶

lerobot_teleoperate(action="stop", session_name="teleop_1234567890")

Replay episode¶

lerobot_teleoperate( action="replay", robot_type="so101_follower", robot_port="/dev/ttyACM0", dataset_repo_id="my_user/cube_picking", replay_episode=5 )

Calibration

Calibrating an arm is LeRobot's own procedure, run from the shell - lerobot-find-port to identify the bus, lerobot-setup-motors to assign motor IDs, then lerobot-calibrate to record the homing offsets and travel limits. The resulting JSON lives under HF_LEROBOT_CALIBRATION and a session here reads it through LeRobot; the interactive prompt LeRobot shows when a device has none is answered by auto_accept_calibration below.

Parameters:

Name	Type	Description	Default
`action`	`str`	Action to perform (start, stop, list, status, replay, dagger)	`'start'`
`session_name`	`str \| None`	Session identifier (auto-generated for start, required for stop/status)	`None`
`background`	`bool`	Run session in background with logging (default: True). Must be a boolean: it selects an execution posture rather than scaling a quantity, so a truthy spelling of off such as `"false"` is refused rather than detaching the session it reads as declining to detach.	`True`
`robot_type`	`str`	Robot type identifier	`'so101_follower'`
`robot_port`	`str \| None`	Serial port for single-arm robots	`'/dev/ttyACM0'`
`robot_id`	`str \| None`	Robot instance identifier	`None`
`robot_cameras`	`dict[str, Any] \| None`	Camera configuration dictionary (see Camera Configuration Format above for the options and their domains)	`None`
`robot_left_arm_port`	`str \| None`	Left arm port for bimanual robots	`None`
`robot_right_arm_port`	`str \| None`	Right arm port for bimanual robots	`None`
`teleop_type`	`str \| None`	Teleoperator type identifier	`'so101_leader'`
`teleop_port`	`str \| None`	Serial port for single-arm teleoperators	`'/dev/ttyACM1'`
`teleop_id`	`str \| None`	Teleoperator instance identifier	`None`
`teleop_left_arm_port`	`str \| None`	Left arm port for bimanual teleoperators	`None`
`teleop_right_arm_port`	`str \| None`	Right arm port for bimanual teleoperators	`None`
`dataset_repo_id`	`str \| None`	HuggingFace dataset repository ID (enables recording mode)	`None`
`dataset_single_task`	`str \| None`	Task description for recordings	`None`
`dataset_num_episodes`	`int`	Number of episodes to record	`50`
`dataset_fps`	`int`	Recording frame rate	`30`
`dataset_episode_time_s`	`int`	Episode duration in seconds	`60`
`dataset_reset_time_s`	`int`	Reset time between episodes	`60`
`dataset_root`	`str \| None`	Local dataset storage directory. When omitted for a recording, an explicit root is still pinned (resolved from `dataset_repo_id` under `$HF_LEROBOT_HOME`) and returned as `dataset_root` in the result. lerobot HEAD stamps a fresh record's `repo_id` with a `_YYYYMMDD_HHMMSS` timestamp (affecting the Hub push target and dataset metadata); pinning the root keeps the on-disk data at the requested location regardless, so downstream train/verify steps find it. Use `record_resume=True` to append instead.	`None`
`dataset_video`	`bool`	Enable video encoding	`True`
`dataset_push_to_hub`	`bool`	Upload dataset to HuggingFace Hub	`False`
`record_resume`	`bool`	Append to an existing dataset at the resolved root (lerobot-record `--resume true`) instead of creating a fresh one. Resume preserves the existing (already-stamped) repo_id rather than re-stamping, so repeated sessions accumulate in one dataset.	`False`
`replay_episode`	`int`	Episode number to replay	`0`
`display_data`	`bool`	Show live camera feeds and telemetry	`False`
`fps`	`int`	Teleoperation control loop frequency	`60`
`teleop_time_s`	`float \| None`	Session duration limit	`None`
`play_sounds`	`bool`	Enable lerobot's spoken event announcements ("Recording episode 3", "Stop recording"). Effective for recording, replay and dagger; plain teleoperation emits no audio and ignores it. Must be a boolean - a string such as `"false"` is refused rather than read by truthiness, since every non-empty string is truthy.	`True`
`auto_accept_calibration`	`bool`	Answer the calibration prompt on the session's behalf, by writing two newlines into the process's stdin shortly after it starts. Withhold it (`False`) to answer the prompt yourself; nothing reports that stdin was written to, so an unintended acceptance is not visible afterwards. A write that fails is reported at WARNING, since by then the start result has already told the caller the session started. Must be a boolean, on the same reasoning as `background`.	`True`
`dagger_record_autonomous`	`bool`	Record the autonomous rollout episodes into the corrections dataset as well, rather than only the teleoperated takeovers. Must be a boolean; a truthy spelling of off would otherwise land autonomous episodes in a corrections dataset.	`False`
`policy_path`	`str \| None`	Checkpoint the `dagger` action rolls out autonomously between human takeovers. Required for `dagger`; ignored by every other action.	`None`
`dagger_input_device`	`str`	How the operator seizes control during a `dagger` rollout - `"keyboard"` (default) or `"pedal"`. Any other value is refused.	`'keyboard'`
`dagger_num_episodes`	`int \| None`	Cap on the corrections collected in one `dagger` session. A positive whole number, or None for no cap.	`None`

Returns:

Type	Description
`dict[str, Any]`	Dict with operation status and results:
`dict[str, Any]`	{ "status": "success\|error", "content": [{"text": "Description of operation"}], "session_name": "session_id", # for start action "pid": 12345, # process ID for background sessions "command": "full_command_executed", "log_file": "/tmp/session.log", # for background sessions "sessions": {...}, # for list action "uptime": 123.45, # session uptime in seconds; None when the # record states no usable start time "is_running": true # for status action
`dict[str, Any]`	}

strands_robots.tools.pose_tool.pose_tool ¶

pose_tool(action: str, robot_id: str = 'so101_follower', port: str | None = '/dev/ttyACM0', calibration: str | None = None, pose_name: str | None = None, motor_name: str | None = None, position: float | None = None, delta: float | None = None, positions: dict[str, float] | None = None, description: str | None = None, smooth: bool = True, steps: int = 20, step_delay: float = 0.05, tool_context: ToolContext | None = None) -> dict[str, Any]

Advanced robot pose management tool with fine motor control.

Actions

Pose Management: - "store_pose": Store current robot pose with a name - "load_pose": Move robot to a stored pose - "list_poses": List all stored poses - "delete_pose": Delete a stored pose - "show_pose": Display pose information

Motor Control: - "move_motor": Move single motor to position - "move_multiple": Move multiple motors simultaneously - "incremental_move": Small incremental motor movement - "read_position": Read current motor position - "read_all": Read all motor positions

System: - "connect": Test robot connection - "emergency_stop": De-energize every motor (Torque_Enable=0). The arm goes LIMP and falls under gravity -- it does not hold position, so anything it is grasping is dropped. Reports an error when any motor could not be released. - "reset_to_home": Move to safe home position

Calibration

This tool records no calibration - it reads the one on disk. Pass calibration the JSON lerobot-calibrate wrote for this arm and every degree and percent here is the number LeRobot quotes for the same servo, and every target is bounded by the travel that run measured. Omitting it reads and commands the servo's full rotation instead, which is off by however far this arm's stops sit inside that rotation.

Recording one is LeRobot's own procedure, run from the shell with lerobot-calibrate (after lerobot-find-port and lerobot-setup-motors), which writes the JSON under HF_LEROBOT_CALIBRATION; the interactive prompt LeRobot shows when a device has none is answered by a lerobot_teleoperate session's auto_accept_calibration.

Parameters:

Name	Type	Description	Default
`action`	`str`	Action to perform	required
`robot_id`	`str`	Robot identifier for pose storage. Becomes part of the pose file's name, so a value resolving outside the storage directory is refused rather than written there.	`'so101_follower'`
`port`	`str \| None`	Serial port for robot communication	`'/dev/ttyACM0'`
`calibration`	`str \| None`	Path of the calibration JSON `lerobot-calibrate` wrote for this arm, e.g. what :func:`~strands_robots.drivers.feetech.bus.lerobot_calibration_path` returns. Unset reads and commands the servo's full rotation.	`None`
`pose_name`	`str \| None`	Name for pose operations	`None`
`motor_name`	`str \| None`	Motor name for single motor operations	`None`
`position`	`float \| None`	Target position in degrees (or 0-100% for gripper). A finite number within the motor's measured travel - a value outside it is refused rather than clamped to the mechanical limit, because the clamp cannot be told apart from a typo and the success text echoes the value asked for.	`None`
`delta`	`float \| None`	Incremental movement in degrees. A finite number whose magnitude is at most the motor's full travel, which no starting position could exceed.	`None`
`positions`	`dict[str, float] \| None`	Dictionary of motor positions {motor_name: degrees}. Every value is held to the same domain as `position`, and the first that is not names the motor it came from.	`None`
`description`	`str \| None`	Description for stored poses	`None`
`smooth`	`bool`	Interpolate towards the targets over `steps * step_delay` seconds instead of writing each goal position once. A boolean: it selects one of two trajectories, so a value that is only truthy or only falsy is refused rather than read as one of them - `smooth=0` would drop the interpolation this defaults to, and `smooth="false"` would keep it. Read only by `load_pose` and `move_multiple`.	`True`
`steps`	`int`	Number of increments for an interpolated move. A positive integer - it divides the travel and bounds the write loop.	`20`
`step_delay`	`float`	Seconds between increments of an interpolated move. A positive finite number - this pause is what makes the move smooth, so `0` is refused; use `smooth=False` to go straight to the target. Together with `steps` it sets the trajectory duration (the default 20 x 0.05s = ~1s).	`0.05`
`tool_context`	`ToolContext \| None`	Supplied by the agent runtime; carries the operator interrupt the motion actions are approved through. Without it a motion is refused unless pre-approved via STRANDS_POSE_COMMAND_ALLOW or BYPASS_TOOL_CONSENT=true	`None`

Operator approval

"move_motor", "move_multiple", "incremental_move", "load_pose" and "reset_to_home" move the arm, so each stops for a human before the motor controller is built; a declined or headless call sends no goal position. Pre-approve with STRANDS_POSE_COMMAND_ALLOW=move_motor,load_pose (or "*"). "connect", the reads, the pose library actions and "emergency_stop" are never gated.

Both interpolation options are read only by load_pose and move_multiple (when smooth is left true) and by reset_to_home, which always interpolates; any other action ignores them and is never refused for them. smooth itself is read only by the first two - reset_to_home supplies its own - so only those two are refused for it.

Returns:

Type	Description
`dict[str, Any]`	Dict containing status and response content, or an error dict when an
`dict[str, Any]`	interpolation option or a joint target the requested action reads cannot
`dict[str, Any]`	be honored.

strands_robots.tools.serial_tool.serial_tool ¶

serial_tool(action: str, port: str | None = None, baudrate: int = 9600, timeout: float = 1.0, data: str | None = None, hex_data: str | None = None, motor_id: int | None = None, position: int | None = None, velocity: int | None = None, read_bytes: int = 1024, tool_context: ToolContext | None = None) -> dict[str, Any]

Advanced serial communication tool for robot control and device communication.

Actions

"list_ports": Discover available serial ports
"send": Send data to serial port
"read": Read data from serial port
"send_read": Send data and read response
"feetech_position": Control STS/SMS servo position
"feetech_velocity": Control STS/SMS servo velocity
"feetech_ping": Ping a Feetech servo motor
"monitor": Monitor serial port (continuous read)

Parameters:

Name	Type	Description	Default
`action`	`str`	Action to perform	required
`port`	`str \| None`	Serial port path (e.g., "/dev/ttyACM0", "COM3")	`None`
`baudrate`	`int`	Communication speed in baud; a positive integer (default: 9600)	`9600`
`timeout`	`float`	Read timeout in seconds; a finite number >= 0, where 0 is pyserial's non-blocking mode (return what is already buffered)	`1.0`
`data`	`str \| None`	String data to send	`None`
`hex_data`	`str \| None`	Hex string data to send (e.g., "FF FF 01 04 03 00 64 92")	`None`
`motor_id`	`int \| None`	Motor ID for Feetech commands; an integer in [1, 254], of which 254 (0xfe) is the broadcast every servo receives. An action that reads a reply back accepts only a single servo, [1, 253]	`None`
`position`	`int \| None`	Target position for STS/SMS-series motors; an integer in [0, 4095]. That full scale and the two-byte order this tool encodes into are both STS/SMS properties: the SCS series is 10-bit and reads the same two bytes in the opposite order, so an SCS-series servo is not addressed by this action at all	`None`
`velocity`	`int \| None`	Target velocity for STS/SMS-series motors; an integer in [0, 32767]. Goal_Velocity is sign-magnitude on that series, so a magnitude reaching bit 15 commands the opposite direction instead of a faster move	`None`
`read_bytes`	`int`	Number of bytes to read; a positive integer	`1024`
`tool_context`	`ToolContext \| None`	Supplied by the agent runtime; carries the operator interrupt the write actions are approved through. Without it a write is refused unless pre-approved via STRANDS_SERIAL_COMMAND_ALLOW or BYPASS_TOOL_CONSENT=true	`None`

Operator approval

"send", "send_read", "feetech_position" and "feetech_velocity" put bytes on the bus, so each stops for a human before the port is opened; a declined or headless call writes nothing. Pre-approve with STRANDS_SERIAL_COMMAND_ALLOW=feetech_position,feetech_velocity (or "*"). Reads, "monitor" and "feetech_ping" are never gated.

Validation

A numeric option the requested action consumes is checked before the port is opened, so a value that cannot be honored is reported instead of being masked into a different servo command, silently coerced by pyserial, or read as no wait at all. An option the action ignores is never checked.

Returns:

Type	Description
`dict[str, Any]`	Dict containing status and response content

Policies and training¶

strands_robots.tools.run_policy.run_policy ¶

run_policy(simulation: Any, *, robot_name: str | None = None, policy_provider: str = 'mock', policy_config: dict[str, Any] | None = None, instruction: str = '', n_episodes: int = 1, n_steps: int = 60, control_frequency: float = 30.0, action_horizon: int = 8, fast_mode: bool = True, dataset_root: str | None = None, dataset_repo_id: str = 'local/run_policy_rollout', dataset_task: str = '', dataset_fps: int = 30, dataset_cameras: list[str] | None = None, seed: int | None = None, policy_kwargs: dict[str, Any] | None = None, video: dict[str, Any] | None = None, stop_when: dict[str, Any] | None = None) -> dict[str, Any]

Roll out a policy for n_episodes x n_steps with per-episode parquet boundaries.

Pass-through wrapper around :meth:Simulation.run_policy that owns the multi-episode loop and the recording lifecycle, so an LLM agent never has to improvise either. Closes the #708 fabrication vector by:

Explicit n_episodes - the loop iterates exactly N times, no narrated counts.
Per-episode save_episode - each rollout lands in its own parquet row via PolicyRunner._finalize_recorder_episode.
Parquet-truth return - final payload carries total_episodes / total_frames read from meta/info.json AFTER stop_recording returns, NOT self-reported by the loop. Mismatch with n_episodes is surfaced as warnings=[...] for the verifier to act on. Both counts are graded by declared_count, so a header that declares something which is not a count reads -1 and is reported as corrupt metadata rather than coerced into agreement with the request.

Parameters:

Name	Type	Description	Default
`simulation`	`Any`	Live `Simulation` (or compatible) handle. Constructed by the orchestrator - pass through a Python partial / closure, not from agent text. LLMs cannot synthesize this argument, which is the point: the episode loop runs in deterministic Python.	required
`robot_name`	`str \| None`	Robot to control. Forwarded to `run_policy`. Required when the simulation hosts more than one robot.	`None`
`policy_provider`	`str`	Provider name passed to `create_policy` inside the engine (`"mock"` / `"lerobot_local"` / `"remote"` / `"wbc"` / ...).	`'mock'`
`policy_config`	`dict[str, Any] \| None`	Provider-specific kwargs forwarded verbatim.	`None`
`instruction`	`str`	Natural-language instruction for the policy.	`''`
`n_episodes`	`int`	Number of reset -> rollout episodes. MUST be a positive int. There is no "guess from duration" fallback.	`1`
`n_steps`	`int`	Hard cap on control steps per episode. Forwarded to `run_policy` as `n_steps`.	`60`
`control_frequency`	`float`	Target Hz for policy queries. Must be a finite number > 0; an unusable rate is reported before the rollout starts instead of aborting every episode mid-flight. When a recording is requested it must also EQUAL `dataset_fps` - see that parameter.	`30.0`
`action_horizon`	`int`	Lower bound on actions consumed per policy call before re-querying; the effective interval is `max(action_horizon, policy.execution_horizon)`, so a chunk-emitting policy always consumes its full chunk and a smaller value has no effect (see `resolve_chunk_length`). Must be a positive integer, reported before the rollout starts for the same reason.	`8`
`fast_mode`	`bool`	Skip real-time sleep between steps (default True for rollouts - wall-clock pacing slows headless eval). Must be a boolean; it selects a posture rather than scaling a quantity, so any other type is reported before the rollout starts rather than read by truthiness - a truthy `"false"` would otherwise run the episodes unpaced.	`True`
`dataset_root`	`str \| None`	When set, the tool drives the full recording cycle: `start_recording(root=dataset_root, ...)` -> N rollouts with per-episode save_episode -> `stop_recording` -> parquet-truth read. When `None` the loop runs without recording (smoke-test mode).	`None`
`dataset_repo_id`	`str`	Forwarded to `start_recording`.	`'local/run_policy_rollout'`
`dataset_task`	`str`	Task label forwarded to `start_recording`.	`''`
`dataset_fps`	`int`	Dataset FPS forwarded to `start_recording`. Must be a positive whole number - reported by `start_recording` itself, which checks the rate before it touches the target directory, so an unusable value costs nothing. It must also EQUAL `control_frequency` whenever `dataset_root` is set: the recorder captures one frame per control step and never decimates, while LeRobot timestamps every frame from the declared rate, so a differing pair cannot be honored, only mislabelled. The disagreement is refused up front (:func:`~strands_robots.simulation.recording.requested_rate_mismatch_reason`) rather than by the per-episode rollout - which this tool reaches only after `start_recording(overwrite=True)` has replaced any dataset already at `dataset_root` with an empty one. Ignored entirely when `dataset_root` is `None`.	`30`
`dataset_cameras`	`list[str] \| None`	Camera names to record into the dataset. When set, forwarded as `start_recording(cameras=...)` (supported by both the MuJoCo and Newton backends) to scope a policy-specific dataset to exactly the views the policy declares (e.g. `["camera1", "camera2", "camera3"]`) and keep the implicit `default` free camera out of `observation.images.*`. When `None` (default) no `cameras` kwarg is forwarded at all, so every scene camera is recorded and the call stays backend-agnostic across the MuJoCo and Newton engines.	`None`
`seed`	`int \| None`	Master RNG seed. Each episode derives a deterministic offset so rollouts are reproducible within a process.	`None`
`policy_kwargs`	`dict[str, Any] \| None`	Optional per-call goal payload forwarded to every `policy.get_actions` call (the #300 goal keys).	`None`
`video`	`dict[str, Any] \| None`	Optional rollout-video config forwarded to :meth:`Simulation.run_policy` (e.g. `{"path": "/tmp/rollout.mp4", "fps": 30, "camera": "camera1", "width": 640, "height": 480}`). `path` is required to enable recording; a falsy/absent path disables it. For `n_episodes > 1` an `_ep<i>` suffix is inserted into the path stem so each episode writes its own MP4 instead of overwriting. The returned payload carries `video_paths` (the MP4s that landed on disk).	`None`
`stop_when`	`dict[str, Any] \| None`	Optional semantic early-return clause forwarded to every per-episode :meth:`Simulation.run_policy` call: the episode ends as soon as the condition holds in the sim, instead of only at the `n_steps` budget - a per-episode success gate for collection loops. Same predicate DSL as a benchmark spec's `success` clause: a single call `{"predicate": "grasped", "body": "cube", "gripper_prefix": "so101/gripper"}` or an `{"all": [...]}` / `{"any": [...]}` group. Validated against the closed predicate registry up front (before any recording is started), so an unknown predicate name is rejected with the valid list while nothing has been set up yet. Each rollout's `stopped_reason` (`'predicate'` \| `'budget'` \| `'cancelled'` \| `'error'`) is reported per episode, as is `stop_when_true_at_reset`: a clause the scene's initial state already satisfies is evaluated only AFTER an applied action, so it fires on the episode's first step whatever the policy commands - one recorded frame for that episode, tagged `stopped_reason='predicate'` and indistinguishable from an episode that reached the condition. Usually a threshold on the wrong side of the initial state (a `body_above_z` below where the object already rests). The payload aggregates `episodes_stop_when_true_at_reset` with `stop_when_reset_warning`; it is deliberately not a `warnings` entry, which would flip `status` to `"error"`, because domain randomisation legitimately satisfies a clause on some draws.	`None`

Returns:

Type Description

dict[str, Any]

Standard {status, content} payload. On success the payload

dict[str, Any]

also carries::

{ "n_episodes_requested": int, "n_episodes_actual": int, # parquet-truth, -1 if unread "n_frames_actual": int, # parquet-truth, -1 if unread "dataset_root": str | None, "recording_save_error": str | None, # None on a healthy run "warnings": [str, ...], # mismatch flags "episodes_stop_when_true_at_reset": int, "stop_when_reset_warning": str | None, "episodes": [ {"index": int, "status": "success" | "error", ...}, ... ], }

strands_robots.tools.train_policy.train_policy ¶

train_policy(action: str = 'train', provider: str = 'lerobot_local', dataset_root: str | None = None, dataset_repo_id: str | None = None, streaming: bool = False, base_model: str = '', output_dir: str | None = None, embodiment: str | None = None, steps: int = 10000, batch_size: int = 32, learning_rate: float | None = None, save_freq: int = 1000, num_gpus: int = 1, num_nodes: int = 1, resume: bool = False, seed: int | None = None, method: str = 'full', lora_r: int | None = None, lora_alpha: int | None = None, lora_target_modules: str | None = None, tune: dict[str, bool] | None = None, val_episodes: int | None = None, augmentation: dict[str, Any] | None = None, fps: int | None = None, extra: dict[str, Any] | None = None, job_id: str | None = None) -> dict[str, Any]

Post-tune (fine-tune) a robot policy on a recorded LeRobotDataset.

Provider-agnostic: provider picks the training backend and the same arguments map onto its native pipeline. Closes the record -> train -> deploy loop - the produced checkpoint loads back via create_policy.

Parameters:

Name	Type	Description	Default
`action`	`str`	One of: - `"train"` : validate + launch training (default). - `"validate"` : pure preflight only; report problems, launch nothing. - `"status"` : "RUNNING != learning" verdict for a job (needs `job_id`). Prefer `metrics['success_rate']` / `task_metrics` over reward when a task reports them; a failed run names its cause in `metrics['failure']`. - `"stop"` : stop a running job, keeping its checkpoints (needs `job_id`; `isaaclab`). - `"play"` : play a finished job's latest checkpoint back and record a video, with the physics it trained on (needs `job_id`; `isaaclab`; `extra` may set `num_envs`, `video_length`, `timeout_s`, `wait`). Poll the returned `job_id` with `status`; `metrics['video']` is the clip. - `"record"` : roll a finished job's policy out and record it as a LeRobotDataset (needs `job_id` and `extra['dataset_dir']`; `isaaclab`; `extra` may set `repo_id`, `episodes`, `frames`, `camera`, `camera_eye`, `camera_target`, `width`, `height`, `task_description`, `timeout_s`, `wait`). Poll the returned `job_id`; `metrics['dataset']` is the dataset root. - `"export"` : produce a loadable artifact from a checkpoint (needs `output_dir`; uses the run's last checkpoint). - `"list"` : list available training providers.	`'train'`
`provider`	`str`	Training backend / policy family - `"lerobot_local"` (act, diffusion, smolvla, pi0, pi05, groot for NVIDIA GR00T N1.7, ...), `"cosmos3"` (NVIDIA Cosmos3), `"isaaclab"` (GPU-parallel RL in a separate Isaac Lab install; needs no dataset), or `"mock"`. Same name as the inference provider in `create_policy`.	`'lerobot_local'`
`dataset_root`	`str \| None`	Path to a LeRobotDataset v3 root (has `meta/info.json`) - exactly what `Robot.stop_recording` writes. Optional when `dataset_repo_id` is set (then it is the local cache root).	`None`
`dataset_repo_id`	`str \| None`	Hugging Face Hub dataset id (`org/name`) to train from the Hub instead of a local root - required to `streaming` a large (50-500 GB) Hub dataset without downloading it in full. lerobot only.	`None`
`streaming`	`bool`	Stream frames instead of materializing the dataset (lerobot `StreamingLeRobotDataset`). With `dataset_repo_id` this streams Hub shards with bounded disk; with a local `dataset_root` it streams from disk with bounded RAM. lerobot only; ignored elsewhere.	`False`
`base_model`	`str`	HF id or local checkpoint to post-tune from. For GR00T N1.7 this is required (`nvidia/GR00T-N1.7-3B`); ACT-from-scratch leaves it "".	`''`
`output_dir`	`str \| None`	Where checkpoints + logs go.	`None`
`embodiment`	`str \| None`	Embodiment tag - which state/action projector head the run trains. Read by any lerobot policy whose config declares `embodiment_tag` (GR00T N1.7, where it is required); refused for a lerobot policy that has no such field, since those take their state/action shape from the dataset features.	`None`
`steps`	`int`	Total optimizer steps.	`10000`
`batch_size`	`int`	Global batch size (summed across GPUs).	`32`
`learning_rate`	`float \| None`	Optimizer learning rate. `None` (default) uses the backend's own default (the policy training preset for lerobot, Cosmos's TOML default); an explicit value must be a positive finite number and is honored by every backend. `0` and `inf` are refused up front: the first trains for the whole run without updating a weight, the second writes a checkpoint of `NaN`, and no backend reports either.	`None`
`save_freq`	`int`	Checkpoint cadence in steps.	`1000`
`num_gpus`	`int`	GPUs on this node (`>1` -> accelerate/torchrun multi-GPU). A positive integer; anything else is refused by preflight.	`1`
`num_nodes`	`int`	Nodes (Cosmos HSDP / torchrun `--nnodes`). A positive integer; anything else is refused by preflight.	`1`
`resume`	`bool`	Resume from the latest checkpoint under `output_dir`.	`False`
`seed`	`int \| None`	Master seed.	`None`
`method`	`str`	Tuning strategy - `"full"` \| `"lora"` \| `"expert_only"` \| `"frozen_backbone"`. `lora` and `expert_only` are mutually exclusive.	`'full'`
`lora_r`	`int \| None`	LoRA adapter rank, read only when `method="lora"`. A positive integer, or None to keep peft's default. It is the denominator of the `lora_alpha / lora_r` scaling, so anything else is refused by preflight.	`None`
`lora_alpha`	`int \| None`	Numerator of the LoRA `lora_alpha / lora_r` scaling, read only when `method="lora"`. A positive integer, or None to keep peft's default. Zero trains an adapter whose scaling is `0.0` and which therefore cannot change the model, so it is refused rather than run.	`None`
`lora_target_modules`	`str \| None`	Comma-separated module names the LoRA adapters are attached to, read only when `method="lora"`. Omit to keep the backend's default target set.	`None`
`tune`	`dict[str, bool] \| None`	Fine-grained component toggles for GR00T (`{"llm","visual","projector","diffusion"}`), honoured by `lerobot_local` with `extra={"policy_type": "groot"}`. A key naming no component (`vision` for `visual`) or a component the policy cannot freeze is refused by preflight, because an unforwarded toggle trains the config default and reports success.	`None`
`val_episodes`	`int \| None`	Hold out the LAST N episodes for validation; the run logs an eval loss over them at the checkpoint cadence. A positive integer below the dataset's episode count, or None for no held-out set - the split is a fraction lerobot takes the ceiling of, so a fractional count would reserve a different number of episodes. The count is read from the dataset's local `meta/info.json`, so a Hub source (`dataset_repo_id` with no populated `dataset_root`) is refused with the two ways to get a split instead of being launched without one.	`None`
`augmentation`	`dict[str, Any] \| None`	Backend-specific augmentation dict.	`None`
`fps`	`int \| None`	Dataset control rate (when a backend needs it).	`None`
`extra`	`dict[str, Any] \| None`	Backend-specific passthrough. lerobot: `policy_type`, `job_name`, any `--key=value`. Cosmos: `cosmos_root`, `sft_toml`. Isaac Lab: `task` (required, e.g. `"Isaac-Cartpole"`), `num_envs`, `physics` (`"newton_mjwarp"` / `"isaacsim_physx"`), `wait` (block until the run ends), `timeout_s`; `steps` is the PPO iteration count and `status` polls the returned `job_id`.	`None`
`job_id`	`str \| None`	Job identifier for `action="status"`, `"stop"`, `"play"` and `"record"`.	`None`

Returns:

Type	Description
`dict[str, Any]`	Canonical Strands result `{status, content:[...]}` (no sibling keys).
`dict[str, Any]`	For `train`/`status`/`export` the structured fields
`dict[str, Any]`	(`job_id`, `checkpoint_dir`, `exported_model`, `metrics`) are in
`dict[str, Any]`	a `{"json": ...}` block inside `content`, alongside a human-readable
`dict[str, Any]`	`{"text": ...}` block.

Dependencies (per provider - the base [lerobot] extra is not always enough): - lerobot_local + ACT/diffusion: pip install 'strands-robots[lerobot]'. - lerobot_local + smolvla/pi0/pi05: add lerobot's [smolvla]/[pi] extra on top of strands-robots[lerobot] (which pins lerobot>=0.6.0). Those extras layer transformers>=5.4.0,<5.6.0 (plus num2words / scipy); do NOT pin transformers==5.3.0 - it conflicts with lerobot 0.6's transformers floor. - lerobot_local + groot (GR00T N1.7): add lerobot's [groot] extra (pip install 'strands-robots[groot]'). - cosmos3: install the upstream package into THIS interpreter (the trainer imports it and calls its library functions in-process - no subprocess). Point extra['cosmos_root']/COSMOS_ROOT at the checkout for runtime config/recipe resolution. - torchcodec's .so must match the installed torch build exactly; a torch nightly load-fails a stable torchcodec (undefined symbol) and lerobot silently yields zero frames. See docs/reference/training/overview.md. - isaaclab: nothing in THIS interpreter. Install Isaac Lab in its own virtual environment and set ISAACLAB_PYTHON to its python and OMNI_KIT_ACCEPT_EULA=YES; the trainer runs its CLI as a subprocess. See docs/learn/training/isaaclab.md.

strands_robots.tools.lerobot_train.lerobot_train ¶

lerobot_train(dataset_root: str | None = None, tool_context: ToolContext | None = None, policy_type: str = 'act', pretrained_path: str | None = None, output_dir: str | None = None, job_name: str = 'strands_ft', steps: int = 20000, batch_size: int = 8, save_freq: int = 5000, device: str = 'cuda', dtype: str | None = None, gradient_checkpointing: bool = False, lora: bool = False, lora_r: int | None = None, lora_alpha: int | None = None, lora_target_modules: str | None = None, train_expert_only: bool = False, val_episodes: int | None = None, num_gpus: int = 1, push_to_hub: bool = False, resume: bool = False, action: str = 'start', session_name: str | None = None, extra_flags: dict[str, Any] | None = None) -> dict[str, Any]

Fine-tune a LeRobot policy on a local dataset by wrapping lerobot-train.

This closes the local record -> train -> deploy loop. After recording a LeRobot v3 dataset, call this with dataset_root pointing at the dataset directory (the one containing meta/info.json). On start it launches python -m lerobot.scripts.lerobot_train (or accelerate launch for num_gpus > 1) as a detached background process and tracks it in the same on-disk session store used by lerobot_teleoperate.

Memory-fit levers

lora and train_expert_only both freeze the VLM and are mutually exclusive; setting both fails fast. lora emits --peft.method_type=LORA plus the supplied --peft.* overrides. train_expert_only applies only to policies whose lerobot config exposes it (currently pi0/pi05/ smolvla; sourced live so it tracks lerobot).

Overfit guard

val_episodes=N reserves the LAST N episodes as a validation set by emitting --dataset.eval_split (the fraction that makes lerobot hold out exactly N) together with --eval_steps, so lerobot both keeps the tail out of training AND logs an eval loss over it at the checkpoint cadence. The episode and task counts are read from meta/info.json; a dataset with several tasks is refused because lerobot applies the split fraction per task, where a global count is not expressible. Passing dataset.eval_split or eval_steps in extra_flags overrides the derived value.

Resume

resume=True emits --config_path=<ckpt>/train_config.json --resume=true only when a checkpoint exists under <output_dir>/checkpoints/last. If no resumable checkpoint exists, a fresh run starts and a stale empty output_dir is cleared so lerobot's "already exists" guard does not trip.

Postures

resume, lora, train_expert_only, gradient_checkpointing and push_to_hub each select a posture rather than scaling a quantity, so each is checked rather than parsed: a truthy spelling of off - "false", "no", "0" - is refused before the run starts, instead of selecting the affirmative posture it reads as the opposite of. Only start reads them, so no other action is refused for one.

Actions

start: launch a new training run (default). status: report a run's PID, uptime, running flag, and recent log tail. stop: terminate a running session by name (SIGTERM then SIGKILL), reported successful only once the process has exited. list: list tracked training sessions.

Parameters:

Name	Type	Description	Default
`dataset_root`	`str \| None`	Local LeRobot v3 dataset directory (must contain meta/info.json). Read by `start` only, which refuses to launch without it; `status`, `stop` and `list` look a session up by name and never read it.	`None`
`policy_type`	`str`	Policy architecture (act, diffusion, vqbet, tdmpc, smolvla, pi0, pi05, pi0_fast, groot, xvla, ...).	`'act'`
`pretrained_path`	`str \| None`	HF id or local path to initialize weights from (gated checkpoints need HF_TOKEN in the environment).	`None`
`output_dir`	`str \| None`	Where to write run outputs; defaults to `<dataset_root>/../train_out/<job_name>`.	`None`
`job_name`	`str`	Run name used in the default output_dir and lerobot logs.	`'strands_ft'`
`steps`	`int`	Number of training steps.	`20000`
`batch_size`	`int`	Training batch size.	`8`
`save_freq`	`int`	Checkpoint save frequency in steps. A whole number; a non-positive value disables periodic saving (only the final checkpoint is written), and a fractional, non-finite, boolean or non-numeric cadence is refused before the run starts because lerobot decodes the flag into an `int` field.	`5000`
`device`	`str`	Torch device type, optionally with an index (cuda, cuda:0, cpu, mps). Refused before launch if torch cannot parse it; only the spelling is graded, so naming a device this machine does not have is allowed.	`'cuda'`
`dtype`	`str \| None`	Policy dtype (bfloat16, float32) for policies whose lerobot config declares a dtype field (e.g. the pi0 family, xvla). Default None lets lerobot pick; a policy whose installed config has no dtype field (ACT on lerobot 0.6.1) raises before launch.	`None`
`gradient_checkpointing`	`bool`	Trade compute for memory on supported policies.	`False`
`lora`	`bool`	Enable LoRA/PEFT fine-tuning (full-VLM fit on one GPU).	`False`
`lora_r`	`int \| None`	LoRA rank.	`None`
`lora_alpha`	`int \| None`	LoRA alpha (scaling = lora_alpha / r).	`None`
`lora_target_modules`	`str \| None`	PEFT target module spec (e.g. "all-linear").	`None`
`train_expert_only`	`bool`	Freeze the VLM, train only the action expert (policies exposing train_expert_only: pi0/pi05/smolvla).	`False`
`val_episodes`	`int \| None`	A positive integer below the dataset's episode count, or None for no held-out set. Reserves the LAST N episodes as a held-out validation split, evaluated every `save_freq` steps so each checkpoint has a validation loss beside it.	`None`
`num_gpus`	`int`	Number of GPUs; >1 launches via accelerate --multi_gpu.	`1`
`push_to_hub`	`bool`	Push the trained checkpoint to the HF Hub at the end. Publishing is an outward-facing action, so a true value requires operator approval through `tool_context`. A headless run pre-approves it with `STRANDS_TRAIN_EXTRA_FLAGS_ALLOW=policy.push_to_hub`: this parameter is gated under the `policy.`-prefixed key, the only spelling LeRobot accepts (`push_to_hub` is a field of its policy config, not of the train config). The bare `push_to_hub` allowlist entry clears the raw `extra_flags={'push_to_hub': True}` passthrough instead -- one flag, two spellings, two entries. The default false value emits the flag unchanged and is not gated.	`False`
`resume`	`bool`	Resume from the latest checkpoint under output_dir when present.	`False`
`action`	`str`	One of start, status, stop, list.	`'start'`
`session_name`	`str \| None`	Session identifier (auto-generated on start; required for status/stop).	`None`
`extra_flags`	`dict[str, Any] \| None`	Passthrough dict of additional lerobot-train flags, e.g. `{"policy.optimizer_lr": 1e-4}` -> `--policy.optimizer_lr=0.0001`. A key that abbreviates a gated flag is gated as that flag, because the trainer's parser honors unambiguous prefixes: `{"ou": "/x"}` reaches `output_dir`, so it needs the same approval, and the same `STRANDS_TRAIN_EXTRA_FLAGS_ALLOW=output_dir` entry clears it.	`None`

Returns:

Type	Description
`dict[str, Any]`	Dict with `status` ("success" or "error") and a `content` list of
`dict[str, Any]`	`{"text": ...}` items, plus action-specific keys (`session_name`,
`dict[str, Any]`	`pid`, `command`, `log_file`, `output_dir`, `sessions`,
`dict[str, Any]`	`is_running`, `uptime`). `uptime` is seconds, and is `None` when
`dict[str, Any]`	the session record states no usable start time - see
`dict[str, Any]`	func:`~strands_robots.tools._process_stop.session_uptime`, which is what
`dict[str, Any]`	the reported `Uptime` field says instead.

Assets and memory¶

strands_robots.tools.download_assets.download_assets ¶

download_assets(action: str = 'download', robots: str | None = None, category: str | None = None, force: bool = False) -> dict[str, Any]

Download and manage robot model assets (MJCF XML + meshes).

Assets are sourced from robot_descriptions (recommended by MuJoCo Menagerie, requires pip install strands-robots[sim-mujoco]). When robot_descriptions is unavailable, falls back to a shallow git clone of the Menagerie repo. Robots with a custom GitHub source in the registry are cloned from their respective repos.

Downloaded assets are cached in ~/.strands_robots/assets/ (override with STRANDS_ASSETS_DIR).

Parameters:

Name	Type	Description	Default
`action`	`str`	`download` \| `list` \| `status`. `status` marks each robot `[ok]` (assets present) or `[--]` (missing). A `download` that fetched nothing is reported as `status="error"` naming the cause - an unknown name, a failed clone, or a selection that matched no robot - rather than as a success reporting three zeros.	`'download'`
`robots`	`str \| None`	Comma-separated names (e.g. `so100,panda`). Omit for all. A non-empty value that names no robot (`","`) is refused rather than read as "all".	`None`
`category`	`str \| None`	Filter: arm, bimanual, hand, humanoid, mobile, mobile_manip	`None`
`force`	`bool`	Re-fetch a robot whose assets are already present, replacing the cached directory. A posture, so a non-boolean is refused rather than read by truthiness - `force="false"` would otherwise select the re-fetch it spells the skipping of.	`False`

strands_robots.tools.harness_memory.harness_memory ¶

harness_memory(action: str, task: str | None = None, trace: list[dict[str, Any]] | None = None, summary: dict[str, Any] | None = None, kind: str | None = None, text: str | None = None, backend: str | None = None, robot: str | None = None) -> dict[str, Any]

Persist and retrieve harness memory: task solution traces + global rules.

Task-Specific Memory stores HOW a task was solved (primitive ordering, where policy calls sit) - never WHERE objects happened to be. Loaded traces carry a re-grounding contract: reuse the procedural structure, but never replay literal coordinates; re-localize every object from the current observation. Global Memory stores cross-task success rules and failure models as plain text.

Actions

Task-Specific Memory: - "save_trace": Store (or replace) a task's solution trace + summary. Requires task, trace (list of action dicts - each entry must name a simulation action or registered tool in its "action" field), and summary (JSON object: strategy, what to avoid). Optional backend / robot are recorded as provenance alongside the library version. - "load_trace": Return a task's trace + summary with the re-grounding contract prepended. Requires task. - "list_tasks": List all stored task keys. - "delete_trace": Remove a task's trace + summary. Requires task.

Global Memory: - "append_rule": Append one plain-text rule. Requires kind ("success_rule" or "failure_model") and text (single line). - "load_rules": Return all success rules and failure models.

Storage: ~/.strands_robots/memory/ (override with STRANDS_MEMORY_DIR). Plain JSONL / JSON / text files - auditable, no embeddings; task keying is exact-name. The store assumes one agent session per store (no concurrent-writer coordination), and everything read back from disk is re-validated before it reaches the response.

Parameters:

Name	Type	Description	Default
`action`	`str`	Action to perform.	required
`task`	`str \| None`	Task key, matched exactly on later runs. Must match ^[a-zA-Z0-9_-]+\Z (max 128 chars; no dots, so use "task_v2" rather than "task.v2").	`None`
`trace`	`list[dict[str, Any]] \| None`	Solution skeleton: list of primitive-invocation dicts, one per step, e.g. {"action": "run_policy", "instruction": "grasp the bowl"}. Spatial values are reference bindings, not replay targets.	`None`
`summary`	`dict[str, Any] \| None`	Semantic summary: task description, strategy, pitfalls to avoid ("avoid" list), success flag.	`None`
`kind`	`str \| None`	Rule kind for append_rule: "success_rule" or "failure_model".	`None`
`text`	`str \| None`	Rule text for append_rule (single line, max 2000 chars).	`None`
`backend`	`str \| None`	Optional provenance: simulation backend the trace was collected on (e.g. "mujoco").	`None`
`robot`	`str \| None`	Optional provenance: robot the trace was collected with (e.g. "so100").	`None`

Returns:

Type	Description
`dict[str, Any]`	Dict containing status and response content.

ROS and mesh¶

strands_robots.tools.use_ros.use_ros ¶

use_ros(action: str, tool_context: ToolContext | None = None, topic: str | None = None, service: str | None = None, action_name: str | None = None, type: str | None = None, fields: dict[str, Any] | None = None, timeout: float = 5.0, count: int = 1, rate: float = 10.0) -> dict[str, Any]

Universal ROS 2 tool - in-process rclpy, dynamic types, no shelling out.

Parameters:

Name	Type	Description	Default
`action`	`str`	One of `status`, `list_topics`, `list_nodes`, `list_services`, `list_actions`, `info`, `echo`, `publish`, `service_call`, `action_send_goal`.	required
`tool_context`	`ToolContext \| None`	Injected agent context, used to ask an operator before a `publish`, `service_call` or `action_send_goal` reaches a safety-critical command surface.	`None`
`topic`	`str \| None`	Topic name (`echo`, `publish`, `info`).	`None`
`service`	`str \| None`	Service name (`service_call`, `info`).	`None`
`action_name`	`str \| None`	Action server name (`action_send_goal`), e.g. `/navigate_to_pose`.	`None`
`type`	`str \| None`	Fully-qualified interface type, e.g. `geometry_msgs/msg/Twist`, `turtlesim/srv/Spawn`, or `nav2_msgs/action/NavigateToPose`. Auto-resolved for `echo` when omitted.	`None`
`fields`	`dict[str, Any] \| None`	JSON field dict applied with `set_message_fields` (`publish`, `service_call`, `action_send_goal`). Booleans and nulls are preserved - the dict is passed straight to rclpy, never serialised through source.	`None`
`timeout`	`float`	Seconds to wait for samples / a service / an action result. For `action_send_goal` this is the end-to-end budget (discovery + acceptance + execution); size it to the goal (e.g. 120 for a Nav2 navigation), and note the goal is cancelled when it expires. A positive finite number of seconds.	`5.0`
`count`	`int`	Number of messages to echo or publish. A positive integer; it is consumed as a `range()` bound, so `0` publishes nothing and a float or a numeric string cannot be honored.	`1`
`rate`	`float`	Publish rate in Hz. A positive finite number - the inter-message period is `1 / rate`, so `0`, a negative value, `nan` and `inf` all leave the burst unthrottled rather than paced.	`10.0`

Returns:

Type	Description
`dict[str, Any]`	A Strands tool result dict `{"status": ..., "content": [{"text": ...}]}`.

strands_robots.tools.use_rosbridge.use_rosbridge ¶

use_rosbridge(action: str, host: str = 'localhost', port: int = 9090, topic: str | None = None, service: str | None = None, type: str | None = None, fields: dict[str, Any] | None = None, timeout: float = 5.0, count: int = 1, rate: float = 10.0, tool_context: ToolContext | None = None) -> dict[str, Any]

Universal rosbridge tool - ROS over a WebSocket, no ROS install needed.

Parameters:

Name	Type	Description	Default
`action`	`str`	One of `status`, `list_topics`, `list_services`, `echo`, `publish`, `service_call`.	required
`host`	`str`	rosbridge server hostname or IP. Held to the shared domain every dialled host in this package shares, then to this transport's own narrower allowlist.	`'localhost'`
`port`	`int`	rosbridge WebSocket port (default 9090).	`9090`
`topic`	`str \| None`	Topic name (`echo`, `publish`). Held to the same name rule as `service`.	`None`
`service`	`str \| None`	Service name (`service_call`). Held to the same name rule as `topic`.	`None`
`type`	`str \| None`	ROS1 two-segment interface type, e.g. `geometry_msgs/Twist`. Required for `publish` - a message cannot be built without it - and auto-resolved for `echo` when omitted.	`None`
`fields`	`dict[str, Any] \| None`	JSON field dict (`publish` message / `service_call` request).	`None`
`timeout`	`float`	Seconds for the WebSocket dial, sample collection, or a service call. A positive finite number of seconds; every action dials the bridge, so every action reads it.	`5.0`
`count`	`int`	Messages to echo or publish. A positive integer; it is consumed as a `range()` bound, so `0` publishes nothing and a float or a numeric string cannot be honored.	`1`
`rate`	`float`	Publish rate in Hz. A positive finite number - the inter-message period is `1 / rate`, so `0`, a negative value, `nan` and `inf` all leave the burst unthrottled rather than paced.	`10.0`
`tool_context`	`ToolContext \| None`	Injected agent context, used to ask an operator before a `publish` or `service_call` reaches a safety-critical command surface.	`None`

Returns:

Type	Description
`dict[str, Any]`	A Strands tool result dict `{"status": ..., "content": [{"text": ...}]}`.

strands_robots.tools.use_rtps.use_rtps ¶

use_rtps(action: str, topic: str | None = None, type: str | None = None, fields: dict[str, Any] | None = None, timeout: float = 5.0, count: int = 1, rate: float = 10.0, tool_context: ToolContext | None = None) -> dict[str, Any]

Pure-RTPS ROS 2 participant tool - no rclpy, all ROS 2 distros.

Parameters:

Name	Type	Description	Default
`action`	`str`	One of `status`, `types`, `advertise`, `publish`, `subscribe`, `echo`.	required
`topic`	`str \| None`	ROS 2 topic name, absolute (e.g. `/turtle1/cmd_vel`).	`None`
`type`	`str \| None`	ROS 2 interface type in the IDL bundle (e.g. `geometry_msgs/msg/Twist`). List with `action="types"`.	`None`
`fields`	`dict[str, Any] \| None`	JSON field dict for `publish`; nested message fields are built recursively. Booleans and nulls are preserved (plain Python values).	`None`
`timeout`	`float`	Seconds to wait for samples (`echo`). A positive finite number.	`5.0`
`count`	`int`	Number of messages to publish or samples to echo. A positive integer; it is consumed as a `range()` bound, so `0` publishes nothing and a float or a numeric string cannot be honored.	`1`
`rate`	`float`	Publish rate in Hz. A positive finite number - the inter-message period is `1 / rate`, so `0`, a negative value, `nan` and `inf` all leave the burst unthrottled rather than paced.	`10.0`
`tool_context`	`ToolContext \| None`	Injected agent context, used to ask an operator before a `publish` reaches a safety-critical command surface.	`None`

Returns:

Type	Description
`dict[str, Any]`	A Strands tool result dict `{"status": ..., "content": [{"text": ...}]}`.

strands_robots.tools.robot_mesh.robot_mesh ¶

robot_mesh(action: str, tool_context: ToolContext | None = None, target: str = '', instruction: str = '', command: str = '', policy_provider: str = 'mock', policy_port: int = 0, duration: float = 30.0, timeout: float = 30.0, name: str = '', limit: int = 50, function: str = '') -> dict[str, Any]

Coordinate every robot, sim, and agent on the local Zenoh mesh.

Parameters:

Name	Type	Description	Default
`action`	`str`	One of `peers` / `status` / `tell` / `send` / `ping` / `rpc` / `broadcast` / `stop` / `emergency_stop` / `subscribe` / `unsubscribe` / `watch` / `inbox`. `ping` asks one peer whether it is reachable and how fast (nothing on the robot is read or moved): `{"status": "ok", "latency_ms", "via": "direct"\|"publish"}`, or `offline` in one round trip when the AWS IoT broker knows the peer is gone. `rpc` calls a device's NATIVE Device Connect function (e.g. the Reachy's `nod` / `look` / `playMove`) directly, bypassing the policy-action allowlist that `tell` / `send` enforce. Pass the function name in `function` and any kwargs as a JSON object in `command`.	required
`target`	`str`	Peer id (for `tell` / `send` / `stop` / `watch`) or Zenoh topic pattern (for `subscribe`).	`''`
`instruction`	`str`	Natural-language instruction for `tell`.	`''`
`command`	`str`	JSON-encoded command body for `send` / `broadcast`.	`''`
`policy_provider`	`str`	Policy provider tag forwarded with `tell`.	`'mock'`
`policy_port`	`int`	Optional policy port forwarded with `tell`.	`0`
`duration`	`float`	Task duration (seconds) forwarded with `tell`.	`30.0`
`timeout`	`float`	Response timeout for RPC actions (seconds). A positive finite number; read by `tell` / `send` / `rpc` / `broadcast` / `stop` and ignored by the rest. `stop` additionally caps it at 5s. Zero or negative would report `{"status": "timeout"}` without waiting at all, so an unusable value is refused rather than reported as a peer that did not answer.	`30.0`
`name`	`str`	Optional subscription name for `subscribe` / `inbox`.	`''`
`limit`	`int`	Max messages returned by `inbox` (default: 50). A positive integer; read by `inbox` only.	`50`
`function`	`str`	Device-native function name for `rpc` (e.g. `nod`).	`''`

Returns:

Type	Description
`dict[str, Any]`	A Strands tool response dict with status and a single text block.

Examples::

robot_mesh(action="peers")
robot_mesh(action="tell", target="so100_sim-a1b2",
           instruction="pick up the cube")
robot_mesh(action="send", target="peer-b",
           command='{"action": "status"}')
robot_mesh(action="emergency_stop")    # raises a HITL interrupt;
                                       # runs only on operator approval

Safety controls

Human-in-the-loop interrupts for emergency_stop and broadcast. The tool calls tool_context.interrupt("robot_mesh-<action>-approval", reason=...) and only proceeds if the operator's response is an affirmative ("y" / "yes" / "approve"). The Strands SDK delivers the response out-of-band of the LLM's tool arguments, so prompt-injection that flips a boolean cannot bypass this gate.
Per-action sliding-window rate limit (e.g. emergency_stop is capped at 3 calls/min). Reject reason includes wait-time estimate.
send / broadcast payloads are validated through :func:strands_robots.mesh.security.validate_command before leaving the agent. The same validator runs on the receiver side, so a malformed or out-of-policy payload is rejected client-side before it hits the wire.
Every tell / send / broadcast / stop / emergency_stop / rpc is audited.

Edit page