Tell the Robot What You Want — Sandhya Subramani, AWS
An AWS demo treats a robot as an agent tool: an LLM selects existing hardware policies while separate agents handle thinking, communication, and voice. The live failures expose the safety gap.
The rover uses **three concurrent Strands agents** for environmental reasoning, text communication, and voice. The agent chooses which existing robot policy to invoke; execution spans cloud training and edge runtime, and the package is said to support **40+ robots across eight categories**.
The transferable pattern is to expose constrained hardware capabilities as tools and keep “what to do” separate from the policy that determines “how.” Coding-agent builders should apply explicit tool descriptions, narrow permissions, and guardrails at that boundary.
The rover uses **three concurrent Strands agents** for environmental reasoning, text communication, and voice. The agent chooses which existing robot policy to invoke; execution spans cloud training and edge runtime, and the package is said to support **40+ robots across eight categories**. The transferable pattern is to expose constrained hardware capabilities as tools and keep “what to do” separate from the policy that determines “how.” Coding-agent builders should apply explicit tool descriptions, narrow permissions, and guardrails at that boundary. The live rover fell from the stage and sometimes spoke without performing the requested action. The demo shows flexible orchestration, but not reliable planning, physical safety, or that untrained motor capabilities emerge from the agent layer.
This makes the tool boundary concrete in robotics: the agent selects among existing policies, but does not thereby acquire new motor competence or safe physical planning. The failed live behavior reinforces prior arguments for narrow authority, external guardrails, and evidence of execution, while adding that a valid request, spoken acknowledgement, and completed physical action are distinct states that the harness must verify.