Documentation
Agent Navigation

The agent can use these actions to manipulate itself and navigate through the environment.

RotateRight

Rotates the agent to the right of its current facing direction. The default rotation amount is determined by rotateStepDegrees set on initialization. This default can be overwritten by passing in degrees set to a non-zero value. Note that if agentControllerType = 'stochastic' is set on initialize, the applyActionNoise bool will allow this action to automatically add noise based on initialize gaussians.

event = controller.step(action='RotateRight', degrees=90.0)
Parameter Type Description Default
degrees float The number of degrees to rotate right relative to the agent’s current forward. This overrides the default rotation value initialized by rotateStepDegrees 90.0
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

RotateLeft

Rotates the agent to the left of its current facing direction. The default rotation amount is determined by rotateStepDegrees set on initialization. This default can be overwritten by passing in degrees set to a non-zero value. Note that if agentControllerType = 'stochastic' is set on initialize, the applyActionNoise bool will allow this action to automatically add noise based on initialize gaussians.

event = controller.step(action='RotateLeft', degrees=90.0)
Parameter Type Description Default
degrees float The number of degrees to rotate left relative to the agent’s current forward. This overrides the default rotation value initialized by rotateStepDegrees 90.0
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

LookUp

Angles the agent’s view up in degree degree increments (max upward angle is 30 degrees above the forward horizon. Increments must be multiples of of tenths of a degree).

event = controller.step(action='LookUp', degrees=25.1)
Parameter Type Description Default
degrees float The number of degrees to rotate the agent’s view upward. This value is automatically clamped to the nearest tenth of a degree 30.0
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

LookDown

Angles the agent’s view down in degree degree increments (max downward angle is 60 degrees below the forward horizon. Increments must be multiples of of tenths of a degree).

event = controller.step(action='LookDown', degrees=45.6)
Parameter Type Description Default
degrees float The number of degrees to rotate the agent’s view downward. This value is automatically clamped to the nearest tenth of a degree 30.0
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

MoveAhead

Moves the agent forward by gridSize.

event = controller.step(action='MoveAhead')
Parameter Type Description Default
moveMagnitude float Specify move distance and overwrite gridSize value 0.0 (will default to gridSize)
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

MoveRight

Moves the agent right by gridSize (without changing its view direction).

event = controller.step(action='MoveRight')
Parameter Type Description Default
moveMagnitude float Specify move distance and overwrite gridSize value 0.0 (will default to gridSize)
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

MoveLeft

Moves the agent left by gridSize (without changing its view direction).

event = controller.step(action='MoveLeft')
Parameter Type Description Default
moveMagnitude float Specify move distance and overwrite gridSize value 0.0 (will default to gridSize)
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

MoveBack

Moves the agent backward by gridSize (without changing its view direction).

event = controller.step(action='MoveBack')
Parameter Type Description Default
moveMagnitude float Specify move distance and overwrite gridSize value 0.0 (will default to gridSize)
manualInteract bool If set to True, any object picked up by the agent will not automatically snap to the default position in front of the agent upon movement, but instead will remain wherever the agent hand has minipulated it. Use this in conjunction with a PickupObject action that also uses manualInteract = True to more realistically simulate picking up and moving objects about the scene without the default abstraction of the object teleporting to the agent’s hand. False

Crouch

Only useable if agent is initialized to agentMode = 'default' and agentControllerType = 'physics'. Makes the Agent crouch, lowering the camera angle. Note this does not change the Agent’s collision, so crouching will not allow the agent to go under things that would block it while standing.

event = controller.step(action='Crouch')

Stand

Only useable if agent is initialized to agentMode = 'default' and agentControllerType = 'physics'. Makes the Agent Stand if it is currently crouching, returning the Camera position to default.

event = controller.step(action='Stand')

Get Reachable Positions

Sets actionReturn in the event’s metadata to an array of xyz dictionary. This array gives valid position coordinates that the Agent can reach without colliding with the environment or Sim Objects in the current scene. This can be used in tandem with Teleport or TeleportFull actions to move the Agent as needed.

event = controller.step(action='GetReachablePositions')

Teleport

Moves the agent to any location in the scene. Using this command it is possible to put the agent into places that would not normally be possible to navigate to, but it can be useful if you need to place an agent in the exact same spot for a task.

controller.step(action='Teleport', x=0.999, y=1.01, z=-0.3541)
Parameter Type Description Default
x float x coordinate in 3D scene space 0.0
y float y coordinate in 3D scene space 0.0
z float z coordinate in 3D scene space 0.0

TeleportFull

Moves the agent to any location in the scene. Using this command it is possible to put the agent into places that would not normally be possible to navigate to, but it can be useful if you need to place an agent in the exact same spot for a task. Identical to Telport, but also allows rotation and horizon to be passed in.

Note that if using a rotation that is not an increment of 90.0 degrees, Movement actions will still automatically snap the agent’s position to the closest grid point based on the initialized gridSize unless snapToGrid is set to False on scene initialization.

event = controller.step(action='TeleportFull', x=0.999, y=1.01, z=-0.3541, rotation=dict(x=0.0, y=90.0, z=0.0), horizon=30.0)
Parameter Type Description Default
x float x coordinate in 3D scene space 0.0
y float y coordinate in 3D scene space 0.0
z float z coordinate in 3D scene space 0.0
rotation xyz dictionary Rotation about the Y-axis to change the forward orientation of the Agent relative to world X/Z axes. Modify the Y value of the xyz dictionary to change rotation (ie: 0, 90, 0).  
horizon float Rotation about the X axis to change the Up/Down look angle of the Agent. Any angle can be used here, but values of -30.0, 0.0, 30.0, 60.0 will mimic the maximum and minimum angles used by the LookUp and LookDown actions 0.0

The horizon angle values describe the rotation about the Agent’s X-axis. This axis has “right hand” facing with respect to the forward Z-Axis, and because of this the values are slightly misleading as the (-30.0) horizon will actually angle the agent’s forward Z direction 30 degrees upward. Because horizon values describe changes about the X-Axis, positive and negative angles can result in the same end position.

Horizon Angle Value Change in Forward Z
-30.0 (330.0) Look 30 Degrees Up
0.0 Look straight ahead
30.0 Look 30 Degrees Down
60.0 Look 60 Degrees Down

FlyAhead

Only useable if the agent is in agentMode = 'drone'. Fly the drone forward by adding thrust of the given magnitude in the agent’s current forward facing direction.

event = controller.step(action='FlyAhead', moveMagnitude=1)
Parameter Type Description Default
moveMagnitude float magnitude of thrust added 0.0

FlyBack

Only useable if the agent is in agentMode = 'drone'. Fly the drone backward by adding thrust of the given magnitude in the agent’s current backward direction relative to the agent’s current forward orientation.

event = controller.step(action='FlyBack', moveMagnitude=1)
Parameter Type Description Default
moveMagnitude float magnitude of thrust added 0.0

FlyLeft

Only useable if the agent is in agentMode = 'drone'. Fly the drone left by adding thrust of the given magnitude in the agent’s current left direction relative to the agent’s current forward orientation.

event = controller.step(action='FlyLeft', moveMagnitude=1)
Parameter Type Description Default
moveMagnitude float magnitude of thrust added 0.0

FlyRight

Only useable if the agent is in agentMode = 'drone'. Fly the drone right by adding thrust of the given magnitude in the agent’s current right direction relative to the agent’s current forward orientation.

event = controller.step(action='FlyRight', moveMagnitude=1)
Parameter Type Description Default
moveMagnitude float magnitude of thrust added 0.0

FlyUp

Only useable if the agent is in agentMode = 'drone'. Fly the drone up by adding thrust of the given magnitude in the agent’s current upward direction.

event = controller.step(action='FlyUp', moveMagnitude=1)
Parameter Type Description Default
moveMagnitude float magnitude of thrust added 0.0

FlyDown

Only useable if the agent is in agentMode = 'drone'. Fly the drone downd by adding thrust of the given magnitude in the agent’s current downard direction.

event = controller.step(action='FlyUp', moveMagnitude=1)
Parameter Type Description Default
moveMagnitude float magnitude of thrust added 0.0

FlyTo

Only useable if the agent is in agentMode = 'drone'. Adds a velocity vector to the agent’s current velocity to simulate thrust. This can also simultaneously rotate the drone’s yaw and change the drone’s camera pitch.

event = controller.step(action='FlyTo', x=1, y=1, z=1, rotation=dict(x=0, y=90, z=0), horizon=30)
Parameter Type Description Default
x float x value of the velocity vector for thrust 0.0
y float y value of the velocity vector for thrust 0.0
z float z value of the velocity vector for thrust 0.0
rotation xyz dictionary Only change the y value of this dictionary in order to rotate the drone’s yaw. The x and z components will be ignored. 0.0
horizon float The value to change the drone’s pitch 0.0