Documentation
Miscellaneous Actions

Pass

This is a Noop Action that does nothing to change the agent or environment, but does generate an event object with metadata about the scene.

event = controller.step(action='Pass')
Parameter Type Description Default
objectId string the string unique id of the target object null

AddThirdPartyCamera

Add a third party camera to the scene. See the Metadata section for information on how to receive the frame.

controller.step(
    action='AddThirdPartyCamera', 
    rotation=dict(x=0, y=90, z=0), 
    position=dict(x=-1.25, y=1.0, z=-1.0),
    fieldOfView=60
    )
Parameter Type Description Default
rotation xyz dictionary x, y, z coordinates for camera rotation  
position xyz dictionary x, y, z coordinates for camera position  
fieldOfView float camera field of view, valid values are (0-180) non inclusive 90

UpdateThirdPartyCamera

Update the position/rotation of a thirdPartyCamera. Both rotation and position are required.

controller.step(
    action='UpdateThirdPartyCamera',
    thirdPartyCameraId=0, # id is available in the metadata response
    rotation=dict(x=0, y=90, z=0),
    position=dict(x=-1.25, y=1.0, z=-1.5)
    )
Parameter Type Description Default
rotation xyz dictionary x, y, z coordinates for camera rotation  
position xyz dictionary x, y, z coordinates for camera position  

Spawn Target Circle

Spawns a TargetCircle object in a Receptacle object in the scene. TargetCircle objects have no collision and will not react to other objects. A TargetCircle itself is Receptacle object, and it can be used to return metadata for any “contained” objects that are on the circle (see metadata’s receptacleObjectIds). TargetCircles can be used as target locations to move other Sim Objects to using various physics interactions (Push, Pull, DirectionalPush etc.).

event = controller.step(action = 'SpawnTargetCircle', objectId = "Table|+1.0|+1.0|+1.0", minDistance = 1.0, anywhere=True, objectVariation=0)
Parameter Type Description Default
minDistance float The minimum distance away from the agent circles can appear. This will enforce a TargetCircle spawns a minimum threshold away from the center of the agent. 0.0
anywhere bool This flags whether all receptacles in the scene will be considered as a valid spawn position for a TargetCircle. By default, anywhere=False which makes circles spawn only within the field of view of the agent. Set to True to allow circles to spawn outside of the agent’s field of view. False
objectId string The string id of a specific receptacle in the scene to spawn a circle on. Only receptacle objects are valid targets to spawn a circle in. Note if anywhere=True the target receptacle object does not need to be in view to spawn a circle in. 0.0
objectVariation int Valid values are: 0, 1, 2. Each corresponds to a different size of TargetCircle to spawn in- small, medium, large respectively. 0

Set Mass Properties

Change mass properties of any Pickupable or Moveable object.

event = controller.step(action = 'SetMassProperties', objectId = "Apple|+1.0|+1.0|+1.0", mass = 22.0, drag=15.0, angularDrag=5.6)
Parameter Type Description Default
objectId string The string id of a Pickupable or Moveable sim object.  
mass float The mass to change the object’s mass to in kilograms 0.0
drag float A value used to slow an object’s motion down. Higher values will slow the object down more 0.0
angularDrag float A value used to slow an object’s rotational motion down. Higher values will slow the object’s rotation down more 0.0

Pause Physics Auto-Simulation

Only to be used if the agent is initialized with agentMode = 'default' and agentControllerType = 'physics'. Pause the physics update from continuing in real time. This can be used in tandem with AdvancePhysicsStep to return metadata event objects that breakdown a physics interaction into stepwise actions. Normally, without puasing the physics simulation and stepping through sections of time, a physics action like Push will only return an metadata event object at the end of the pushed object’s motion. This can be used to capture segments of the motion in progress by pausing physics, and then executing a series of AdvancePhysicsStep actions to generate many frames.

event = controller.step(action = 'PausePhysicsAutoSim')

Advance Physics Step

Only to be used if the agent is initialized with agentMode = 'default' and agentControllerType = 'physics'. This can be used to manually advance the physics update by a specified timeStep after physics simulation has been paused with PausePhysicsAutoSim. Object metadata like whether an object is in motion or not can be tracked on a per-object basis after each step.

event = controller.step(action = 'AdvancePhysicsStep', timeStep=0.05)
Parameter Type Description Default
timeStep float The amount of time in seconds to advance the physics simulation manually. Valid values must be greater than 0.0 and no greater than 0.05 0.01

Unpause Physics Auto-Simulation

Only to be used if the agent is initialized with agentMode = 'default' and agentControllerType = 'physics'. Use this to unpause the physics update and allow it to continue in real time. Note that any in-progress changes to the environment will not be reflected in this action’s metadata return. If physics auto-simulation is unpaused while an object is still moving (inMotion = True), the metadata return will not wait for the object to stop moving, but will instead return immediately. This will cause the remaining motion of that object to be lost. To capture the entire movement of an object, do not unpause physics with this action until all objects return inMotion = False in their object metadata or if the boolean isSceneAtRest returns True in the event metadata.

event = controller.step(action = 'UnpausePhysicsAutoSim')

Remove Object from Scene

Remove an object from the current scene by objectId. This can be used to edit scenes by removing specific objects if desired.

event = controller.step(action='RemoveFromScene', objectId="Mug|0.25|-0.27")
Parameter Type Description Default
objectId string the string unique id of the target object null

Disable Object

Disable an object from being active and visible in the scene. Unlike the RemoveFromScene action, this does not permenantly destroy an object in the scene, as it can be toggled back on later in the episode using EnableObject.

event = controller.step(action = 'DisableObject', objectId=DiningTable|+1.0|+1.0|+1.0)
Parameter Type Description Default
objectId string The string id of the sim object to disable. Note that this may cause unintended interactions with other objects after an object is disabled. For example, a table with a Plate and Apple resting on top of it will cause both the Plate and Apple to fall to the floor if the table object supporting them is disabled.  

Enable Object

Activates an object if it has been previously disabled by DisableObject.

event = controller.step(action = 'EnableObject', objectId=DiningTable|+1.0|+1.0|+1.0)
Parameter Type Description Default
objectId string The string id of the sim object to reactivate. Note that this may cause unintended interactions with other objects after an object is reactivated. Enabled objects will return to their original location, which may cause clipping or weird collision with other objects if another sim object or the agent has moved into the area where the enabled object is.  

SpawnDroneLauncher

Only to be used if the agent is initialized to agentMode = 'drone' and agentControllerType = 'drone'. Spawn a drone object launcher that acts as the origin point for spawning drone-objects. Drone-objects are made to be caught in the agent’s basket if the agent has been initialized in agentMode = 'drone'. After spawning a drone object launcher, the LaunchDroneObject action can be used to spawn objects fromt he drone launcher’s position.

event = controller.step(action='SpawnDroneLauncher', position=dict(x=1, y=1.8, z=1.5))
Parameter Type Description Default
position xyz dictionary x, y, z coordinates to spawn the drone launcher  

LaunchDroneObject

Only to be used if the agent is initialized to agentMode = 'drone' and agentControllerType = 'drone'. After spawning in a drone launcher with SpawnDroneLauncher, launch objects from the drone launcher’s position with a given magnitude and in the specified direction.

event = controller.step(action='LaunchDroneObject', moveMagnitude=30, x=0.7, y=0.7, z=-1, objectName='Apple', objectRandom=True))
Parameter Type Description Default
moveMagnitude float force magnitude to launch the drone object 0.0
x float x direction to launch the object 0.0
y float y direction to launch the object 0.0
z float z direction to launch the object 0.0
objectName string the name of the object type that can be launched by the drone launcher. Valid object types are: ‘Cup’, ‘Toaster’, ‘Bowl’, ‘WateringCan’, ‘Book’, ‘SaltShaker’, ‘Bottle’, ‘Vase’, ‘Statue’, ‘SoapBottle’, ‘ToiletPaper’, ‘AlarmClock’, ‘Bread’, ‘Newspaper’, ‘TissueBox’, ‘BasketBall’, ‘Apple’, ‘Mug’, ‘Lettuce’, ‘Candle’  
objectRandom bool If set to True the object type specified by objectName will select a random instance of the object type from the library of all variations of that object type False

FlyAssignStart

Only to be used if the agent is initialized to agentMode = 'drone' and agentControllerType = 'drone'. A helper function to be used after initializing the agent in agentMode = 'drone' and spawning a drone object launcher with SpawnDroneLauncher. This moves both the drone and the object launcher to different coordinates in world space.

event = controller.step(action='FlyAssignStart', x=1, y=1.5, z=1, position=dict(x=1, y=1.8, z=1.5))
Parameter Type Description Default
x float x position to move the agent to 0.0
y float y position to move the agent to 0.0
z float z position to move the agent to 0.0
position xyz dictionary x, y, z coordinates to move the drone object launcher to  

FlyRandomStart

Only to be used if the agent is initialized to agentMode = 'drone' and agentControllerType = 'drone'. Selects random positions in the scene from valid positions found the same was as the GetReachablePositions action and teleports the drone and launcher to that position. The object launcher will be in front of the drone with [1.5, 2.5] longitudinal distance and [-0.5, 0.5] horizontal distance with respective of drone. Note that you do not need to call the GetReachablePositions action to use FlyRandomStart.

event = controller.step(action='FlyRandomStart', y=1.5)
Parameter Type Description Default
y float y distance off the ground to move the drone and launcher to. The x and z coordinates are determined randomly from the set of valid positions in the scene. 0.0

ChangeDronePositionRandomNoiseSigma

Only to be used if the agent is initialized to agentMode = 'drone' and agentControllerType = 'drone'. Change the standard deviation of the drone’s movement noise to 0.1. The noise is defined by a Gaussin distribuion. The default is 0.0

event = controller.step(action='ChangeDronePositionRandomNoiseSigma', dronePositionRandomNoiseSigma=0.01)
Parameter Type Description Default
dronePositionRandomNoiseSigma float standard deviation of the drone’s movement noise 0.0

ChangeFixedDeltaTime

Only to be used if the agent is initialized to agentMode = 'drone' and agentControllerType = 'drone'. Change the update cycle of the Unity Physics Engine while in drone mode. Drone mode pauses the physics update to sync up with the drone movements, to simulate continuous thrust. The default is 0.02.

event = controller.step(action='ChangeFixedDeltaTime', fixedDeltaTime=0.02)
Parameter Type Description Default
fixedDeltaTime float the update cycle of the Unity Physics Engine while in drone mode 0.02