Actions

The agent within THOR can perform several actions that allow it to interact with the scenes within the environment. New actions such as turning on faucet or slicing a loaf of bread can be easily added to the API. Actions are defined in unity/Assets/Scripts/DiscreteRemoteFPSAgentController.cs. This page shows the state of different actionable objects.

Prior to running the commands below it is assumed you have run the following:

import ai2thor.controller
controller = ai2thor.controller.Controller()
controller.start()
# can be any one of the scenes FloorPlan###
controller.reset('FloorPlan28')
controller.step(dict(action='Initialize', gridSize=0.25))

Initialize

Initialize must be called after resetting a scene to set fields such as gridSize.

controller.step(dict(action='Initialize', gridSize=0.25))
Parameter Type Description Default
gridSize float Size of the grid that the agent navigates in. This determines the step size that the agent takes when the actions MoveAhead, MoveBack, MoveRight, MoveLeft are taken. 0.25
renderDepthImage bool When enabled a depth image is sent and made available on the returned Event as the attribute depth_frame. False
renderClassImage bool When enabled a class segmentation image is sent and made available on the returned Event as the attribute class_segmentation_frame. False
renderObjectImage bool When enabled an object segmentation image is sent and made available on the returned Event as the attribute instance_segmentation_frame. False
visibilityDistance float Distance in meters from the agent’s camera (positioned near the top of the agent) that an object should be considered visibile. 1.0
cameraY float Height of the camera attached to the agent. 0.675


RandomInitialize

This is used to shuffle the objects in a scene as well as increase the number of objects in a scene. A convenience function on the controller wraps the call to RandomInitialize.

controller.random_initialize(
    random_seed=None,
    randomize_open=False,
    unique_object_types=False,
    exclude_receptacle_object_pairs=[],
    max_num_repeats=1,
    remove_prob=0.5)
Parameter Type Description Default
random_seed int Optional random seed passed to the randomizer used to make the randomization deterministic for reproducing scene configurations. None
randomize_open bool Whether to randomly open/close receptacles that are openable. False
unique_object_types bool When True only one instance of an object type (mug, bowl, knife, etc.) is placed into the scene. False
exclude_receptacle_object_pairs list list of dicts of the form dict(receptacleObjectId=, objectId=) that is used to prevent the specified objects from being placed in a receptacle. []
max_num_repeats int If this number is greater than 1, a random number between 1 and max_num_repeats is picked for each object to determine the number of copies of an object to place in a scene. 1
remove_prob float Probability that a pickupable object will be removed. Setting this value to 1.0 would remove all pickupable objects and setting to 0.0 would remove no objects. 0.5


MoveAhead

Move the agent forward by the gridSize.

controller.step(dict(action='MoveAhead'))


MoveBack

Move the agent backward by the gridSize.

controller.step(dict(action='MoveBack'))


MoveRight

Move the agent right by the gridSize (without changing the viewing direction).

controller.step(dict(action='MoveRight'))


MoveLeft

Move the agent left by the gridSize (without changing the viewing direction).

controller.step(dict(action='MoveLeft'))


LookUp

Increase the tilt angle the agent is looking by 30 degrees.

controller.step(dict(action='LookUp'))


LookDown

Decrease the tilt angle the agent is looking by 30 degrees.

controller.step(dict(action='LookDown'))


RotateRight

Rotate the agent by 90 degrees to the right.

controller.step(dict(action='RotateRight'))


RotateLeft

Rotate the agent by 90 degrees to the left.

controller.step(dict(action='RotateLeft'))


OpenObject

Open an object (assuming object is visible to the agent). In the case of the Refrigerator, the door will open.

controller.step(dict(action='OpenObject', objectId="Fridge|0.25|0.75|0.05"))


CloseObject

Close an object (assuming object is visible to the agent). In the case of the Refrigerator, the door will close.

controller.step(dict(action='CloseObject', objectId="Fridge|0.25|0.75|0.05"))


PickupObject

Pick up a visible object and place it into the Agent’s inventory. Currently the Agent can only have a single object in its inventory. See below for a more complex example.

controller.step(dict(action='PickupObject', objectId="Mug|0.25|-0.27|0.95"))


PutObject

Put an object in the Agent’s inventory into a visible receptacle. In order for this to work, the agent must pick up a visible Mug and open a visible Fridge.

controller.step(dict(
    action="PutObject",
    objectId="Mug|0.25|-0.27|0.95",
    receptacleObjectId="Fridge|0.05|0.75|0.05"))


Teleport

Move the agent to any location in the scene. Using this command it is possible to put the agent into places that would not normally be possible to navigate to, but it can be useful if you need to place an agent in the exact same spot for a task.

controller.step(dict(action='Teleport', x=0.999, y=1.01, z=-0.3541))


TeleportFull

Move the agent to any location in the scene. Using this command it is possible to put the agent into places that would not normally be possible to navigate to, but it can be useful if you need to place an agent in the exact same spot for a task. Identical to Telport, but also allows rotation and horizon to be passed in.

controller.step(dict(action='TeleportFull', x=0.999, y=1.01, z=-0.3541, rotation=90.0, horizon=30.0))

Failures

It is possible depending on where the agent is located within a scene that an action fails. In the case of the Move actions, it will likely be due to some collision with an object in the agent’s path. PutObject or PickupObject failures are due to the visibility of either the target object to be picked up or the receptacle object in the case of PutObject.

Next Steps

Continue on to the Event/Metadata tutorial.