Actions

The agent within THOR can perform several actions that allow it to interact with the scenes within the environment. New actions such as turning on faucet or slicing a loaf of bread can be easily added to the API. Actions are defined in unity/Assets/Scripts/DiscreteRemoteFPSAgentController.cs. This page shows the state of different actionable objects.

Prior to running the commands below it is assumed you have run the following:

import ai2thor.controller
controller = ai2thor.controller.Controller()
controller.start()
# can be any one of the scenes FloorPlan###
controller.reset('FloorPlan28')
controller.step(dict(action='Initialize', gridSize=0.25))

MoveAhead

Move the agent forward by the gridSize.

controller.step(dict(action='MoveAhead'))

MoveBack

Move the agent backward by the gridSize.

controller.step(dict(action='MoveBack'))

MoveRight

Move the agent right by the gridSize (without changing the viewing direction).

controller.step(dict(action='MoveRight'))

MoveLeft

Move the agent left by the gridSize (without changing the viewing direction).

controller.step(dict(action='MoveLeft'))

LookUp

Increase the tilt angle the agent is looking by 30 degrees.

controller.step(dict(action='LookUp'))

LookDown

Decrease the tilt angle the agent is looking by 30 degrees.

controller.step(dict(action='LookDown'))

RotateRight

Rotate the agent by 90 degrees to the right.

controller.step(dict(action='RotateRight'))

RotateLeft

Rotate the agent by 90 degrees to the left.

controller.step(dict(action='RotateLeft'))

OpenObject

Open an object (assuming object is visible to the agent). In the case of the Refrigerator, the door will open.

controller.step(dict(action='OpenObject', objectId="Fridge|0.25|0.75"))

CloseObject

Close an object (assuming object is visible to the agent). In the case of the Refrigerator, the door will close.

controller.step(dict(action='CloseObject', objectId="Fridge|0.25|0.75"))

PickupObject

Pick up a visible object and place it into the Agent’s inventory. Currently the Agent can only have a single object in its inventory. See below for a more complex example.

controller.step(dict(action='PickupObject', objectId="Mug|0.25|-0.27"))

PutObject

Put an object in the Agent’s inventory into a visible receptacle. In order for this to work, the agent must pick up a visible Mug and open a visible Fridge.

controller.step(dict(
    action="PutObject",
    objectId="Mug|0.25|-0.27",
    receptacleObjectId="Fridge|0.05|0.75"))

Teleport

Move the agent to any location in the scene. Using this command it is possible to put the agent into places that would not normally be possible to navigate to, but it can be useful if you need to place an agent in the exact same spot for a task.

controller.step(dict(action='Teleport', x=0.999, y=1.01, z=-0.3541))

Failures

It is possible depending on where the agent is located within a scene that an action fails. In the case of the Move actions, it will likely be due to some collision with an object in the agent’s path. PutObject or PickupObject failures are due to the visibility of either the target object to be picked up or the receptacle object in the case of PutObject.

Next Steps

Continue on to the Event/Metadata tutorial.