Documentation
Held Object Manipulation

After the agent has picked up a Sim Object that is pickupable, these actions can be used to manipulate held items in various ways. These actions are only useable if the agent is initialized with agentControllerType = 'physics' on initialization.

Move Hand Forward

Moves the Agent’s hand and held object forward relative to the direction the agent is currently facing. The hand can only be moved if it is holding an object.

event = controller.step(action='MoveHandAhead', moveMagnitude=0.1)
Parameter Type Description Default
moveMagnitude float The distance, in meters, to move the hand in this direction 0.0
forceVisible bool By default, the agent can move held objects through the environment in a way that could cause the held object to become not visible. If set to True, the object in the hand will fail to move if it would move in a way that would make it not visible, preventing the agent from moving objects behind/inside other objects so that the moved object becomes not visible. False

Move Hand Back

Moves the Agent’s hand and held object backward relative to the direction the agent is currently facing. The hand can only be moved if it is holding an object.

event = controller.step(action='MoveHandBack', moveMagnitude=0.1)
Parameter Type Description Default
moveMagnitude float The distance, in meters, to move the hand in this direction 0.0
forceVisible bool By default, the agent can move held objects through the environment in a way that could cause the held object to become not visible. If set to True, the object in the hand will fail to move if it would move in a way that would make it not visible, preventing the agent from moving objects behind/inside other objects so that the moved object becomes not visible. False

Move Hand Left

Moves the Agent’s hand and held object left relative to the direction the agent is currently facing. The hand can only be moved if it is holding an object.

event = controller.step(action='MoveHandLeft', moveMagnitude=0.1)
Parameter Type Description Default
moveMagnitude float The distance, in meters, to move the hand in this direction 0.0
forceVisible bool By default, the agent can move held objects through the environment in a way that could cause the held object to become not visible. If set to True, the object in the hand will fail to move if it would move in a way that would make it not visible, preventing the agent from moving objects behind/inside other objects so that the moved object becomes not visible. False

Move Hand Right

Moves the Agent’s hand and held object right relative to the direction the agent is currently facing. The hand can only be moved if it is holding an object.

event = controller.step(action='MoveHandRight', moveMagnitude=0.1)
Parameter Type Description Default
moveMagnitude float The distance, in meters, to move the hand in this direction 0.0
forceVisible bool By default, the agent can move held objects through the environment in a way that could cause the held object to become not visible. If set to True, the object in the hand will fail to move if it would move in a way that would make it not visible, preventing the agent from moving objects behind/inside other objects so that the moved object becomes not visible. False

Move Hand Up

Moves the Agent’s hand and held object up relative to the direction the agent is currently facing. The hand can only be moved if it is holding an object.

event = controller.step(action='MoveHandUp', moveMagnitude=0.1)
Parameter Type Description Default
moveMagnitude float The distance, in meters, to move the hand in this direction 0.0
forceVisible bool By default, the agent can move held objects through the environment in a way that could cause the held object to become not visible. If set to True, the object in the hand will fail to move if it would move in a way that would make it not visible, preventing the agent from moving objects behind/inside other objects so that the moved object becomes not visible. False

Move Hand Down

Moves the Agent’s hand and held object down relative to the direction the agent is currently facing. The hand can only be moved if it is holding an object.

event = controller.step(action='MoveHandDown', moveMagnitude=0.1)
Parameter Type Description Default
moveMagnitude float The distance, in meters, to move the hand in this direction 0.0
forceVisible bool By default, the agent can move held objects through the environment in a way that could cause the held object to become not visible. If set to True, the object in the hand will fail to move if it would move in a way that would make it not visible, preventing the agent from moving objects behind/inside other objects so that the moved object becomes not visible. False

Move Hand Delta

Moves the agent hand constrained to x, y, z (axes relative to the agent’s current forward facing) with a given magnitude along each axis. This allows the agent to combine the above hand movements into a single action.

event = controller.step(action='MoveHandDelta', action.x=0.5, action.y=-0.5, action.z=0.5)
Parameter Type Description Default
action.x float The magnitude in the x direction relative to the agent’s forward facing to move the object 0.0
action.y float The magnitude in the y direction relative to the agent’s forward facing to move the object 0.0
action.z float The magnitude in the z direction relative to the agent’s forward facing to move the object 0.0
forceVisible bool By default, the agent can move held objects through the environment in a way that could cause the held object to become not visible. If set to True, the object in the hand will fail to move if it would move in a way that would make it not visible, preventing the agent from moving objects behind/inside other objects so that the moved object becomes not visible. False

Rotate Hand

Rotates the hand and held object about the specified axes (x, y, z) to the specified degree. These examples rotate a held object to 90 degrees about each axis.

event = controller.step(action='RotateHand', x=90)
event = controller.step(action='RotateHand', y=90)
event = controller.step(action='RotateHand', z=90)

Multiple Axes can be specified at once as well.

event = controller.step(action='RotateHand', x=90, y=-15, z=28)
Parameter Type Description Default
x float rotation about the object’s x axis 0.0
y float rotation about the object’s y axis 0.0
z float rotation about the object’s z axis 0.0