Event Metadata

Each call to the controller.step() function returns an Event object that contains a rich amount of information about the state of the environment and each of the objects within the environment.

import ai2thor.controller
controller = ai2thor.controller.Controller()
controller.start()
# can be any one of the scenes FloorPlan###
controller.reset('FloorPlan28')
event = controller.step(dict(action='Initialize', gridSize=0.25))


# Numpy Array (width, height, channels) channels are in RGB order
event.frame

# Numpy Array in BGR order suitable for use with OpenCV
event.cv2image()


Metadata attributes

# retrieved by using the instance variable 'metadata'
event.metadata
Attribute Type Description Example
agent agent attributes pertaining to agent’s location, camera position and rotation  
errorMessage string string explaining why the last action failed (if lastActionSuccess is false)  
lastAction string The action that was issued to the agent to generate the response MoveAhead
lastActionSuccess boolean True/False whether the last action suceeded True
objects array of objects Array of all objects in the scene  
screenHeight number Height of the image rendered by Unity 300
screenWidth number Width of the image rendered by Unity 300
sequenceId number Used to ensure that commands and responses are aligned  


Agent attributes

event.metadata['agent']
Attribute Type Description Example
cameraHorizon float Position of camera relative to the horizon. 0.0 is looking straight ahead, 30.0 degrees is looking down by 30 degrees and 330 is looking up by 30.0 degrees. 0.0
position vector3 X,Y,Z coordinates of the agent in the world reference frame  
rotation vector3 X,Y,Z rotations of the agent in degrees in global space  


Object attributes

Attribute Type Description Example
distance float Distance from centerpoint of object to the agent’s camera 3.541793
isopen boolean Boolean indicating whether the object is open or closed True
name string Name of the object, not guaranteed to be unique  
objectId string Unique id for the object within the scene TableTop|-02.08|+00.94|-03.62
openable boolean Boolean indicating whether the object can be opened, such as a cabinet or drawer True
pickupable boolean Boolean indicating whether the object can be picked up by the agent. It will only be possible to actually pick up the object if it is also visible to the agent True
receptacle boolean Boolean indicating whether the object is a receptacle that can contain other objects True
position vector3 X,Y,Z coordinates of the object in global space  
rotation vector3 X,Y,Z rotations of the object in degrees in global space  
visible boolean Boolean indicating whether the object is visible to the agent True
receptacleCount number If an object is a receptacle, there are a number of predefined positions within the receptacle where objects can be placed. This number is the total of those positions. 3
receptacleObjectIds array of strings If the object is a receptacle, this is an array of objectIds that the receptacle contains  
pivotSimObjects array of pivotSimObjects Each receptacle contains a number of locations or pivots that can contain objects. This list identifies which objects are in what pivots.  


PivotSimObject attributes

Attribute Type Description Example
objectId string objectId of the object in this particular pivot Lettuce|-00.33|+00.74|-00.69
pivotId number Pivot position in the receptacle 1


Vector3 attributes

Attribute Type Description Example
x float    
y float    
z float    

Next Steps

Continue on to the Object Types tutorial.