An agent is a capsule shaped entity that can navigate within scenes and interact with objects.


A scene within AI2-THOR represents a virtual room that an agent can navigate in and interact with. There are 4 scene Categories, each with 30 unique Scenes within them: Kitchen, Living Room, Bedroom, Bathroom.


An action is a discrete command for the Agent to perform within a scene (e.g. MoveAhead, RotateRight, PickupObject)

Sim Object

A sim object is an object that the can be interacted with by the Agent. Objects have a range of interactions based on affordanced defined by the Object Type.

Object Visibility

An object is considered Visible when it satisfies three conditions: It must be within the Camera’s viewport (camera field of view), it must be within a radius around the agent’s central vertical axis (default: 1.5 meters radius), and a ray emitted from the camera must hit the object without first hitting another obstruction.

A note about the “threshold of distance” for the second condition: The initialization value used to determine visibility distance is visibilityDistance. This value defines the radius of a cylinder centered about the y axis of the agent.

Note that an object rendered in an image will not always be Visible to the Agent. For example, an object outside the 1.5 meter threshold could be seen in an image, but will be reported as not-visible to the Agent.

Object Interactability

An object is said to be Interactable if it is flagged as Visible and if it is unobstructed by any other objects. Most objects will be Interactable as long as they are also Visible, but some objects have transparency which can cause objects to be reported as Visible through them. An example is a Glass Shower Door with a Sponge object behind it. The glass door will be flagged as Visible and Interactable, but the sponge will only be Visible. Attempting to interact with the sponge will throw an error as it cannot be reached through the glass door, only seen.


A type of object that can contain another object. Some examples of receptacles are: TableTop, Cup, Sofa, Bed, Desk, Bowl, etc. Some Receptacles cannot be moved within the scene, and are mostly large objects that don’t make sense to move (Countertop, Sink, etc.). Some Receptacles can also open and close (Microwave, Cabinet, Drawer, etc.) while others can also be picked up and moved around by the Agent (Plate, Bowl, Box, etc.).