Documentation
Examples

This page aims to provide basic starting examples to begin using AI2-THOR.

MoveAhead

A simple example that moves the agent ahead by one step and returns the corresponding image and metadata.

from ai2thor.controller import Controller

# Kitchens: FloorPlan1 - FloorPlan30
# Living rooms: FloorPlan201 - FloorPlan230
# Bedrooms: FloorPlan301 - FloorPlan330
# Bathrooms: FloorPLan401 - FloorPlan430

controller = Controller(scene='FloorPlan28', gridSize=0.25)

event = controller.step(action='MoveAhead')

# Numpy Array - shape (width, height, channels), channels are in RGB order
event.frame

# Numpy Array in BGR order suitable for use with OpenCV
event.cv2image

# current metadata dictionary that includes the state of the scene
event.metadata

# shuts down the controller
controller.stop()

Context Managers

An alternate approach to initialize a controller is to use a context manager. Here, the previous MoveAhead example could be written as:

from ai2thor.controller import Controller

with Controller(scene='FloorPlan28', gridSize=0.25) as c:
    event = c.step(action='MoveAhead')
    event.frame
    event.cv2image
    event.metadata

If there is ever a bug inside of the with command or everything inside of the with command executes, the Controller will immediately stop.

Object interaction

Our goal is to pickup a mug, then open a microwave, and then put the mug in microwave. To pickup an object the agent must first navigate to an area where there are pickupable/visible objects. Normally, it should be done through a series of MoveAhead, RotateLeft, RotateRight commands. Here we are teleporting directly to a known location that a mug exists.

from ai2thor.controller import Controller
controller = Controller(scene='FloorPlan28', gridSize=0.25)

# change starting locations
controller.step(action='Teleport', x=-2.5, y=0.900998235, z=-3.0)
controller.step(action='LookDown')
event = controller.step(action='Rotate', rotation=180)

# in FloorPlan28, the agent should now be looking at a mug
for o in event.metadata['objects']:
    if o['visible'] and o['pickupable'] and o['objectType'] == 'Mug':
        # pick up the mug
        event = controller.step(action='PickupObject',
                                objectId=o['objectId'],
                                raise_for_failure=True)
        mug_object_id = o['objectId']
        break

# the agent now has the Mug in its inventory
# to put it into the Microwave, we need to open the microwave first

# move to the microwave
event = controller.step(action='LookUp')
event = controller.step(action='RotateLeft')
event = controller.step(action='MoveLeft', moveMagnitude=0.25 * 4)
event = controller.step(action='MoveAhead', moveMagnitude=0.25 * 6)

# the agent should now be looking at the microwave
for o in event.metadata['objects']:
    if o['visible'] and o['openable'] and o['objectType'] == 'Microwave':
        # open the microwave
        event = controller.step(action='OpenObject',
                                objectId=o['objectId'],
                                raise_for_failure=True)
        receptacle_object_id = o['objectId']
        break

# put the object in the microwave
event = controller.step(
    action='PutObject',
    receptacleObjectId=receptacle_object_id,
    objectId=mug_object_id,
    raise_for_failure=True)

# close the microwave
event = controller.step(
    action='CloseObject',
    objectId=receptacle_object_id,
    raise_for_failure=True)

Multi-agent

This example shows how to run AI2-THOR in the multi-agent setting.

from ai2thor.controller import Controller
controller = Controller()

# agentCount specifies the number of agents in a scene
event = controller.step(action='Initialize', gridSize=0.25, agentCount=2)

# print out agentIds
for e in event.events:
    print(e.metadata['agentId'])

# move the second agent ahead, agents are 0-indexed
multi_agent_event = controller.step(action='MoveAhead', agentId=1)

Multi-threaded

This example shows how to run multiple instances of an agent in a multi-threaded fashion.

import threading
import time
from ai2thor.controller import Controller

thread_count = 8
episodes = 100

def run():
    controller = Controller(scene='FloorPlan1', gridSize=0.25)
    for _ in range(episodes):
        t_start = time.time()
        controller.reset('FloorPlan1')
        print('init time', time.time() - t_start)
        t_start_total = time.time()
        for _ in range(10):
            controller.step(action='MoveAhead')
            controller.step(action='RotateRight')
        total_time = time.time() - t_start_total
        print('total time', total_time, 20 / total_time, 'fps')

threads = [threading.Thread(target=run) for _ in range(thread_count)]
for t in threads:
    t.daemon = True
    t.start()
    time.sleep(1)

for t in threads:
    # calling join() in a loop/timeout to allow for Python 2.7
    # to be interrupted with SIGINT
    while t.isAlive():
        t.join(1)

print('done')