This page aims to provide basic starting examples to begin using AI2-THOR.
A simple example that moves the agent ahead by one step and returns the corresponding image and metadata.
from ai2thor.controller import Controller # Kitchens: FloorPlan1 - FloorPlan30 # Living rooms: FloorPlan201 - FloorPlan230 # Bedrooms: FloorPlan301 - FloorPlan330 # Bathrooms: FloorPLan401 - FloorPlan430 controller = Controller(scene='FloorPlan28', gridSize=0.25) event = controller.step(action='MoveAhead') # Numpy Array - shape (width, height, channels), channels are in RGB order event.frame # Numpy Array in BGR order suitable for use with OpenCV event.cv2image # current metadata dictionary that includes the state of the scene event.metadata # shuts down the controller controller.stop()
An alternate approach to initialize a controller is to use a context manager. Here, the previous MoveAhead example could be written as:
from ai2thor.controller import Controller with Controller(scene='FloorPlan28', gridSize=0.25) as c: event = c.step(action='MoveAhead') event.frame event.cv2image event.metadata
If there is ever a bug inside of the
with command or everything inside of the
with command executes, the Controller will immediately stop.
Our goal is to pickup a mug, then open a microwave, and then put the mug in microwave. To pickup an object the agent must first navigate to an area where there are pickupable/visible objects. Normally, it should be done through a series of
RotateRight commands. Here we are teleporting directly to a known location that a mug exists.
from ai2thor.controller import Controller controller = Controller(scene='FloorPlan28', gridSize=0.25) # change starting locations controller.step(action='Teleport', x=-2.5, y=0.900998235, z=-3.0) controller.step(action='LookDown') event = controller.step(action='Rotate', rotation=180) # in FloorPlan28, the agent should now be looking at a mug for o in event.metadata['objects']: if o['visible'] and o['pickupable'] and o['objectType'] == 'Mug': # pick up the mug event = controller.step(action='PickupObject', objectId=o['objectId'], raise_for_failure=True) mug_object_id = o['objectId'] break # the agent now has the Mug in its inventory # to put it into the Microwave, we need to open the microwave first # move to the microwave event = controller.step(action='LookUp') event = controller.step(action='RotateLeft') event = controller.step(action='MoveLeft', moveMagnitude=0.25 * 4) event = controller.step(action='MoveAhead', moveMagnitude=0.25 * 6) # the agent should now be looking at the microwave for o in event.metadata['objects']: if o['visible'] and o['openable'] and o['objectType'] == 'Microwave': # open the microwave event = controller.step(action='OpenObject', objectId=o['objectId'], raise_for_failure=True) receptacle_object_id = o['objectId'] break # put the object in the microwave event = controller.step( action='PutObject', receptacleObjectId=receptacle_object_id, objectId=mug_object_id, raise_for_failure=True) # close the microwave event = controller.step( action='CloseObject', objectId=receptacle_object_id, raise_for_failure=True)
This example shows how to run AI2-THOR in the multi-agent setting.
from ai2thor.controller import Controller controller = Controller() # agentCount specifies the number of agents in a scene event = controller.step(action='Initialize', gridSize=0.25, agentCount=2) # print out agentIds for e in event.events: print(e.metadata['agentId']) # move the second agent ahead, agents are 0-indexed multi_agent_event = controller.step(action='MoveAhead', agentId=1)
This example shows how to run multiple instances of an agent in a multi-threaded fashion.
import threading import time from ai2thor.controller import Controller thread_count = 8 episodes = 100 def run(): controller = Controller(scene='FloorPlan1', gridSize=0.25) for _ in range(episodes): t_start = time.time() controller.reset('FloorPlan1') print('init time', time.time() - t_start) t_start_total = time.time() for _ in range(10): controller.step(action='MoveAhead') controller.step(action='RotateRight') total_time = time.time() - t_start_total print('total time', total_time, 20 / total_time, 'fps') threads = [threading.Thread(target=run) for _ in range(thread_count)] for t in threads: t.daemon = True t.start() time.sleep(1) for t in threads: # calling join() in a loop/timeout to allow for Python 2.7 # to be interrupted with SIGINT while t.isAlive(): t.join(1) print('done')