Documentation
Actionable Properties

Objects in this framework can have a number of Actionable Properties associated with the object type. Actionable Properties are specific properties that have an Action associated with them. For example, an object that is Openable means that the OpenObject and CloseObject actions can be used to interact with that object. Below is a list of all Actionable Properties and their detailed descriptions.

Openable

These objects can be opened or closed using the OpenObject and CloseObject actions. Receptacles that are Openable allow for objects to be placed inside them.

All Openable objects will return openable = True in their object metadata to indicate that they have this property. The object metadata also indicates an Openable object’s current open/closed state. If isOpen = True, the object is open. If isOpen = False, the object is closed.

Pickupable

These objects are able to be picked up or put down into Receptacles objects by the Agent using the PickupObject and PutObject actions. Picked up objects can also be dropped using DropHandObject which will remove the object from the Agent’s hand without needing a target receptacle. Throw is an extension of dropping an object, where an additional force is added to throw object about the scene. Pickupable objects can also be shoved around using the Push and Pull actions. Some Receptacle objects are also pickupable. Any objects placed inside a pickupable receptacle are moved all at once, so complex sequences like (Put Apple on Plate -> Move Plate with Apple to Sink) are possible.

All Pickupable objects will return pickupable = True in their object metadata to indicate that they have this property. The object metadata also indicates a Pickupable object’s current state of whether it is currently picked up by the agent or not. If isPickedUp = True the agent is actively holding the object. If isPickedUp = False then the object is not being picked up by the agent.

Moveable

These are non-static objects that can be moved around the scene by using actions like Push and Pull. They can also be repositioned due to forces from other objects, such as using Throw to toss a Pickupable object at a Moveable object. The key difference between Moveable and Pickupable objects are that Moveable objects are too large to be held in the agent’s hand as Pickupable objects are, so actions like PickupObject will not work on Moveable objects.

All Moveable objects will return moveable = True in their object metadata to indicate that they have this property. The object metadata also indicates if an object is currently in motion. If isMoving = True then the object is actively moving. If isMoving = False the object is not moving. Note that both Moveable and Pickupable objects can be moving, so the isMoving metadata value is not restricted to only Moveable objects.

Toggleable (On/Off)

Interact with these objects using the ToggleObjectOn and ToggleObjectOff actions. All Toggleable objects have a visible state change that occurs when toggled on or off (i.e., laptop screen will be on with an image or blank when off, a lamp will be lit when on and dim when off).

All Toggleable objects will return toggleable = True in their object metadata to indicate that they have this property. The object metadata also indicates if a Toggleable object is currently toggled On or Off. If isToggled = True, the object is turned on. If isToggled = False, the object is turned off.

Receptacle

Receptacle objects allow other objects to be placed on or in them if the other object can physically fit the receptacle. Some receptacles are restricted to specific object types (i.e. ToiletPaper is the only type that can go on a ToiletPaperHanger). Note that any receptacles that are also Pickupable can be moved about the scene with any sim objects they actively contain (ie: Pick up a Plate with an Apple on it to move both the Plate and Apple around the scene simultaneously).

All Receptacle objects will return receptacle = True in their object metadata to indicate they have this property. The object metadata also indicates if a Receptacle object contains any other objects via the receptacleObjectIds metadata string list, which lists all objectIds of any sim objects contained by the Receptacle. Additionally, objects report back any receptacles that they are contained by via the parentReceptacles list of strings in the object metadata.

Fillable

These objects can be filled with various liquids using the FillObjectWithLiquid action. Fillable objects can be filled with Water, Coffee, or Wine. If an object is filled with one of the liquid types and is rotated greater than 90 degrees from the global upward axis, the object will automatically empty itself because the liquid has “spilled.” Additionally, some fill interactions are context sensative. For example, placing an empty mug in a coffee maker object that is turned on will automatically fill the mug with coffee. Similarly, moving an empty mug under a running faucet will automatically fill the mug with water.

All Fillable objects will return canFillWithLiquid = True in their object metadata to indicate they have this property. The object metadata also indicates if an object is actively filled with a liquid. If isFilledWithLiquid = True the object is filled by some liquid. If isFilledWithLiquid = False, the object is empty.

Sliceable

These objects can be sliced into smaller pieces using the SliceObject action. This destroys the source object and spawns in multiple “sliced” pieces of the source object in the exact same location. This is a one-way state change, so only a scene reset will revert sliced objects to their whole versions. Sliced objects will still report metadata information even after being destroyed. This allows you to check the last position the source object was before the Slice action finished.

All Sliceable objects will return sliceable = True in their object metadata to indicate if they have this property. The object metadata also indicates if a Sliceable object has been sliced or not. If isSliced = True the object has been sliced and is no longer interactable (although the pieces it spawns are interactable). If isSliced = False the object has not been sliced and is whole.

Cookable

These objects have a cooked state that can be switched to with the CookObject action. This is a one-way state change, so only a scene reset will revert cooked objects back to their uncooked state. Cookable objects can interact with objects that are heat sources (canChangeTempToHot = True) to automatically change the object to a cooked state. This means interactions like turning on a Microwave with a Potato in it will turn the Potato to the cooked state without using the CookObject action.

All Cookable objects will return cookable = True in their object metadata to indicate that they have this property. The object metadata also indicates if a Cookable object is currently cooked. If isCooked = True, the object is cooked. If isCooked = False, the object is not cooked.

Breakable

These objects have a broken state that can be switched to with the BreakObject action. This is a one-way state change, so only a scene reset will revert broken objects to their unbroken state. Breakable objects also break automatically if they collide with a high enough force. This force threshold is different between objects because some objects are more fragile than others. This automatic breaking can be toggled off via the MakeObjectsOfTypeUnbreakable action. When some objects break, they will shatter into pieces that will not be interacatble, as the fractured remains of the object are not themselves sim objects. Objects that can shatter like certain Cup or Vase objects made of glass or ceramic will shatter in this way when broken. Other objects may remain interactable in a limited way after they break. A Laptop, for example, when broken will have its screen become cracked. You will still be able to open, close, and pickup the Laptop if it is broken, but you will no longer be able to toggle it on.

All Breakable objects will return breakable = True in their object metadata to indicate that they have this property. The object metadata also indicates if a Breakable object is currently broken. If isBroken = True, the object has broken. If isBroken = False, the object is whole and not broken.

Dirty

These objects have a clean and dirty state that can be toggled between using the DirtyObject action. This includes objects like Mugs that can have grime on them, or a Bed that has the covers messy or made. Certain Dirty objects can contextually be switched to their clean state if they are moved under running water. (a grimy Bowl moved under a Faucet that is toggled on will clean the bowl automatically).

All Dirty objects will return dirtyable = True in their object metadata to indicate that they have this property. The object metadata also indicates if a Dirty object is currently in the dirty or clean state. If isDirty = True, the object is in its dirty state. if isDirty = False, the object is in its clean state.

UsedUp

These objects can have parts of themselves used up with the UseUpObject action. This is a one-way interaction and can not be reversed unless the scene is reset. This can change the overall look of the object to show that its contents have been “Used Up.” ToiletPaper, TissueBox, and PaperTowel objects are examples of objects that, when used upp, change form in some way.

All UsedUp objects will return canBeUsedUp = True in their object metadata to indicate that they have this property. The object metadata also indicates if a UsedUp object is currently used up or full. If isUsedUp = True, the object has been used up. If isUsedUp = False, the object is full.

Table of Object Actionable Properties

If an Actionable Property has an * next to it, it means not all instances of that object type are gauranteed to have that property. Please check a specific object instance’s Object Metadata for details on if that particular instance has a given property.

# Object Type Actionable Properties
1 AlarmClock Pickupable
2 AluminumFoil Pickupable
3 Apple Pickupable, Sliceable
4 AppleSliced Pickupable
5 ArmChair Receptacle, Moveable
6 BaseballBat Pickupable
7 BasketBall Pickupable
8 Bathtub Receptacle*
9 BathtubBasin Receptacle
10 Bed Receptacle, Dirty
11 Blinds Openable
12 Book Openable, Pickupable
13 Boots Pickupable
14 Bottle Pickupable, Fillable, Breakable
15 Bowl Pickupable, Receptacle, Fillable, Breakable*, Dirty
16 Box Openable, Pickupable, Receptacle
17 Bread Pickupable, Sliceable
18 BreadSliced Pickupable, Cookable
19 ButterKnife Pickupable
20 Cabinet Openable, Receptacle
21 Candle Pickupable, Toggleable
22 CD Pickupable
23 CellPhone Pickupable, Toggleable, Breakable
24 Chair Moveable
25 Cloth Pickupable, Dirty
26 CoffeeMachine Toggleable, Receptacle, Moveable
27 CoffeeTable Receptacle, Moveable
28 CounterTop Receptacle
29 CreditCard Pickupable
30 Cup Pickupable, Receptacle, Fillable, Breakable*, Dirty
31 Curtains -
32 Desk Receptacle, Moveable
33 DeskLamp Toggleable, Moveable
34 Desktop Moveable
35 DiningTable Receptacle, Moveable
36 DishSponge Pickupable
37 DogBed Moveable
38 Drawer Openable, Receptacle
39 Dresser Receptacle, Moveable
40 Dumbbell Pickupable
41 Egg Pickupable, Sliceable, Breakable
42 EggCracked Pickupable, Cookable
43 Faucet Toggleable
44 Floor -
45 FloorLamp Toggleable, Moveable
46 Footstool Moveable
47 Fork Pickupable
48 Fridge Openable, Receptacle
49 GarbageBag Moveable
50 GarbageCan Receptacle, Moveable
51 HandTowel Pickupable
52 HandTowelHolder Receptacle
53 HousePlant Fillable, Moveable
54 Kettle Openable, Pickupable, Fillable
55 KeyChain Pickupable
56 Knife Pickupable
57 Ladle Pickupable
58 Laptop Openable, Pickupable, Toggleable, Breakable
59 LaundryHamper Receptacle, Moveable
60 Lettuce Pickupable, Sliceable
61 LettuceSliced Pickupable
62 LightSwitch Toggleable
63 Microwave Openable, Toggleable, Receptacle, Moveable
64 Mirror Breakable, Dirty
65 Mug Pickupable, Receptacle, Fillable, Breakable, Dirty
66 Newspaper Pickupable
67 Ottoman Receptacle, Moveable
68 Painting -
69 Pan Pickupable, Receptacle, Dirty
70 PaperTowel Pickupable, UsedUp
71 Pen Pickupable
72 Pencil Pickupable
73 PepperShaker Pickupable
74 Pillow Pickupable
75 Plate Pickupable, Receptacle, Breakable*, Dirty
76 Plunger Pickupable
77 Poster -
78 Pot Pickupable, Receptacle, Fillable, Dirty
79 Potato Pickupable, Sliceable, Cookable
80 PotatoSliced Pickupable, Cookable
81 RemoteControl Pickupable
82 RoomDecor Moveable
83 Safe Openable, Receptacle, Moveable
84 SaltShaker Pickupable
85 ScrubBrush Pickupable
86 Shelf Receptacle
87 ShelvingUnit Moveable
88 ShowerCurtain Openable
89 ShowerDoor Openable, Breakable
90 ShowerGlass Breakable
91 ShowerHead Toggleable
92 SideTable Receptacle, Moveable
93 Sink Receptacle*
94 SinkBasin Receptacle
95 SoapBar Pickupable
96 SoapBottle Pickupable, UsedUp
97 Sofa Receptacle, Moveable
98 Spatula Pickupable
99 Spoon Pickupable
100 SprayBottle Pickupable
101 Statue Pickupable, Breakable
102 Stool Moveable
103 StoveBurner Toggleable, Receptacle
104 StoveKnob Toggleable
105 TableTopDecor Pickupable
106 TargetCircle -
107 TeddyBear Pickupable
108 Television Toggleable, Breakable, Moveable
109 TennisRacket Pickupable
110 TissueBox Pickupable, UsedUp
111 Toaster Toggleable, Receptacle, Moveable
112 Toilet Openable, Receptacle
113 ToiletPaper Pickupable, UsedUp
114 ToiletPaperHanger Receptacle
115 Tomato Pickupable, Sliceable
116 TomatoSliced Pickupable
117 Towel Pickupable
118 TowelHolder Receptacle
119 TVStand Receptacle, Moveable
120 VacuumCleaner Moveable
121 Vase Pickupable, Breakable
122 Watch Pickupable
123 WateringCan Pickupable, Fillable
124 Window Breakable
125 WineBottle Pickupable, Fillable, Breakable

Default Receptacle Restrictions

By default, all Pickupable objects can only be placed in a limited set of Receptacle typed objects via the PutObject or InitialRandomSpawn actions. These actions have this default behavior so that objects will only be placed in receptacles that make sense (ie: it is unnatural to find a Egg inside a Cabinet, but it makes sense to find one in a Fridge).

Object Type Default Compatible Receptacles
AlarmClock Box, Dresser, Desk, SideTable, DiningTable, TVStand, CoffeeTable, CounterTop, Shelf
Apple Pot, Pan, Bowl, Microwave, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, Desk, CounterTop, GarbageCan, Dresser
AppleSliced Pot, Pan, Bowl, Microwave, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, Desk, CounterTop, GarbageCan, Dresser
BaseballBat Bed, DiningTable, TVStand, CoffeeTable, SideTable, Desk, CounterTop
BasketBall Sofa, ArmChair, Dresser, Desk, Bed, DiningTable,TVStand, CoffeeTable, SideTable, CounterTop
Book Sofa, ArmChair, Box, Ottoman, Dresser, Desk, Bed, Cabinet, DiningTable,TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer
Boots -
Bottle Fridge, Box, Dresser, Desk, Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, GarbageCan
Bowl Microwave, Fridge, Dresser, Desk, Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf
Box Sofa, ArmChair, Dresser, Desk, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Ottoman
Bread Microwave, Fridge, DiningTable, TVStand, CoffeeTable, SideTable, Desk, CounterTop, GarbageCan, Plate
BreadSliced Microwave, Fridge, DiningTable,TVStand, CoffeeTable, SideTable, Desk, CounterTop, GarbageCan, Toaster, Plate
ButterKnife Pot, Pan, Bowl, Mug, Plate, Cup, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, Desk, CounterTop, Drawer
Candle Box, Dresser, Desk, Toilet, Cart, Bathtub, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer
CD Box, Ottoman, Dresser, Desk, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan, Safe, Sofa, ArmChair
CellPhone Sofa, ArmChair, Box, Ottoman, Dresser, Desk, Bed, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, Safe
Cloth Sofa, ArmChair, Box, Ottoman, Dresser, LaundryHamper, Desk, Toilet, Cart, BathtubBasin, Bathtub, Sink, SinkBasin, Cabinet, DiningTable, TVStand,CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
CreditCard Sofa, ArmChair, Box, Ottoman, Dresser, Desk, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, Shelf
Cup Microwave, Fridge, Dresser, Desk, Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf
DishSponge Pot, Pan, Bowl, Plate, Box, Toilet, Cart, Cart, BathtubBasin, Bathtub, Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
Egg Pot, Pan, Bowl, Microwave, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, GarbageCan
EggCracked Pot, Pan, Bowl, Microwave, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, GarbageCan
Fork Pot, Pan, Bowl, Mug, Plate, Cup, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Drawer
HandTowel HandTowelHolder
Kettle DiningTable,TVStand, CoffeeTable, SideTable, CounterTop, Sink, SinkBasin, Cabinet, StoveBurner, Shelf
KeyChain Sofa, ArmChair, Box, Ottoman, Dresser, Desk, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, Safe
Knife Pot, Pan, Bowl, Mug, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Drawer
Ladle Pot, Pan, Bowl, Plate, Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Drawer
Laptop Sofa, ArmChair, Ottoman, Dresser, Desk, Bed, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop
Lettuce Pot, Pan, Bowl, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, GarbageCan
LettuceSliced Pot, Pan, Bowl, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, GarbageCan
Mug SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf
Newspaper Sofa, ArmChair, Ottoman, Dresser, Desk, Bed, Toilet, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
Pan DiningTable, CounterTop, TVStand, CoffeeTable, SideTable, Sink, SinkBasin, Cabinet, StoveBurner, Fridge
PaperTowelRoll Box, Toilet, Cart, Bathtub, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, GarbageCan
Pen Mug, Box, Dresser, Desk, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
Pencil Mug, Box, Dresser, Desk, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
PepperShaker DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Drawer, Cabinet, Shelf
Pillow Sofa, ArmChair, Ottoman, Bed
Plate Microwave, Fridge, Dresser, Desk, Sink,Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf
Plunger Cart, Cabinet
Pot StoveBurner, Fridge, Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf
Potato Pot, Pan, Bowl, Microwave, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, GarbageCan
PotatoSliced Pot, Pan, Bowl, Microwave, Fridge, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, GarbageCan
RemoteControl Sofa, ArmChair, Box, Ottoman, Dresser, Desk, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer
SaltShaker DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Drawer, Cabinet, Shelf
SoapBar Toilet, Cart, Bathtub, BathtubBasin, Sink, SinkBasin, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
SoapBottle Dresser, Desk, Toilet, Cart, Bathtub, Sink, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
Spatula Pot, Pan, Bowl, Plate, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Drawer
Spoon Pot, Pan, Bowl, Mug, Plate, Cup, Sink, SinkBasin, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Drawer
SprayBottle Dresser, Desk, Toilet, Cart, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
Statue Box, Dresser, Desk, Cart, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Safe
TeddyBear Bed, Sofa, ArmChair, Ottoman, Dresser, Desk, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Safe
TennisRacket Dresser, Desk, Bed, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop
TissueBox Box, Dresser, Desk, Toilet, Cart, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
ToiletPaper Dresser, Desk, Toilet, ToiletPaperHanger, Cart, Bathtub, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
ToiletPaperRoll Dresser, Desk, Toilet, ToiletPaperHanger, Cart, Bathtub, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, GarbageCan
Tomato DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Sink, SinkBasin, Pot, Bowl, Fridge, GarbageCan, Plate
TomatoSliced DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Sink, SinkBasin, Pot, Bowl, Fridge, GarbageCan, Plate
Towel TowelHolder
Vase Box, Dresser, Desk, Cart, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Safe
Watch Box, Dresser, Desk, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer, Safe
WateringCan Dresser, Desk, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, Drawer
WineBottle Fridge, Dresser, Desk, Cabinet, DiningTable, TVStand, CoffeeTable, SideTable, CounterTop, Shelf, GarbageCan