Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
reasoning_about_objects [2013/01/15 15:55] – [Reasoning about objects] tenorthreasoning_about_objects [2014/06/05 11:38] (current) – external edit 127.0.0.1
Line 1: Line 1:
-====== Reasoning about objects ====== +#REDIRECT doc:reasoning_about_objects
- +
-Before starting this tutorial, you should have completed the following ones: +
-  * [[Tutorial: KnowRob basics]] +
-  * [[Loading files and ROS packages]] +
- +
-===== Load environment specification ===== +
-We have split the description of the semantic environment map from the set of (smaller) objects in the environment. In realistic applications, the map will be rather static, while the set of objects is usually detected by the robot during operation and updated over time. With the following commands, you can launch KnowRob including the semantic map of the IAS kitchen, and then load the OWL file containing a set of demo objects: +
-<code> +
- roscd knowrob_tutorial +
- rosrun rosprolog rosprolog knowrob_tutorial +
- owl_parse('owl/ccrl2_map_objects.owl', false, false, true). +
-</code> +
-If you need to create or edit a semantic map, you can either use the graphical [[Mod_semantic_map#SemanticMapEditor|Semantic Map editor]] or the [[Mod_semantic_map#SemanticMapToOWL|SemanticMapToOWL service]] using the SemMap message. +
- +
-===== Load CAD model definition for objects ===== +
- +
-Some of the objects in the demo set have CAD models attached. They are used for visualization as well as for reasoning (see below). These models are automatically used when they are specified for either a complete class of objects (using an OWL hasValue restriction) or for an object instance as shown below: +
-<code> +
- <owl:NamedIndividual rdf:about="&map_obj;milk1"> +
-     <rdf:type rdf:resource="&knowrob;CowsMilk-Product"/> +
-     <knowrob:widthOfObject rdf:datatype="&xsd;double">0.1</knowrob:widthOfObject> +
-     <knowrob:depthOfObject rdf:datatype="&xsd;double">0.1</knowrob:depthOfObject> +
-     <knowrob:heightOfObject rdf:datatype="&xsd;double">0.25</knowrob:heightOfObject> +
-     <knowrob:pathToCadModel rdf:datatype="&xsd;string"> +
-                             package://knowrob_tutorial/cad/milk.kmz</knowrob:pathToCadModel> +
- </owl:NamedIndividual> +
-</code> +
- +
-The %%package://...%% identifier references files inside ROS packages. +
- +
-===== Visualize environment including object CAD models ===== +
- +
-{{ :map-fmi-hospital-room.png?nolink&400|}} +
- +
-Once the respective CAD model definitions are loaded, they are automatically used for visualizing object instances instead of the default primitive models (boxes, cylinders etc). You should see a map including CAD models such as the following when calling: +
-<code> +
- visualisation_canvas(_). +
-</code> +
-Similarly, we can also visualize small objects in the kitchen: +
-<code> +
- (owl_individual_of(A, knowrob:'FoodUtensil'); +
-  owl_individual_of(A, knowrob:'FoodVessel'); +
- owl_individual_of(A, knowrob:'FoodOrDrink')), +
- add_object(A, _). +
-</code> +
-===== Query for objects with certain properties ===== +
- +
-Since all objects in the map are instances of the respective object classes, one can query for objects that have certain properties or belong to a certain class, for example: +
-<code> +
- % perishable objects: +
- ?- owl_individual_of(A, knowrob:'Perishable'). +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#butter1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#buttermilk1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#cheese1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#milk1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#yogurt1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#sausage1' +
-  +
- % all HandTools (e.g. silverware) +
- ?- owl_individual_of(A, knowrob:'HandTool'). +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#knife1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#fork1' +
-  +
- % all FoodVessels (i.e. pieces of tableware) +
- ?- owl_individual_of(A, knowrob:'FoodVessel'). +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#cup1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#plate1' ; +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#saucer1' +
-  +
- % everything with a handle: +
- ?- owl_has(A, knowrob:properPhysicalParts, H), owl_individual_of(H,  knowrob:'Handle'). +
- A = 'http://ias.cs.tum.edu/kb/knowrob.owl#Dishwasher37', +
- H = 'http://ias.cs.tum.edu/kb/knowrob.owl#Handle145' +
-</code> +
- +
-===== Infer likely storage locations ===== +
- +
-Apart from the current environment setup, robots sometimes need to reason about the nominal locations of objects, for example when cleaning up or when unpacking a shopping basket. There are different techniques for inferring the location where an object should be placed.  +
- +
-  * Using assertions of the [https://ias.cs.tum.edu/_media/spezial/bib/pancakes11humanoids.pdf storagePlaceFor] property is a rather generic, though not very adaptive technique that allows to state e.g. that perishable items belong into the refrigerator. It does not require any knowledge about the environment, but since it works on the level of object classes, it cannot choose between containers of the same type, e.g. different cupboards. +
-  * A finer-grained solution is based on [https://ias.cs.tum.edu/people/tenorth/publications?key=schuster12orgprinciples organizational principles] that places objects at the location where semantically similar objects are stored. It requires some knowledge about the distribution of other objects in the environment. +
- +
-=== Query for likely storage location === +
- +
-The simple option based on the storagePlaceFor predicate can be queried as follows in order to determine where an object (instance or class) shall be stored, or which known objects are to be stored in a given container: +
-<code> +
- ?- storagePlaceFor(Place, map_obj:'butter1'). +
- Place = 'http://ias.cs.tum.edu/kb/knowrob.owl#Refrigerator67' +
- +
- ?- storagePlaceFor(knowrob:'Refrigerator67', Obj). +
- Obj = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#butter1' ; +
- Obj = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#buttermilk1' ; +
- Obj = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#cheese1' ; +
- Obj = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#milk1'; +
- [...] +
-</code> +
- +
-An extension of the storagePlaceFor predicate also reads why the system came to the conclusion that something is to be stored at some place: +
-<code> +
- ?- storagePlaceForBecause(Place, map_obj:'butter1', Because). +
- Place = 'http://ias.cs.tum.edu/kb/knowrob.owl#Refrigerator67', +
- Because = 'http://ias.cs.tum.edu/kb/knowrob.owl#Perishable' +
-</code> +
- +
-TODO: add orgprinciples calls +
- +
-=== Check for objects that are not at their storage location === +
- +
-By combining the semantic map, giving the current object locations, with the knowledge about the locations where objects should be, the robot can determine which objects are mis-placed. This may for instance be useful when tidying up an environment. +
- +
-The following code first compute which objects are inside which other ones, then selects those that are food or drink, computes their most likely storage place (which, in this example, is usually the refrigerator), and compares the two places. As a result, the system finds out that the butter and the buttermilk are misplaced in a drawer and in the oven, respectively. +
- +
-<code> +
- ?- rdf_triple(knowrob:'in-ContGeneric', Obj, ActualPlace),  +
-    owl_individual_of(Obj, knowrob:'FoodOrDrink'),  +
-    storagePlaceFor(StoragePlace,Obj),  +
-    ActualPlace\=StoragePlace. +
- Obj = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#butter1', +
- ActualPlace = 'http://ias.cs.tum.edu/kb/knowrob.owl#Drawer31', +
- StoragePlace = 'http://ias.cs.tum.edu/kb/knowrob.owl#Refrigerator67' ; +
- Obj = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#buttermilk1', +
- ActualPlace = 'http://ias.cs.tum.edu/kb/knowrob.owl#Oven19', +
- StoragePlace = 'http://ias.cs.tum.edu/kb/knowrob.owl#Refrigerator67' +
-</code> +
- +
-===== Read object component hierarchy ===== +
- +
-Composed objects and their parts are linked by an inverse part-of hierarchy, described using the properPhysicalParts property in OWL. This property is transitive, i.e. a part of a part of an object is also a part of the object itself. You can read all parts and sub-parts of an object using owl_has, which takes the transitivity into account. In the example below, Handle160 is a part of Door70, which by itself is part of Refrigerator67. +
-<code> +
- ?- owl_has(knowrob:'Refrigerator67', knowrob:properPhysicalParts, P). +
- P = 'http://ias.cs.tum.edu/kb/knowrob.owl#Door70' ; +
- P = 'http://ias.cs.tum.edu/kb/knowrob.owl#Handle160' ; +
- P = 'http://ias.cs.tum.edu/kb/knowrob.owl#Hinge70' +
-</code> +
- +
-=== Special case: articulated objects like cupboards === +
- +
-{{ :part-of-hierarchy-map.png?nolink&472| }} +
- +
-Articulated objects like cupboards that have doors or drawers are represented in a special way to describe, on the one hand, the component hierarchy, and on the other hand the fixed and moving connections. Like for other composed objects, there is a part-of hierarchy (properPhysicalParts). Joints are parts of the cupboard/parent object, and are fixed-to (connectedTo-Rigidly) both the parent and the child (e.g. the door). In addition, the child is hingedTo or prismaticallyConnectedTo the parent. +
- +
- +
-Joints are described using the following properties, which are compatible to the representation used by the [http://www.ros.org/wiki/articulation ROS articulation stack]. +
-  * Type: At the moment, rotational and prismatic joints are supported (knowrob:'Hinge' and knowrob:'PrismaticJoint') +
-  * Parent, Child: resp. object instances +
-  * Pose: like for [[Modeling_perceived_or_inferred_objects|normal objects]] using e.g. a SemanticMapPerception instance +
-  * Direction: vector giving the opening direction of a prismatic joint +
-  * Radius: radius of a rotational joint (e.g. between handle and hinge) +
-  * Qmin, Qmax: lower and upper joint limits +
- +
- +
-===== Read articulation information ===== +
- +
-There are some convenience predicates for reading, creating, updating and deleting joints from articulated objects. This task is on the one hand rather common, on the other hand somewhat complex because the structure visualized in the previous image needs to be established. +
- +
-To create a joint of type knowrob:'HingedJoint' between two object instances roboearth:'IkeaExpedit2x40' and roboearth:'IkeaExpeditDoor13' at position (1,1,1) with unit orientation, radius 0.33m and joint limits 0.1 and 0.5 respectively, one can use the following statement: +
-<code> +
- create_joint_information('HingedJoint', roboearth:'IkeaExpedit2x40', roboearth:'IkeaExpeditDoor13',  +
-                           [1,0,0,1,0,1,0,1,0,0,1,1,0,0,0,1], [], '0.33', '0.1', '0.5', Joint). +
-</code> +
- +
-If a prismatic joint is to be created instead, the empty list [] needs to be replaced with a unit vector describing the joint's direction, e.g. [0,0,1] for a joint opening in z-direction, and the joint type needs to be set as 'PrismaticJoint'. +
- +
-Joint information can conveniently be read using the following predicate that requires a joint instance as argument: +
-<code> +
- read_joint_information(Joint, Type, Parent, Child, Pose, Direction, Radius, Qmin, Qmax). +
-</code> +
- +
-To update joint information, one can use the following predicate: +
-<code> +
- update_joint_information(Joint, 'HingedJoint', [1,0,0,2,0,1,0,2,0,0,1,2,0,0,0,1], [1,2,3], 0.32, 1.2, 1.74). +
-</code> +
- +
-===== Read and convert units of measure ===== +
- +
-All numerical values can optionally be annotated with a unit of measure. To keep the system backwards compatible, other values are interpreted to be given in the respective SI unit (e.g. meter, second). +
- +
-Full article incl. explanation of design choices and links to further information: [[Measurement_units]] +
-<code> +
- $ roscd knowrob_common/owl +
- $ rosrun rosprolog rosprolog ias_knowledge_base +
-  +
- ?- owl_parse('knowrob_units.owl', false, false, true). +
- ?- consult('../prolog/knowrob_units.pl'). +
-  +
- % read information that is asserted for a test instance +
- ?- rdf_has('http://ias.cs.tum.edu/kb/knowrob_units.owl#test-inst', +
-            'http://ias.cs.tum.edu/kb/knowrob_units.owl#length', O). +
- O = literal(type('http://qudt.org/vocab/unit#Centimeter','12.0')) . +
-  +
- % manual conversion into other units +
- ?- convert_to_unit($O, 'http://qudt.org/vocab/unit#Kilometer', P). +
- P = 0.00012. +
-  +
- ?- convert_to_unit($O, 'http://qudt.org/vocab/unit#Meter', P). +
- P = 0.12. +
-  +
- ?- convert_to_unit($O, 'http://qudt.org/vocab/unit#Millimeter', P). +
- P = 120.0. +
-</code> +
- +
-The integration with the rdf_triple computables allows to transparently convert values into the desired unit of measure:  +
-<code> +
- % transparent conversion during the query   +
- ?- rdf_triple('http://ias.cs.tum.edu/kb/knowrob_units.owl#length',  +
-               'http://ias.cs.tum.edu/kb/knowrob_units.owl#test-inst',  +
-                literal(type('http://qudt.org/vocab/unit#Meter', Val))). +
- Val = 0.12 ; +
-  +
- ?- rdf_triple('http://ias.cs.tum.edu/kb/knowrob_units.owl#length',  +
-               'http://ias.cs.tum.edu/kb/knowrob_units.owl#test-inst',  +
-                literal(type('http://qudt.org/vocab/unit#Kilometer', Val))). +
- Val = 0.00012 ; +
-</code> +
- +
- +
-===== Convert between relative and global coordinates ===== +
-see here for now: [[Coordinate_systems]] +
- +
-===== Query for qualitative spatial relations ===== +
- +
-Using [[Tutorial:_Computables|computables]] that calculate qualitative spatial relations between objects, we can query e.g. in which container we expect to find sausage1, ask for the content of Refrigerator67, or ask what is on top of Dishwasher37: +
- +
-<code> +
- ?- rdf_triple(knowrob:'in-ContGeneric', map_obj:sausage1, C). +
- C = 'http://ias.cs.tum.edu/kb/knowrob.owl#Refrigerator67' +
-  +
- ?- rdf_triple(knowrob:'in-ContGeneric', O, knowrob:'Refrigerator67'). +
- O = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#cheese1' ; +
- O = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#milk1' ; +
- O = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#sausage1' +
-  +
- rdf_triple(knowrob:'on-Physical', A, knowrob:'Dishwasher37'). +
- A = 'http://ias.cs.tum.edu/kb/ccrl2_map_objects.owl#cup1' ; +
- A = 'http://ias.cs.tum.edu/kb/knowrob.owl#CounterTop205' +
-</code> +