Differences

This shows you the differences between two versions of the page.

--- doc:writing_an_interface_to_your_perception_system [2014/11/27 12:19] – [Setting up the perception tutorial] admin
+++ doc:writing_an_interface_to_your_perception_system [2014/11/27 14:54] – [Perception service] admin
@@ Line 15: / Line 15: @@
 The knowrob_perception tutorial is part of the knowrob_tutorials repository. You need to check it out into your catkin workspace and compile it using catkin_make:
-<code>
+<code bash>
 cd /path/to/catkin_ws/src
 git clone https://github.com/knowrob/knowrob_tutorials.git
@@ Line 22: / Line 22: @@
 </code>
+Since a large part of the tutorial is written in Java, it is recommended to set up an Eclipse project and to inspect and edit the Java files from there. You can generate the Eclipse project files using the following command and then only need to create a new "Java Project" for that directory inside Eclipse. The classpath will then already be set up to include all dependencies of the knowrob_perception_tutorial package.
+<code bash>
+./gradlew eclipse
+</code>
 ====== Interfacing topic-based perception systems ======
 ===== Publisher =====
-The file src/edu/tum/cs/ias/knowrob/tutorial/DummyPublisher.java implements a simple dummy publisher that simulates a perception system that regularly detects objects and publishes these detections on the /dummy_object_detections topic once a second. Try to understand how these detections are generated by the generateDummyObjectDetection() method. You can start the publisher using
+The file src/edu/tum/cs/ias/knowrob/tutorial/DummyPublisher.java implements a simple publisher of object detections. It simulates a perception system that regularly detects objects and publishes these detections on the /dummy_object_detections topic once a second. Try to understand how these detections are generated by looking at the generateDummyObjectDetection() method. You can start the publisher using
 <code>
 rosrun knowrob_perception_tutorial dummy_publisher
@@ Line 58: / Line 62: @@
 ===== Subscriber =====
-The counterpart on the client side that consumes the object detections is implemented in the file ''src/edu/tum/cs/ias/knowrob/tutorial/DummySubscriber.java''. The subscriber has been realized using two threads: The ''listenToObjDetections'' thread subscribes to the topic and puts the incoming messages into the ''callback'' queue. The ''updateKnowRobObjDetections'' thread processes all object detections in this queue and creates the corresponding representations in KnowRob. The rationale behind this setup is to keep the subscriber thread as light-weight as possible to avoid problems of missing messages when the program is occupied updating the knowledge base. While the creation of an object detection in KnowRob is quite fast, this structure is crucial if the processing becomes more complex.
+The counterpart on the client side that consumes the object detections is implemented in the file ''src/edu/tum/cs/ias/knowrob/tutorial/DummySubscriber.java''. The subscriber uses two threads: The main thread subscribes to the topic and puts the incoming messages into the ''detections'' queue. The ''updateKnowRobObjDetections'' thread processes all object detections in this queue and creates the corresponding representations in KnowRob. The rationale behind this setup is to keep the subscriber thread as light-weight as possible to avoid problems of missing messages when the program is occupied updating the knowledge base. While, in this case, the creation of an object detection in KnowRob is quite fast, this two-thread structure is crucial once the processing becomes more complex.
-The following code snippet is the main part of the ''updateKnowRobObjDetections'' thread. It pops the an ''ObjectDetection'' message from the ''callback'' queue, converts the pose quaternion into a 4x4 pose matrix and calls the ''create_object_perception'' predicate.
+The following code snippet is the main part of the ''updateKnowRobObjDetections'' thread. It pops an ''ObjectDetection'' message from the ''detections'' queue, converts the pose quaternion into a 4x4 pose matrix and calls the ''create_object_perception'' predicate.
-<code>
+<code java>
-  while (n.isValid()) {
+node.executeCancellableLoop(new CancellableLoop() {
+  @Override
+  protected void loop() throws InterruptedException {
-    obj = callback.pop();
+    ObjectDetection obj = detections.take();
-    Matrix4d p = quaternionToMatrix(obj.pose.pose);
+    Matrix4d p = quaternionToMatrix(obj.getPose().getPose());
     String q = "create_object_perception(" +
-          "'http://ias.cs.tum.edu/kb/knowrob.owl#"+obj.type+"', ["
+                      "'http://knowrob.org/kb/knowrob.owl#"+obj.getType()+"', ["
-          + p.m00 + ","+ p.m01 + ","+ p.m02 + ","+ p.m03 + ","
+                      + p.m00 + ","+ p.m01 + ","+ p.m02 + ","+ p.m03 + ","
-          + p.m10 + ","+ p.m11 + ","+ p.m12 + ","+ p.m13 + ","
+                      + p.m10 + ","+ p.m11 + ","+ p.m12 + ","+ p.m13 + ","
-          + p.m20 + ","+ p.m21 + ","+ p.m22 + ","+ p.m23 + ","
+                      + p.m20 + ","+ p.m21 + ","+ p.m22 + ","+ p.m23 + ","
-          + p.m30 + ","+ p.m31 + ","+ p.m32 + ","+ p.m33 +
+                      + p.m30 + ","+ p.m31 + ","+ p.m32 + ","+ p.m33 +
-          "], ['DummyObjectDetection'], ObjInst)";
+                      "], ['DummyObjectDetection'], ObjInst)";
     PrologInterface.executeQuery(q);
-    n.spinOnce();
   }
+});
 </code>
+This predicate from the knowrob_perception package is defined as below. It creates a new object instance for the given object type (''rdf_instance_from_class''), creates an instance describing the perception event using the ''PerceptionTypes'' given as argument, links the object instance to this perception event, and sets the pose at which the object was detected. This convenience predicate assumes that the detected objects are always novel. By calling the different predicates individually, one can also add perception events to existing objects instances if the identity of the object instance is known.
-This predicate, defined in the knowrob_perception package, is defined as below. It creates a new object instance for the given object type (''rdf_instance_from_class''), creates an instance describing the perception event using the ''PerceptionTypes'' given as argument, links the object instance to this perception event, and sets the pose at which the object was detected. This convenience predicate assumes that the detected objects are always novel. By calling the different predicates individually, one can also add perception events to existing objects instances if the identity of the object instance is known.
+<code prolog>
-<code>
 create_object_perception(ObjClass, ObjPose, PerceptionTypes, ObjInst) :-
     rdf_instance_from_class(ObjClass, ObjInst),
@@ Line 96: / Line 102: @@
 Whenever the subscriber is started, it creates the KnowRob-internal representations for all object detections that are received via the topic. It can be run from Prolog via the [[http://www.swi-prolog.org/packages/jpl/|JPL Java-Prolog interface]] as defined in prolog/perception_tutorial.pl:
-<code>
+<code prolog>
 obj_detections_listener(Listener) :-
-  jpl_new('edu.tum.cs.ias.knowrob.tutorial.DummySubscriber', ['knowrob_tutorial_listener'], Listener),
+  jpl_new('org.knowrob.tutorials.DummySubscriber', [], Listener),
-  jpl_call(Listener, 'startObjDetectionsListener', [], _).
+  jpl_list_to_array(['org.knowrob.tutorials.DummySubscriber'], Arr),
+  jpl_call('org.knowrob.utils.ros.RosUtilities', runRosjavaNode, [Listener, Arr], _).
 </code>
-If the dummy publisher is running, the following sequence of commands starts the topic listener, queries for object instances and their poses.
+To start a rosjava node from KnowRob, you first need to instantiate the class containing the node definition, create an array with the fully-qualified class name, and pass it to the ''runRosjavaNode'' method. After starting the dummy publisher in another terminal, you can start KnowRob, create the topic listener, and query for object instances and their poses.
-<code>
+<code prolog>
 ?- obj_detections_listener(L).
-L = @'J#00000000000173056232'.
+L = @'J#00000000000034668496'
-Attaching 0x8afd1010
+% ... several rosjava INFO messages ...
-<wait for a few seconds...>
+% wait for a few seconds...
 ?- owl_individual_of(A, knowrob:'HumanScaleObject').
-A = 'http://ias.cs.tum.edu/kb/knowrob.owl#Cup_vUXiHMJy' ;
+A = knowrob:'Cup_uGmuwKPo' ;
-A = 'http://ias.cs.tum.edu/kb/knowrob.owl#Cup_bneXbLGX' ;
+A = knowrob:'DrinkingBottle_TaVWzXre' ;
-A = 'http://ias.cs.tum.edu/kb/knowrob.owl#DinnerFork_TaVWzXre'
+A = knowrob:'DrinkingBottle_rMsqkRjP' ;
+A = knowrob:'DinnerFork_tDjYwuhx'
-?- current_object_pose('http://ias.cs.tum.edu/kb/knowrob.owl#DinnerFork_TaVWzXre', P).
+?- current_object_pose(knowrob:'DinnerFork_tDjYwuhx', P).
 P = [1.0,0.0,0.0,2.9473,0.0,1.0,0.0,2.6113,0.0,0.0,1.0,0.2590,0.0,0.0,0.0,1.0].
 </code>
@@ Line 124: / Line 132: @@
 ===== Perception service =====
-The dummy perception service is very similar to the dummy publisher. Whenever a request for an object detection is received, it responds with a simulated detection of a random object type at a random pose. In real scenarios, the request will probably not be empty, but specify properties of the perception method to be used. The code of the dummy service can be found in the file src/edu/tum/cs/ias/knowrob/tutorial/DummyService.java. It can be started with the following command:
+The dummy perception service is very similar to the dummy publisher. Whenever a request for an object detection is received, it responds with a simulated detection of a random object type at a random pose. In the example, the 'request' part of the service is empty -- in real scenarios, the request will often specify properties of the perception method to be used. The code of the dummy service can be found in the file src/edu/tum/cs/ias/knowrob/tutorial/DummyService.java. It can be started with the following command:
 <code>
 rosrun knowrob_perception_tutorial dummy_service