Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
doc:marker_visualization [2013/11/20 07:39] – [Example queries] admindoc:marker_visualization [2015/10/19 15:07] (current) – [Marker-based visualization canvas] daniel86
Line 1: Line 1:
 ====== Marker-based visualization canvas ====== ====== Marker-based visualization canvas ======
 +~~NOTOC~~
 +\\
 +^ This page describes the 'catkinized' version of KnowRob that uses the [[http://wiki.ros.org/catkin/|catkin buildsystem]] and the pure Java-based [[http://wiki.ros.org/rosjava|rosjava]]. The older version, which was based on the rosbuild buildsystem and rosjava_jni, used a [[http://wiki.ros.org/mod_vis|local visualization canvas implemented in Java]].^
 +\\
  
 {{ :doc:web-visualization.png?nolink&650 |}} {{ :doc:web-visualization.png?nolink&650 |}}
  
-Based on [[http://www.ros.org/wiki/rviz/DisplayTypes/Marker|rviz Markers]], this visualization module can display the current or past belief states in the rviz visualizer as well as in the ros3djs web-based visualization.+Based on [[http://www.ros.org/wiki/rviz/DisplayTypes/Marker|rviz Markers]], this visualization module can display 3D information in a browser-based canvas that has been built using the [[https://github.com/RobotWebTools/ros3djs|ros3djs library]].
  
-An example can be seen here (only from the local network): http://tenorth.ai.loc/~tenorth/test-vis.html+===== Marker Types =====
  
-**This module is currently under development and not publicly available.**+The set of supported marker primitive types can be queried using the ''marker_prop_type(Type,_)'' predicate. 
 +The primitive types include arrow, cube, sphere, cylinder, line_strip, line_list, cube_list, sphere_list, points, text_view_facing, mesh_resource and triangle_list. 
 +Primitive markers can be created using ''marker($type('Name'),_)'', where $type must be replaced by one of the primitive types.
  
 +In addition, KnowRob supports a set of complex marker types.
 +  * **cylinder_tf(Frame0,Frame1)**: A cylinder between 2 links
 +  * **link(Frame)**: A marker that takes the pose of a link
 +  * **trajectory(Frame)**: A trajectory of a link
 +  * **trail(Frame)**: A trail of a link (trajectory with alpha fade)
 +  * **average_trajectory(Frame)**: The average over multiple trajectories
 +  * **pointer(Frame0,Frame1)**: Arrow that points from Frame0 in the direction of Frame1
 +  * **object(Name)**: A marker that represents an OWL individual (including children objects)
 +  * **agent(Name)**: A marker that represents an OWL agent individual (including all links)
 +  * **stickman(Name)**: A special human skeleton visualization, kinematic structure is spanned by the OWL individual indentified by Name
 +===== Usage =====
  
-===== System overview ===== +First, setup the rosbridge communication interface and initialiaze the `knowrob_vis` package.
-{{ :doc:web-visualization-architecture.png?nolink |}}+
  
-**Note:** Though the visualization looks much like a standard web page, it's more similar to a web-based rvizThis means that there is still a single ROS graph and a single KnowRob instance in the background that is shared among all website visitors.+  $ roslaunch rosbridge_server rosbridge_websocket.launch 
 +  $ rosrun rosprolog  rosprolog knowrob_vis 
 +   
 +Markers must be explicitly published by the user with following query ([[http://www.open-ease.org/|openEASE]] does this automatically).
  
 +  % publish modified marker
 +  ?- marker_publish.
  
-===== Installation ===== 
-Setting up your computer as server for the web-based marker visualization involves a few steps to make everything work properly: 
-  * Install a web server, e.g. apache2 by ''sudo apt-get install apache2'' 
-  * [[http://wiki.ubuntuusers.de/Apache#Benutzerspezifisch|Set up the system]] to serve web pages from ~/public_html to %%http://localhost/~username%% 
-  * Put the contents of the [[https://github.com/code-iai/knowrob_dev/tree/master/knowrob_vis/html|'html' folder in knowrob_marker_vis]] into the ~/public_html folder. This is needed since some browsers don't accept loading Javascript and HTML files partly from the web and partly from your local harddisk for security reasons. You therefore always have to use the HTML file served by your web browser. 
-  * To use CAD models for visualization, put the respective ROS packages into your global web root, usually /var/www. The %%package://xyz%% URLs describing the models are mapped to %%http://localhost/xyz%%. The pr2_description, for example, should be put into /var/www/pr2_description. 
  
-===== Example queries ===== 
  
-The following queries read data from the semantic map and from the logged data and visualize it in the browser-based visualization. The robot pose at time 1377780296 is read from the MongoDB using the tfMemory extension.+==== Semantic Map Marker ==== 
 + 
 +The following queries read data from semantic map and visualize it in the browser-based visualization.
  
-  $ roslaunch knowrob_vis knowrob_vis.launch 
-  $ rosrun rosprolog  rosprolog knowrob_vis 
-   
-  % load packages, start marker publisher 
-  ?- register_ros_package(knowrob_mongo), register_ros_package(mod_srdl), visualisation_canvas. 
-   
   % load and visualize semantic map   % load and visualize semantic map
-  ?- owl_individual_of(A, knowrob:'SemanticEnvironmentMap'), add_object_with_children(A).+  ?- owl_parse('package://knowrob_map_data/owl/ccrl2_semantic_map.owl'). 
 +  ?- owl_individual_of(A, knowrob:'SemanticEnvironmentMap'), marker_update(object(A)).
      
-  % read robot pose at time 1377766521 from MongoDB +  % highlight all electrical household appliances 
-  ?- mng_robot_pose_at_time(pr2:'PR2Robot1', '/map', knowrob:timepoint_1377780296Pose)+  ?- owl_individual_of(A, knowrob:'ElectricalHouseholdAppliance'), marker_highlight(object(A)). 
-   + 
-  % add robot to the visualization +==== Agent Marker ==== 
-  ?- add_object_with_children(pr2:'PR2Robot1', knowrob:timepoint_1377780296). + 
-   +The following queries read data from a robot description, look up the robot pose in the mongo DB 
-  % highlight all cameras that are part of the SRDL model of the robot +and visualize it in the browser-based visualization. 
-  ?- owl_individual_of(A, srdl2comp:'Camera'), highlight_object(A@(true), 0255255255).+ 
 +  % load and visualize the pr2 robot for given timepoint T 
 +  ?- owl_parse('package://knowrob_srdl/owl/PR2.owl'). 
 +  ?- mng_robot_pose_at_time(pr2:'PR2Robot1','/map',T,_), marker_update(object(pr2:'PR2Robot1'),End). 
 + 
 +==== Trajectory Marker ==== 
 + 
 +The following queries read data from a robot description, sample the '/base_link' between T0 and T1 
 +and visualize it in the browser-based visualization. 
 + 
 +  % load and visualize the base link of the pr2 robot for given timerange (T0,T1) 
 +  ?- owl_parse('package://knowrob_srdl/owl/PR2.owl'). 
 +  ?- marker_update(trajectory('/base_link'), interval(T0,T1,dt(Interval))). 
 +===== Visualizing 3D object meshes ===== 
 + 
 +KnowRob supports the visualization of objects using 3D surface meshes in the STL or Collada file formats. These meshes can either be attached to an object instance or to an object classin which case they are used for visualization of all of its instances. The following OWL snippet shows how an instance can be linked to its CAD model using the //pathToCadModel// property: 
 + 
 +<code xml> 
 + <owl:NamedIndividual rdf:about="&map_obj;milk1"> 
 +     <rdf:type rdf:resource="&knowrob;CowsMilk-Product"/> 
 +     <knowrob:pathToCadModel rdf:datatype="&xsd;string"> 
 +                             package://knowrob_tutorial/cad/milk.kmz</knowrob:pathToCadModel> 
 + </owl:NamedIndividual> 
 +</code> 
 + 
 +The package //knowrob_cad_models// in //knowrob_addons// provides such definitions for several object typesand also contains CAD models that are downloaded at build time. 
 + 
 +Inside the ROS filesystemresources such as binary files can be described using different kinds of URLs: The common file:%%//%% and http:%%//%% URLsand the package:%%//%% URL used above that describes a path relative to a ROS package. These URLs can be resolved using the [[http://wiki.ros.org/resource_retriever|resource_retriever]] tool
  
 +Since the client of a webserver cannot access the whole file system, this mechanism does not work as such if requests are made from a website. The ros3djs library therefore rewrites package:%%//%% URLs for object meshes to subfolders of the website's docroot. For example, meshes in the package pr2_description are searched at http://localhost/pr2_description/...
  
 +If you would like to visualize meshes contained in ROS packages in the browser, you have to put these meshes into the docroot of the webserver, i.e. as a subfolder of the directory 'html' in //knowrob_vis//. Please make sure that the folder is named exactly like the ROS package containing the meshes and that is has the same subfolders. You can remove all files except the meshes themselves.
  
-The following is to be performed from the roslog_2013_08_29_1445 folder: 
  
-  # Is some part of the right arm blocking the view on the object? 
-  ?- register_ros_package(mod_srdl), owl_parse('perceive-no-details.owl', false, false, true). 
-  ?- mng_obj_pose_by_desig('http://ias.cs.tum.edu/kb/cram_log.owl#VisualPerception_Z9fXhEae_object_0', Pose). 
-  ?- sub_component(pr2:pr2_right_arm, Part),  
-     once(owl_individual_of(Part, srdl2comp:'UrdfLink')), 
-     obj_blocked_by_in_camera(log:'VisualPerception_Z9fXhEae_object_0', Part,  
-     pr2:pr2_head_mount_kinect_rgb_link, log:'timepoint_1377780296'). 
-      
-  # Is the object visible in the camera, given the camera's field of view? 
-  ?- obj_visible_in_camera(log:'VisualPerception_Z9fXhEae_object_0',  
-                           pr2:pr2_head_mount_kinect_rgb_link, 
-                           log:'timepoint_1377780271'). 
-  true . 
 ===== Links ===== ===== Links =====
   * [[http://robotwebtools.org/|Overview page robotwebtools]]   * [[http://robotwebtools.org/|Overview page robotwebtools]]