Object Tracking using ViSP in ROS - 1st demo

At this point, the goal is to achieve the same object tracking tool (using RGB and Depth data) within ROS environment. 

For this, I used some required packages to use ViSP platform in ROS; an RGB-D camera (Intel RealSense D435i), and a woodblock as the object to be tracked.
The RGB-D tracking already achieved of this same woodblock is described in one of my previous posts.

Having all the required packages installed, I created a new package where the RGB-D camera is launched and then a tracking node will be responsible for reading the camera images and track the woodblock. The tracking node will, simultaneously and continuously, publish the woodblock position and orientation in reference to the camera frame.
We can see in this next video the result:


 As it's possible to see, there are two axis system: the optical_camera and the wood_block. The wood_block link is continuously extracted by the tracking node (ViSP_node) in reference to the optical_camera link.

 Analyzing the node graph, we can see that the tracking node (ViSP_node) is created to: 

  1.  subscribe the /camera/color/image_raw topic
  2.  use this image data to track the woodblock
  3.  publish, in real time, the woodblock pose (in reference to the camera link)

 


 The TF tree, as usual, provides useful information to a better understanding:

As we can see,

  •  /camera/realsense2_camera_manager publishes the camera_aligned_depth_to_color_frame link pose in reference to the camera_link link
  • ViSP_node publishes the wood_block link pose in reference to the camera_color_optical_frame link
  • All the remaining transformations are being published by the robot_state_publisher

 

Non solved issue

This was the first fully working  demonstration, where we achieved the goal of tracking an object and publish its pose in ROS environment. However, the tracking process was very unstable and easily lost the woodblock. 
In this experiment, the tracker only used RGB data to compute the woodblock pose, and the Depth map was never considered.
The lack of depth information considered by the tracker caused the lack of robustness felt during the demo.
 
The depth information was not included since there are no method in vpROSGrabber class that imports depth maps data directly. It is required to create a method for acquisition of 16UC1 data type, in order to open the depth map and enable the tracker to use this information alongside with the RGB data. 
This will be the very next step.

Comments

Post a Comment

Popular posts from this blog

Remote Control of UR10e via MoveIt (ROS)

Image
Installation First, install the External Control URCap on the UR10e Teach Pendant. See this post , to further instructions. Then, you will need to install a few packages on your Ubuntu system: sudo apt install ros-noetic-moveit sudo apt-get install ros-noetic-industrial-robot-status-interface sudo apt-get install ros-noetic-scaled-controllers sudo apt-get install ros-noetic-pass-through-controllers sudo apt-get install ros-noetic-ur-client-library sudo apt-get install ros-noetic-velocity-controllers sudo apt-get install ros-noetic-force-torque-sensor-controller ( Note: At this moment, if you do not have a catkin workspace, you should now create one, by following the steps described here ) cd catkin_ws/src git clone https://github.com/afonsocastro/larcc_interface.git git clone https://github.com/ros-industrial/ur_msgs.git cd ~/catkin_ws catkin_make  Finally, to establish the communication between the robot and the computer, it is required to connect an Ethernet cable from the UR1...
Read more

UR10e control architecture

Image
Now that I am able to control the arm of the robot as well as the gripper, it is now time to create an interface that can be used to control both the arm and the gripper.  This interface should be completely agnostic to any use case that is being developed. Since gripper control and arm control are two independent procedures that can be running simultaneously, this interface should put these two controllers at the same level. Following this philosophy, any job that I want to develop in the future will be able to call any one of these controllers by the same method! Here we can see the structure of the first complete job.  In this case, an object that is continuously being tracked through a RGB-D camera is followed by the robot arm and then is picked up and stored in a box.  As we can see, both controllers (arm and gripper) are at the same level, and the job1 ROS node is communicating with both of them simultaneously. The gripper controller is described in this previous p...
Read more

Real-time UR10e following a tracked object

Image
For a first trial, it was developed this demonstration of a real-time followed object that is being tracked using one RGB-D camera (Intel RealSense D435).   As seen in this previous post , ViSP is used to process the images acquired by the RealSense RGB-D camera and to continuously track the object . A TCP/IP Socket Connection is established (inside the same computer) between ViSP and ROS . This socket communication is responsible for bringing the geometric transformation between the camera and the object to the ROS environment. This previous post describes with more detailed information this connection. Besides the transformation between the camera and the object, it is also required to know the transformation between the robot and the camera . For the robot to understand the position and orientation of the object in reference to the robot himself, this transformation becomes crucial. For getting it, I performed a manual calibration , as described in this previous post . Fina...
Read more