RGB-D tracking

Since RGB object tracking did not work perfectly, it was now time to study the tracking procedure using not only a RGB set of frames, but a RGB-D data combination. Luckily, ViSP (being used since the beginning of tracking) has its own modules for live tracking, using an Intel Realsense RGB-D camera. 

I have used the Intel Realsense D435 RGB-D camera. 

Besides changing from RGB to RGB-D tracking, I also changed the study object, from the lego piece to a wood block. This object change has the purpose of having a perfectly matched CAD model (.cao file) for the real object shape. Remember that the lego piece was not exactly a parallelepiped as the CAD model was describing. CAD file of wood block: wood block file.

The results from this RGB-D study were quite better, having the object pose almost always correctly tracked. More than that, with the depth map, the user can grab the object with no problem because the tracker can now easily estimate the object edges. 

Here is the result:

This experiment was built upon the tutorial-mb-generic-tracker-rgbd-realsense.cpp from the ViSP tutorials: https://visp-doc.inria.fr/doxygen/visp-3.4.0/tutorial-tracking-mb-generic-rgbd.html 

Now, I will try to save a dataset of this object, to include the auto initialization option on this tracker. With the auto initialization option, it is possible to re-find the object every time the tracker (for some reason) loses the object pose. This dataset should not be too long, and should ensure the most distinct object poses within the controlled environment. 

Comments

Post a Comment

Popular posts from this blog

Remote Control of UR10e via MoveIt (ROS)

Image
Installation First, install the External Control URCap on the UR10e Teach Pendant. See this post , to further instructions. Then, you will need to install a few packages on your Ubuntu system: sudo apt install ros-noetic-moveit sudo apt-get install ros-noetic-industrial-robot-status-interface sudo apt-get install ros-noetic-scaled-controllers sudo apt-get install ros-noetic-pass-through-controllers sudo apt-get install ros-noetic-ur-client-library sudo apt-get install ros-noetic-velocity-controllers sudo apt-get install ros-noetic-force-torque-sensor-controller ( Note: At this moment, if you do not have a catkin workspace, you should now create one, by following the steps described here ) cd catkin_ws/src git clone https://github.com/afonsocastro/larcc_interface.git git clone https://github.com/ros-industrial/ur_msgs.git cd ~/catkin_ws catkin_make  Finally, to establish the communication between the robot and the computer, it is required to connect an Ethernet cable from the UR1...
Read more

Installing External Control URCap on robot Teach Pendant

Image
For working on a real robot you need to install the externalcontrol-1.0.5.urcap which can be found inside the resources folder of this repository . Using a USB pen drive, follow: Format the flash drive Download and save the externalcontrol-1.0.5.urcap on the USB pen drive Insert the USB drive on UR10e controller (the controller has two USB ports)        4. Turn on the Teach Pendant      Click on Menu (top right corner) + System + URCaps + Select External Control and press "+"        6.  Configure the remote host's IP to 192.168.56.1 Click on Menu (top right corner) + System + Network Configure: Network method : Static Address IP address: 192.168.56.2 Subnet mask: 255.255.255.0 Default gateway: 192.168.56.2 Click on Apply    10. Disable EtherNet/IP fieldbus: Installation > Fieldbus > EtherNet/IP > Disable  
Read more

UR10e control architecture

Image
Now that I am able to control the arm of the robot as well as the gripper, it is now time to create an interface that can be used to control both the arm and the gripper.  This interface should be completely agnostic to any use case that is being developed. Since gripper control and arm control are two independent procedures that can be running simultaneously, this interface should put these two controllers at the same level. Following this philosophy, any job that I want to develop in the future will be able to call any one of these controllers by the same method! Here we can see the structure of the first complete job.  In this case, an object that is continuously being tracked through a RGB-D camera is followed by the robot arm and then is picked up and stored in a box.  As we can see, both controllers (arm and gripper) are at the same level, and the job1 ROS node is communicating with both of them simultaneously. The gripper controller is described in this previous p...
Read more