Human Tracking - first tips

There are some different systems for estimating the pose of a human in a real time video (for /human_tracker node). The most known (and better documented) is OpenPose.

OpenPose allows:

  • 2D real-time multi-person keypoint detection
  • 3D real-time single-person keypoint detection (compatible with PointGrey cameras, requiring multiple views)

The best advantage of OpenPose is that it can also detect the hand shape, being useful for the /human_decision node as well. With many keypoints all over the human body, including its hand, it is easily extracted the human decision based upon the hand gestures.

link to their github: OpenPose github

link to their paper: OpenPose paper

Other alternatives for this human tracking purpose are HRNet and MMPose, both of them based on PyTorch (a python library that eases building deep learning projects). These two algorithm had used COCO and MPII human poses datasets for training their trackers.

The HRnet has proved to be more accurate than OpenPose, however it do not take the human hand in consideration, and it is not so well integrated in different hardware and sources as it is OpenPose. For a more detailed comparison, see this.

The real challenge, as far as I understand, it will be how to get the 3D spacial pose of the human body, from this 2D keypoints of one camera image... More to be discussed further!



Comments

Post a Comment

Popular posts from this blog

Remote Control of UR10e via MoveIt (ROS)

Image
Installation First, install the External Control URCap on the UR10e Teach Pendant. See this post , to further instructions. Then, you will need to install a few packages on your Ubuntu system: sudo apt install ros-noetic-moveit sudo apt-get install ros-noetic-industrial-robot-status-interface sudo apt-get install ros-noetic-scaled-controllers sudo apt-get install ros-noetic-pass-through-controllers sudo apt-get install ros-noetic-ur-client-library sudo apt-get install ros-noetic-velocity-controllers sudo apt-get install ros-noetic-force-torque-sensor-controller ( Note: At this moment, if you do not have a catkin workspace, you should now create one, by following the steps described here ) cd catkin_ws/src git clone https://github.com/afonsocastro/larcc_interface.git git clone https://github.com/ros-industrial/ur_msgs.git cd ~/catkin_ws catkin_make  Finally, to establish the communication between the robot and the computer, it is required to connect an Ethernet cable from the UR1...
Read more

Installing External Control URCap on robot Teach Pendant

Image
For working on a real robot you need to install the externalcontrol-1.0.5.urcap which can be found inside the resources folder of this repository . Using a USB pen drive, follow: Format the flash drive Download and save the externalcontrol-1.0.5.urcap on the USB pen drive Insert the USB drive on UR10e controller (the controller has two USB ports)        4. Turn on the Teach Pendant      Click on Menu (top right corner) + System + URCaps + Select External Control and press "+"        6.  Configure the remote host's IP to 192.168.56.1 Click on Menu (top right corner) + System + Network Configure: Network method : Static Address IP address: 192.168.56.2 Subnet mask: 255.255.255.0 Default gateway: 192.168.56.2 Click on Apply    10. Disable EtherNet/IP fieldbus: Installation > Fieldbus > EtherNet/IP > Disable  
Read more

UR10e control architecture

Image
Now that I am able to control the arm of the robot as well as the gripper, it is now time to create an interface that can be used to control both the arm and the gripper.  This interface should be completely agnostic to any use case that is being developed. Since gripper control and arm control are two independent procedures that can be running simultaneously, this interface should put these two controllers at the same level. Following this philosophy, any job that I want to develop in the future will be able to call any one of these controllers by the same method! Here we can see the structure of the first complete job.  In this case, an object that is continuously being tracked through a RGB-D camera is followed by the robot arm and then is picked up and stored in a box.  As we can see, both controllers (arm and gripper) are at the same level, and the job1 ROS node is communicating with both of them simultaneously. The gripper controller is described in this previous p...
Read more