Synthetic Dataset Generation (real texture)

YOLOv7 training for our own object has proven to be very powerful if a wide enough dataset can be provided. In order to achieve this massive number of images, collected from the same source (homogeneous dataset), a synthetic dataset was generated, following these steps:

  1. Collect real wood block measures and texture.
  2. Using Blender software, create the object CAD model, importing its own real texture.
  3. Calibrate the camera intrinsic parameters, using some known pattern (I used a large chessboard).
  4. Collect several online background (materials) scenes.
  5. With the Wood Block CAD model + camera intrinsic parameters + background materials, generate the dataset, choosing the amount of different scenes, target objects and distractors (using BlenderProc).

This generated dataset is most-close to reality comparing to other previous synthetic datasets mentioned in this blog, since it considers the real camera intrinsic parameters and includes the real texture on the object CAD model. The dataset result can be summarized as follows:

210 000 images:

  • Synthetic images
  • wood block CAD model with real texture
     
  • 4200 different scenes
  • 50 images per scene
  • 1 image per scene without wood block (no target)
     
  • min. target objects/image: 1
  • max. target objects/image: 5
     
  • min. distractors/image: 2
  • max. distractors/image: 30
     
  • images generated with real camera intrinsic parameters

 

Here are some examples of this type of synthetic images:

          

 

 

 

 

Comments

Post a Comment

Popular posts from this blog

Real-time UR10e following a tracked object

Image
For a first trial, it was developed this demonstration of a real-time followed object that is being tracked using one RGB-D camera (Intel RealSense D435).   As seen in this previous post , ViSP is used to process the images acquired by the RealSense RGB-D camera and to continuously track the object . A TCP/IP Socket Connection is established (inside the same computer) between ViSP and ROS . This socket communication is responsible for bringing the geometric transformation between the camera and the object to the ROS environment. This previous post describes with more detailed information this connection. Besides the transformation between the camera and the object, it is also required to know the transformation between the robot and the camera . For the robot to understand the position and orientation of the object in reference to the robot himself, this transformation becomes crucial. For getting it, I performed a manual calibration , as described in this previous post . Finally,
Read more

RGB-D tracking + UR10e following & picking/placing

Image
Following the first use case described in this last post , the result of the experiment can be seen in this video: There are still several improvements to be made to this job1 task: The job should wait for the response of each controller before sending the next request The robot xacro should be extended to include every object that compose the real scenario  The procedure should have an interactive way of telling the robot that is time to pick up the object The use case should ensure that the object pick was successful (or not) This issues can be followed here .
Read more

UR10e control architecture

Image
Now that I am able to control the arm of the robot as well as the gripper, it is now time to create an interface that can be used to control both the arm and the gripper.  This interface should be completely agnostic to any use case that is being developed. Since gripper control and arm control are two independent procedures that can be running simultaneously, this interface should put these two controllers at the same level. Following this philosophy, any job that I want to develop in the future will be able to call any one of these controllers by the same method! Here we can see the structure of the first complete job.  In this case, an object that is continuously being tracked through a RGB-D camera is followed by the robot arm and then is picked up and stored in a box.  As we can see, both controllers (arm and gripper) are at the same level, and the job1 ROS node is communicating with both of them simultaneously. The gripper controller is described in this previous post . The arm
Read more