Neural Network Analysis

With the new dataset collected on 1 August 2022 (see this post), we optimized some parameters to achieve a proper and valid neural network.

On a Universe of 2558 samples (640 pull, 642 push, 632 shake, 644 twist). Where 1790 for training (1253 train, 537 validation) and 768 for testing.

Searched combinations:
n_hidden_layers = 2 or 3
neurons_in_layer = 32 or 64
dropout = 0.2 or 0.5
activation_function = relu or selu

Fixed values:
model_optimizer = Adam
learning_rate = 0.001
max_epochs = 300
batch_size = 96
model_loss = sparse_categorical_crossentropy
kernel_regularizer_per_layer = L1
early_stopping = val_loss ; patience = 30

 

model configurations: 160
searched time: 16m55s

Best Result

image

 image

https://user-images.githubusercontent.com/47828797/188680392-d883fb9f-6e0a-411e-9513-46c92be02084.png

Epoch 218/300:
loss: 0.7502 - accuracy: 0.8859
val_loss: 0.7890 - val_accuracy: 0.8566

Using 1253 samples for training and 537 for validation

 

Confusion Matrix

 

image

Using 768 new samples for predicting

 

10 best results (optimization)

image

 

Analyzing the Confusion Matrix, we can see that the forth interaction (TWIST) had much more mismatched predictions than the other three interactions. Because of this, we can suspect that this specific movement could be ambiguous...


Let's now exclude the TWIST interaction from our study by creating a new neural network, with only 3 output neurons; and removing all twist samples from our dataset.

Study without TWIST

On a Universe of 1914 samples (640 pull, 642 push, 632 shake). Where 133 for training (937 train, 401 validation) and 575 for testing.

 

Optimization

Searched combinations:

batch_size = 64 or 96
n_hidden_layers = 2 or 3
neurons_in_layer = 16 or 32 or 64
dropout = 0.2 or 0.5
activation_function = relu or selu

Fixed values:
model_optimizer = Adam
learning_rate = 0.001
max_epochs = 300
model_loss = sparse_categorical_crossentropy
kernel_regularizer_per_layer = L1
early_stopping = val_loss ; patience = 30

 

model configurations: 1008
searched time: 2h59m44s

 

 

BEST Result 

Epoch 223/300

loss: 0.3423 - accuracy: 0.9840 
val_loss: 0.3785 - val_accuracy: 0.9502 


Using 937 samples for training and 401 for validation

 

Confusion Matrix


Using 575 new samples for predicting

 

 10 best results (optimization)

Comments

Post a Comment

Popular posts from this blog

Real-time UR10e following a tracked object

Image
For a first trial, it was developed this demonstration of a real-time followed object that is being tracked using one RGB-D camera (Intel RealSense D435).   As seen in this previous post , ViSP is used to process the images acquired by the RealSense RGB-D camera and to continuously track the object . A TCP/IP Socket Connection is established (inside the same computer) between ViSP and ROS . This socket communication is responsible for bringing the geometric transformation between the camera and the object to the ROS environment. This previous post describes with more detailed information this connection. Besides the transformation between the camera and the object, it is also required to know the transformation between the robot and the camera . For the robot to understand the position and orientation of the object in reference to the robot himself, this transformation becomes crucial. For getting it, I performed a manual calibration , as described in this previous post . Finally,
Read more

RGB-D tracking + UR10e following & picking/placing

Image
Following the first use case described in this last post , the result of the experiment can be seen in this video: There are still several improvements to be made to this job1 task: The job should wait for the response of each controller before sending the next request The robot xacro should be extended to include every object that compose the real scenario  The procedure should have an interactive way of telling the robot that is time to pick up the object The use case should ensure that the object pick was successful (or not) This issues can be followed here .
Read more

RGB-D tracking (ViSP/ROS) - 2nd demo

Image
  Using the infrastructure described in my last post , where ViSP communicates with ROS via a TCP/IP socket connection, this is the result of the RGB-D tracking:     I believe that this result is good enough for now. On the ROS side, we simply have the robot state publisher (responsible for the camera links) and the client node (responsible for the woodblock link). These two agents are continuously publishing the transformations.     One issue that we can already notice is that, in ROS, now we have neither the RGB image topic nor the depth map topic, which restricts us to create the pointcloud where we could clearly see the woodblock. A possible way around this issue is to send the images through the socket communication channel as well. For now, let's consider this step halfway concluded. Now it's time to control the robot!  The challenge for the next few days is to remotely control the UR10e collaborative robot.  
Read more