Using a dual arm collaborative robot and RGB-D camera with YOLOv5 for pick and place application

Abstract

In many industries nowadays, robots and cameras are used together to detect certain objects and perform specific tasks. However, misdetection can be occurred due to uncertainty of lighting conditions, background, and environment. Using a dual arm 6-DOF collaborative robot, ABB YuMi, and RGB-D camera with YOLOv5 in the pick-and-place application is proposed in this research in order to resolve the aforementioned problems. The image is collected and labeled in preparation for the dataset. The dataset is trained with a machine learning algorithm, YOLOv5. It became the weight for real-time detection. When RGB images from the camera are sent to YOLOv5, data in regard to position x-y and the color of the bottle are extracted from the depth and the color images. The position is used to control the movement of the robot. The experiment consists of 3 parts. First, YOLOv5 is tested with trained images and without trained images. Second, YOLOv5 is tested for real-time images from the camera. Finally, we assume that YOLOv5 could be 100 % detection and test the ability to grasp the bottle. The result was 95%, 90%, and 90%, respectively.