Experiments with cooperative robots that can detect object’s shape, color and size to perform tasks in industrial workplaces

Abstract

Automation and human-robot collaboration are increasing in modern workplaces such as industrial manufacturing. Nowadays, humans rely heavily on advanced robotic devices to perform tasks quickly and accurately. Modern robots with computer vision and artificial intelligence are gaining attention and popularity rapidly. This paper demonstrates how a robot can automatically detect an object’s shape, color, and size using computer vision techniques and act based on information feedback. In this work, a powerful computational model for a robot has been developed that distinguishes an object’s shape, size, and color in real time with high accuracy. Then it can integrate a robotic arm to pick a specific object. A dataset of 6558 images of various monochromatic objects has been developed, containing three colors against a white background and five shapes for the research. The designed system for detection has achieved 99.8% success in an object’s shape detection. Also, the system demonstrated 100% success in the object’s color and size detection with the OpenCV image processing framework. On the other hand, the prototype robotic system based on Raspberry Pi-4B has achieved 80.7% accuracy for geometrical shape detection and 81.07%, and 59.77% accuracy for color recognition and distance measurement, respectively. Moreover, the system guided a robotic arm to pick up the object based on its color and shape with a mean response time of 19 seconds. The idea is to simulate a workplace environment where a worker will ask the robotic systems to perform a task on a specific object. Our robotic system can accurately identify the object’s attributes (e.g., 100%) and is able to perform the task reliably (81%). However, reliability can be improved by using a more powerful computing system, such as the robotic prototype. The article’s contribution is to use a cutting-edge computer vision technique to detect and categorize objects with the help of a small private dataset to shorten the training duration and enable the suggested system to adapt to components that may be needed for creating a new industrial product in a shorter period. The source code and images of the collected dataset can be found at: https://github.com/TituShahoriar/cse499B_Hardware_Proposed_System.