Autonomous Pick And Place Robotic Arm With Object Detection And Recognition

Abstract

Current robotics pick and place automation is done specifically for each environment and task without much human interaction in a workspace. A collaborative pick-and-place robot is a type of robot that can work alongside humans safely to perform tasks such as picking up objects and placing them in desired locations while also being able to adapt to different environments and scenarios. This project is designed to demonstrate how robotics and AI synergize to create a practical solution to enhance collaborative automation and reduce cost by using low-cost edge microcontrollers for all processing. Using a novel method for planar grasp detection and planar pose estimation of irregular objects using a robotic arm equipped with an AI vision system. The main objectives of this project are: Deploy the project on a low-cost edge micro-controller ”Jetson Nano”; real-time object detection and recognition using a deep neural network model planar pose estimation from 2D RGB image planar grasp planning algorithm using a 2D image arm manipulation and motion planning; a failure handling strategy that can detect and recover from unsuccessful grasps using feedback from force sensors and vision; safety handing strategy that can detect conditions where human and robot will have unsafe interaction and prevent them using feedback from vision. The designed project can handle multiple objects in cluttered scenes and perform robust pick and place tasks with failure handling mechanisms and has achieved real-time object detection and recognition on Jetson Nano as well as a pick and place success rate of over 330 mean picks per hour which is faster than previous implementations.