Development of a Deep Learning Model for Object Detection

Autonomous cleaning robots are increasingly vital in keeping large-scale environments like warehouses and factories clean. This project presents the development and evaluation of a machine learning-based object detection system for KEMARO’s industrial cleaning robots. The primary objective was to address the robot’s current limitations in detecting small, valuable, or dangerous objects, such as keys or cables, which are overlooked by the current computer vision algorithms. The project explored state-of-the-art sensor technologies, including depth cameras, alongside advanced deep learning models such as YOLO and FastSAM.
High-quality real-world data was recorded using the Intel RealSense D435 with various configurations, including color recording and different resolutions, while maintaining the same approximate height and tilt to the camera placement of KEMARO’s K900 cleaning robot.
A key innovation of the study was the adaptation of YOLO for RGB-D input, allowing depth information to complement color image data. Data scarcity was addressed through a combination of real-world data collection and synthetic dataset generation using Blender. Various training configurations were tested to evaluate the generalization of the object detection models in real-world scenarios.
The results demonstrate that synthetic data can improve training outcomes when combined with real-world data, but challenges remain in model stability and the quality of depth information. Feeding RGB instead of grayscale images to the model achieved better results in object detection. While incorporating depth (RGB-D) increased model object detection performance when using perfect depth-maps, it proofed to be disadvantageous with imperfect and noisy real-world depth maps.
A roadmap for deep learning based object detection was also developed for KEMARO, providing recommendations for data governance, sensor improvements, and iterative hardware upgrades.