FlexiGroBots - People action recognition
This application enhances agricultural safety by facilitating collaboration between human operators and autonomous robots on the same land sections simultaneously. Addressing the increasing need for robotic solutions in agriculture due to labor shortages and economic efficiency, it focuses on adding safety layers and enhancing robot perception of their surroundings.
This tool is a component of the European project, Flexigrobots. It consists of various models that together provide services for recognizing action in agricultural environments when robot interaction such as walking, bending, running, sitting or carrying an object. Users can utilize this service through Docker technology, accessible in a public GitHub repository associated with this application.
The human action detection tool explores alternatives using leading models. It uses an updated SlowFast-ResNet50 model from the MMaction2 library, pretrained with the Kinetics400 dataset, to detect actions relevant in agricultural contexts. The tool's effectiveness is validated qualitatively through video analysis from pilot data and quantitatively with a 27.71 mAP on the corresponding dataset.
The tool accurately identifies common actions like walking, bending, and carrying objects in agricultural settings During development, other state-of-the-art models like LART (with a 42.6 mAP) were tested. However, LART's longer processing time, unsuitable for real-time application, led to the selection of the current model. For instance, the implemented model processes a 26-second video in 13.4 seconds on a specific hardware setup, while LART takes significantly longer.
The focus remains on practical, real-time applications for autonomous agricultural robots to improve decision-making and ensure safety in human-robot collaborative environments. Although more accurate models exist, the chosen solution balances accuracy with real-time processing needs, aligning with the project's goals to enhance agricultural safety and efficiency. The ongoing exploration of new models supports this evolving field, but the current implementation remains a robust and practical choice.