CHEIRONOM (Gesture Recognition Challenge)
The main goal of this project is to have an AI system translate a streaming video of the actions of a driver, captured from within the car, to a stream of action recognition labels, i.e. have the car understand what the driver needs done from the gestures of the driver. This AI system needs to be robust, reliable and fast enough to be executed on an embedded system at the same time where other resource hungry processes are also being executed, on the same device.
Categories
What is the challenge that is being addressed?
Drivers should be able to perform non-driving activity within the car (e.g. media center control, peripheral control, signaling) in a way that does not undermine safety. To prioritize safety, the gesturing on behalf of the driver should be simple to perform, so that the cognitive burden does not interfere with driving, and the system should pick up on the gestures robustly, so as to not stress the driver while trying to register gestures. At the same time, the gesture recognition should be lightweight enough so that even more AI related processes can also be accommodated.
What is the AI solution the project plans to implement?
The employed AI solution, for any given point in time, considers the last 3 seconds of video input to infer the instantaneous gesture. The temporal horizon makes the inference more robust, when comparing to smaller horizons or single shot inference. The inferred gesture can vary between “no action” and several other relevant actions, e.g. swiping, sliding, zooming, etc. (27 categories in total). The video segment of the short history is intelligently processed so that all information within is considered simultaneously, but done carefully so as to keep the inference cost low enough.
How will BonsAPPs support you in implementing this solution?
BonsAPPs is helpful in at least two directions. Firstly, the BonsAPPs tooling allows for the streamlined porting of AI solutions to embedded devices, including optimization and benchmarking. This porting process is on rails which aid the AI talent in the decision making during development. Then, the platform establishes a commercialization prospect for the adopters of the BonsAPPs platform, and a clear path to monetization of AI skill. The experience of the BonsAPPs platform is thus diffused to the adopters and, the combined value is routed towards real problem solving.
