Detecting Human Alertness on System onChip (SoC) using Dual Cues

Abstract

Diminishing alertness in human subjects impacts safety-critical operations severely. Although the perception of a low level of alertness is inherently subjective, we very often lose the required self-awareness during demanding jobs such as automobile driving, operation control of nuclear power plants, etc. In such contexts, we need a continuous and real-time measurement of alertness levels in human operators to avoid any adverse consequences. Literature suggests that alertness level can be influenced by several factors and essentially need a multidimensional approach for measurement. The current state of the art in this regard is expensive and very specific to vehicular technology. Therefore, some of the cues may not be usable for other safety-critical operations. On the other hand, the low-cost solutions of alertness detection systems lack effectiveness due to the usage of a single cue. Therefore, in this work, we aim to design a portable and robust alertness-level detection system using two cascaded cues, which utilize variations in sensory-motor neuron coordinated response and ocular parameters like PERcentage CLOSure (PERCLOS) of eyelids over time. To attain this, as well as to keep provision for future augmentation of other ocular parameters like saccadic ratio, we have chosen System-on-Chip (SoC) —essentially a heterogeneous digital system— to mitigate the multi-processing requirement using hardware accelerator and soft processing units. Out of the two cues used, the sensory-motor neuron coordination is measured using the response of human subjects from the Choice Reaction Test (CRT). Generally, such a test is carried out with a predefined time-bound, which may introduce possibilities of manipulation of response during time-pressing jobs. We have designed trial-bound in a short time span CRT and proposed a new reaction time metric, Peak-to-Spread Ratio (PSd), to measure the alertness level of the subject. Experimental results on the database generated from 65 subjects —for audio and visual response to the CRT— confirm that PSd has better resolution and sensitivity than state-of-the-art metrics. A combined hardware-software co-designed architecture with Viola-Jones face detector as a pre processor is adopted for PERCLOS detection. Integral image computation and cascaded classifier sub-modules are implemented on Programmable Logic (PL-FPGA), while the image scaling, non-maximum suppression, sub-modules of face detection along with PERCLOS measurement module are implemented on Processing System (PS-ARM). To validate the system, we have generated a database with various levels of challenges associated with eyelid motion and high-speed saccadic eye movement measurement at different levels of alertness. Performance with eyelid motion at various alertness levels —measured with PERCLOS— is tested with the proposed digital architecture. A preliminary experiment is carried out on the algorithmic level for saccadic eye movement analysis and the hardware implementation of it is kept for future augmentation to improve the efficacy of the overall system.