Development of Machine Learning-Based Advanced Driver Assistance System for Smart Vehicular Applications

Abstract

Advanced Driver Assistance System (ADAS) is an intelligent way of accident prevention in road traffic. ADAS is used for abnormal driving activity prediction and to guide the driver for its rectification. Most road accidents are caused due to careless driving techniques adopted by human drivers. Advanced computer vision tools and powerful Graphical Processing Units (GPUs) are capable of developing end-to-end tracking systems for complex traffic scenarios. The primary objective is to predict abnormal driving activities to prevent road traffic accidents. Monitoring and detecting driving activities are the critical roles performed by computer vision-based systems. Human beings can do this work quickly because of their superior visual and cognitive abilities. However, continuously watching a driver’s video feed for long periods is a tedious task for humans, which is sometimes impossible, not interesting, and may result in errors. Consequently, this makes automated machine vision-based techniques an ideal choice. The State-of-the-Art (SOTA) literature has several studies that can be used to create systems that automatically detect and monitor driver distraction. The application of ADAS involves monitoring the driver’s activity based on driver pose analysis, driver’s behavior analysis, basic emotion recognition to increase driver comfort, monitoring driver health conditions for risk assessment, etc. The main focus of recent ADAS research has been the relationship between the increased risk of Cardiovascular Disease (CVD) and traffic accidents. Advances in the Internet of Things (IoT) and automotive technology offer the possibility of integrating healthcare into vehicles for driver safety and health. This thesis aims to investigate and analyze some new algorithms with Machine Learning (ML) and Deep Learning (DL) based approaches to predict the driver’s activity in an in-vehicle scenario. This thesis considers both heart rate and in-vehicle camera data for the development of ADAS system. A two-stage framework is proposed for the preliminary screening of commercial drivers prior to actual driving evaluation using ML techniques. First, it aims to address the healthcare of cardiac drivers in resource-constrained scenarios, such as bus terminals with paramedic staff. Model performance is estimated using the publicly available cardiovascular disease benchmark databases and shows superiority over the SOTA techniques. The system then generates a warning for no driving in the event of predicted heart disease and stores the expected abnormal parameters in a Comma-Separated Value (CSV) file for screening by experienced cardiologists working at nearby super specialty hospitals. Email-based data communication has been established to transfer the CSV file to the hospital, reducing the burden on drivers, who regularly visit the hospital. Analyzing driving behavior is also vital to ensure safety. The proposed work on driver behavior analysis uses DL approaches such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) to identify driving activity and categorize driving behavior as normal, aggressive, and drowsy. The UAH-DriveSet dataset, a publicly accessible smartphone dataset is used to model this work as a time-series classification task. Further, the identification of driving abnormalities is supported by Facial Expression Recognition (FER), eye-closure analysis, head position estimation, etc. Six basic emotions are used as the basis for facial expression identifications: happiness, surprise, anger, sadness, fear, and disgust. The FER system will continuously monitor the driver through the dashboard camera to identify the driver’s irresponsibility and provide timely assistance for safety. The FER framework is studied using different pre-trained Convolutional Neural Network (CNN) models such as AlexNet, SqueezeNet, and VGG19. The performance of the proposed model has been verified on six publicly accessible benchmark databases namely FER2013, JAFFE, KDEF, CK+, SFEW, and KMU-FED and shows superiority over the SOTA techniques. Literature studies on driver distraction monitoring use in-vehicle DL-based ADAS to reduce the risk of traffic accidents. DL algorithms are difficult to implement in resource-constrained low-cost embedded devices such as Raspberry Pi or mobile phones due to the computational resource requirements for in-vehicle use. Hence, in this work, lightweight models are designed using SqueezeNet 1.1 with last-layer modification to employ it in Raspberry Pi for FER and Distracted Driving Detection (DDD) tasks. Each work is carried out by testing it on different benchmark databases with its performance evaluated by comparing the results obtained by it with those of the benchmark and recent SOTA techniques.