Automotive Back

  • Intelligent Dashboard Control System(IDCS), Distraction and Cognitive Load Detection, Faurecia Services Groupe, France (INR 57.3L, 2018-20)
  • Ride Quality Monitoring of Drivers and Passengers of Autonomous Vehicles, Wipro as part of the WIRIN project (INR 10L, 2018-19)
  • Ashok Leyland Course on Automotive User Interface (INR 2L, 2018)

We have developed a finger movement and eye gaze controlled interactive Head Up Display (HUD) for car that can also simultaneously estimate drivers’ cognitive load using a new machine learning algorithm involving ocular parameters. We validated the interactive HUD through simulation and in-car studies. The work is extended to autonomous vehicles too by integrating the cognitive load estimation module with object detection Convolutional Neural Network (CNN) models. As part of the study, we compared existing CNNs like RetinaNet, YOLO and maskR-CNN for Indian road dataset with respect to accuracy and latency of detecting different traffic participants.

Main Results

  • User studies show the interactive HUD improves driving performance in terms of mean deviation from lane in an ISO 26022 lane changing task compared to touchscreen system and participants can undertake ISO 9241 pointing tasks in less than 2 secs on average inside a Toyota Etios car.
  • Average velocities of saccadic intrusion increases with increase in cognitive load and recording saccadic intrusion and eye blinks for 6 s duration can predict drivers’ instantaneous perception of developing road hazards.
  • Accuracy of Mask R-CNN is significantly higher than YOLOv3 and RetinaNet while evaluating these models on novel classes of objects such as animals, autorickshaws, caravan in Indian road dataset. We have also tested latency of the CNN models and found that latency of YOLOv3 is significantly lower than other two models and RetinaNet is significantly faster than Mask R-CNN.

Publications

  • P. Biswas, Exploring the use of Eye Gaze Controlled Interfaces in Automotive Environments Springer 2016, ISBN: 978-3-319-40708-1
  • G. Prabhakar, A. Ramakrishnan, L. R. D Murthy, V. K. Sharma, M. Madan, S. Deshmukh and P. Biswas, Interactive Gaze & Finger controlled HUD for Cars, Journal of Multimodal User Interface, Springer, 2019
  • G. Prabhakar, A. Mukhopadhyay, LRD Murthy, M. Madan, S. Deshmukh and P. Biswas, Cognitive load estimation using Ocular Parameters in Automotive, Transportation Engineering, Elsevier 2020
  • P. Biswas and G. Prabhakar, Detecting Drivers’ Cognitive Load from Saccadic Intrusion, Transportation Research Part F: Traffic Psychology and Behaviour 54, Elsevier, 2018
  • G. Prabhakar and P. Biswas, Eye Gaze Controlled Projected Display in Automotive and Military Aviation Environments. Multimodal Technologies and Interaction, 2(1), 2018
  • P. Biswas, S. Deshmukh, G. Prabhakar, M. Modiksha and A. Mukhopadhyay, System and Method for Monitoring Cognitive Load of a Driver of a Vehicle, Indian Patent Application No.: 201941052358
  • P. Biswas, S. Deshmukh, G. Prabhakar, M. Modiksha, V. K. Sharma and A. Ramakrishnan,A System for Man- Machine Interaction in Vehicles , Indian Patent Application No.: 201941009219, PCT International Application No. PCT/IB2020/050253
  • G. Prabhakar and P. Biswas, Wearable Laser Pointer for Automotive User Interface Application No.: 201741035044, PCT International Application No. PCT/IB2018/057680
  • P. Biswas, Interactive Gaze Controlled Projected Display , Indian Patent Application No.: 201641037828
  • P. Biswas, G. Prabhakar, J. Rajesh, K. Pandit and A. Halder, Improving Eye Gaze Controlled Car Dashboard using Simulated Annealing, Proceedings of the 31st British Human Computer Interaction Conference (British HCI 17, selected to exhibit in the Interactions Gallery)
  • G. Prabhakar, N. Madhu and P. Biswas, Comparing Pupil Dilation, Head Movement, and EEG for Distraction Detection of Drivers, Proceedings of the 32nd British Human Computer Interaction Conference 2018 (British HCI 18)
  • P. Biswas, S. Twist and S. Godsill, Intelligent Finger Movement Controlled Interface for Automotive Environment, Proceedings of the 31st British Human Computer Interaction Conference 2017 (British HCI 17)
  • A. Mukhopadhyay, A. Agarwal, I. Mukherjee and P. Biswas, Performance Comparison of Different CNN models for Indian Road Dataset, 3rd International Conference on Graphics and Signal Processing (ICGSP 2019, Best Presentation Award)
  • A. Mukhopadhay and P. Biswas, Comparing CNNs for Non-Conventional Traffic Participants, ACM Automotive UI 2019 [Recipient of Global Fellowship]
  • P. Biswas, Multimodal Gaze Controlled Dashboard, ACM AutomotiveUI 2016
  • G. Prabhakar and P. Biswas, Evaluation of Laser Pointer as a Pointing Device in Automotive, IEEE International Conference on Intelligent Computing, Instrumentation and Control Technologies [Best Paper Award]
  • G. Prabhakar, J. Rajesh and P.Biswas, Comparison of three Hand Movement Tracking Sensors as Cursor Controllers, IEEE International Conference on Control, Instrumentation, Communication & Computational Technologies (ICCICCT-2016) [Best Paper Award]
  • A. Ramakrishnan, G. Prabhakar, M. Madan, S. Deshmukh, P. Biswas, Eye Gaze Controlled HUD, International Conference on ICT for Sustainable Development 2019