- Hackathon for rehabilitation of severely disabled young adults, India-EU ICT Standardisation Collaboration Project (€20,000, 2022)
- Privacy Respecting AR/VR Enabling Differently-Abled in Multi-Cultural Society, Facebook Reality Labs, USA ($75000, 2021-2022)
- DST DUO Fellowship with University of Barcelona , Spain (€6000, 2020)
- Gaze Controlled Applications for eInclusion of users with SSMI, Microsoft Research India (INR 10L, 2019-2020)
- Social Networking Application for Children with Severe Speech and Motor Impairment (SSMI), Microsoft AI for Accessibility Grant ($15000, 2019-2020)
- Cognitive and Auditory Rehabilitation for Elderly, Department of BioTechnology (DBT, Govt of India) (INR 10L, 2018-2021)
- IT4All, DST SERB Early Career Fellowship, (INR 14.5L, 2017-2020)
Our projects on Assistive Technology and Inclusive Design are covering a wide range of abilities including users with severe speech and motor impairment, students with learning disabilities and elderly users with hearing impairment. We deployed eye trackers and a set of eye gaze controlled assistive software and robotic manipulator at the Spastic Society of India, Chennai, India. We compared ocular parameters of students with learning disabilities with their able bodied counterpart, deployed online multilingual cognitive and auditory test battery and investigated website and smartphone usage patterns by elderly users. We work with International Telecommunication Union (Telecom Branch of UNO) to promote accessibility of audio visual media by chairing an Inter-Rapporteur Group on Audio Visual Media Accessibility (IRG AVA).
Persons with severe speech and motor impairment (SSMI) face significant challenges in daily interactions, necessitating innovative technological support. My research led to the development of a safe, eye-gaze-controlled human-robot interaction system tailored for SSMI individuals. A groundbreaking hackathon was organized, exclusively featuring participants who utilize assistive technology and demonstrate high levels of impairment. These individuals underwent training on an eye gaze-controlled interface, facilitating the emergence of new applications aimed at enhancing their rehabilitation process. For instance, the system enables users to perform tasks like drawing, typing, stamping through low-cost robotics. This initiative was complemented by an Indo EU workshop on Inclusive Design and Standardization focused on Inclusive Design and Standardization, highlighting advancements in the field. We were granted patent on a Multimodal Joystick Controller, that enabled children with SSMI to operate standard RC cars and even toy drones. This playfulness was found to improve their interaction quality with gaze-controlled interface. My research included the development of a robot operated via a bespoke eye gaze tracking algorithm and evaluated various safe distance maintenance algorithms through simulations and user trials. Findings indicated that users could control a robotic arm effectively within three minutes. Additionally, a gaze-controlled robotic arm with a task-specific gripper demonstrated efficient performance in user studies, particularly with the video see-through interface, which yielded the best results for task execution. This research presents practical solutions for SSMI children, promoting their independence in engaging with daily activities.
Main Results
- Users with SSMI can statistically significantly select correct elements on screen in an eye gaze controlled interface faster when elements were organized at their preferred screen positions than random organization.
- Using our bespoke eye gaze controlled interface, able bodied users can select one of nine regions of screen at a median of less than 2 secs and users with SSMI can do so at a median of 4 secs. Using the eye gaze controlled human-robot AR display, users with SSMI could undertake representative pick and drop task at an average duration less than 15 secs and reach a randomly designated target within 60 secs using a COTS eye tracker and at an average time of 2 mins using the webcam based eye gaze tracker.
- Users with severe speech and motor impairment could bring the robotic arm at any designated point within its working envelope in less than 3 minutes.
- Results showed that VST had the average stamping time of 28.45s(SD = 15.44s) and the average stamp count 7.36(SD = 3.83), outperforming VPT and OST.
- Total number of eye gaze fixations and regressions (backward gaze movement) were statistically significantly correlated to the reading ability of students.
Video Demonstrations and Presentations
Publications
- P. Biswas, Requirements for the semantics of a Common User Profile used to personalize audiovisual media, 17th International Conference on COMmunication Systems and NETworks (IEEE COMSNETS 25), Bengaluru, India, 2025, pp. 1250-1253
- P. Biswas (Lead Editor), ITU Technical Report Concept of a common user profile format used to personalize audiovisual media
- Himanshu , M Mitra, VK Sharma, J Sulthan, A Atulkar, D Bhathad, P Biswas, Unlocking Potential: Gaze-Based Interfaces in Assistive Robotics for SSMI Users, IEEE International Conference on Robotics and Automation (ICRA 25)
- VK Sharma, LRD Murthy, and P Biswas, Comparing two safe distance maintenance algorithms for a gaze controlled HRI involving users with SSMI, ACM Transactions on Accessible Computing 2022
- VK Sharma, LRD Murthy and P Biswas, Enabling Learning through Play for Persons with speech and motor impairment: Inclusive and Eye-Gaze Controlled Human-Robot Interface for interaction with Joystick operated generic Toys, International Conference on Social Robotics (ICSR) 2022
- S Arjun, B Kothari, NK Shah, P Biswas, Do weak readers in rural India automatically read Same Language Subtitles on Bollywood films? : An eye gaze analysis, Journal of Eye Movement Research 2022
- LRD Murthy and P. Biswas, Appearance-based Gaze Estimation using Attention and Difference Mechanism, 3rd International Workshop on Gaze Estimation and Prediction in the Wild (GAZE 2021) at CVPR 2021
- S. Kumar, JeevithaShree DV, and P. Biswas, Comparing ten WCAG Tools for accessibility evaluation of websites ,Technology and Disability 2021
- U Singh, Jeevithashree DV, A Muraleedharan, Anamika, KPS Saluja, P Biswas, Accessibility Analysis Of Educational Websites Using WCAG 2.0, ACM Digital Government: Research and Practice, 2024
- Jeevithashree DV, KPS Saluja and P. Biswas, A case study of developing gaze controlled interface for users with severe speech and motor impairment, Technology and Disability 31(1), IOS Press, 2019
- JeevithaShree DV, P Jain, A Mukhopadhyay, KPS Saluja, P Biswas, Eye Gaze Controlled Adaptive Virtual Keyboard for Users with SSMI, Technology and Disability, IOS Press, 2021
- VK Sharma and P. Biswas, Gaze Controlled Safe HRI for Users with SSMI, IEEE RAS International Conference on Advanced Robotics Systems (ICAR 2021)
- VK Sharma, KPS Saluja, LRD Murthy, G Sharma and P Biswas, Webcam Controlled Robotic Arm for Persons with SSMI, Technology and Disability 32 (3), IOS Press 2020
- N. Blow and P. Biswas, A pointing facilitation system for motor-impaired users combining polynomial smoothing and time weighted gradient target prediction models, Assistive Technology 29 (1), ISSN: 1040-0435, Taylors and Francis, 2016
- P.Biswas and P. O. Looms, From disability to capability: strategies for reconciling solutions for all, Encyclopaedia of Computer Science and Technology (2nd Edition), Taylor and Francis 2018
- P. Biswas and P. Langdon, Applications of Intelligent and Multimodal Eye Gaze Controlled Interfaces, Wiley Handbook of Human-Computer Interaction 2018
- P. Biswas and M. Springett, Inclusive User Modelling, Wiley Handbook of Human-Computer Interaction 2018
- VK Sharma, KPS Saluja, V Mollyn and P Biswas, Eye Gaze Controlled Robotic Arm for Persons with Severe Speech and Motor Impairment, ACM International Conference on Eye Tracking Research and Applications (ETRA 2020) (Acceptance rate 30%)
- Jeevithashree DV, K. P. Saluja and P. Biswas, Gaze Controlled Interface for Limited Mobility Environment, ACM International Conference on Designing Interactive Systems (DIS) 2018
- S. Arjun, Personalizing data visualization and interaction, ACM International conference on User Modeling and Adaptive Personalization (UMAP 2018) Doctoral Consortium
- A. Sahay and P. Biswas, Webcam based eye gaze tracking using a landmark detector, ACM Compute 2017, Annual Conference of ACM India
- KPS Saluja, Jeevithashree Dv, S. Arjun and P. Biswas, Analyzing Eye Gaze Movement of Users with Different Reading Abilities due to Learning Disability, 3rd International Conference on Graphics and Signal Processing (ICGSP 2019, Best Presentation Award)
- K. Maheswary, S. Arjun and P. Biswas, Inclusive Crowd Sourcing based Disaster Reporting System, International Conference on Control, IEEE Instrumentation, Communication & Computational Technologies (ICCICCT-2016)
- A. Agarwal, JeevithaShree DV, K S Saluja, A Sahay, P Mounika, A Sahu, R Bhaumik, V K Rajendran and P. Biswas, Comparing Two Webcam based Eye Gaze Trackers for Users with Severe Speech and Motor Impairment, International Conference on Research into Design (ICoRD 2019) [Distinguished Paper Award]
- P. Mounika, D. Karia D., K. Sharma, and P. Biswas, Accessibility evaluation of three important Indian websites, International Conference on Research into Design (ICoRD 2019)
- P.Biswas, A. Halder, K. Maheswary and S. Arjun, Inclusive Personlization of User Interfaces, 6th International Conference on Research into Design (ICoRD 2017)
