Introduction to Multimodal Interaction

Instructor

Pradipta Biswas, PhD (Cantab)

Assistant Professor

Email: pradipta AT cpdm DOT iisc DOT ernet DOT in

Prerequisite

· Basic knowledge of Mathematics

· C++ and preferably C# or VB.Net programming skill

· Computer with Windows 7/8/10

· MS Visual Studio 2010/2012/2015

· Basic knowledge of multimodal interaction

· Exposure to state-of-the-art eye gaze, hand, head and finger movement and EEG trackers

· Developing new input modalities tracking eye gaze, hand, finger, head movement of users

· Hands on experience on developing software using Tobii, Leap, Kinect and Speech SDKs

· Multimodal fusion algorithms

· Validating new input modality

· Data analytics relevant to multimodal interaction

1. Introduction to Multimodal Interaction [Slides]

2. A case study of developing multimodal middleware [Slides]

3. Gesture recognition systems, hand and finger movement tracking, Affective computing [Slides]

4. Multimodal interaction for limited mobility environment [Slides]

5. Multimodal fusion algorithms [Slides]

6. Eye gaze tracking, iris recognition and head movement tracking [Slides]

7. Multimodal interaction for automotive environment [Slides]

8. Image processing algorithm for face and eye detection [Slides]

9. Experiment design and data analytics [Slides]

Instructor’s Bio: Pradipta Biswas is an Assistant Professor at the Centre for Product Design and Manufacturing of Indian Institute of Science. His research focuses on user modelling and multimodal human-machine interaction for aviation and automotive environments and for assistive technology. He set up and leads the Interaction Design Lab at CPDM, IISc. Earlier, he was a Senior Research Associate at Engineering Department, Research Fellow at Wolfson College and Research Associate at Trinity Hall of University of Cambridge. He completed PhD in Computer Science at the Rainbow Group of University of Cambridge Computer Laboratory and Trinity College in 2010 and was awarded a Gates-Cambridge Scholarship in 2006. He undertook a master degree at the Indian Institute of Technology, Kharagpur. He conducted a course on Human Computer Interaction at Indian Institute of Technology, Mandi, guest lecture at Indian Institute of Technology, Madras and was a vice chairman of ITU-T Focus Group on Smart TV.

I³D Lab Demo Video