- eXtended Reality (XR) based Aircraft Maintenance System with Cognitive Load Estimation Feature, Aeronautical Development Agency (INR 90.91L, 2024-2026)
- eXtended Reality CoBot controller for Manufacturing Processes, DST, Govt of India (INR 50L, 2023-25)
- AI - AR Integration for Manufacturing Assembly Process, Collins Aerospace Systems, USA (INR 70.8L, 2022-2023)
- Holoportation and UI/UX Design for Metaverse, Siemens, Germany (INR 76.8L, 2022-2027)
- Interactive AR/VR digital twin for navigation and monitoring of office spaces, British Telecom (INR 45L, 2020-2022), part of BTIRC
- Mixer ML/AI -FCM IISc, Faurecia Services Groupe, France (INR 6.5L, 2020-2021)
- Smart Sensor Dashboard, Department of Heavy Industries, Common Engineering Facility Centre (CEFC) (INR 10L, 2019-2020)
- Multimodal CoBot, Department of Heavy Industries, Common Engineering Facility Centre (CEFC) (INR 10L, 2019-2020)
- Smart Manufacturing Test Bed, Robert Bosch Centre for Cyber-Physical Systems (INR 15L, 2016-2017)
- Honeywell Course on Multimodal Interaction (INR 0.75L, 2017)
We are developing an interactive sensor dashboard and multimodal CoBot for supporting Industry 4.0 initiative. We are investigating different visualization techniques of sensor reading in 2D and 3D environment. Presently we are implementing and deploying a 3D sensor dashboard for BT offices in India. The multimodal CoBot allows to control a robotic manipulator through multiple modalities like eye gaze, gesture and speech.
Earlier we also developed image processing algorithms for automated inspection of rubber sheet for a shoe making MSME and Printed Circuit Boards (PCB) for detecting correct orientation and type of IC chips using webcam and shape detection algorithms. We have developed a prototype system using a Kinect that can detect bad leaning posture by measuring the difference between the neck-spine and hip-spine joints in the skeleton figure of Kinect. Following RULA guidelines, it raises an alert when the neck or torso is inclined more than 60 ͦ.
Main Results
- We developed and validated a new way of generating synthetic image and video based data for training deep learning models like Convolutional Neural Networks using VR digital twins which offers more customization than Generative Adversial Network (GAN) and Variable Auto Encoders (VAE).
- We evaluated shape detection algorithms in different lighting conditions (indoor, outdoor, and controlled light source) and best shape detection result is obtained in YCbCr color space using bounding box shape descriptors for 2500 Lux using LED.
Publications
- S Raj, N Beri, D Patel, Y Sinha, A Chakrabarti, P Biswas, HaM3D: Generalized XR-Based Multimodal HRI Framework with Haptic Feedback for Industry 4.0, Journal of Multimodal User Interface 2024
- A. Mukhopadhyay, GS Rajshekar Reddy, S. Ghosh, P. Biswas, Validating Social Distancing through Deep Learning and VR-Based Digital Twins, ACM VRST 2021
- A. Mukhopadhyay, GS Rajshekar Reddy, KPS Saluja, S. Ghosh, A. Peña-Rios, G. K. Gopal, P. Biswas, A Virtual Reality-Based Digital Twin of Office Spaces with Social Distance Measurement Feature, Virtual Reality & Intelligent Hardware, Elsevier
- A. Mukhopadhay, I. Mukherjee and P. Biswas, Comparing Shape Descriptor Methods for Different Colour Space and Lighting Conditions, Artificial Intelligence in Engineering Design and Manufacturing, Cambridge University Press 33 (4) 2019
- S. Arjun, GS Rajshekar, A. Mukhopadhyay, S. Vinod and P. Biswas, Evaluating Visual Variables in a Virtual Reality Environment, British HCI 2021
- S. Arjun, LRD Murthy, P. Biswas, Interactive Sensor Dashboard for Smart Manufacturing, International Conference on Industry 4.0 and Smart Manufacturing, Procedia Computer Science, Elsevier 2022
- V. K. Sharma and P. Biswas, System for Operating Joystick Indian Patent Application no. 201941044740, PCT International Application No. PCT/IB2020/059959
- A. Mukhopadhyay, GS Rajshekar Reddy, I. Mukherjee, G. K. Gopal, A. Peña-Rios, P. Biswas, Generating Synthetic Data for Deep Learning using VR Digital Twin, 3rd International Conference on Virtual Reality and Image Processing (VRIP 2021)
- P. Biswas, S. Roy, G. Prabhakar, J. Rajesh, S. Arjun, M. Arora, B. Gurumoorthy and A. Chakrabarti, Interactive Sensor Visualization for Smart Manufacturing System, Proceedings of the 31st British Human Computer Interaction Conference 2017 (British HCI 17)
- A. Mukhopadhyay, LRD Murthy, M. Arora, A. Chakrabarti, I. Mukherjee and P. Biswas, PCB Inspection in the context of Smart Manufacturing, International Conference on Research into Design (ICoRD 2019)
- LRD Murthy, Somnath Arjun, Kamalpreet Singh Saluja, Pradipta Biswas, Smart Sensor Dashboard, 7th International Conference on PLMSS (Product Life Cycle Modelling, Simulation and Synthesis) 2019, https://arxiv.org/abs/2005.05025
- K. Puneeth, A. Sahay, G, Shreya, P. Biswas, M. Arora, A. Chakrabarti, Designing an Affordable System for Early Defect Detection Using Image Processing, Advances in Manufacturing Technology XXXIII: Proceedings of the 17th International Conference on Manufacturing Research, incorporating the 34th National Conference on Manufacturing Research, 2019
