Scholarly record
IMPLEMENTATION OF MULTIMODAL INTERFACE FOR HUMAN-COMPUTER INTERACTION SYSTEM TO CONTROL A ROBOTIC WHEELCHAIR
Abstract
The research on the development of new robot control systems is currently underway. The interest in this area stems from the need for practical, user-friendly means of transport adapted to people with physical disabilities. The application of these devices will allow disabled people to manipulate external devices using EEG of both brain activity and eye movement. It is still possible for people with disabilities to partially apply their gaze in order to control and communicate with an assistive device. However, the application of eyetracker - based interfaces technology encounters the problem of involuntary eye movements leading to unwanted item selection (the Midas-Touch problem). The following issue can be addressed by the development of multimodal and hybrid management interfaces. Brain-computer interfaces implement translation of brain activity patterns into commands designed to control interactive applications, with recognition of motor imagination patterns. Thus, this study proposes a multimodal architecture for wheelchair gaze-control system for people with mobility impairments using gaze control and intention confirmation technology using brain-computer interfaces. In this study, a hybrid model was proposed to classify EEG motor imagery signals and eye tracker signals to implement a control system with a neural network architecture consisting of pre-trained convolutional neural network and gated recurrent unit. The performance of the adapted approach is determined using a multiclass imaginary motion dataset and the corresponding swipes and classification results.
Publication Impact Profile
- Citations
- Scopus - Citation Indexes: 1
- Captures
- Mendeley - Readers: 16
Publication details
References9
Vinay Krishna Sharma, Murthy, KamalPreet Singh Saluja, Vimal Mollyn, Eye Gaze Controlled Robotic Arm for Persons with SSMI. Computer science. 2020. DOI: 10.3233/TAD-200264. https://doi.org/10.1145/3379155.3391324
Sunny, M.S.H., Zarif, M.I.I., Rulik, I. et al. Eye-gaze control of a wheelchair mounted 6DOF assistive robot for activities of daily living. J NeuroEngineering Rehabil 18, 173, 2021. DOI: 10.1186/s12984-021-00969-2.
Ban, S., Lee, Y.J., Yu, K.J., Chang, J.W., Kim, J. and Yeo, W. (2023), Persistent Human�Machine Interfaces for Robotic Arm Control Via Gaze and Eye Direction Tracking. Adv. Intell. Syst.200408. DOI: 10.1002/aisy.202200408.
Zhang X, Liu X, Yuan S-M, Lin S-F. Eye tracking based control system for natural human�computer interaction. Comput Intell Neurosci. 2017; 2017:1� 9. DOI: 10.1155/2017/5739301.
Olivas, Brenda E & Chacon, Mario. Classification of multiple motor imagery using deep convolutional neural networks and spatial filters. Applied Soft Computing. 2018, 75. DOI: 10.1016/j.asoc.2018.11.031.
Umar Amin, Mansour Alsulaiman, Ghulam Muhammad, Mohamed Amine Mekhtiche, M. Shamim Hossain. Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion. Future Generation Computer Systems, Volume 101, 2019, Pages 542-554, ISSN 0167-739X, DOI: 10.1016/j.future.2019.06.027.
Liu C, Jin J, Daly I, Li S, Sun H, Huang Y, Wang X, Cichocki A. SincNet-Based Hybrid Neural Network for Motor Imagery EEG Decoding. IEEE Trans Neural Syst Rehabil Eng. 2022; 30:540-549. DOI: 10.1109/TNSRE.2022.3156076. Epub 2022 Mar 14. PMID: 35235515.
Hu H, Pu Z, Li H, Liu Z, Wang P. Learning Optimal Time-Frequency-Spatial Features by the CiSSA-CSP Method for Motor Imagery EEG Classification. Sensors (Basel). 2022 Nov 5;22(21):8526. DOI: 10.3390/s22218526.
Khademi, Zahra Ebrahimi, Farideh Montazery Kordy, Hussain. A transfer learningbased CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals. Computers in Biology and Medicine. 2020. 143. 105288. DOI: 10.1016/j.compbiomed.2022.105288.
View or Download full articleAccess options
SWS access login
Login as SWS Scientific CommitteeLogin as SWS Scientific PartnerLogin as SWS AuthorAuthors and approved SWS contributors will read and export their own linked papers after identity matching by SWS profile, email and SGEM GlobalID.
For librarian assistance: [email protected]
Purchase Instant Access
- Article can be downloaded after successful payment.
- Article may be used according to SWS library access terms.
- Article cannot be redistributed.