Speech recognition and intelligent translation under multimodal human–computer interaction system
Abstract: The traditional translation robot is limited to the translation of single-mode text images and text videos, which has the problem of low translation accuracy. Therefore, speech recognition and intelligent translation in multimodal human–computer interaction (HCI) system are proposed. First, the network structure of speech recognition model in multi-channel HCI system is established, and the multi-head self-attention mechanism is constructed. Then, the artificial intelligence voice wake-up function is designed, and a multimodal machine translation model is constructed. On this basis, selective attention is added to obtain visual recognition of perceived text, and the decoder is used for multimodal gating fusion to realize the output of encoder translation results. Experimental results show that this method has high BLUE value and high translation accuracy.
- Location
-
Deutsche Nationalbibliothek Frankfurt am Main
- Extent
-
Online-Ressource
- Language
-
Englisch
- Bibliographic citation
-
Speech recognition and intelligent translation under multimodal human–computer interaction system ; volume:33 ; number:1 ; year:2024 ; extent:14
Journal of intelligent systems ; 33, Heft 1 (2024) (gesamt 14)
- Creator
-
Huang, Danhua
Xiang, Shuaiqiu
- DOI
-
10.1515/jisys-2023-0192
- URN
-
urn:nbn:de:101:1-2409071652440.194184105618
- Rights
-
Open Access; Der Zugriff auf das Objekt ist unbeschränkt möglich.
- Last update
-
15.08.2025, 7:27 AM CEST
Data provider
Deutsche Nationalbibliothek. If you have any questions about the object, please contact the data provider.
Associated
- Huang, Danhua
- Xiang, Shuaiqiu
Other Objects (12)
Multimodal machine translation through visuals and speech
Intelligent support mechanisms in adaptable human-computer interfaces
Exploiting unconscious user signals in multimodal human-computer interaction
Incremental speech translation
Speech communication and multimodal interfaces
Modeling modality selection in multimodal human-computer interaction : extending automated usability evaluation tools for multimodal input
Human-computer interaction, Pt. 3.. Towards mobile and intelligent interaction environments
Intelligent Vehicle Violation Detection System Under Human–Computer Interaction and Computer Vision
Multimodal human–computer interaction in interventional radiology and surgery: a systematic literature review
Machine Translation of Spontaneous Speech
Learning speech translation from interpretation
SpeakQL: Towards Speech-driven Multimodal Querying
Multimodal machine translation through visuals and speech
Intelligent support mechanisms in adaptable human-computer interfaces
Exploiting unconscious user signals in multimodal human-computer interaction
Incremental speech translation
Speech communication and multimodal interfaces
Modeling modality selection in multimodal human-computer interaction : extending automated usability evaluation tools for multimodal input
Human-computer interaction, Pt. 3.. Towards mobile and intelligent interaction environments
Intelligent Vehicle Violation Detection System Under Human–Computer Interaction and Computer Vision
Multimodal human–computer interaction in interventional radiology and surgery: a systematic literature review
Machine Translation of Spontaneous Speech
Learning speech translation from interpretation
SpeakQL: Towards Speech-driven Multimodal Querying
Multimodal machine translation through visuals and speech
Intelligent support mechanisms in adaptable human-computer interfaces
Exploiting unconscious user signals in multimodal human-computer interaction
Incremental speech translation
Speech communication and multimodal interfaces
Modeling modality selection in multimodal human-computer interaction : extending automated usability evaluation tools for multimodal input
Human-computer interaction, Pt. 3.. Towards mobile and intelligent interaction environments
Intelligent Vehicle Violation Detection System Under Human–Computer Interaction and Computer Vision
Multimodal human–computer interaction in interventional radiology and surgery: a systematic literature review
Machine Translation of Spontaneous Speech
Learning speech translation from interpretation