A Strategy for Multilingual Spoken Language Understanding Based on Graphs of Linguistic Units
Abstract
In this thesis, the problem of multilingual spoken language understanding is addressed using graphs to model and combine the different knowledge sources that take part in the understanding process. As a result of this work, a full multilingual spoken language understanding system has been developed, in which statistical models and graphs of linguistic units are used. One key feature of this system is its ability to combine and process multiple inputs provided by one or more sources such as speech recognizers or machine translators. A graph-based monolingual spoken language understanding system was developed as a starting point. The input to this system is a set of sentences that is provided by one or more speech recognition systems. First, these sentences are combined by means of a grammatical inference algorithm in order to build a graph of words. Next, the graph of words is processed to construct a graph of concepts by using a dynamic programming algorithm that identifies the lexical structures that represent the different concepts of the task. Finally, the graph of concepts is used to build the best sequence of concepts. The multilingual case happens when the user speaks a language different to the one natively supported by the system. In this thesis, a test-on-source approach was followed. This means that the input sentences are translated into the system’s language, and then they are processed by the monolingual system. For this purpose, two speech translation systems were developed. The output of these speech translation systems are graphs of words that are then processed by the monolingual graph-based spoken language understanding system. Both in the monolingual case and in the multilingual case, the experimental results show that a combination of several inputs allows to improve the results obtained with a single input. In fact, this approach outperforms the current state of the art in many cases when several inputs are combined.
Keywords
- Spoken language understanding
- Multilingual spoken language understanding
- Graph of words
- Algorithm
- Dynamic programming
- Grammatical inference
Publications during the thesis (17)
- 2016 ConferenceA Strategy for Multilingual Spoken Language Understanding Based on Graphs of Linguistic Units
Marcos Calvo Lance , Fernando García Granada , Emilio Sanchis Arnal
International Conference IberSPEECH (Formerly Jornadas en Tecnología del Habla), 349-358
- 2016 JournalMultilingual Spoken Language Understanding using graphs and multiple translations
Marcos Calvo Lance , Lluís Felip Hurtado Oliver , Fernando García Granada , Emilio Sanchis Arnal , Encarna Segarra Soriano
Computer Speech and Language, 38, 86-103 · DOI: 10.1016/j.csl.2016.01.002
- 2015 ConferenceCombining Several ASR Outputs in a Graph-Based SLU System
Marcos Calvo Lance , Lluís Felip Hurtado Oliver , Fernando García Granada , Emilio Sanchis Arnal
Iberoamerican Congress on Pattern Recognition (CIARP), 551-558
- 2015 ConferenceSelection and Aggregation Techniques for Crowdsourced Semantic Annotation Task
Marcos Calvo Lance , Riccardi, Giussepe , Fernando García Granada , Emilio Sanchis Arnal
Annual Conference of the International Speech Communication Association (INTERSPEECH), 2779-2783
- 2015 JournalDiseño y comparación de varias aproximaciones estadísticas a la Comprensión del Habla en dos tareas e idiomas distintos
Fernando García Granada , Marcos Calvo Lance , Lluís Felip Hurtado Oliver , Emilio Sanchis Arnal , Encarna Segarra Soriano
PROCESAMIENTO DEL LENGUAJE NATURAL, 31-38
- 2014 ConferenceA Multilingual Spoken Language Understanding System
Sergio Laguna Bello , Maite Giménez Fayos , Marcos Calvo Lance , Fernando García Granada , Encarna Segarra Soriano , Emilio Sanchis Arnal , Lluís Felip Hurtado Oliver
International Conference IberSPEECH (Formerly Jornadas en Tecnología del Habla), 348-353
- 2014 ConferenceObtaining parallel corpora for Multilingual Spoken Language Understanding tasks
Fernando García Granada , Marcos Calvo Lance , Emilio Sanchis Arnal , Lluís Felip Hurtado Oliver , Encarna Segarra Soriano
International Conference IberSPEECH (Formerly Jornadas en Tecnología del Habla), 208-215
- 2013 ConferenceExploiting multiple hypotheses for Multilingual Spoken Language Understanding
Marcos Calvo Lance , Fernando García Granada , Lluís Felip Hurtado Oliver , Santiago Jiménez Serrano , Emilio Sanchis Arnal
Conference on Computational Natural Language Learnings (CoNLL), 193-201
- 2013 JournalA Phonetic-Based Approach to Query-by-Example Spoken Term Detection
Lluís Felip Hurtado Oliver , Marcos Calvo Lance , Jon Ander Gomez Adrian , Fernando García Granada , Emilio Sanchis Arnal
Lecture Notes in Computer Science, 504-511
- 2013 JournalELIRF at MEDIAEVAL 2013: Similar Segments of Social Speech Task
Fernando García Granada , Emilio Sanchis Arnal , Marcos Calvo Lance , Ferran Pla Santamaría , Lluís Felip Hurtado Oliver
CEUR Workshop Proceedings, 1043, 135-136
- 2013 JournalELIRF at MEDIAEVAL 2013: Spoken Web Search Task
Jon Ander Gomez Adrian , Lluís Felip Hurtado Oliver , Marcos Calvo Lance , Emilio Sanchis Arnal
CEUR Workshop Proceedings, 59-60
- 2013 JournalExploiting Multiple ASR Outputs for a Spoken Language Understanding Task
Marcos Calvo Lance , Fernando García Granada , Lluís Felip Hurtado Oliver , Santiago Jiménez Serrano , Emilio Sanchis Arnal
Lecture Notes in Computer Science, 8113, 138-145
- 2012 ConferenceThe ELiRF Query-by-Example STD systems for the Albayzin 2012 Search on Speech Evaluation
Emilio Sanchis Arnal , Lluís Felip Hurtado Oliver , Jon Ander Gomez Adrian , Marcos Calvo Lance , Raul Fabra Boluda
International Conference IberSPEECH (Formerly Jornadas en Tecnología del Habla), 611-618
- 2012 ConferenceA methodology for obtaining concept graphs from word graphs
Marcos Calvo Lance , Jon Ander Gomez Adrian , Lluís Felip Hurtado Oliver , Emilio Sanchis Arnal
International Conference on Finite-State Methods and Natural Language Processing (FSMNLP) (Formerly: International Workshop on Finite-State Methods and Natural Language Processing), 75-79
- 2012 JournalA Multilingual SLU System Based on Semantic Decoding of Graphs of Words
Marcos Calvo Lance , Lluís Felip Hurtado Oliver , Fernando García Granada , Emilio Sanchis Arnal
Communications in Computer and Information Science, 328, 158-167
- 2012 JournalUn algoritmo para la comprensión automática del habla sobre grafos de palabras
Emilio Sanchis Arnal , Marcos Calvo Lance , Jon Ander Gomez Adrian , Lluís Felip Hurtado Oliver
PROCESAMIENTO DEL LENGUAJE NATURAL, 105-112
- 2012 JournalVoice-QA: Evaluating the Impact of Misrecognized Words on Passage Retrieval
Marcos Calvo Lance , Davide Buscaldi , Paolo Rosso
Lecture Notes in Computer Science, 7637, 462-471