Personal assistants, automatic speech recognizers and dialogue understanding systems are becoming more critical in our interconnected digital world. A clear example is air traffic control (ATC) communications. ATC aims at guiding aircraft and controlling the airspace in a safe and optimal manner. These voice-based dialogues are carried between an air traffic controller (ATCO) and pilots via very-high frequency radio channels. In order to incorporate these novel technologies into ATC (low-resource domain), large-scale annotated datasets are required to develop the data-driven AI systems. Two examples are automatic speech recognition (ASR) and natural language understanding (NLU). In this paper, we introduce the ATCO2 corpus, a dataset that aims at fostering research on the challenging ATC field, which has lagged behind due to lack of annotated data. The ATCO2 corpus covers 1) data collection and pre-processing, 2) pseudo-annotations of speech data, and 3) extraction of ATC-related named entities. The ATCO2 corpus is split into three subsets. 1) ATCO2-test-set corpus contains 4 hours of ATC speech with manual transcripts and a subset with gold annotations for named-entity recognition (callsign, command, value). 2) The ATCO2-PL-set corpus consists of 5281 hours of unlabeled ATC data enriched with automatic transcripts from an in-domain speech recognizer, contextual information, speaker turn information, signal-to-noise ratio estimate and English language detection score per sample. Both available for purchase through ELDA at http://catalog.elra.info/en-us/repository/browse/ELRA-S0484. 3) The ATCO2-test-set-1h corpus is a one-hour subset from the original test set corpus, that we are offering for free at https://www.atco2.org/data. We expect the ATCO2 corpus will foster research on robust ASR and NLU not only in the field of ATC communications but also in the general research community.

This paper describes a simple yet efficient repetition-based modular system for speeding up air-traffic controllers (ATCos) training. E.g., a human pilot is still required in EUROCONTROL's ESCAPE lite simulator (see https://www.eurocontrol.int/simulator/escape) during ATCo training. However, this need can be substituted by an automatic system that could act as a pilot. In this paper, we aim to develop and integrate a pseudo-pilot agent into the ATCo training pipeline by merging diverse artificial intelligence (AI) powered modules. The system understands the voice communications issued by the ATCo, and, in turn, it generates a spoken prompt that follows the pilot's phraseology to the initial communication. Our system mainly relies on open-source AI tools and air traffic control (ATC) databases, thus, proving its simplicity and ease of replicability. The overall pipeline is composed of the following: (1) a submodule that receives and pre-processes the input stream of raw audio, (2) an automatic speech recognition (ASR) system that transforms audio into a sequence of words; (3) a high-level ATC-related entity parser, which extracts relevant information from the communication, i.e., callsigns and commands, and finally, (4) a speech synthesizer submodule that generates responses based on the high-level ATC entities previously extracted. Overall, we show that this system could pave the way toward developing a real proof-of-concept pseudo-pilot system. Hence, speeding up the training of ATCos while drastically reducing its overall cost.

Automatic Speech Recognition (ASR) for air traffic control is generally trained by pooling Air Traffic Controller (ATCO) and pilot data into one set. This is motivated by the fact that pilot's voice communications are more scarce than ATCOs. Due to this data imbalance and other reasons (e.g., varying acoustic conditions), the speech from ATCOs is usually recognized more accurately than from pilots. Automatically identifying the speaker roles is a challenging task, especially in the case of the noisy voice recordings collected using Very High Frequency (VHF) receivers or due to the unavailability of the push-to-talk (PTT) signal, i.e., both audio channels are mixed. In this work, we propose to (1) automatically segment the ATCO and pilot data based on an intuitive approach exploiting ASR transcripts and (2) subsequently consider an automatic recognition of ATCOs' and pilots' voice as two separate tasks. Our work is performed on VHF audio data with high noise levels, i.e., signal-to-noise (SNR) ratios below 15 dB, as this data is recognized to be helpful for various speech-based machine-learning tasks. Specifically, for the speaker role identification task, the module is represented by a simple yet efficient knowledge-based system exploiting a grammar defined by the International Civil Aviation Organization (ICAO). The system accepts text as the input, either manually verified annotations or automatically generated transcripts. The developed approach provides an average accuracy in speaker role identification of about 83%. Finally, we show that training an acoustic model for ASR tasks separately (i.e., separate models for ATCOs and pilots) or using a multitask approach is well suited for the noisy data and outperforms the traditional ASR system where all data is pooled together.

自动语音识别（ASR）是一个复杂和具有挑战性的任务。近年来，该地区出现了重大进展。特别是对于巴西葡萄牙语（BP）语言，在2020年的下半年，有大约376小时的公众可供ASR任务。在2021年初发布新数据集，这个数字增加到574小时。但是，现有资源由仅包含读取和准备的演讲的Audios组成。缺少数据集包括自发性语音，这在不同的ASR应用中是必不可少的。本文介绍了Coraa（注释Audios语料库）V1。使用290.77小时，在包含验证对（音频转录）的BP中ASR的公共可用数据集。科拉还含有欧洲葡萄牙音像（4.69小时）。我们还提供了一个基于Wav2VEC 2.0 XLSR-53的公共ASR模型，并通过CoraA进行微调。我们的模型在CoraA测试集中实现了24.18％的单词误差率，并且在常见的语音测试集上为20.08％。测量字符错误率时，我们分别获得11.02％和6.34％，分别为CoraA和常见声音。 Coraa Corpora在自发言论中与BP中的改进ASR模型进行了组装，并激励年轻研究人员开始研究葡萄牙语的ASR。所有Corpora都在CC By-NC-ND 4.0许可证下公开提供Https://github.com/nilc-nlp/coraa。

自动语音识别和文本到语音系统主要以监督方式培训，需要高质量，准确标记的语音数据集。在这项工作中，我们研究语音数据的常见问题，并为语音数据集的构建和交互式错误分析引入工具箱。施工工具基于K \“urzinger等。工作，并且，尽我们所知，数据集探索工具是世界上第一个这类开源工具。我们演示了如何应用这些工具来创建一个俄语语音数据集并分析现有语音数据集（多语种LibrisPeech，Mozilla Common语音）。该工具是开放的，作为Nemo框架的一部分。

构建可用的无线电监控自动语音识别（ASR）系统是资源不足的语言的一项挑战性任务，但这在广播是公众沟通和讨论的主要媒介的社会中至关重要。联合国在乌干达的最初努力证明了如何理解被社交媒体排除在社交媒体中的农村人的看法在国家规划中很重要。但是，由于缺乏转录的语音数据集，这些努力正受到挑战。在本文中，Makerere人工智能研究实验室发布了155小时的Luganda Radio演讲语料库。据我们所知，这是撒哈拉以南非洲第一个公开可用的广播数据集。本文描述了语音语料库的开发，并使用开源语音识别工具包Coqui STT Toolkit提出了基线Luganda ASR绩效结果。

口语理解（SLU）是大多数人机相互作用系统中的核心任务。随着智能家居，智能手机和智能扬声器的出现，SLU已成为该行业的关键技术。在经典的SLU方法中，自动语音识别（ASR）模块将语音信号转录为文本表示，自然语言理解（NLU）模块从中提取语义信息。最近，基于深神经网络的端到端SLU（E2E SLU）已经获得了动力，因为它受益于ASR和NLU部分的联合优化，因此限制了管道架构的误差效应的级联反应。但是，对于E2E模型用于预测语音输入的概念和意图的实际语言特性知之甚少。在本文中，我们提出了一项研究，以确定E2E模型执行SLU任务的信号特征和其他语言特性。该研究是在必须处理非英语（此处法语）语音命令的智能房屋的应用领域进行的。结果表明，良好的E2E SLU性能并不总是需要完美的ASR功能。此外，结果表明，与管道模型相比，E2E模型在处理背景噪声和句法变化方面具有出色的功能。最后，更细粒度的分析表明，E2E模型使用输入信号的音调信息来识别语音命令概念。本文概述的结果和方法提供了一个跳板，以进一步分析语音处理中的E2E模型。

We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning. When compared to humans, the models approach their accuracy and robustness. We are releasing models and inference code to serve as a foundation for further work on robust speech processing.

通过共享数据集和基准，已经促进了语音处理的进展。历史上，这些都集中在自动语音识别（ASR），扬声器标识或其他较低级别的任务上。兴趣在更高层次的口语中越来越多，理解任务，包括使用端到端模型，但是此类任务的注释数据集较少。与此同时，最近的工作显示了预先培训通用表示的可能性，然后使用相对较少标记的数据进行微调的多个任务。我们建议为口语语言理解（屠宰）创建一套基准任务，由有限尺寸标记的培训集和相应的评估集组成。该资源将允许研究界跟踪进度，评估高级任务的预先接受预期的表示，并研究开放的问题，例如管道与端到端方法的实用性。我们介绍了雪橇基准套件的第一阶段，包括指定实体识别，情感分析和相应数据集上的ASR。我们专注于自然产生的（未读取或综合）语音和自由可用的数据集。我们为VoxceReb和Voxpopuli数据集的子集提供新的转录和注释，基线模型的评估指标和结果，以及重现基线的开源工具包，并评估新模型。

人民的言论是自由下载的30,000小时，并在CC-BY-SA下进行学术和商业用途的许可的受监管的会话英语语音识别数据集（具有CC-by子集）。通过使用现有转录搜索适当许可的音频数据来通过搜索互联网来收集数据。我们描述了我们的数据收集方法，并在Apache 2.0许可证下发布了我们的数据收集系统。我们表明，在此数据集上培训的模型在Librispeech的测试清洁测试集上实现了9.98％的单词错误率。最后，我们讨论了围绕创建一个相当大量的机器学习的法律和道德问题，并计划继续维护项目的计划根据MLCommons的赞助。

开发语音技术是对低资源语言的挑战，其中注释和原始语音数据稀疏。马耳他是一种这样的语言。近年来，对马耳他的计算处理有所增加，包括语音技术，但后者的资源仍然稀疏。在本文中，我们考虑提高这些语言的语音识别的数据增强技术，专注于马耳他作为测试用例。我们考虑三种不同类型的数据增强：无监督的培训，多语言培训和合成演讲的使用作为培训数据。目标是确定这些技术或它们的组合，是改善起始点是大约7小时转录语音的语言的语言的最有效。我们的结果表明，在这里研究了三种数据增强技术，导致我们在不使用语言模型的情况下实现15％的绝对增长。

This paper presents the OPUS ecosystem with a focus on the development of open machine translation models and tools, and their integration into end-user applications, development platforms and professional workflows. We discuss our on-going mission of increasing language coverage and translation quality, and also describe on-going work on the development of modular translation models and speed-optimized compact solutions for real-time translation on regular desktops and small devices.

扬声器日流是一个标签音频或视频录制的任务，与扬声器身份或短暂的任务标记对应于扬声器标识的类，以识别“谁谈到何时发表讲话”。在早期，对MultiSpeaker录音的语音识别开发了扬声器日益衰退算法，以使扬声器自适应处理能够实现扬声器自适应处理。这些算法还将自己的价值作为独立应用程序随着时间的推移，为诸如音频检索等下游任务提供特定于扬声器的核算。最近，随着深度学习技术的出现，这在讲话应用领域的研究和实践中引起了革命性的变化，对扬声器日益改善已经进行了快速进步。在本文中，我们不仅审查了扬声器日益改善技术的历史发展，而且还审查了神经扬声器日益改善方法的最新进步。此外，我们讨论了扬声器日复速度系统如何与语音识别应用相结合，以及最近深度学习的激增是如何引领联合建模这两个组件互相互补的方式。通过考虑这种令人兴奋的技术趋势，我们认为本文对社区提供了有价值的贡献，以通过巩固具有神经方法的最新发展，从而促进更有效的扬声器日益改善进一步进展。

确保适当的标点符号和字母外壳是朝向应用复杂的自然语言处理算法的关键预处理步骤。这对于缺少标点符号和壳体的文本源，例如自动语音识别系统的原始输出。此外，简短的短信和微博的平台提供不可靠且经常错误的标点符号和套管。本调查概述了历史和最先进的技术，用于恢复标点符号和纠正单词套管。此外，突出了当前的挑战和研究方向。

接受社会辅助机器人的基本功能之一是其与环境中其他代理商的通信能力。在Robin项目的背景下，调查了通过与机器人的语音互动的情境对话。本文介绍了具有深度神经网络的不同语音识别实验，专注于生产快速（从网络本身的100ms延迟下），而仍然可靠的型号。即使关键所需特性之一是低延迟，最终的深度神经网络模型也能实现识别罗马尼亚语的最新状态，以获得9.91％的字错误率（WER），当与语言模型相结合，从而改善以前的结果同时提供了改进的运行时性能。此外，我们探索了两个模块，用于校正ASR输出（连字符和大写恢复和未知单词校正），针对Robin项目的目标（在封闭的微观世界中对话）。我们根据API设计模块化架构，允许整合引擎（机器人或外部）根据需要将可用模块链接在一起。最后，我们通过将其集成在相关平台中并通过上传文件或录制新的语音来测试所提出的设计。

Spoken language understanding (SLU) tasks have been studied for many decades in the speech research community, but have not received as much attention as lower-level tasks like speech and speaker recognition. In particular, there are not nearly as many SLU task benchmarks, and many of the existing ones use data that is not freely available to all researchers. Recent work has begun to introduce such benchmark datasets for several tasks. In this work, we introduce several new annotated SLU benchmark tasks based on freely available speech data, which complement existing benchmarks and address gaps in the SLU evaluation landscape. We contribute four tasks: question answering and summarization involve inference over longer speech sequences; named entity localization addresses the speech-specific task of locating the targeted content in the signal; dialog act classification identifies the function of a given speech utterance. We follow the blueprint of the Spoken Language Understanding Evaluation (SLUE) benchmark suite. In order to facilitate the development of SLU models that leverage the success of pre-trained speech representations, we will be publishing for each task (i) annotations for a relatively small fine-tuning set, (ii) annotated development and test sets, and (iii) baseline models for easy reproducibility and comparisons. In this work, we present the details of data collection and annotation and the performance of the baseline models. We also perform sensitivity analysis of pipeline models' performance (speech recognizer + text model) to the speech recognition accuracy, using more than 20 state-of-the-art speech recognition models.

Modern speech recognition systems exhibits rapid performance degradation under domain shift. This issue is especially prevalent in data-scarce settings, such as low-resource languages, where diversity of training data is limited. In this work we propose M2DS2, a simple and sample-efficient finetuning strategy for large pretrained speech models, based on mixed source and target domain self-supervision. We find that including source domain self-supervision stabilizes training and avoids mode collapse of the latent representations. For evaluation, we collect HParl, a $120$ hour speech corpus for Greek, consisting of plenary sessions in the Greek Parliament. We merge HParl with two popular Greek corpora to create GREC-MD, a test-bed for multi-domain evaluation of Greek ASR systems. In our experiments we find that, while other Unsupervised Domain Adaptation baselines fail in this resource-constrained environment, M2DS2 yields significant improvements for cross-domain adaptation, even when a only a few hours of in-domain audio are available. When we relax the problem in a weakly supervised setting, we find that independent adaptation for audio using M2DS2 and language using simple LM augmentation techniques is particularly effective, yielding word error rates comparable to the fully supervised baselines.

本文介绍了对土耳其语可用于的语料库和词汇资源的全面调查。我们审查了广泛的资源，重点关注公开可用的资源。除了提供有关可用语言资源的信息外，我们还提供了一组建议，并确定可用于在土耳其语言学和自然语言处理中进行研究和建筑应用的数据中的差距。

AI研究中的基石是创建和采用标准化培训和测试数据集，以指定最新模型的进度。一个特别成功的例子是用于培训和评估英语自然语言理解（NLU）模型的胶水数据集。围绕基于BERT的语言模型的大量研究围绕着胶水中NLU任务的性能改进。为了评估其他语言的语言模型，创建了几个特定语言的胶水数据集。语音语言理解（SLU）的领域遵循了类似的轨迹。大型自我监督模型（例如WAV2VEC2）的成功实现了具有相对易于访问的未标记数据的语音模型。然后可以在SLU任务（例如出色的基准测试）上评估这些模型。在这项工作中，我们将其扩展到通过释放Indicsuperb基准测试来指示语言。具体来说，我们做出以下三项贡献。 （i）我们收集了Kathbath，其中包含来自印度203个地区的1,218个贡献者的12个印度语言的1,684小时的标记语音数据。 （ii）使用Kathbath，我们在6个语音任务中创建基准：自动语音识别，扬声器验证，说话者识别（单声道/多），语言识别，逐个示例查询以及对12种语言的关键字发现。 （iii）在发布的基准测试中，我们与常用的基线Fbank一起训练和评估不同的自我监督模型。我们表明，在大多数任务上，特定于语言的微调模型比基线更准确，包括对于语言识别任务的76 \％差距。但是，对于说话者识别，在大型数据集上训练的自我监督模型证明了一个优势。我们希望Indicsuperb有助于发展印度语言的语音语言理解模型的进步。

The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 3rd International Workshop on Reading Music Systems, held in Alicante on the 23rd of July 2021.