本文提出了创造和管理12个主要印度语言的大型并行语言（即将扩展到23种语言）的挑战，作为由信息技术部（DIT），政府部门资助的主要财团项目的一部分。印度，并在印度的10所不同大学中平行运行。为了有效地管理这些巨大的Corpora的创建和传播过程，基于Web的（具有减少的独立版本）的注释工具ILCiann（印度语言语料集团倡议注释工具）已经开发出来。它主要是为POS注释制定的，以及由具有不同竞争力和物理位于相距远的地点的人员的管理器的管理。为了维持在创建Corpora中的一致性和标准，有必要每个人都在这个工具提供的共同平台上。

由于它们在自然语言处理工具的开发中所扮演的关键作用，因此优质树仓的价值正在稳步增长。这种树仓的创造是劳动密集型且耗时的。尤其是当考虑树库的大小时，支持注释过程的工具至关重要。但是，已经提出了各种注释工具，但是它们通常不适合土耳其语等凝集性语言。 V1是用于注释依赖关系的船，随后被用于创建手动注释的Boun Treebank（UD_TURKISH-BOUN）。在这项工作中，我们根据使用船V1获得的经验报告了依赖性注释工具船V2的设计和实施，这揭示了一些改进的机会。 V2是一种多用户和基于Web的依赖性注释工具，设计为注释用户体验以产生有效的注释。该工具的主要目标是：（1）支持以提高速度创建有效且一致的注释，（2）显着改善注释者的用户体验，（3）支持注释者之间的协作，（4）提供开放 - 通过灵活的应用程序编程接口（API）来源和易于部署的基于Web的注释工具，以使科学界受益。本文讨论了船V2的启发，设计和实施以及示例。

Magahi是一种印度东部地区的印度雅典语言。尽管具有大量扬声器，但对于语言而言，几乎没有语言资源（LR）或语言技术（LT），主要是因为其状态为非预定语言。本文介绍了开发Magahi的注释语料库的尝试。这些数据主要从Magahi中的几个博客中获取，Magahi中的一些故事集合以及Magahi的录音，它使用BIS Tagset在POS级别注释。

土著非洲语言在人工智能中被归类为服务不足，并且数字包容性和信息获取差。挑战是如何在没有必要数据的情况下使用机器学习和深度学习模型。 Kencorpus是一种肯尼亚语言语料库，打算弥合有关如何收集和存储文本和语音数据的差距，足以启用数据驱动的解决方案，例如机器翻译，多语言社区中的问题回答和转录。 Kencorpus是一种主要在肯尼亚说的三种语言的语料库（文本和语音）：斯瓦希里语，Dholuo和Luhya（方言Lumarachi，Lulogooli和Lubukusu）。该语料库打算填补开发数据集的空白，该数据集可用于低资源语言的自然语言处理和机器学习任务。这些语言中的每一种都为语言语料库贡献了文本和语音数据。数据收集是由社区，学校和合作伙伴（媒体，出版商）的研究人员完成的。 Kencorpus有5,594个项目的集合，为4,442个文本（560万字）和1,152个语音文件（177小时）。基于这些数据，还开发了其他数据集，例如Dholuo和Luhya的POS标记集（分别为50,000和93,000个单词），来自Swahili文本（7,537 QA对）的问答对，以及将文本转换为Swahili（12,400句子）。数据集可用于机器学习任务，例如文本处理，注释和翻译。该项目还在QA任务的文本和机器学习语音和机器学习中为概念系统提供了证明，最初的结果证实了Kencorpus对机器学习社区的可用性。 Kencorpus是这些低资源语言的第一个此类语料库，并且是学习和共享类似作品的经验的基础。

本文介绍了对土耳其语可用于的语料库和词汇资源的全面调查。我们审查了广泛的资源，重点关注公开可用的资源。除了提供有关可用语言资源的信息外，我们还提供了一组建议，并确定可用于在土耳其语言学和自然语言处理中进行研究和建筑应用的数据中的差距。

在本文中，我们介绍了在阿拉伯语编码的突尼斯阿拉伯语项目的最终结果，该项目是基于拉丁语的数字对话写作系统。该项目导致创建了两个集成和独立的资源：一个语料库和一个NLP工具，以通过各种语言信息来注释前者：单词分类，音译，标记，标记，pos tagging，lemmatization。我们从计算和语言方法论以及为改善结果而采用的策略中讨论我们的选择。我们报告了执行的实验，以概述我们的研究路径。最后，我们解释了为什么我们相信这些资源对计算和语言研究的潜力。关键词：突尼斯阿拉伯语，注释语料库，神经网络体系结构

We describe the design and use of the Stanford CoreNLP toolkit, an extensible pipeline that provides core natural language analysis. This toolkit is quite widely used, both in the research NLP community and also among commercial and government users of open source NLP technology. We suggest that this follows from a simple, approachable design, straightforward interfaces, the inclusion of robust and good quality analysis components, and not requiring use of a large amount of associated baggage.

The need for Question Answering datasets in low resource languages is the motivation of this research, leading to the development of Kencorpus Swahili Question Answering Dataset, KenSwQuAD. This dataset is annotated from raw story texts of Swahili low resource language, which is a predominantly spoken in Eastern African and in other parts of the world. Question Answering (QA) datasets are important for machine comprehension of natural language for tasks such as internet search and dialog systems. Machine learning systems need training data such as the gold standard Question Answering set developed in this research. The research engaged annotators to formulate QA pairs from Swahili texts collected by the Kencorpus project, a Kenyan languages corpus. The project annotated 1,445 texts from the total 2,585 texts with at least 5 QA pairs each, resulting into a final dataset of 7,526 QA pairs. A quality assurance set of 12.5% of the annotated texts confirmed that the QA pairs were all correctly annotated. A proof of concept on applying the set to the QA task confirmed that the dataset can be usable for such tasks. KenSwQuAD has also contributed to resourcing of the Swahili language.

We present POTATO, the Portable text annotation tool, a free, fully open-sourced annotation system that 1) supports labeling many types of text and multimodal data; 2) offers easy-to-configure features to maximize the productivity of both deployers and annotators (convenient templates for common ML/NLP tasks, active learning, keypress shortcuts, keyword highlights, tooltips); and 3) supports a high degree of customization (editable UI, inserting pre-screening questions, attention and qualification tests). Experiments over two annotation tasks suggest that POTATO improves labeling speed through its specially-designed productivity features, especially for long documents and complex tasks. POTATO is available at https://github.com/davidjurgens/potato and will continue to be updated.

This paper presents the OPUS ecosystem with a focus on the development of open machine translation models and tools, and their integration into end-user applications, development platforms and professional workflows. We discuss our on-going mission of increasing language coverage and translation quality, and also describe on-going work on the development of modular translation models and speed-optimized compact solutions for real-time translation on regular desktops and small devices.

The primary obstacle to developing technologies for low-resource languages is the lack of representative, usable data. In this paper, we report the deployment of technology-driven data collection methods for creating a corpus of more than 60,000 translations from Hindi to Gondi, a low-resource vulnerable language spoken by around 2.3 million tribal people in south and central India. During this process, we help expand information access in Gondi across 2 different dimensions (a) The creation of linguistic resources that can be used by the community, such as a dictionary, children's stories, Gondi translations from multiple sources and an Interactive Voice Response (IVR) based mass awareness platform; (b) Enabling its use in the digital domain by developing a Hindi-Gondi machine translation model, which is compressed by nearly 4 times to enable it's edge deployment on low-resource edge devices and in areas of little to no internet connectivity. We also present preliminary evaluations of utilizing the developed machine translation model to provide assistance to volunteers who are involved in collecting more data for the target language. Through these interventions, we not only created a refined and evaluated corpus of 26,240 Hindi-Gondi translations that was used for building the translation model but also engaged nearly 850 community members who can help take Gondi onto the internet.

即使在高度发达的国家，多达15-30％的人口只能理解使用基本词汇编写的文本。他们对日常文本的理解是有限的，这阻止了他们在社会中发挥积极作用，并就医疗保健，法律代表或民主选择做出明智的决定。词汇简化是一项自然语言处理任务，旨在通过更简单地替换复杂的词汇和表达方式来使每个人都可以理解文本，同时保留原始含义。在过去的20年中，它引起了极大的关注，并且已经针对各种语言提出了全自动词汇简化系统。该领域进步的主要障碍是缺乏用于构建和评估词汇简化系统的高质量数据集。我们提出了一个新的基准数据集，用于英语，西班牙语和（巴西）葡萄牙语中的词汇简化，并提供有关数据选择和注释程序的详细信息。这是第一个可直接比较三种语言的词汇简化系统的数据集。为了展示数据集的可用性，我们将两种具有不同体系结构（神经与非神经）的最先进的词汇简化系统适应所有三种语言（英语，西班牙语和巴西葡萄牙语），并评估他们的表演在我们的新数据集中。为了进行更公平的比较，我们使用多种评估措施来捕获系统功效的各个方面，并讨论其优势和缺点。我们发现，最先进的神经词汇简化系统优于所有三种语言中最先进的非神经词汇简化系统。更重要的是，我们发现最先进的神经词汇简化系统对英语的表现要比西班牙和葡萄牙语要好得多。

卡雷利亚共和国的波罗的海语言的研究越来越重视是语料库语言学的方法和工具。自2016年以来，Karelian研究中心的语言学家，数学家和程序员一直在与VEPS和Karelian语言的开放语料库（VEPKAR）合作，这是2009年创建的VEPS Corpus的扩展。和VEP，与它们相关的多功能字典以及具有高级搜索系统的软件，使用各种文本（语言，流派等）和许多语言类别（在文本中实现了文本中的词汇和语法搜索，这要归功于Word的生成器我们之前创建的表单）。编译了3000个文本的语料库，上传和标记了文本，将文本分类为语言，方言，类型和流派的系统，并创建了单词形式的生成器。未来的计划包括开发用于使用音频记录的语音模块和使用形态分析输出的句法标记模块。由于语料库管理器和正在进行的VEPKAR的持续功能进步，并具有新的材料和文本标记，用户可以处理广泛的科学和应用任务。在创建全国性国家VEPKAR语料库时，其开发商和经理在19-21世纪努力保护和展示VEP和Karelian语言状态。

This article presents morphologically-annotated Yemeni, Sudanese, Iraqi, and Libyan Arabic dialects Lisan corpora. Lisan features around 1.2 million tokens. We collected the content of the corpora from several social media platforms. The Yemeni corpus (~ 1.05M tokens) was collected automatically from Twitter. The corpora of the other three dialects (~ 50K tokens each) came manually from Facebook and YouTube posts and comments. Thirty five (35) annotators who are native speakers of the target dialects carried out the annotations. The annotators segemented all words in the four corpora into prefixes, stems and suffixes and labeled each with different morphological features such as part of speech, lemma, and a gloss in English. An Arabic Dialect Annotation Toolkit ADAT was developped for the purpose of the annation. The annotators were trained on a set of guidelines and on how to use ADAT. We developed ADAT to assist the annotators and to ensure compatibility with SAMA and Curras tagsets. The tool is open source, and the four corpora are also available online.

While the NLP community is generally aware of resource disparities among languages, we lack research that quantifies the extent and types of such disparity. Prior surveys estimating the availability of resources based on the number of datasets can be misleading as dataset quality varies: many datasets are automatically induced or translated from English data. To provide a more comprehensive picture of language resources, we examine the characteristics of 156 publicly available NLP datasets. We manually annotate how they are created, including input text and label sources and tools used to build them, and what they study, tasks they address and motivations for their creation. After quantifying the qualitative NLP resource gap across languages, we discuss how to improve data collection in low-resource languages. We survey language-proficient NLP researchers and crowd workers per language, finding that their estimated availability correlates with dataset availability. Through crowdsourcing experiments, we identify strategies for collecting high-quality multilingual data on the Mechanical Turk platform. We conclude by making macro and micro-level suggestions to the NLP community and individual researchers for future multilingual data development.

In this work, we introduce IndicXTREME, a benchmark consisting of nine diverse tasks covering 18 languages from the Indic sub-continent belonging to four different families. Across languages and tasks, IndicXTREME contains a total of 103 evaluation sets, of which 51 are new contributions to the literature. To maintain high quality, we only use human annotators to curate or translate\footnote{for IndicXParaphrase, where an automatic translation system is used, a second human verification and correction step is done.} our datasets. To the best of our knowledge, this is the first effort toward creating a standard benchmark for Indic languages that aims to test the zero-shot capabilities of pretrained language models. We also release IndicCorp v2, an updated and much larger version of IndicCorp that contains 20.9 billion tokens in 24 languages. We pretrain IndicBERT v2 on IndicCorp v2 and evaluate it on IndicXTREME to show that it outperforms existing multilingual language models such as XLM-R and MuRIL.

我们介绍了游戏（丁）的对话，这是一本关于现实生活，口头，自发的多方对话的手动抄录，棋盘游戏Catan的法语玩家之间的对话。我们的目标是为法语提供高质量的资源，由长时间的对话组成，以促进他们的研究风格（Asher等，2016）。在一般的对话环境中，参与者共享个人信息，这使得不可能自由公开地传播资源。在丁（Ding）中，参与者的注意力集中在游戏上，这阻止了他们谈论自己。此外，我们正在通过注释（Cruz Blandon等，2019）对对话中问题的性质进行研究，以开发更自然的自动对话系统。

意大利的特征是欧洲一种一种独一无二的语言多样性格局，该景观暗中编码了当地知识，文化传统，艺术表达及其演讲者的历史。但是，意大利的30多种语言品种有几代人内消失的风险。语言技术在保存濒危语言方面具有主要作用，但是目前，它在资源不足，主要缺乏标准拼写术的品种中挣扎，主要用于口语环境。在本文中，我们介绍了意大利的语言背景，并讨论了意大利语言品种开发NLP技术面临的挑战。我们提供潜在的方向，并倡导从以机器为中心转向以说话者为中心的NLP的范式转变。最后，我们建议建立一个当地社区，旨在为意大利语言和方言的言语和语言技术负责，参与式发展。

We study politeness phenomena in nine typologically diverse languages. Politeness is an important facet of communication and is sometimes argued to be cultural-specific, yet existing computational linguistic study is limited to English. We create TyDiP, a dataset containing three-way politeness annotations for 500 examples in each language, totaling 4.5K examples. We evaluate how well multilingual models can identify politeness levels -- they show a fairly robust zero-shot transfer ability, yet fall short of estimated human accuracy significantly. We further study mapping the English politeness strategy lexicon into nine languages via automatic translation and lexicon induction, analyzing whether each strategy's impact stays consistent across languages. Lastly, we empirically study the complicated relationship between formality and politeness through transfer experiments. We hope our dataset will support various research questions and applications, from evaluating multilingual models to constructing polite multilingual agents.

近年来，基于变压器的模型已导致自然语言处理的语言建模取得重大进步。但是，他们需要大量的数据接受（预先）训练，并且除英语以外的语言中缺乏语料库。最近，一些计划提出了从自动网络爬行获得的多语言数据集。但是，西班牙语的结果具有重要的缺点，因为与其他语言相比，它们要么太小，要么呈现出较低的质量，从而获得了次优的清洁和重复数据删除。在本文中，我们介绍了Escorpius，这是一种西班牙爬行语料库，该语料库是从附近的1 pb普通爬网数据中获得的。它是西班牙语中最广泛的语料库，其提取，纯化和重复数据删除的质量水平。我们的数据策划过程涉及一条新型的高度平行清洁管道，并包含一系列重复数据删除机制，以确保文档和段落边界的完整性。此外，我们同时维护源网页URL和WARC Shard Origin URL，以抱怨欧盟法规。 Escorpius已根据CC BY-NC-ND 4.0许可发布，可在HuggingFace上获得。