现代实体链接（EL）系统构成了流行偏见，但是没有数据集以英语以外的其他语言上关注尾巴和新兴实体。我们向Hansel展示了中国人的新基准，它填补了非英国几乎没有射击和零击EL挑战的空缺。Hansel的测试集经过人工注释和审查，并采用了一种用于收集零照片EL数据集的新方法。它涵盖了新闻，社交媒体帖子和其他网络文章中的10k多种文档，Wikidata作为目标知识库。我们证明，现有的最新EL系统在Hansel上的表现不佳（R@1中的36.6％，几乎没有射击）。然后，我们建立了一个强大的基线，该基线在我们的数据集上的零射门上为46.2％的R@1分之1。我们还表明，我们的基线在TAC-KBP2015中国实体链接任务上取得了竞争成果。

Wikidata是一个经常更新，社区驱动和多语言知识图形。因此，Wikidata是实体联系的一个有吸引力的基础，这是最近发表论文的增加显而易见的。该调查侧重于四个主题：（1）存在哪些Wikidata实体链接数据集，它们是多么广泛使用，它们是如何构建的？ （2）对实体联系数据集的设计进行Wikidata的特点，如果是的话，怎么样？ （3）当前实体链接方法如何利用Wikidata的特定特征？ （4）现有实体链接方法未开发哪种Wikidata特征？本次调查显示，当前的Wikidata特定实体链接数据集在其他知识图表中的方案中的注释方案中没有不同。因此，没有提升多语言和时间依赖数据集的可能性，是自然适合维基帽的数据集。此外，我们表明大多数实体链接方法使用Wikidata以与任何其他知识图相同的方式，因为任何其他知识图都缺少了利用Wikidata特定特征来提高质量的机会。几乎所有方法都使用标签等特定属性，有时是描述，而是忽略超关系结构等特征。因此，例如，通过包括超关系图嵌入或类型信息，仍有改进的余地。许多方法还包括来自维基百科的信息，这些信息很容易与Wikidata组合并提供有价值的文本信息，Wikidata缺乏。

我们介绍了关于多语言信息访问（MIA）2022共享任务的研讨会的结果，评估了16种类型上多样性的语言中的跨语性开放回程答案（QA）系统。在此任务中，我们在14种类型上多样化的语言中调整了两个大规模的跨语性开放式质疑QA数据集，并使用了2种代表性不足的语言中的新注释的开放式QA数据：Tagalog和Tamil。四个团队提交了他们的系统。利用迭代开采的最佳系统是不同的负面示例和较大的预审慎模型达到32.2 F1，表现优于我们的基线4.5分。第二最佳系统使用实体感知的上下文化表示文档检索，并在泰米尔语（20.8 F1）方面取得了重大改进，而其他大多数系统的得分几乎为零。

我们提出了将事件链接到知识库的多语言链接的任务。我们会自动编译一个针对此任务的大型数据集，包括180万个涉及Wikidata超过10.9k事件的44种语言提及。我们提出了事件链接任务的两个变体：1）多语言，其中事件描述来自与提及的语言相同的语言，以及2）交叉语言，其中所有事件描述均以英语为单位。在提出的两个任务上，我们比较了包括BM25+（LV和Zhai，2011年）在内的多个事件链接系统以及Blink（Wu等，2020）的Biencoder和Crossencoder体系结构的多语言改编。在我们对两个任务变体的实验中，我们发现Biencoder和Crossencoder模型均显着优于BM25+基线。我们的结果还表明，跨语言任务通常比多语言任务更具挑战性。为了测试所提出的链接系统的室外概括，我们还创建了基于Wikinews的评估集。我们提出了定性分析，强调了拟议数据集捕获的各个方面，包括需要在上下文上进行时间推理并解决跨语言的各种事件描述。

We present, Naamapadam, the largest publicly available Named Entity Recognition (NER) dataset for the 11 major Indian languages from two language families. In each language, it contains more than 400k sentences annotated with a total of at least 100k entities from three standard entity categories (Person, Location and Organization) for 9 out of the 11 languages. The training dataset has been automatically created from the Samanantar parallel corpus by projecting automatically tagged entities from an English sentence to the corresponding Indian language sentence. We also create manually annotated testsets for 8 languages containing approximately 1000 sentences per language. We demonstrate the utility of the obtained dataset on existing testsets and the Naamapadam-test data for 8 Indic languages. We also release IndicNER, a multilingual mBERT model fine-tuned on the Naamapadam training set. IndicNER achieves the best F1 on the Naamapadam-test set compared to an mBERT model fine-tuned on existing datasets. IndicNER achieves an F1 score of more than 80 for 7 out of 11 Indic languages. The dataset and models are available under open-source licenses at https://ai4bharat.iitm.ac.in/naamapadam.

我们提出了多语言数据集的Multiconer，用于命名实体识别，涵盖11种语言的3个域（Wiki句子，问题和搜索查询），以及多语言和代码混合子集。该数据集旨在代表NER中的当代挑战，包括低文字方案（简短和未添加的文本），句法复杂的实体（例如电影标题）和长尾实体分布。使用基于启发式的句子采样，模板提取和插槽以及机器翻译等技术，从公共资源中汇编了26M令牌数据集。我们在数据集上应用了两个NER模型：一个基线XLM-Roberta模型和一个最先进的Gemnet模型，该模型利用了Gazetteers。基线实现了中等的性能（Macro-F1 = 54％），突出了我们数据的难度。 Gemnet使用Gazetteers，显着改善（Macro-F1 =+30％的平均改善）。甚至对于大型预训练的语言模型，多功能人也会构成挑战，我们认为它可以帮助进一步研究建立强大的NER系统。 Multiconer可在https://registry.opendata.aws/multiconer/上公开获取，我们希望该资源将有助于推进NER各个方面的研究。

GPT-3等大型自回归语言模型是几秒钟的学习者，可以在没有微调的情况下执行各种语言任务。虽然已知这些模型能够共同代表许多不同的语言，但他们的培训数据由英语主导，可能限制了它们的交叉概括。在这项工作中，我们在覆盖多种语言的平衡语料库上培训多语言自回归语言模型，并在广泛的任务中研究他们几乎没有零点的学习能力。我们最大的模型，具有75亿参数，在20多种代表语言中，在几种代表语言中，在几种代表性语言中，在几种代表性语言中，在多语言型号推理中表现出可比大小的GPT-3（在0次设置和0次拍摄设置中的绝对精度改善+ 7.4％ 4-拍摄设置中的9.4％）和自然语言推理（每次拍摄和4次设置中的每一个+ 5.4％）。在Flores-101机器翻译基准测试中，我们的模型优于GPT-3在182个翻译方向上有32个培训例子，同时超过45个方向的官方监督基线。我们介绍了模型成功和失败的位置的详细分析，特别是它尤其显示在某些任务中实现交叉语境的内容学习，而仍然存在改善表面的鲁棒性和适应没有a的任务的余地自然冻结形式。最后，我们评估我们在仇恨语音检测中以五种语言的仇恨语音检测的模型，并发现它具有与可比大小的GPT-3模型类似的限制。

Event Extraction (EE) is one of the fundamental tasks in Information Extraction (IE) that aims to recognize event mentions and their arguments (i.e., participants) from text. Due to its importance, extensive methods and resources have been developed for Event Extraction. However, one limitation of current research for EE involves the under-exploration for non-English languages in which the lack of high-quality multilingual EE datasets for model training and evaluation has been the main hindrance. To address this limitation, we propose a novel Multilingual Event Extraction dataset (MEE) that provides annotation for more than 50K event mentions in 8 typologically different languages. MEE comprehensively annotates data for entity mentions, event triggers and event arguments. We conduct extensive experiments on the proposed dataset to reveal challenges and opportunities for multilingual EE.

数据增强是自然语言处理（NLP）模型的鲁棒性评估的重要组成部分，以及增强他们培训的数据的多样性。在本文中，我们呈现NL-Cogmenter，这是一种新的参与式Python的自然语言增强框架，它支持创建两个转换（对数据的修改）和过滤器（根据特定功能的数据拆分）。我们描述了框架和初始的117个变换和23个过滤器，用于各种自然语言任务。我们通过使用其几个转换来分析流行自然语言模型的鲁棒性来证明NL-Upmenter的功效。基础架构，Datacards和稳健性分析结果在NL-Augmenter存储库上公开可用（\ url {https://github.com/gem-benchmark/nl-augmenter}）。

Event Detection (ED) is the task of identifying and classifying trigger words of event mentions in text. Despite considerable research efforts in recent years for English text, the task of ED in other languages has been significantly less explored. Switching to non-English languages, important research questions for ED include how well existing ED models perform on different languages, how challenging ED is in other languages, and how well ED knowledge and annotation can be transferred across languages. To answer those questions, it is crucial to obtain multilingual ED datasets that provide consistent event annotation for multiple languages. There exist some multilingual ED datasets; however, they tend to cover a handful of languages and mainly focus on popular ones. Many languages are not covered in existing multilingual ED datasets. In addition, the current datasets are often small and not accessible to the public. To overcome those shortcomings, we introduce a new large-scale multilingual dataset for ED (called MINION) that consistently annotates events for 8 different languages; 5 of them have not been supported by existing multilingual datasets. We also perform extensive experiments and analysis to demonstrate the challenges and transferability of ED across languages in MINION that in all call for more research effort in this area.

Translating training data into many languages has emerged as a practical solution for improving cross-lingual transfer. For tasks that involve span-level annotations, such as information extraction or question answering, an additional label projection step is required to map annotated spans onto the translated texts. Recently, a few efforts have utilized a simple mark-then-translate method to jointly perform translation and projection by inserting special markers around the labeled spans in the original sentence. However, as far as we are aware, no empirical analysis has been conducted on how this approach compares to traditional annotation projection based on word alignment. In this paper, we present an extensive empirical study across 42 languages and three tasks (QA, NER, and Event Extraction) to evaluate the effectiveness and limitations of both methods, filling an important gap in the literature. Experimental results show that our optimized version of mark-then-translate, which we call EasyProject, is easily applied to many languages and works surprisingly well, outperforming the more complex word alignment-based methods. We analyze several key factors that affect end-task performance, and show EasyProject works well because it can accurately preserve label span boundaries after translation. We will publicly release all our code and data.

Visual Entity Linking (VEL) is a task to link regions of images with their corresponding entities in Knowledge Bases (KBs), which is beneficial for many computer vision tasks such as image retrieval, image caption, and visual question answering. While existing tasks in VEL either rely on textual data to complement a multi-modal linking or only link objects with general entities, which fails to perform named entity linking on large amounts of image data. In this paper, we consider a purely Visual-based Named Entity Linking (VNEL) task, where the input only consists of an image. The task is to identify objects of interest (i.e., visual entity mentions) in images and link them to corresponding named entities in KBs. Since each entity often contains rich visual and textual information in KBs, we thus propose three different sub-tasks, i.e., visual to visual entity linking (V2VEL), visual to textual entity linking (V2TEL), and visual to visual-textual entity linking (V2VTEL). In addition, we present a high-quality human-annotated visual person linking dataset, named WIKIPerson. Based on WIKIPerson, we establish a series of baseline algorithms for the solution of each sub-task, and conduct experiments to verify the quality of proposed datasets and the effectiveness of baseline methods. We envision this work to be helpful for soliciting more works regarding VNEL in the future. The codes and datasets are publicly available at https://github.com/ict-bigdatalab/VNEL.

We present NusaCrowd, a collaborative initiative to collect and unite existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, we have has brought together 137 datasets and 117 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their effectiveness has been demonstrated in multiple experiments. NusaCrowd's data collection enables the creation of the first zero-shot benchmarks for natural language understanding and generation in Indonesian and its local languages. Furthermore, NusaCrowd brings the creation of the first multilingual automatic speech recognition benchmark in Indonesian and its local languages. Our work is intended to help advance natural language processing research in under-represented languages.

可靠的评估基准是为了可复制性和全面性而设计的，在机器学习方面取得了进步。但是，由于缺乏多语言基准，视觉和语言研究主要集中在英语任务上。为了填补这一空白，我们介绍了图像的语言理解评估基准。 Iglue通过汇总已有的数据集并创建新的数据来汇集 - 视觉问题回答，跨模式检索，扎根的推理以及跨20种不同语言的扎根成本。我们的基准测试能够评估多语言多模型用于转移学习的模型，不仅在零弹位设置中，而且还以新定义的少数图学习设置。根据对可用最新模型的评估，我们发现翻译测试转移优于零弹性转移，并且对于许多任务而言，很难利用射击的学习。此外，下游性能部分用可用的未标记文本数据进行预处理来解释，并且仅通过目标源语言的类型学距离而微弱。我们希望通过向社区释放基准来鼓励该领域的未来研究工作。

实体消除歧义（ED）是实体链接（EL）的最后一步，当候选实体根据它们出现的上下文重新登录。所有数据集，用于培训和评估EL的模型，包括方便样本，如新闻文章和推文，将实体分布的现有概率偏置传播到更频繁发生的实体。前面表明，在这种数据集上的EL系统的性能高估，因为可以通过仅仅在学习之前获得更高的精度分数。为了提供更具足够的评估基准，我们介绍了ShadowLink数据集，其中包括16K短文本代码段，其中包含实体提出。我们评估并报告在Shadowlink基准上的流行EL系统的表现。结果表明，在评估中所有EL系统的越来越少的常见实体之间的准确性差异相当差异，证明了现有概率偏差和实体的效果。

Early detection of relevant locations in a piece of news is especially important in extreme events such as environmental disasters, war conflicts, disease outbreaks, or political turmoils. Additionally, this detection also helps recommender systems to promote relevant news based on user locations. Note that, when the relevant locations are not mentioned explicitly in the text, state-of-the-art methods typically fail to recognize them because these methods rely on syntactic recognition. In contrast, by incorporating a knowledge base and connecting entities with their locations, our system successfully infers the relevant locations even when they are not mentioned explicitly in the text. To evaluate the effectiveness of our approach, and due to the lack of datasets in this area, we also contribute to the research community with a gold-standard multilingual news-location dataset, NewsLOC. It contains the annotation of the relevant locations (and their WikiData IDs) of 600+ Wikinews articles in five different languages: English, French, German, Italian, and Spanish. Through experimental evaluations, we show that our proposed system outperforms the baselines and the fine-tuned version of the model using semi-supervised data that increases the classification rate. The source code and the NewsLOC dataset are publicly available for being used by the research community at https://github.com/vsuarezpaniagua/NewsLocation.

由于看不见和新兴实体的频率，新闻中的命名实体链接（NEL）是一项具有挑战性的努力，因此需要使用无监督或零摄像的方法。但是，这种方法往往会带来警告，例如不整合新兴实体的合适知识库（例如Wikidata），缺乏可扩展性和不良的可解释性。在这里，我们考虑在Quotebank中的人歧义，这是新闻中大量的说话者归类的语言，并调查了NEL在网络规模的语料库中直观，轻巧且可扩展的启发式方法的适用性。我们表现最好的启发式歧义分别在Quotebank和Aida-Conll基准上分别占94％和63％。此外，提出的启发式方法与最先进的无监督和零摄像方法，本本系和MGenRE相比，从而成为无监督和零照片实体链接的强基础。

In this work, we introduce IndicXTREME, a benchmark consisting of nine diverse tasks covering 18 languages from the Indic sub-continent belonging to four different families. Across languages and tasks, IndicXTREME contains a total of 103 evaluation sets, of which 51 are new contributions to the literature. To maintain high quality, we only use human annotators to curate or translate\footnote{for IndicXParaphrase, where an automatic translation system is used, a second human verification and correction step is done.} our datasets. To the best of our knowledge, this is the first effort toward creating a standard benchmark for Indic languages that aims to test the zero-shot capabilities of pretrained language models. We also release IndicCorp v2, an updated and much larger version of IndicCorp that contains 20.9 billion tokens in 24 languages. We pretrain IndicBERT v2 on IndicCorp v2 and evaluate it on IndicXTREME to show that it outperforms existing multilingual language models such as XLM-R and MuRIL.

有关应答数据集和模型的研究在研究界中获得了很多关注。其中许多人释放了自己的问题应答数据集以及模型。我们在该研究领域看到了巨大的进展。本调查的目的是识别，总结和分析许多研究人员释放的现有数据集，尤其是在非英语数据集以及研究代码和评估指标等资源中。在本文中，我们审查了问题应答数据集，这些数据集可以以法语，德语，日语，中文，阿拉伯语，俄语以及多语言和交叉的问答数据集进行英语。

Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject-verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems which represent the current dominant state of the art. In this survey paper, we condense the field into a single article and first outline some of the linguistic challenges of the task, introduce the most popular datasets that are available to researchers (for both English and other languages), and summarise the various methods and techniques that have been developed with a particular focus on artificial error generation. We next describe the many different approaches to evaluation as well as concerns surrounding metric reliability, especially in relation to subjective human judgements, before concluding with an overview of recent progress and suggestions for future work and remaining challenges. We hope that this survey will serve as comprehensive resource for researchers who are new to the field or who want to be kept apprised of recent developments.