复合现象在梵语中无处不在。它用于表达思想的简洁性，同时丰富语言的词汇和结构形成。在这项工作中，我们专注于梵语复合类型标识（SACTI）任务，在其中我们考虑了识别复合词组件之间语义关系的问题。早期的方法仅依赖于从组件获得的词汇信息，而忽略最关键的上下文和句法信息，对SACTI有用。但是，SACTI任务主要是由于化合物组件之间隐式编码的上下文敏感语义关系。因此，我们提出了一种新颖的多任务学习体系结构，该体系结构结合了上下文信息，并使用形态标记和依赖性解析作为两个辅助任务来丰富互补的句法信息。与最新系统相比，SACTI基准数据集上的实验显示了6.1分（准确性）和7.7点（F1得分）绝对增益。此外，我们的多语言实验证明了拟议的架构在英语和马拉地语中的功效。代码和数据集可在https://github.com/ashishgupta2598/sacti上公开获得。

Much recent progress in applications of machine learning models to NLP has been driven by benchmarks that evaluate models across a wide variety of tasks. However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing. To this end, we introduce the Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME) benchmark, a multi-task benchmark for evaluating the cross-lingual generalization capabilities of multilingual representations across 40 languages and 9 tasks. We demonstrate that while models tested on English reach human performance on many tasks, there is still a sizable gap in the performance of cross-lingually transferred models, particularly on syntactic and sentence retrieval tasks. There is also a wide spread of results across languages. We release the benchmark 1 to encourage research on cross-lingual learning methods that transfer linguistic knowledge across a diverse and representative set of languages and tasks.

数据饥饿的深度神经网络已经将自己作为许多NLP任务的标准建立为包括传统序列标记的标准。尽管他们在高资源语言上表现最先进的表现，但它们仍然落后于低资源场景的统计计数器。一个方法来反击攻击此问题是文本增强，即，从现有数据生成新的合成训练数据点。虽然NLP最近目睹了一种文本增强技术的负载，但该领域仍然缺乏对多种语言和序列标记任务的系统性能分析。为了填补这一差距，我们调查了三类文本增强方法，其在语法（例如，裁剪子句子），令牌（例如，随机字插入）和字符（例如，字符交换）级别上执行更改。我们系统地将它们与语音标记，依赖解析和语义角色标记的分组进行了比较，用于使用各种模型的各种语言系列，包括依赖于诸如MBERT的普赖金的多语言语境化语言模型的架构。增强最显着改善了解析，然后是语音标记和语义角色标记的依赖性解析。我们发现实验技术通常在形态上丰富的语言，而不是越南语等分析语言。我们的研究结果表明，增强技术可以进一步改善基于MBERT的强基线。我们将字符级方法标识为最常见的表演者，而同义词替换和语法增强仪提供不一致的改进。最后，我们讨论了最大依赖于任务，语言对和模型类型的结果。

Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, whereby it enables effective zero-shot cross-lingual transfer of syntactic knowledge. The transfer is more successful between some languages, but it is not well understood what leads to this variation and whether it fairly reflects difference between languages. In this work, we investigate the distributions of grammatical relations induced from mBERT in the context of 24 typologically different languages. We demonstrate that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms. Such difference learnt via self-supervision plays a crucial role in the zero-shot transfer performance and can be predicted by variation in morphosyntactic properties between languages. These results suggest that mBERT properly encodes languages in a way consistent with linguistic diversity and provide insights into the mechanism of cross-lingual transfer.

We present, Naamapadam, the largest publicly available Named Entity Recognition (NER) dataset for the 11 major Indian languages from two language families. In each language, it contains more than 400k sentences annotated with a total of at least 100k entities from three standard entity categories (Person, Location and Organization) for 9 out of the 11 languages. The training dataset has been automatically created from the Samanantar parallel corpus by projecting automatically tagged entities from an English sentence to the corresponding Indian language sentence. We also create manually annotated testsets for 8 languages containing approximately 1000 sentences per language. We demonstrate the utility of the obtained dataset on existing testsets and the Naamapadam-test data for 8 Indic languages. We also release IndicNER, a multilingual mBERT model fine-tuned on the Naamapadam training set. IndicNER achieves the best F1 on the Naamapadam-test set compared to an mBERT model fine-tuned on existing datasets. IndicNER achieves an F1 score of more than 80 for 7 out of 11 Indic languages. The dataset and models are available under open-source licenses at https://ai4bharat.iitm.ac.in/naamapadam.

As an important fine-grained sentiment analysis problem, aspect-based sentiment analysis (ABSA), aiming to analyze and understand people's opinions at the aspect level, has been attracting considerable interest in the last decade. To handle ABSA in different scenarios, various tasks are introduced for analyzing different sentiment elements and their relations, including the aspect term, aspect category, opinion term, and sentiment polarity. Unlike early ABSA works focusing on a single sentiment element, many compound ABSA tasks involving multiple elements have been studied in recent years for capturing more complete aspect-level sentiment information. However, a systematic review of various ABSA tasks and their corresponding solutions is still lacking, which we aim to fill in this survey. More specifically, we provide a new taxonomy for ABSA which organizes existing studies from the axes of concerned sentiment elements, with an emphasis on recent advances of compound ABSA tasks. From the perspective of solutions, we summarize the utilization of pre-trained language models for ABSA, which improved the performance of ABSA to a new stage. Besides, techniques for building more practical ABSA systems in cross-domain/lingual scenarios are discussed. Finally, we review some emerging topics and discuss some open challenges to outlook potential future directions of ABSA.

Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks. Building on recent token-level probing work, we introduce a novel edge probing task design and construct a broad suite of sub-sentence tasks derived from the traditional structured NLP pipeline. We probe word-level contextual representations from four recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena. We find that existing models trained on language modeling and translation produce strong representations for syntactic phenomena, but only offer comparably small improvements on semantic tasks over a non-contextual baseline.

对于自然语言处理应用可能是有问题的，因为它们的含义不能从其构成词语推断出来。缺乏成功的方法方法和足够大的数据集防止了用于检测成语的机器学习方法的开发，特别是对于在训练集中不发生的表达式。我们提出了一种叫做小鼠的方法，它使用上下文嵌入来实现此目的。我们展示了一个新的多字表达式数据集，具有文字和惯用含义，并使用它根据两个最先进的上下文单词嵌入式培训分类器：Elmo和Bert。我们表明，使用两个嵌入式的深度神经网络比现有方法更好地执行，并且能够检测惯用词使用，即使对于训练集中不存在的表达式。我们展示了开发模型的交叉传输，并分析了所需数据集的大小。

We present DualNER, a simple and effective framework to make full use of both annotated source language corpus and unlabeled target language text for zero-shot cross-lingual named entity recognition (NER). In particular, we combine two complementary learning paradigms of NER, i.e., sequence labeling and span prediction, into a unified multi-task framework. After obtaining a sufficient NER model trained on the source data, we further train it on the target data in a {\it dual-teaching} manner, in which the pseudo-labels for one task are constructed from the prediction of the other task. Moreover, based on the span prediction, an entity-aware regularization is proposed to enhance the intrinsic cross-lingual alignment between the same entities in different languages. Experiments and analysis demonstrate the effectiveness of our DualNER. Code is available at https://github.com/lemon0830/dualNER.

我们提出了一种具有有限目标语言数据的交叉语言内容标记的新颖框架，这在预测性能方面显着优于现有的工作。该框架基于最近的邻居架构。它是Vanilla K-最近邻模型的现代实例化，因为我们在所有组件中使用变压器表示。我们的框架可以适应新的源语言实例，而无需从头开始侦察。与基于邻域的方法的事先工作不同，我们基于查询邻的交互对邻居信息进行编码。我们提出了两个编码方案，并使用定性和定量分析显示其有效性。我们的评估结果是来自两个不同数据集的八种语言，用于滥用语言检测，在强大的基线上，可以在F1中显示最多9.5（对于意大利语）的大量改进。平均水平，我们在拼图式多语言数据集中的三种语言中实现了3.6的F1改进，2.14在WUL数据集的F1中的改进。

多语言语言模型（\ mllms），如mbert，xlm，xlm-r，\ textit {etc。}已成为一种可行的选择，使预先估计到大量语言的力量。鉴于他们的成功在零射击转移学习中，在（i）建立更大的\ mllms〜覆盖了大量语言（ii）创建覆盖更广泛的任务和语言来评估的详尽工作基准mllms〜（iii）分析单音零点，零拍摄交叉和双语任务（iv）对Monolingual的性能，了解\ mllms〜（v）增强（通常）学习的通用语言模式（如果有的话）有限的容量\ mllms〜以提高他们在已见甚至看不见语言的表现。在这项调查中，我们审查了现有的文学，涵盖了上述与\ MLLMS有关的广泛研究领域。根据我们的调查，我们建议您有一些未来的研究方向。

首字母缩略词和长形式通常在研究文件中发现，更多的资料来自科学和法律领域的文件。在此文件中使用的许多首字母缩略词是特定于域的，很少在正常文本语料库中找到。由于这一点，基于变压器的NLP模型经常检测缩略词令牌的OOV（词汇），特别是对于非英语语言，它们的性能在提取期间将首字母缩略词与它们的长形式联系起来。此外，像BERT这样的预磨削变压器模型不专注于处理科学和法律文件。随着这些积分是这项工作背后的总体动机，我们提出了一种新颖的框架尚非：缩写式提取的字符感知BERT，其考虑文本中的字符序列，并通过屏蔽语言建模进行了科学和法律域。我们进一步使用了一个增强损失功能的目标，将最大损耗和掩码丢失术语添加到培训人物的标准交叉熵损失。我们进一步利用伪标记和对抗性数据生成来提高框架的普遍性。与各种基线相比，实验结果证明了所提出的框架的优越性。此外，我们表明，所提出的框架更适合基线模型，用于对非英语的零拍摄概括，从而加强了我们方法的有效性。我们的Team BackGprop在法国数据集中获得了最高分，丹麦和越南的最高分，在全球排行榜上的英语合法数据集中获得了第三高，用于SDU AAAI-22的Althym提取（AE）共享任务。

由于低资源语言缺乏培训数据，交叉语言机器阅读理解（XMRC）是挑战。最近的方法仅使用培训数据，以资源丰富的语言，如英语到微调大规模的跨语法预训练的语言模型。由于语言之间的巨大差异，仅由源语言微调的模型可能无法对目标语言表现良好。有趣的是，我们观察到，虽然先前方法预测的前1个结果可能经常无法达到地面真理答案，但是正确的答案通常包含在Top-K预测结果中。基于这种观察，我们开发了一种两级方法来提高模型性能。召回的第一阶段目标：我们设计一个艰难的学习（HL）算法，以最大化顶级预测包含准确答案的可能性。第二阶段专注于精确：开发了答案感知对比学习（AA-CL）机制，以了解准确答案和其他候选者之间的细差异。我们的广泛实验表明，我们的模型在两个交叉语言MRC基准数据集上显着优于一系列强大的基线。

对任何人类语言的文本的语法分析通常涉及许多基本的处理任务，例如令牌化，形态标记和依赖性解析。最先进的系统可以在具有大数据集的语言上实现这些任务的高精度，但是对于几乎没有带注释的数据的他的他加禄语等语言的结果很差。为了解决他加禄语语言的此问题，我们研究了在没有带注释的他加禄语数据的情况下使用辅助数据源来创建特定于任务模型的使用。我们还探索了单词嵌入和数据扩展的使用，以提高性能，而只有少量带注释的他加禄语数据可用。我们表明，与最先进的监督基线相比，这些零射击和几乎没有射击的方法在对域内和域外的塔加尔teact文本进行了语法分析方面进行了实质性改进。

Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferable features of language. To shed light on the linguistic knowledge they capture, we study the representations produced by several recent pretrained contextualizers (variants of ELMo, the OpenAI transformer language model, and BERT) with a suite of seventeen diverse probing tasks. We find that linear models trained on top of frozen contextual representations are competitive with state-of-the-art task-specific models in many cases, but fail on tasks requiring fine-grained linguistic knowledge (e.g., conjunct identification). To investigate the transferability of contextual word representations, we quantify differences in the transferability of individual layers within contextualizers, especially between recurrent neural networks (RNNs) and transformers. For instance, higher layers of RNNs are more taskspecific, while transformer layers do not exhibit the same monotonic trend. In addition, to better understand what makes contextual word representations transferable, we compare language model pretraining with eleven supervised pretraining tasks. For any given task, pretraining on a closely related task yields better performance than language model pretraining (which is better on average) when the pretraining dataset is fixed. However, language model pretraining on more data gives the best results.

Multi-lingual language models (LM), such as mBERT, XLM-R, mT5, mBART, have been remarkably successful in enabling natural language tasks in low-resource languages through cross-lingual transfer from high-resource ones. In this work, we try to better understand how such models, specifically mT5, transfer *any* linguistic and semantic knowledge across languages, even though no explicit cross-lingual signals are provided during pre-training. Rather, only unannotated texts from each language are presented to the model separately and independently of one another, and the model appears to implicitly learn cross-lingual connections. This raises several questions that motivate our study, such as: Are the cross-lingual connections between every language pair equally strong? What properties of source and target language impact the strength of cross-lingual transfer? Can we quantify the impact of those properties on the cross-lingual transfer? In our investigation, we analyze a pre-trained mT5 to discover the attributes of cross-lingual connections learned by the model. Through a statistical interpretation framework over 90 language pairs across three tasks, we show that transfer performance can be modeled by a few linguistic and data-derived features. These observations enable us to interpret cross-lingual understanding of the mT5 model. Through these observations, one can favorably choose the best source language for a task, and can anticipate its training data demands. A key finding of this work is that similarity of syntax, morphology and phonology are good predictors of cross-lingual transfer, significantly more than just the lexical similarity of languages. For a given language, we are able to predict zero-shot performance, that increases on a logarithmic scale with the number of few-shot target language data points.

State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models. These models are generally trained on data in a single language (usually English), and cannot be directly used beyond that language. Since collecting data in every language is not realistic, there has been a growing interest in crosslingual language understanding (XLU) and low-resource cross-language transfer. In this work, we construct an evaluation set for XLU by extending the development and test sets of the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 15 languages, including low-resource languages such as Swahili and Urdu. We hope that our dataset, dubbed XNLI, will catalyze research in cross-lingual sentence understanding by providing an informative standard evaluation task. In addition, we provide several baselines for multilingual sentence understanding, including two based on machine translation systems, and two that use parallel data to train aligned multilingual bag-of-words and LSTM encoders. We find that XNLI represents a practical and challenging evaluation suite, and that directly translating the test data yields the best performance among available baselines.

虽然对多语言视觉语言预测的模型实现了一些好处，但是当将多句预训练的视力语言模型应用于非英语数据时，各种任务和语言的最新基准测试表明，跨语性概括不佳，并且在有监督之间存在很大的差距（ ）英语表现和（零射）跨语性转移。在这项工作中，我们探讨了这些模型在零拍的跨语性视觉响应（VQA）任务上的糟糕性能，其中模型在英语视觉问题数据上进行了微调，并对7种类型上多样的语言进行了评估。我们通过三种策略改善了跨语性转移：（1）我们引入了语言的先验目标，以增加基于相似性损失以指导模型在培训期间的跨渗透损失，（2）我们学习了一个特定于任务的子网络，改善跨语性概括并减少不修改模型的方差，（3）我们使用合成代码混合来扩大培训示例，以促进源和目标语言之间的嵌入。我们使用预审计的多语言多模式变压器UC2和M3P进行的XGQA实验证明了针对7种语言提出的微调策略的一致有效性，以稀疏模型优于现有的转移方法。复制我们发现的代码和数据已公开可用。

与辅助语言的元学习已经表明了对交叉语言自然语言处理的有希望的改进。然而，以前的研究采样使用相同语言的元培训和元测试数据，这限制了模型交叉传输的能力。在本文中，我们提出了XLA-MAML，在元学习阶段执行直接交叉调整。我们对自然语言推理和问题进行零射击和几次拍摄实验。实验结果表明了我们在不同语言，任务和预磨料模型中的方法的有效性。我们还对元学习的各种交叉特定设置进行了分析，包括采样策略和并行性。

我们探索无监督模型解释和语法诱导的文本表示的深度聚类。由于这些表示是高维的，因此邮件中的开箱即用的方法不起作用。因此，我们的方法共同将表示转换为较低的聚类友好空间并群集它们。我们考虑了两种语法：在这项工作中的语音感应（POSI）和选区标签（Colab）的一部分。有趣的是，我们发现多语言伯爵（Mbert）包含令人惊讶的英语句法知识;甚至可能和英语伯特（Ebert）一样多。我们的模型可用作无可自由的探针，可使可以是较少偏见的探测方式。我们发现与监督探针相比，无监督探针显示出较高层次的益处。我们进一步注意，我们无监督的探测器使用Ebert和Mbert表示不同，特别是对于POSI。我们通过将其作为无监督的语法感应技术证明其能力来验证探针的功效。通过简单地调整输入表示，我们的探针适用于句法形式主义。我们举报了我们在45标签的英语POSI上探讨的竞争性表现，在10个语言上的12标签POSI上的最先进性能，以及Colab上的竞争结果。我们还对资源贫困语言进行零拍语法归纳，并报告强劲的结果。