通常,在自然语言处理领域,识别指定实体是一项实用且具有挑战性的任务。由于混合的性质导致语言复杂性,因此在代码混合文本上命名的实体识别是进一步的挑战。本文介绍了CMNERONE团队在Semeval 2022共享任务11 Multiconer的提交。代码混合的NER任务旨在识别代码混合数据集中的命名实体。我们的工作包括在代码混合数据集上的命名实体识别(NER),来利用多语言数据。我们的加权平均F1得分为0.7044,即比基线大6%。
translated by 谷歌翻译
我们利用预训练的语言模型来解决两种低资源语言的复杂NER任务:中文和西班牙语。我们使用整个单词掩码(WWM)的技术来提高大型和无监督的语料库的掩盖语言建模目标。我们在微调的BERT层之上进行多个神经网络体系结构,将CRF,Bilstms和线性分类器结合在一起。我们所有的模型都优于基线,而我们的最佳性能模型在盲目测试集的评估排行榜上获得了竞争地位。
translated by 谷歌翻译
多字表达式(MWES)呈现单词组,其中整体的含义不是源于其部分的含义。处理MWE的任务在许多自然语言处理(NLP)应用中至关重要,包括机器翻译和术语提取。因此,检测MWE是一个流行的研究主题。在本文中,我们在检测MWES的任务中探索了最新的神经变压器。我们在数据集中凭经验评估了Semeval-2016任务10:检测最小的语义单元及其含义(DIMSUM)。我们表明,变压器模型的表现优于先前基于长期记忆(LSTM)的神经模型。该代码和预培训模型将免费提供给社区。
translated by 谷歌翻译
我们介绍ASNER,这是一种使用基线阿萨姆语NER模型的低资源阿萨姆语言的命名实体注释数据集。该数据集包含大约99k代币,其中包括印度总理和阿萨姆人戏剧演讲中的文字。它还包含个人名称,位置名称和地址。拟议的NER数据集可能是基于深神经的阿萨姆语言处理的重要资源。我们通过训练NER模型进行基准测试数据集并使用最先进的体系结构评估被监督的命名实体识别(NER),例如FastText,Bert,XLM-R,Flair,Muril等。我们实施了几种基线方法,标记BI-LSTM-CRF体系结构的序列。当使用Muril用作单词嵌入方法时,所有基线中最高的F1得分的准确性为80.69%。带注释的数据集和最高性能模型公开可用。
translated by 谷歌翻译
Part of Speech (POS) tagging is crucial to Natural Language Processing (NLP). It is a well-studied topic in several resource-rich languages. However, the development of computational linguistic resources is still in its infancy despite the existence of numerous languages that are historically and literary rich. Assamese, an Indian scheduled language, spoken by more than 25 million people, falls under this category. In this paper, we present a Deep Learning (DL)-based POS tagger for Assamese. The development process is divided into two stages. In the first phase, several pre-trained word embeddings are employed to train several tagging models. This allows us to evaluate the performance of the word embeddings in the POS tagging task. The top-performing model from the first phase is employed to annotate another set of new sentences. In the second phase, the model is trained further using the fresh dataset. Finally, we attain a tagging accuracy of 86.52% in F1 score. The model may serve as a baseline for further study on DL-based Assamese POS tagging.
translated by 谷歌翻译
自动言论(POS)标记是许多自然语言处理(NLP)任务的预处理步骤,例如名称实体识别(NER),语音处理,信息提取,单词sense sisse disampigation和Machine Translation。它已经在英语和欧洲语言方面取得了令人鼓舞的结果,但是使用印度语言,尤其是在Odia语言中,由于缺乏支持工具,资源和语言形态丰富性,因此尚未得到很好的探索。不幸的是,我们无法为ODIA找到一个开源POS标记,并且仅尝试为ODIA语言开发POS标记器的尝试。这项研究工作的主要贡献是介绍有条件的随机场(CRF)和基于深度学习的方法(CNN和双向长期短期记忆)来开发ODIA的语音部分。我们使用了一个公开访问的语料库,并用印度标准局(BIS)标签设定了数据集。但是,全球的大多数语言都使用了带有通用依赖项(UD)标签集注释的数据集。因此,要保持统一性,odia数据集应使用相同的标签集。因此,我们已经构建了一个从BIS标签集到UD标签集的简单映射。我们对CRF模型进行了各种特征集输入,观察到构造特征集的影响。基于深度学习的模型包括BI-LSTM网络,CNN网络,CRF层,角色序列信息和预训练的单词向量。通过使用卷积神经网络(CNN)和BI-LSTM网络提取角色序列信息。实施了神经序列标记模型的六种不同组合,并研究了其性能指标。已经观察到具有字符序列特征和预训练的单词矢量的BI-LSTM模型取得了显着的最新结果。
translated by 谷歌翻译
预先训练的上下文化文本表示模型学习自然语言的有效表示,以使IT机器可以理解。在注意机制的突破之后,已经提出了新一代预磨模的模型,以便自变压器引入以来实现了良好的性能。来自变压器(BERT)的双向编码器表示已成为语言理解的最先进的模型。尽管取得了成功,但大多数可用的型号已经在印度欧洲语言中培训,但是对代表性的语言和方言的类似研究仍然稀疏。在本文中,我们调查了培训基于单语言变换器的语言模型的可行性,以获得代表语言的特定重点是突尼斯方言。我们评估了我们的语言模型对情感分析任务,方言识别任务和阅读理解问答任务。我们表明使用嘈杂的Web爬网数据而不是结构化数据(维基百科,文章等)更方便这些非标准化语言。此外,结果表明,相对小的Web爬网数据集导致与使用较大数据集获得的那些表现相同的性能。最后,我们在所有三个下游任务中达到或改善了最先进的Tunbert模型。我们释放出Tunbert净化模型和用于微调的数据集。
translated by 谷歌翻译
文本分类是具有各种有趣应用程序的典型自然语言处理或计算语言学任务。随着社交媒体平台上的用户数量的增加,数据加速促进了有关社交媒体文本分类(SMTC)或社交媒体文本挖掘的新兴研究。与英语相比,越南人是低资源语言之一,仍然没有集中精力并彻底利用。受胶水成功的启发,我们介绍了社交媒体文本分类评估(SMTCE)基准,作为各种SMTC任务的数据集和模型的集合。借助拟议的基准,我们实施和分析了各种基于BERT的模型(Mbert,XLM-R和Distilmbert)和基于单语的BERT模型(Phobert,Vibert,Vibert,Velectra和Vibert4news)的有效性SMTCE基准。单语模型优于多语言模型,并实现所有文本分类任务的最新结果。它提供了基于基准的多语言和单语言模型的客观评估,该模型将使越南语言中有关贝尔特兰的未来研究有利。
translated by 谷歌翻译
This paper investigates the problem of Named Entity Recognition (NER) for extreme low-resource languages with only a few hundred tagged data samples. NER is a fundamental task in Natural Language Processing (NLP). A critical driver accelerating NER systems' progress is the existence of large-scale language corpora that enable NER systems to achieve outstanding performance in languages such as English and French with abundant training data. However, NER for low-resource languages remains relatively unexplored. In this paper, we introduce Mask Augmented Named Entity Recognition (MANER), a new methodology that leverages the distributional hypothesis of pre-trained masked language models (MLMs) for NER. The <mask> token in pre-trained MLMs encodes valuable semantic contextual information. MANER re-purposes the <mask> token for NER prediction. Specifically, we prepend the <mask> token to every word in a sentence for which we would like to predict the named entity tag. During training, we jointly fine-tune the MLM and a new NER prediction head attached to each <mask> token. We demonstrate that MANER is well-suited for NER in low-resource languages; our experiments show that for 100 languages with as few as 100 training examples, it improves on state-of-the-art methods by up to 48% and by 12% on average on F1 score. We also perform detailed analyses and ablation studies to understand the scenarios that are best-suited to MANER.
translated by 谷歌翻译
Code-Switching, a common phenomenon in written text and conversation, has been studied over decades by the natural language processing (NLP) research community. Initially, code-switching is intensively explored by leveraging linguistic theories and, currently, more machine-learning oriented approaches to develop models. We introduce a comprehensive systematic survey on code-switching research in natural language processing to understand the progress of the past decades and conceptualize the challenges and tasks on the code-switching topic. Finally, we summarize the trends and findings and conclude with a discussion for future direction and open questions for further investigation.
translated by 谷歌翻译
We present, Naamapadam, the largest publicly available Named Entity Recognition (NER) dataset for the 11 major Indian languages from two language families. In each language, it contains more than 400k sentences annotated with a total of at least 100k entities from three standard entity categories (Person, Location and Organization) for 9 out of the 11 languages. The training dataset has been automatically created from the Samanantar parallel corpus by projecting automatically tagged entities from an English sentence to the corresponding Indian language sentence. We also create manually annotated testsets for 8 languages containing approximately 1000 sentences per language. We demonstrate the utility of the obtained dataset on existing testsets and the Naamapadam-test data for 8 Indic languages. We also release IndicNER, a multilingual mBERT model fine-tuned on the Naamapadam training set. IndicNER achieves the best F1 on the Naamapadam-test set compared to an mBERT model fine-tuned on existing datasets. IndicNER achieves an F1 score of more than 80 for 7 out of 11 Indic languages. The dataset and models are available under open-source licenses at https://ai4bharat.iitm.ac.in/naamapadam.
translated by 谷歌翻译
This paper presents the work of restoring punctuation for ASR transcripts generated by multilingual ASR systems. The focus languages are English, Mandarin, and Malay which are three of the most popular languages in Singapore. To the best of our knowledge, this is the first system that can tackle punctuation restoration for these three languages simultaneously. Traditional approaches usually treat the task as a sequential labeling task, however, this work adopts a slot-filling approach that predicts the presence and type of punctuation marks at each word boundary. The approach is similar to the Masked-Language Model approach employed during the pre-training stages of BERT, but instead of predicting the masked word, our model predicts masked punctuation. Additionally, we find that using Jieba1 instead of only using the built-in SentencePiece tokenizer of XLM-R can significantly improve the performance of punctuating Mandarin transcripts. Experimental results on English and Mandarin IWSLT2022 datasets and Malay News show that the proposed approach achieved state-of-the-art results for Mandarin with 73.8% F1-score while maintaining a reasonable F1-score for English and Malay, i.e. 74.7% and 78% respectively. Our source code that allows reproducing the results and building a simple web-based application for demonstration purposes is available on Github.
translated by 谷歌翻译
命名实体识别是一项信息提取任务,可作为其他自然语言处理任务的预处理步骤,例如机器翻译,信息检索和问题答案。命名实体识别能够识别专有名称以及开放域文本中的时间和数字表达式。对于诸如阿拉伯语,阿姆哈拉语和希伯来语之类的闪族语言,由于这些语言的结构严重变化,指定的实体识别任务更具挑战性。在本文中,我们提出了一个基于双向长期记忆的Amharic命名实体识别系统,并带有条件随机字段层。我们注释了一种新的Amharic命名实体识别数据集(8,070个句子,具有182,691个令牌),并将合成少数群体过度采样技术应用于我们的数据集,以减轻不平衡的分类问题。我们命名的实体识别系统的F_1得分为93%,这是Amharic命名实体识别的新最新结果。
translated by 谷歌翻译
非洲语言最近是几项自然语言处理(NLP)研究的主题,这导致其在该领域的代表性大大增加。但是,在评估模型在诸如命名实体识别(NER)等任务中的性能时,大多数研究往往比数据集的质量更多地关注模型。尽管这在大多数情况下效果很好,但它并不能说明使用低资源语言进行NLP的局限性,即我们可以使用的数据集的质量和数量。本文根据数据集质量提供了各种模型的性能的分析。我们根据某些非洲NER数据集的每个句子的实体密度评估了不同的预训练模型。我们希望这项研究能够改善在低资源语言的背景下进行NLP研究的方式。
translated by 谷歌翻译
Transformer language models (TLMs) are critical for most NLP tasks, but they are difficult to create for low-resource languages because of how much pretraining data they require. In this work, we investigate two techniques for training monolingual TLMs in a low-resource setting: greatly reducing TLM size, and complementing the masked language modeling objective with two linguistically rich supervised tasks (part-of-speech tagging and dependency parsing). Results from 7 diverse languages indicate that our model, MicroBERT, is able to produce marked improvements in downstream task evaluations relative to a typical monolingual TLM pretraining approach. Specifically, we find that monolingual MicroBERT models achieve gains of up to 18% for parser LAS and 11% for NER F1 compared to a multilingual baseline, mBERT, while having less than 1% of its parameter count. We conclude reducing TLM parameter count and using labeled data for pretraining low-resource TLMs can yield large quality benefits and in some cases produce models that outperform multilingual approaches.
translated by 谷歌翻译
在本文中,我们介绍了TweetNLP,这是社交媒体中自然语言处理(NLP)的集成平台。TweetNLP支持一套多样化的NLP任务,包括诸如情感分析和命名实体识别的通用重点领域,以及社交媒体特定的任务,例如表情符号预测和进攻性语言识别。特定于任务的系统由专门用于社交媒体文本的合理大小的基于变压器的语言模型(尤其是Twitter)提供动力,无需专用硬件或云服务即可运行。TweetNLP的主要贡献是:(1)使用适合社会领域的各种特定于任务的模型,用于支持社交媒体分析的现代工具包的集成python库;(2)使用我们的模型进行无编码实验的交互式在线演示;(3)涵盖各种典型社交媒体应用的教程。
translated by 谷歌翻译
在大量人员中,在线社交媒体(OSMS)消费的广泛上升构成了遏制这些平台上仇恨内容的传播的关键问题。随着多种语言的效果越来越多,检测和表征仇恨的任务变得更加复杂。代码混合文本的微妙变化以及切换脚本仅增加了复杂性。本文介绍了哈索克2021多语种推特仇恨语音检测挑战的解决方案,由Team Precog IIIT Hyderabad。我们采用基于多语言变压器的方法,并为所有6个子任务描述了我们的架构作为挑战的一部分。在参加所有子特设券的6支球队中,我们的提交总体排名第3。
translated by 谷歌翻译
我们提供了一个新的Twitter数据语料库,该数据注释了西班牙语和英语之间的代码开关和借用。该语料库包含带有代码开关,借款和命名实体的令牌级别注释的9,500条推文。该语料库与先前的代码开关情况有所不同,因为我们试图清楚地定义和注释codeswitching and Loarding和借贷之间的边界,并且在其他单语上下文中使用时,请不要将常见的“互联网说话”('lol'等)视为代码开关。结果是一个语料库,可以在一个数据集中的Twitter上进行西班牙语 - 英语借款和代码开关的研究和建模。我们提出了使用基于变压器的语言模型对该语料库的标签进行建模的基线得分。注释本身由CC by 4.0许可发布,而其适用的文本则根据Twitter服务条款分发。
translated by 谷歌翻译
多词表达式(MWE)是一系列单词,共同提出的含义不是从其单个单词中得出的。处理MWE的任务在许多自然语言处理(NLP)应用中至关重要,包括机器翻译和术语提取。因此,在不同领域中检测MWE是一个重要的研究主题。在本文中,我们探索了最新的神经变压器,以检测花和植物名称中的MWES。我们在由植物和花朵百科全书创建的数据集上评估了不同的变压器模型。我们从经验上表明,Transformer模型模型优于基于长期记忆(LSTM)的先前神经模型。
translated by 谷歌翻译
本文报告了对十个低资源南非(SA)语言的命名实体识别(NER)的深度学习(DL)变压器架构模型的评估。此外,这些DL变压器模型与其他神经网络和机器学习(ML)NER模型进行了比较。调查结果表明,在每种语言应用离散微调参数时,变压器模型显着提高了性能。此外,微调变压器模型在低资源SA语言上与NER相比优于其他神经网络和机器学习模型。例如,变压器模型为十个SA语言中的六种语言产生了最高的F分数,包括超过条件随机字段ML型号的最高平均F变量。其他研究可以评估最近的其他自然语言处理任务和应用程序的变压器架构模型,例如短语块,机器翻译和语音段标记。
translated by 谷歌翻译