这项工作检查了使用音节嵌入的可能性,而不是经常使用的$ n $ -gram embedings,作为子字嵌入。我们调查这两种语言:英语和荷兰语。为此,我们还翻译了两个标准的英语单词嵌入评估数据集,WordsIm353和Semeval-2017,达到荷兰语。此外,我们为研究界提供了两种语言的音节分解数据集。我们将我们的方法与Full Word和$ n $ -ghmm嵌入式进行比较。与完整的单词嵌入式相比,我们获得了英语模型,距离较小的20至30倍,同时保留80%的性能。对于荷兰语,型号为70%性能保留的15倍。虽然比我们使用的$ N $ -gram基线更准确,但我们的型号可以在几分钟内接受培训,而不是$ n $ -gram方法的小时。我们在未来的工作中确定了升级性能的道路。所有代码都被公开可用,以及我们收集的英语和荷兰音节分解和荷兰评估集转换。
translated by 谷歌翻译
对于自然语言处理应用可能是有问题的,因为它们的含义不能从其构成词语推断出来。缺乏成功的方法方法和足够大的数据集防止了用于检测成语的机器学习方法的开发,特别是对于在训练集中不发生的表达式。我们提出了一种叫做小鼠的方法,它使用上下文嵌入来实现此目的。我们展示了一个新的多字表达式数据集,具有文字和惯用含义,并使用它根据两个最先进的上下文单词嵌入式培训分类器:Elmo和Bert。我们表明,使用两个嵌入式的深度神经网络比现有方法更好地执行,并且能够检测惯用词使用,即使对于训练集中不存在的表达式。我们展示了开发模型的交叉传输,并分析了所需数据集的大小。
translated by 谷歌翻译
我们介绍了Paranames,这是一种多语言并行名称资源,由1.18亿个名称组成,涉及400种语言。为1360万个实体提供了名称,这些实体映射到标准化实体类型(每/loc/org)。使用Wikidata作为来源,我们创建了此类类型的最大资源。我们描述了我们过滤和标准化数据以提供最佳质量的方法。PANAMES对于多语言语言处理非常有用,既可以定义名称翻译/音译的任务,又可以作为任务的补充数据,例如命名实体识别和链接。我们通过训练与英文和英语的规范名称翻译的多语言模型来展示对照群的应用。我们的资源是根据https://github.com/bltlab/paranames发布的创意共享许可证(CC By 4.0)发布的。
translated by 谷歌翻译
Subword-level models have been the dominant paradigm in NLP. However, character-level models have the benefit of seeing each character individually, providing the model with more detailed information that ultimately could lead to better models. Recent works have shown character-level models to be competitive with subword models, but costly in terms of time and computation. Character-level models with a downsampling component alleviate this, but at the cost of quality, particularly for machine translation. This work analyzes the problems of previous downsampling methods and introduces a novel downsampling method which is informed by subwords. This new downsampling method not only outperforms existing downsampling methods, showing that downsampling characters can be done without sacrificing quality, but also leads to promising performance compared to subword models for translation.
translated by 谷歌翻译
在本文中,我们分享了我们努力建立能够翻译一千多种语言的实用机器翻译(MT)系统的发现。我们在三个研究领域中描述了结果:(i)通过利用半监督预训练的语言识别和开发数据驱动的过滤技术来构建1500多种语言的清洁,网挖数据集; (ii)通过利用大规模的多语言模型来开发用于服务不足的语言的实用MT模型,该模型训练了有监督的并行数据,以使用100多种高资源语言和单语言数据集,以增加1000多种语言; (iii)研究这些语言的评估指标的局限性,并对我们MT模型的输出进行定性分析,突出显示了这些类型模型的几种频繁误差模式。我们希望我们的工作为旨在为当前研究的语言构建MT系统的从业者提供有用的见解,并突出显示可以补充Data-Sparse设置中大量多语言模型的弱点的研究方向。
translated by 谷歌翻译
Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.
translated by 谷歌翻译
本文介绍了Persian的HMBLOGS语料库,作为一种低资源语言。此语料库已根据波斯博客的空间的一段时间内基于大约15岁的近2000万博客帖子编制的,包括超过68亿令牌。可以声称,此语料库目前是最大的波斯语料库,这些语料库是为波斯语而独立制定的。该语料库以原始和预处理的形式呈现,并且基于预处理的语料库,产生了一些单词嵌入模型。通过提供的模型,将HMBlogs与波斯中可用的一些最重要的公司进行比较,结果表明了HMBLOGS语料库的优势。这些评估还提供了语料库,评估数据集,模型生产方法,不同的高参数甚至评估方法的重要性和影响。除了评估语料库及其产生的语言模型之外,该研究还提供了一个语义类比数据集。
translated by 谷歌翻译
音译是NLP域中的一项任务,其中输出单词是使用任何外语字母编写的类似单词。如今,该系统已针对多种语言对开发,涉及英语作为源或目标单词,并在Google Translate和聊天机器人等多个地方部署。但是,在指示语言的领域进行的研究很少进行,将其译为其他指示语言。本文展示了一个基于变压器(具有一些修改)的多语言模型,该模型比该域中的所有现有模型都可以显着更高的性能和准确性,并且比最先进的模型获得了更好的结果。本文显示了一个模型,该模型可以在以下五种语言之间进行任何一对 - 英语,印地语,孟加拉语,卡纳达语和泰米尔语之间的音译。它适用于语言在任何书面任务中都是通信的障碍的情况。该模型击败了最先进的(对于上述五种语言中的所有对 - 英语,印地语,孟加拉语,卡纳达语和泰米尔语),并获得了80.7%的前1位准确性得分,比比当前最佳结果。此外,该模型在语音准确性方面达到了93.5%(音译主要是基于语音/声音的任务)。
translated by 谷歌翻译
The relationship between words in a sentence often tells us more about the underlying semantic content of a document than its actual words, individually. In this work, we propose two novel algorithms, called Flexible Lexical Chain II and Fixed Lexical Chain II. These algorithms combine the semantic relations derived from lexical chains, prior knowledge from lexical databases, and the robustness of the distributional hypothesis in word embeddings as building blocks forming a single system. In short, our approach has three main contributions: (i) a set of techniques that fully integrate word embeddings and lexical chains; (ii) a more robust semantic representation that considers the latent relation between words in a document; and (iii) lightweight word embeddings models that can be extended to any natural language task. We intend to assess the knowledge of pre-trained models to evaluate their robustness in the document classification task. The proposed techniques are tested against seven word embeddings algorithms using five different machine learning classifiers over six scenarios in the document classification task. Our results show the integration between lexical chains and word embeddings representations sustain state-of-the-art results, even against more complex systems.
translated by 谷歌翻译
链接的开放数据实践导致了过去十年中网络上结构化数据的显着增长。这样的结构化数据以机器可读的方式描述了现实世界实体,并为自然语言处理领域的研究创造了前所未有的机会。但是,缺乏有关如何使用此类数据,哪种任务以及它们在多大程度上对这些任务有用的研究。这项工作着重于电子商务领域,以探索利用此类结构化数据来创建可能用于产品分类和链接的语言资源的方法。我们以RDF N四分之一的形式处理数十亿个结构化数据点,以创建数百万个与产品相关的语料库单词,后来以三种不同的方式用于创建语言资源:培训单词嵌入模型,继续预训练类似于Bert的语言模型和训练机器翻译模型,这些模型被用作生成产品相关的关键字的代理。我们对大量基准测试的评估表明,嵌入单词是提高这两个任务准确性的最可靠和一致的方法(在某些数据集中,宏观 - 平均F1中最高6.9个百分点)。但是,其他两种方法并不那么有用。我们的分析表明,这可能是由于许多原因,包括结构化数据中的偏置域表示以及缺乏词汇覆盖范围。我们分享我们的数据集,并讨论如何将我们所学到的经验教训朝着这一方向介绍未来的研究。
translated by 谷歌翻译
这项工作介绍了我们培训神经网络的方法,以检测印地语和孟加利的仇恨语篇文本。我们还探讨了如何将转移学习应用于学习这些语言,因为它们具有相同的原点,因此类似于某些延伸。尽管整个实验以低计算能力进行,但所获得的结果也与其他,更昂贵,模型的结果相当。此外,由于使用中的培训数据相对较小,而这两种语言几乎完全不为人知,这项工作可以是概括的,以努力恶化而使没有人能够理解的丢失或外星语语言。
translated by 谷歌翻译
本教程展示了工作流程,将文本数据纳入精算分类和回归任务。主要重点是采用基于变压器模型的方法。平均长度为400个单词的车祸描述的数据集,英语和德语可用,以及具有简短财产保险索赔的数据集用来证明这些技术。案例研究应对与多语言环境和长输入序列有关的挑战。他们还展示了解释模型输出,评估和改善模型性能的方法,通过将模型调整到应用程序领域或特定预测任务。最后,该教程提供了在没有或仅有少数标记数据的情况下处理分类任务的实用方法。通过使用最少的预处理和微调的现成自然语言处理(NLP)模型的语言理解技能(NLP)模型实现的结果清楚地证明了用于实际应用的转移学习能力。
translated by 谷歌翻译
情绪感知智能系统对于广泛的应用是必不可少的。这些系统由语言模型驱动,这主要落入两个范式:基于词汇和上下文。虽然最近的上下文模型越来越占主导地位,但由于它们的可解释性和易用性,我们仍然可以看到基于词汇的模型的需求。例如,基于词汇的模型允许研究人员容易地确定哪些单词和短语对测量情绪的变化有贡献。任何基于词汇的方法的挑战是,词典需要通过新的单词和表达进行常规扩展。在这里,我们提出了两个用于自动词典扩展的模型。我们的第一个模型建立了一种基线,采用简单而浅的神经网络,使用非上下文方法初始化了预先训练的单词嵌入。我们的第二种模式改进了我们的基线,具有深度变压器的网络,它带来了估计其词汇极性的单词定义。我们的评估表明,两种模型都能够以与亚马逊机械土耳其人的评论者相似的准确度,但是在成本的一小部分中,可以获得类似的准确性。
translated by 谷歌翻译
确保适当的标点符号和字母外壳是朝向应用复杂的自然语言处理算法的关键预处理步骤。这对于缺少标点符号和壳体的文本源,例如自动语音识别系统的原始输出。此外,简短的短信和微博的平台提供不可靠且经常错误的标点符号和套管。本调查概述了历史和最先进的技术,用于恢复标点符号和纠正单词套管。此外,突出了当前的挑战和研究方向。
translated by 谷歌翻译
我们使用不同的语言支持特征预处理方法研究特征密度(FD)的有效性,以估计数据集复杂性,这又用于比较估计任何训练之前机器学习(ML)分类器的潜在性能。我们假设估计数据集复杂性允许减少所需实验迭代的数量。这样我们可以优化ML模型的资源密集型培训,这是由于可用数据集大小的增加以及基于深神经网络(DNN)的模型的不断增加的普及而成为一个严重问题。由于训练大规模ML模型引起的令人惊叹的二氧化碳排放量,不断增加对更强大的计算资源需求的问题也在影响环境。该研究是在多个数据集中进行的,包括流行的数据集,例如用于培训典型情感分析模型的Yelp业务审查数据集,以及最近的数据集尝试解决网络欺凌问题,这是一个严重的社会问题,也是一个严重的社会问题一个更复杂的问题,形成了语言代表的观点。我们使用收集多种语言的网络欺凌数据集,即英语,日语和波兰语。数据集的语言复杂性的差异允许我们另外讨论语言备份的单词预处理的功效。
translated by 谷歌翻译
State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models. These models are generally trained on data in a single language (usually English), and cannot be directly used beyond that language. Since collecting data in every language is not realistic, there has been a growing interest in crosslingual language understanding (XLU) and low-resource cross-language transfer. In this work, we construct an evaluation set for XLU by extending the development and test sets of the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 15 languages, including low-resource languages such as Swahili and Urdu. We hope that our dataset, dubbed XNLI, will catalyze research in cross-lingual sentence understanding by providing an informative standard evaluation task. In addition, we provide several baselines for multilingual sentence understanding, including two based on machine translation systems, and two that use parallel data to train aligned multilingual bag-of-words and LSTM encoders. We find that XNLI represents a practical and challenging evaluation suite, and that directly translating the test data yields the best performance among available baselines.
translated by 谷歌翻译
Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem. Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary. In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units. This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations). We discuss the suitability of different word segmentation techniques, including simple character ngram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back-off dictionary baseline for the WMT 15 translation tasks English→German and English→Russian by up to 1.1 and 1.3 BLEU, respectively.
translated by 谷歌翻译
编码单词语义属性的密集词向量或“Word Embeddings”现在已成为机器翻译(MT),问题应答(QA),字感消解(WSD)和信息检索(IR)中的NLP任务的积分。在本文中,我们使用各种现有方法为14个印度语言创建多个单词嵌入。我们将这些嵌入的嵌入式为所有这些语言,萨姆萨姆,孟加拉,古吉拉蒂,印地教派,kannada,konkani,malayalam,marathi,尼泊尔,odiya,punjabi,梵语,泰米尔和泰雅古士在一个单一的存储库中。相对较新的方法,强调迎合上下文(BERT,ELMO等),表明了显着的改进,但需要大量资源来产生可用模型。我们释放使用上下文和非上下文方法生成的预训练嵌入。我们还使用Muse和XLM来培训所有上述语言的交叉语言嵌入。为了展示我们嵌入的效果,我们为所有这些语言评估了我们对XPOS,UPOS和NER任务的嵌入模型。我们使用8种不同的方法释放了436个型号。我们希望他们对资源受限的印度语言NLP有用。本文的标题是指最初在1924年出版的福斯特的着名小说“一段是印度”。
translated by 谷歌翻译
Transformer language models (TLMs) are critical for most NLP tasks, but they are difficult to create for low-resource languages because of how much pretraining data they require. In this work, we investigate two techniques for training monolingual TLMs in a low-resource setting: greatly reducing TLM size, and complementing the masked language modeling objective with two linguistically rich supervised tasks (part-of-speech tagging and dependency parsing). Results from 7 diverse languages indicate that our model, MicroBERT, is able to produce marked improvements in downstream task evaluations relative to a typical monolingual TLM pretraining approach. Specifically, we find that monolingual MicroBERT models achieve gains of up to 18% for parser LAS and 11% for NER F1 compared to a multilingual baseline, mBERT, while having less than 1% of its parameter count. We conclude reducing TLM parameter count and using labeled data for pretraining low-resource TLMs can yield large quality benefits and in some cases produce models that outperform multilingual approaches.
translated by 谷歌翻译
Natural Language Understanding has seen an increasing number of publications in the last few years, especially after robust word embeddings models became prominent, when they proved themselves able to capture and represent semantic relationships from massive amounts of data. Nevertheless, traditional models often fall short in intrinsic issues of linguistics, such as polysemy and homonymy. Any expert system that makes use of natural language in its core, can be affected by a weak semantic representation of text, resulting in inaccurate outcomes based on poor decisions. To mitigate such issues, we propose a novel approach called Most Suitable Sense Annotation (MSSA), that disambiguates and annotates each word by its specific sense, considering the semantic effects of its context. Our approach brings three main contributions to the semantic representation scenario: (i) an unsupervised technique that disambiguates and annotates words by their senses, (ii) a multi-sense embeddings model that can be extended to any traditional word embeddings algorithm, and (iii) a recurrent methodology that allows our models to be re-used and their representations refined. We test our approach on six different benchmarks for the word similarity task, showing that our approach can produce state-of-the-art results and outperforms several more complex state-of-the-art systems.
translated by 谷歌翻译