分布式文档表示是自然语言处理中的基本问题之一。目前分布式文档表示方法主要考虑单词或句子的上下文信息。这些方法不考虑文件作为整体的一致性,例如文档之间的关系,文档中的纸张标题和抽象,标题和描述或相邻机构之间的关系。一致性显示文档是否有意义,逻辑和句法,尤其是科学文档(论文或专利等)。在本文中,我们提出了一个耦合文本对嵌入(CTPE)模型来学习科学文档的表示,其通过分割文档来维护文档与耦合文本对的相干性。首先,我们将文档划分为构造耦合文本对的两个部分(例如,标题和抽象等)。然后,我们采用负面采样来构建两个部分来自不同文档的未耦合文本对。最后,我们训练模型以判断文本对是否被耦合或解耦并使用所获得的耦合文本对的嵌入作为嵌入文档。我们在三个数据集上执行实验,以获得一个信息检索任务和两个推荐任务。实验结果验证了所提出的CTPE模型的有效性。
translated by 谷歌翻译
专利数据是创新研究知识的重要来源。尽管专利对之间的技术相似性是用于专利分析的关键指标。最近,研究人员一直在使用基于不同NLP嵌入模型的专利矢量空间模型来计算专利对之间的技术相似性,以帮助更好地了解创新,专利景观,技术映射和专利质量评估。据我们所知,没有一项全面的调查来建立嵌入模型的性能以计算专利相似性指标的大图。因此,在这项研究中,我们根据专利分类性能概述了这些算法的准确性。在详细的讨论中,我们报告了部分,类和子类级别的前3个算法的性能。基于专利的第一个主张的结果表明,专利,贝特(Bert-For)和tf-idf加权单词嵌入具有最佳准确性,可以在亚类级别计算句子嵌入。根据第一个结果,不同类别中模型的性能各不相同,这表明专利分析中的研究人员可以利用本研究的结果根据他们使用的专利数据的特定部分选择最佳的适当模型。
translated by 谷歌翻译
本地引文建议的目标是推荐从本地引文上下文中缺少的参考,也可以从全球背景下选择。为了在大规模纸质数据库的背景下平衡引文建议的速度和准确性之间的权衡,一种可行的方法是使用有效的排名方法来预先取代有限数量的相关文件,然后使用更多复杂的模型。在那种静脉中,BM25已被发现是一种艰难的预取方法,这就是为什么最近的工作主要集中在重新登记的步骤中。即便如此,我们探讨了由分层注意网络构造的文本嵌入的最近邻南搜索的预取。当耦合与速度regered在本地引文推荐任务上进行微调时,我们的分层关注编码器(Hatten)实现了高预备回忆,以便重新登记给定数量的候选候选者。因此,我们的Reranker需要重新命名更少的预取候选者,但仍然在各种本地引文推荐数据集上实现最先进的性能,例如ACL-200,FullTextPeerread,Refse和Arxiv。
translated by 谷歌翻译
The relationship between words in a sentence often tells us more about the underlying semantic content of a document than its actual words, individually. In this work, we propose two novel algorithms, called Flexible Lexical Chain II and Fixed Lexical Chain II. These algorithms combine the semantic relations derived from lexical chains, prior knowledge from lexical databases, and the robustness of the distributional hypothesis in word embeddings as building blocks forming a single system. In short, our approach has three main contributions: (i) a set of techniques that fully integrate word embeddings and lexical chains; (ii) a more robust semantic representation that considers the latent relation between words in a document; and (iii) lightweight word embeddings models that can be extended to any natural language task. We intend to assess the knowledge of pre-trained models to evaluate their robustness in the document classification task. The proposed techniques are tested against seven word embeddings algorithms using five different machine learning classifiers over six scenarios in the document classification task. Our results show the integration between lexical chains and word embeddings representations sustain state-of-the-art results, even against more complex systems.
translated by 谷歌翻译
测量不同文本的语义相似性在数字人文研究中具有许多重要应用,例如信息检索,文档聚类和文本摘要。不同方法的性能取决于文本,域和语言的长度。本研究侧重于试验一些目前的芬兰方法,这是一种形态学丰富的语言。与此同时,我们提出了一种简单的方法TFW2V,它在处理长文本文档和有限的数据时显示出高效率。此外,我们设计了一种客观评估方法,可以用作基准标记文本相似性方法的框架。
translated by 谷歌翻译
Natural Language Understanding has seen an increasing number of publications in the last few years, especially after robust word embeddings models became prominent, when they proved themselves able to capture and represent semantic relationships from massive amounts of data. Nevertheless, traditional models often fall short in intrinsic issues of linguistics, such as polysemy and homonymy. Any expert system that makes use of natural language in its core, can be affected by a weak semantic representation of text, resulting in inaccurate outcomes based on poor decisions. To mitigate such issues, we propose a novel approach called Most Suitable Sense Annotation (MSSA), that disambiguates and annotates each word by its specific sense, considering the semantic effects of its context. Our approach brings three main contributions to the semantic representation scenario: (i) an unsupervised technique that disambiguates and annotates words by their senses, (ii) a multi-sense embeddings model that can be extended to any traditional word embeddings algorithm, and (iii) a recurrent methodology that allows our models to be re-used and their representations refined. We test our approach on six different benchmarks for the word similarity task, showing that our approach can produce state-of-the-art results and outperforms several more complex state-of-the-art systems.
translated by 谷歌翻译
我们研究了检查问题的事实,旨在识别给定索赔的真实性。具体而言,我们专注于事实提取和验证(发烧)及其伴随数据集的任务。该任务包括从维基百科检索相关文件(和句子)并验证文件中的信息是否支持或驳斥所索赔的索赔。此任务至关重要,可以是假新闻检测和医疗索赔验证等应用程序块。在本文中,我们以通过以结构化和全面的方式呈现文献来更好地了解任务的挑战。我们通过分析不同方法的技术视角并讨论发热数据集的性能结果,描述了所提出的方法,这是最熟悉的和正式结构化的数据集,就是事实提取和验证任务。我们还迄今为止迄今为止确定句子检索组件的有益损失函数的最大实验研究。我们的分析表明,采样负句对于提高性能并降低计算复杂性很重要。最后,我们描述了开放的问题和未来的挑战,我们激励了未来的任务研究。
translated by 谷歌翻译
学术研究是解决以前从未解决过的问题的探索活动。通过这种性质,每个学术研究工作都需要进行文献审查,以区分其Novelties尚未通过事先作品解决。在自然语言处理中,该文献综述通常在“相关工作”部分下进行。鉴于研究文件的其余部分和引用的论文列表,自动相关工作生成的任务旨在自动生成“相关工作”部分。虽然这项任务是在10年前提出的,但直到最近,它被认为是作为科学多文件摘要问题的变种。然而,即使在今天,尚未标准化了自动相关工作和引用文本生成的问题。在这项调查中,我们进行了一个元研究,从问题制定,数据集收集,方法方法,绩效评估和未来前景的角度来比较相关工作的现有文献,以便为读者洞察到国家的进步 - 最内容的研究,以及如何进行未来的研究。我们还调查了我们建议未来工作要考虑整合的相关研究领域。
translated by 谷歌翻译
指定的实体识别任务是信息提取的核心任务之一。单词歧义和单词缩写是命名实体低识别率的重要原因。在本文中,我们提出了一种名为“实体识别模型WCL-BBCD”(与Bert-Bilstm-Crf-Dbpedia的单词对比学习),结合了对比度学习的概念。该模型首先在文本中训练句子对,计算句子对通过余弦的相似性中的单词对之间的相似性,以及通过相似性通过相似性来命名实体识别任务的BERT模型,以减轻单词歧义。然后,将微调的BERT模型与Bilstm-CRF模型相结合,以执行指定的实体识别任务。最后,将识别结果与先验知识(例如知识图)结合使用,以减轻单词缩写引起的低速问题的识别。实验结果表明,我们的模型在Conll-2003英语数据集和Ontonotes V5英语数据集上优于其他类似的模型方法。
translated by 谷歌翻译
诸如学术文章和商业报告之类的长期文件一直是详细说明重要问题和需要额外关注的复杂主题的标准格式。自动汇总系统可以有效地将长文档置于简短而简洁的文本中,以封装最重要的信息,从而在帮助读者的理解中很重要。最近,随着神经体系结构的出现,已经做出了重大的研究工作,以推动自动文本摘要系统,以及有关将这些系统扩展到长期文档领域的挑战的大量研究。在这项调查中,我们提供了有关长期文档摘要的研究的全面概述,以及其研究环境的三个主要组成部分的系统评估:基准数据集,汇总模型和评估指标。对于每个组成部分,我们在长期汇总的背景下组织文献,并进行经验分析,以扩大有关当前研究进度的观点。实证分析包括一项研究基准数据集的内在特征,摘要模型的多维分析以及摘要评估指标的综述。根据总体发现,我们通过提出可能在这个快速增长的领域中提出未来探索的方向来得出结论。
translated by 谷歌翻译
Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest. To model the semantic correlation between words and seeds for discovering topic-indicative terms, existing seed-guided approaches utilize different types of context signals, such as document-level word co-occurrences, sliding window-based local contexts, and generic linguistic knowledge brought by pre-trained language models. In this work, we analyze and show empirically that each type of context information has its value and limitation in modeling word semantics under seed guidance, but combining three types of contexts (i.e., word embeddings learned from local contexts, pre-trained language model representations obtained from general-domain training, and topic-indicative sentences retrieved based on seed information) allows them to complement each other for discovering quality topics. We propose an iterative framework, SeedTopicMine, which jointly learns from the three types of contexts and gradually fuses their context signals via an ensemble ranking process. Under various sets of seeds and on multiple datasets, SeedTopicMine consistently yields more coherent and accurate topics than existing seed-guided topic discovery approaches.
translated by 谷歌翻译
近年来,在应用预训练的语言模型(例如Bert)上,取得了巨大进展,以获取信息检索(IR)任务。在网页中通常使用的超链接已被利用用于设计预训练目标。例如,超链接的锚文本已用于模拟查询,从而构建了巨大的查询文档对以进行预训练。但是,作为跨越两个网页的桥梁,尚未完全探索超链接的潜力。在这项工作中,我们专注于建模通过超链接连接的两个文档之间的关系,并为临时检索设计一个新的预训练目标。具体而言,我们将文档之间的关系分为四组:无链接,单向链接,对称链接和最相关的对称链接。通过比较从相邻组采样的两个文档,该模型可以逐渐提高其捕获匹配信号的能力。我们提出了一个渐进的超链接预测({php})框架,以探索预训练中超链接的利用。对两个大规模临时检索数据集和六个提问数据集的实验结果证明了其优于现有的预训练方法。
translated by 谷歌翻译
生物医学问题的回答旨在从生物医学领域获得对给定问题的答案。由于其对生物医学领域知识的需求很高,因此模型很难从有限的培训数据中学习域知识。我们提出了一种上下文嵌入方法,该方法结合了在生物医学域数据上预先训练的开放域QA模型\ AOA和\ biobert模型。我们对大型生物医学语料库采用无监督的预培训,并在生物医学问题答案数据集上进行了微调。此外,我们采用基于MLP的模型加权层自动利用两个模型的优势以提供正确的答案。由PubMed语料库构建的公共数据集\ BIOMRC用于评估我们的方法。实验结果表明,我们的模型以大幅度优于最先进的系统。
translated by 谷歌翻译
Machine-Generated Text (MGT) detection, a task that discriminates MGT from Human-Written Text (HWT), plays a crucial role in preventing misuse of text generative models, which excel in mimicking human writing style recently. Latest proposed detectors usually take coarse text sequence as input and output some good results by fine-tune pretrained models with standard cross-entropy loss. However, these methods fail to consider the linguistic aspect of text (e.g., coherence) and sentence-level structures. Moreover, they lack the ability to handle the low-resource problem which could often happen in practice considering the enormous amount of textual data online. In this paper, we present a coherence-based contrastive learning model named CoCo to detect the possible MGT under low-resource scenario. Inspired by the distinctiveness and permanence properties of linguistic feature, we represent text as a coherence graph to capture its entity consistency, which is further encoded by the pretrained model and graph neural network. To tackle the challenges of data limitations, we employ a contrastive learning framework and propose an improved contrastive loss for making full use of hard negative samples in training stage. The experiment results on two public datasets prove our approach outperforms the state-of-art methods significantly.
translated by 谷歌翻译
名人认可是品牌交流中最重要的策略之一。如今,越来越多的公司试图为自己建立生动的特征。因此,他们的品牌身份交流应符合人类和法规的某些特征。但是,以前的作品主要是通过假设停止的,而不是提出一种特定的品牌和名人之间匹配的方式。在本文中,我们建议基于自然语言处理(NLP)技术的品牌名人匹配模型(BCM)。鉴于品牌和名人,我们首先从互联网上获得了一些描述性文档,然后总结了这些文档,最后计算品牌和名人之间的匹配程度,以确定它们是否匹配。根据实验结果,我们提出的模型以0.362 F1得分和精度的6.3%优于最佳基线,这表明我们模型在现实世界中的有效性和应用值。更重要的是,据我们所知,拟议的BCM模型是使用NLP解决认可问题的第一项工作,因此它可以为以下工作提供一些新颖的研究思想和方法。
translated by 谷歌翻译
随着互联网技术的发展,信息超载现象变得越来越明显。用户需要花费大量时间来获取所需的信息。但是,汇总文档信息的关键词非常有助于用户快速获取和理解文档。对于学术资源,大多数现有研究通过标题和摘要提取关键纸张。我们发现引用中的标题信息还包含作者分配的密钥次。因此,本文使用参考信息并应用两种典型的无监督的提取方法(TF * IDF和Textrank),两个代表传统监督学习算法(NA \“IVE贝叶斯和条件随机场)和监督的深度学习模型(Bilstm- CRF),分析参考信息对关键症提取的具体性能。从扩大源文本的角度来提高关键术识别的质量。实验结果表明,参考信息可以提高精度,召回和F1自动关键肾上腺瓶在一定程度上提取。这表明了参考信息关于学术论文的关键症提取的有用性,并为以下关于自动关键正萃取的研究提供了新的想法。
translated by 谷歌翻译
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models (Peters et al., 2018a;Radford et al., 2018), BERT is designed to pretrain deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be finetuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial taskspecific architecture modifications.BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).
translated by 谷歌翻译
Future work sentences (FWS) are the particular sentences in academic papers that contain the author's description of their proposed follow-up research direction. This paper presents methods to automatically extract FWS from academic papers and classify them according to the different future directions embodied in the paper's content. FWS recognition methods will enable subsequent researchers to locate future work sentences more accurately and quickly and reduce the time and cost of acquiring the corpus. The current work on automatic identification of future work sentences is relatively small, and the existing research cannot accurately identify FWS from academic papers, and thus cannot conduct data mining on a large scale. Furthermore, there are many aspects to the content of future work, and the subdivision of the content is conducive to the analysis of specific development directions. In this paper, Nature Language Processing (NLP) is used as a case study, and FWS are extracted from academic papers and classified into different types. We manually build an annotated corpus with six different types of FWS. Then, automatic recognition and classification of FWS are implemented using machine learning models, and the performance of these models is compared based on the evaluation metrics. The results show that the Bernoulli Bayesian model has the best performance in the automatic recognition task, with the Macro F1 reaching 90.73%, and the SCIBERT model has the best performance in the automatic classification task, with the weighted average F1 reaching 72.63%. Finally, we extract keywords from FWS and gain a deep understanding of the key content described in FWS, and we also demonstrate that content determination in FWS will be reflected in the subsequent research work by measuring the similarity between future work sentences and the abstracts.
translated by 谷歌翻译
大多数无监督的NLP模型代表了语义空间中单点或单个区域的每个单词,而现有的多感觉单词嵌入物不能代表像素序或句子等更长的单词序列。我们提出了一种用于文本序列(短语或句子)的新型嵌入方法,其中每个序列由一个不同的多模码本嵌入物组表示,以捕获其含义的不同语义面。码本嵌入式可以被视为集群中心,该中心总结了在预训练的单词嵌入空间中的可能共同出现的单词的分布。我们介绍了一个端到端的训练神经模型,直接从测试时间内从输入文本序列预测集群中心集。我们的实验表明,每句话码本嵌入式显着提高无监督句子相似性和提取摘要基准的性能。在短语相似之处实验中,我们发现多面嵌入物提供可解释的语义表示,但不优于单面基线。
translated by 谷歌翻译
识别跨语言抄袭是挑战性的,特别是对于遥远的语言对和感知翻译。我们介绍了这项任务的新型多语言检索模型跨语言本体论(CL \ nobreakdash-osa)。 CL-OSA表示从开放知识图Wikidata获得的实体向量的文档。反对其他方法,Cl \ nobreakdash-osa不需要计算昂贵的机器翻译,也不需要使用可比较或平行语料库进行预培训。它可靠地歧义同音异义和缩放,以允许其应用于Web级文档集合。我们展示了CL-OSA优于从五个大局部多样化的测试语料中检索候选文档的最先进的方法,包括日语英语等遥控语言对。为了识别在角色级别的跨语言抄袭,CL-OSA主要改善了感觉识别翻译的检测。对于这些挑战性案例,CL-OSA在良好的Plagdet得分方面的表现超过了最佳竞争对手的比例超过两种。我们研究的代码和数据公开可用。
translated by 谷歌翻译