Web搜索是人类获取信息的重要方法,但是对于了解网页内容的机器仍然是一个巨大的挑战。在本文中,我们介绍了对网上结构阅读理解(SRC)的任务。鉴于网页和关于它的问题,任务是从网页找到答案。此任务要求系统不仅要了解文本的语义,还需要了解文本的语义,还需要网页的结构。此外,我们提出了一种新的基于Web的结构阅读理解数据集。 WebSRC由400K问答对组成,从6.4K网页收集。与QA对一起,我们的数据集还提供了相应的HTML源代码,屏幕截图和元数据。 WebSRC中的每个问题都需要对网页的某种结构理解来回答,并且答案是网页或是/否的文本跨度。我们评估我们数据集的各种基线,以显示我们的任务难度。我们还研究了结构信息和视觉功能的有用性。我们的数据集和基线已在HTTPS://x-lance.github.io/websrc/上公开提供。
translated by 谷歌翻译
文档视觉问题回答(VQA)旨在了解视觉上富裕的文档,以自然语言回答问题,这是自然语言处理和计算机视觉的新兴研究主题。在这项工作中,我们介绍了一个名为TAT-DQA的新文档VQA数据集,该数据集由3,067个文档页面组成,其中包含半结构化表和非结构化文本以及16,558个问答,通过扩展Tat-QA Dataset。这些文档是从现实世界中的财务报告中取样的,并包含大量数字,这意味着要求离散的推理能力回答该数据集上的问题。基于TAT-DQA,我们进一步开发了一个名为MHST的新型模型,该模型在包括文本,布局和视觉图像在内的多模式中考虑了信息,以智能地以相应的策略(即提取或推理)智能地解决不同类型的问题。广泛的实验表明,MHST模型明显优于基线方法,证明其有效性。但是,表演仍然远远落后于专家人类。我们预计,我们的新Tat-DQA数据集将有助于研究对视觉和语言结合的视觉丰富文档的深入理解,尤其是对于需要离散推理的场景。另外,我们希望拟议的模型能够激发研究人员将来设计更高级的文档VQA模型。
translated by 谷歌翻译
由于其有效的模型架构以及大规模未标记的扫描/数字出生的文件的优势,在各种视觉上丰富的文档理解任务中已经证明了文本和布局的预先培训。我们提出了具有新的预培训任务的Layoutlmv2架构,以在单个多模态框架中模拟文本,布局和图像之间的交互。具体地,对于双流多模态变压器编码器,LayOutLMV2不仅使用现有屏蔽的视觉语言建模任务,还使用新的文本图像对齐和文本图像匹配任务,这使得它更好地捕获跨模块交互在预训练阶段。同时,它还将空间感知的自我注意机制集成到变压器架构中,以便模型可以完全理解不同文本块之间的相对位置关系。实验结果表明,LayoutLMV2优于大幅度的LayOutlm,并在大量下游的下游富有的文件理解任务中实现了新的最先进的结果,包括Funsd(0.7895 $ \至0.8420美元),电源线(0.9493 $ \至0.9601美元),Srie(0.9524 $ \至0.9781美元),Kleister-NDA(0.8340 $ \ 0.8520美元),RVL-CDIP(0.9443 $ \至0.9564美元),DOCVQA(0.7295 $ \至0.8672美元) 。我们使我们的模型和代码公开可用于\ url {https://aka.ms/layoutlmv2}。
translated by 谷歌翻译
由于文件传达了丰富的人类知识,并且通常存在于企业中,因此建筑文档的对话系统已经越来越兴趣。其中,如何理解和从文档中检索信息是一个具有挑战性的研究问题。先前的工作忽略了文档的视觉属性,并将其视为纯文本,从而导致不完整的方式。在本文中,我们提出了一个布局感知文档级信息提取数据集,以促进从视觉上丰富文档(VRD)中提取结构和语义知识的研究,以在对话系统中产生准确的响应。 Lie包含来自4,061页的产品和官方文件的三个提取任务的62K注释,成为我们最大的知识,成为最大的基于VRD的信息提取数据集。我们还开发了扩展基于令牌的语言模型的基准方法,以考虑像人类这样的布局功能。经验结果表明,布局对于基于VRD的提取至关重要,系统演示还验证了提取的知识可以帮助找到用户关心的答案。
translated by 谷歌翻译
在本报告中,我们展示了ICDAR 2021版文档视觉问题挑战的结果。此版本在单个文档VQA和Document Collection VQA上补充了以前的任务,并在Infographics VQA上进行了新引入的。信息图表VQA基于超过5,000个信息图表图像和30,000个问题答案对的新数据集。获胜者方法在Infographics VQA任务中获得了0.6120个ANL,0.7743 anlsl在文档集中的VQA任务和单个文档VQA中的0.8705 ANL中。我们展示了用于每个任务的数据集的摘要,每个提交的方法的描述以及它们的性能的结果和分析。由于还提出了自从第一版DocVQA 2020挑战以来在单个文档VQA上取得的摘要。
translated by 谷歌翻译
问题回答(QA)是最重要的自然语言处理(NLP)任务之一。它旨在使用NLP技术根据大规模的非结构化语料库生成对给定问题的相应答案。随着深度学习的发展,正在提出越来越具有挑战性的质量检查数据集,并且许多用于解决它们的新方法也正在出现。在本文中,我们研究了在深度学习时代发布的有影响力的质量检查数据集。具体来说,我们首先引入两个最常见的质量检查任务 - 文本问题答案和视觉问题 - 分别涵盖最具代表性的数据集,然后给出质量检查研究的一些当前挑战。
translated by 谷歌翻译
We present TriviaQA, a challenging reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K questionanswer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions. We show that, in comparison to other recently introduced large-scale datasets, TriviaQA (1) has relatively complex, compositional questions, (2) has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and (3) requires more cross sentence reasoning to find answers. We also present two baseline algorithms: a featurebased classifier and a state-of-the-art neural network, that performs well on SQuAD reading comprehension. Neither approach comes close to human performance (23% and 40% vs. 80%), suggesting that Trivi-aQA is a challenging testbed that is worth significant future study. 1
translated by 谷歌翻译
ClueWeb22, the newest iteration of the ClueWeb line of datasets, provides 10 billion web pages affiliated with rich information. Its design was influenced by the need for a high quality, large scale web corpus to support a range of academic and industry research, for example, in information systems, retrieval-augmented AI systems, and model pretraining. Compared with earlier ClueWeb corpora, the ClueWeb22 corpus is larger, more varied, of higher-quality, and aligned with the document distributions in commercial web search. Besides raw HTML, ClueWeb22 includes rich information about the web pages provided by industry-standard document understanding systems, including the visual representation of pages rendered by a web browser, parsed HTML structure information from a neural network parser, and pre-processed cleaned document text to lower the barrier to entry. Many of these signals have been widely used in industry but are available to the research community for the first time at this scale.
translated by 谷歌翻译
有关应答数据集和模型的研究在研究界中获得了很多关注。其中许多人释放了自己的问题应答数据集以及模型。我们在该研究领域看到了巨大的进展。本调查的目的是识别,总结和分析许多研究人员释放的现有数据集,尤其是在非英语数据集以及研究代码和评估指标等资源中。在本文中,我们审查了问题应答数据集,这些数据集可以以法语,德语,日语,中文,阿拉伯语,俄语以及多语言和交叉的问答数据集进行英语。
translated by 谷歌翻译
Spoken language understanding (SLU) tasks have been studied for many decades in the speech research community, but have not received as much attention as lower-level tasks like speech and speaker recognition. In particular, there are not nearly as many SLU task benchmarks, and many of the existing ones use data that is not freely available to all researchers. Recent work has begun to introduce such benchmark datasets for several tasks. In this work, we introduce several new annotated SLU benchmark tasks based on freely available speech data, which complement existing benchmarks and address gaps in the SLU evaluation landscape. We contribute four tasks: question answering and summarization involve inference over longer speech sequences; named entity localization addresses the speech-specific task of locating the targeted content in the signal; dialog act classification identifies the function of a given speech utterance. We follow the blueprint of the Spoken Language Understanding Evaluation (SLUE) benchmark suite. In order to facilitate the development of SLU models that leverage the success of pre-trained speech representations, we will be publishing for each task (i) annotations for a relatively small fine-tuning set, (ii) annotated development and test sets, and (iii) baseline models for easy reproducibility and comparisons. In this work, we present the details of data collection and annotation and the performance of the baseline models. We also perform sensitivity analysis of pipeline models' performance (speech recognizer + text model) to the speech recognition accuracy, using more than 20 state-of-the-art speech recognition models.
translated by 谷歌翻译
Recent research shows synthetic data as a source of supervision helps pretrained language models (PLM) transfer learning to new target tasks/domains. However, this idea is less explored for spatial language. We provide two new data resources on multiple spatial language processing tasks. The first dataset is synthesized for transfer learning on spatial question answering (SQA) and spatial role labeling (SpRL). Compared to previous SQA datasets, we include a larger variety of spatial relation types and spatial expressions. Our data generation process is easily extendable with new spatial expression lexicons. The second one is a real-world SQA dataset with human-generated questions built on an existing corpus with SPRL annotations. This dataset can be used to evaluate spatial language processing models in realistic situations. We show pretraining with automatically generated data significantly improves the SOTA results on several SQA and SPRL benchmarks, particularly when the training data in the target domain is small.
translated by 谷歌翻译
视觉问题应答(VQA)任务利用视觉图像和语言分析来回回答图像的文本问题。它是一个流行的研究课题,在过去十年中越来越多的现实应用。本文介绍了我们最近对AliceMind-MMU的研究(阿里巴巴的编码器 - 解码器来自Damo Academy - 多媒体理解的机器智能实验室),其比人类在VQA上获得相似甚至略微更好的结果。这是通过系统地改善VQA流水线来实现的,包括:(1)具有全面的视觉和文本特征表示的预培训; (2)与学习参加的有效跨模型互动; (3)一个新颖的知识挖掘框架,具有专门的专业专家模块,适用于复杂的VQA任务。处理不同类型的视觉问题,需要具有相应的专业知识在提高我们的VQA架构的表现方面发挥着重要作用,这取决于人力水平。进行了广泛的实验和分析,以证明新的研究工作的有效性。
translated by 谷歌翻译
最近,由于其广泛的商业价值,从视觉丰富的文档(例如门票和简历)中自动提取信息已成为一个热门而重要的研究主题。大多数现有方法将此任务分为两个小节:用于从原始文档图像中获取纯文本的文本阅读部分以及用于提取密钥内容的信息提取部分。这些方法主要集中于改进第二个方法,同时忽略了这两个部分高度相关。本文提出了一个统一的端到端信息提取框架,从视觉上富含文档中提出,文本阅读和信息提取可以通过精心设计的多模式上下文块相互加强。具体而言,文本阅读部分提供了多模式功能,例如视觉,文本和布局功能。开发了多模式上下文块,以融合生成的多模式特征,甚至是从预训练的语言模型中获得的先验知识,以提供更好的语义表示。信息提取部分负责使用融合上下文功能生成密钥内容。该框架可以以端到端的可训练方式进行培训,从而实现全球优化。更重要的是,我们将视觉丰富的文档定义为跨两个维度的四个类别,即布局和文本类型。对于每个文档类别,我们提供或推荐相应的基准,实验设置和强大的基准,以弥补该研究领域缺乏统一评估标准的问题。报告了对四种基准测试的广泛实验(从固定布局到可变布局,从完整的文本到半未结构化的文本),证明了所提出的方法的有效性。数据,源代码和模型可用。
translated by 谷歌翻译
在视觉上丰富的文件(VRD)上的结构化文本理解是文档智能的重要组成部分。由于VRD中的内容和布局的复杂性,结构化文本理解是一项有挑战性的任务。大多数现有的研究将此问题与两个子任务结尾:实体标记和实体链接,这需要整体地了解令牌和段级别的文档的上下文。但是,很少的工作已经关注有效地从不同层次提取结构化数据的解决方案。本文提出了一个名为structext的统一框架,它对于处理两个子任务是灵活的,有效的。具体地,基于变压器,我们引入了一个段令牌对齐的编码器,以处理不同粒度水平的实体标记和实体链接任务。此外,我们设计了一种具有三个自我监督任务的新型预训练策略,以学习更丰富的代表性。 Structext使用现有屏蔽的视觉语言建模任务和新句子长度预测和配对框方向任务,以跨文本,图像和布局结合多模态信息。我们评估我们在分段级别和令牌级别的结构化文本理解的方法,并表明它优于最先进的同行,在Funsd,Srie和Ephoie数据集中具有显着优越的性能。
translated by 谷歌翻译
我们提出了一种用于场景文本视觉问题的新型多模式架构(STVQA),命名为布局感知变压器(LatR)。 STVQA的任务需要模型以推理不同的方式。因此,我们首先调查每种方式的影响,并揭示语言模块的重要性,尤其是在丰富布局信息时。考虑到这一点,我们提出了一种客观预培训计划,只需要文本和空间线索。我们表明,尽管域间隙差距,但仍然对扫描文件进行了对扫描文件的培训方案具有某些优点。扫描的文档易于采购,文本密集并具有各种布局,帮助模型通过捆绑语言和布局信息来学习各种空间线索(例如,下面等等)。与现有方法相比,我们的方法执行无词汇解码,如图所示,概括到超出培训词汇。我们进一步证明Latr改善了对OCR错误的鲁棒性,在STVQA失败的常见原因。另外,通过利用视觉变压器,我们消除了对外部物体检测器的需求。 Latr在多个数据集上赢得最先进的STVQA方法。特别是+ 7.6%的TextVQA,ST-VQA上的10.8%,+ 4.0%在OCR-VQA(所有绝对精度数字)。
translated by 谷歌翻译
Document Visual Question Answering (DocVQA) refers to the task of answering questions from document images. Existing work on DocVQA only considers single-page documents. However, in real scenarios documents are mostly composed of multiple pages that should be processed altogether. In this work we extend DocVQA to the multi-page scenario. For that, we first create a new dataset, MP-DocVQA, where questions are posed over multi-page documents instead of single pages. Second, we propose a new hierarchical method, Hi-VT5, based on the T5 architecture, that overcomes the limitations of current methods to process long multi-page documents. The proposed method is based on a hierarchical transformer architecture where the encoder summarizes the most relevant information of every page and then, the decoder takes this summarized information to generate the final answer. Through extensive experimentation, we demonstrate that our method is able, in a single stage, to answer the questions and provide the page that contains the relevant information to find the answer, which can be used as a kind of explainability measure.
translated by 谷歌翻译
问题答案(QA)是自然语言处理中最具挑战性的最具挑战性的问题之一(NLP)。问答(QA)系统试图为给定问题产生答案。这些答案可以从非结构化或结构化文本生成。因此,QA被认为是可以用于评估文本了解系统的重要研究区域。大量的QA研究致力于英语语言,调查最先进的技术和实现最先进的结果。然而,由于阿拉伯QA中的研究努力和缺乏大型基准数据集,在阿拉伯语问答进展中的研究努力得到了很大速度的速度。最近许多预先接受的语言模型在许多阿拉伯语NLP问题中提供了高性能。在这项工作中,我们使用四个阅读理解数据集来评估阿拉伯QA的最先进的接种变压器模型,它是阿拉伯语 - 队,ArcD,AQAD和TYDIQA-GoldP数据集。我们微调并比较了Arabertv2基础模型,ArabertV0.2大型型号和ARAElectra模型的性能。在最后,我们提供了一个分析,了解和解释某些型号获得的低绩效结果。
translated by 谷歌翻译
Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets. In this regard, first, the multi-hop MRC problem definition will be presented, then the evaluation metrics based on their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have been reviewed in detail from 2017 to 2022, and a comprehensive analysis has been prepared at the end. Finally, open issues in this field have been discussed.
translated by 谷歌翻译
Table-and-text hybrid question answering (HybridQA) is a widely used and challenging NLP task commonly applied in the financial and scientific domain. The early research focuses on migrating other QA task methods to HybridQA, while with further research, more and more HybridQA-specific methods have been present. With the rapid development of HybridQA, the systematic survey is still under-explored to summarize the main techniques and advance further research. So we present this work to summarize the current HybridQA benchmarks and methods, then analyze the challenges and future directions of this task. The contributions of this paper can be summarized in three folds: (1) first survey, to our best knowledge, including benchmarks, methods and challenges for HybridQA; (2) systematic investigation with the reasonable comparison of the existing systems to articulate their advantages and shortcomings; (3) detailed analysis of challenges in four important dimensions to shed light on future directions.
translated by 谷歌翻译
This paper proposes to tackle opendomain question answering using Wikipedia as the unique knowledge source: the answer to any factoid question is a text span in a Wikipedia article. This task of machine reading at scale combines the challenges of document retrieval (finding the relevant articles) with that of machine comprehension of text (identifying the answer spans from those articles). Our approach combines a search component based on bigram hashing and TF-IDF matching with a multi-layer recurrent neural network model trained to detect answers in Wikipedia paragraphs. Our experiments on multiple existing QA datasets indicate that (1) both modules are highly competitive with respect to existing counterparts and (2) multitask learning using distant supervision on their combination is an effective complete system on this challenging task.
translated by 谷歌翻译