生物医学机器阅读理解（生物医学MRC）旨在理解复杂的生物医学叙事，并协助医疗保健专业人员从中检索信息。现代神经网络的MRC系统的高性能取决于高质量的大规模，人为宣传的培训数据集。在生物医学领域中，创建此类数据集的一个至关重要的挑战是域知识的要求，引起了标记数据的稀缺性以及从标记的通用（源）域转移学习到生物医学（目标）域的需求。然而，由于主题方差，通用和生物医学领域之间的边际分布存在差异。因此，从在通用域上训练的模型到生物医学领域的模型直接转移学会的表示可能会损害模型的性能。我们为生物医学机器阅读理解任务（BioAdapt-MRC）提供了基于对抗性学习的域适应框架，这是一种基于神经网络的方法，可解决一般和生物医学域数据之间边际分布中的差异。 Bioadapt-MRC松弛了生成伪标签的需求，以训练表现出色的生物医学MRC模型。我们通过将生物ADAPT-MRC与三种广泛使用的基准生物医学MRC数据集进行比较，从而广泛评估了生物ADAPT-MRC的性能-Bioasq-7B，BioASQ-8B和BioASQ-9B。我们的结果表明，如果不使用来自生物医学领域的任何合成或人类通知的数据，Bioadapt-MRC可以在这些数据集中实现最先进的性能。可用性：bioadapt-MRC可作为开放源项目免费获得，\ url {https://github.com/mmahbub/bioadapt-mrc}。

Most previous unsupervised domain adaptation (UDA) methods for question answering(QA) require access to source domain data while fine-tuning the model for the target domain. Source domain data may, however, contain sensitive information and may be restricted. In this study, we investigate a more challenging setting, source-free UDA, in which we have only the pretrained source model and target domain data, without access to source domain data. We propose a novel self-training approach to QA models that integrates a unique mask module for domain adaptation. The mask is auto-adjusted to extract key domain knowledge while trained on the source domain. To maintain previously learned domain knowledge, certain mask weights are frozen during adaptation, while other weights are adjusted to mitigate domain shifts with pseudo-labeled samples generated in the target domain. %As part of the self-training process, we generate pseudo-labeled samples in the target domain based on models trained in the source domain. Our empirical results on four benchmark datasets suggest that our approach significantly enhances the performance of pretrained QA models on the target domain, and even outperforms models that have access to the source data during adaptation.

问答（QA）在回答定制域中的问题方面表现出了令人印象深刻的进展。然而，域的适应性仍然是质量检查系统最难以捉摸的挑战之一，尤其是当质量检查系统在源域中训练但部署在不同的目标域中时。在这项工作中，我们调查了问题分类对质量检查域适应的潜在好处。我们提出了一个新颖的框架：问题回答的问题分类（QC4QA）。具体而言，采用问题分类器将问题类分配给源数据和目标数据。然后，我们通过伪标记以自我监督的方式进行联合培训。为了优化，源和目标域之间的域间差异通过最大平均差异（MMD）距离降低。我们还最大程度地减少了同一问题类别的质量质量适应性表现的QA样本中的类内部差异。据我们所知，这是质量检查域适应中的第一部作品，以通过自我监督的适应来利用问题分类。我们证明了拟议的QC4QA的有效性，并在多个数据集上针对最先进的基线进行了一致的改进。

预审前的语言模型通过提供高质量的上下文化单词嵌入来显着改善了下游语言理解任务（包括提取性问题）的性能。但是，培训问答模型仍然需要大量特定域的注释数据。在这项工作中，我们提出了一个合作的自我训练框架RGX，用于自动生成更非平凡的问题 - 解答对以提高模型性能。 RGX建立在带有答案实体识别器，问题生成器和答案提取器的交互式学习环境的蒙版答案提取任务上。给定带有蒙版实体的段落，生成器会在实体周围生成一个问题，并培训了提取器，以提取蒙面实体，并使用生成的问题和原始文本。该框架允许对任何文本语料库的问题产生和回答模型进行培训，而无需注释。实验结果表明，RGX优于最先进的语言模型（SOTA）的语言模型，并在标准提问基准的基准上采用转移学习方法，并在给定的模型大小和传输学习设置下产生新的SOTA性能。

问题答案（QA）是自然语言处理中最具挑战性的最具挑战性的问题之一（NLP）。问答（QA）系统试图为给定问题产生答案。这些答案可以从非结构化或结构化文本生成。因此，QA被认为是可以用于评估文本了解系统的重要研究区域。大量的QA研究致力于英语语言，调查最先进的技术和实现最先进的结果。然而，由于阿拉伯QA中的研究努力和缺乏大型基准数据集，在阿拉伯语问答进展中的研究努力得到了很大速度的速度。最近许多预先接受的语言模型在许多阿拉伯语NLP问题中提供了高性能。在这项工作中，我们使用四个阅读理解数据集来评估阿拉伯QA的最先进的接种变压器模型，它是阿拉伯语 - 队，ArcD，AQAD和TYDIQA-GoldP数据集。我们微调并比较了Arabertv2基础模型，ArabertV0.2大型型号和ARAElectra模型的性能。在最后，我们提供了一个分析，了解和解释某些型号获得的低绩效结果。

当前在提取问题答案（EQA）中进行的研究对单跨度提取设置进行了建模，其中单个答案跨度是可以预测给定问题对对的标签。对于通用域EQA来说，这种设置是自然的，因为可以单个跨度可以回答通用域中的大多数问题。遵循通用域EQA模型，当前的生物医学EQA（BIOEQA）模型利用单跨度提取设置，采用后处理步骤。在本文中，我们调查了整个普通和生物医学领域的问题分布，发现生物医学问题更可能需要列表型答案（多个答案），而不是Factoid-type答案（单个答案）。这需要能够为问题提供多个答案的模型。基于这项初步研究，我们为Bioeqa提出了一种序列标记方法，Bioeqa是一种多跨度提取设置。我们的方法直接以不同数量的短语作为答案来解决问题，并可以学会从培训数据中确定问题的答案数量。我们在BioASQ 7B和8B列表类型问题上的实验结果优于表现最佳的现有模型，而无需进行后处理步骤。源代码和资源可免费下载，网址为https://github.com/dmis-lab/seqtagqa

预训练的语言模型（PLM）在各种自然语言理解任务上取得了巨大的成功。另一方面，对PLM的简单微调对于特定于领域的任务可能是次优的，因为它们不可能涵盖所有域中的知识。尽管PLM的自适应预培训可以帮助他们获得特定于领域的知识，但需要大量的培训成本。此外，自适应预训练可能会通过造成灾难性忘记其常识来损害PLM在下游任务上的表现。为了克服PLM适应性适应性预训练的这种局限性，我们提出了一个新颖的域名适应框架，用于将PLMS创造为知识增强语言模型适应性（KALA），该框架调节了PLM的中间隐藏表示与域中的中间隐藏表示，由实体和实体和实体和实体和实体构成他们的关系事实。我们验证了Kala在问题答案中的性能，并在各个域的多个数据集上命名实体识别任务。结果表明，尽管在计算上有效，但我们的Kala在很大程度上优于适应性预训练。代码可在以下网址获得：https：//github.com/nardien/kala/。

Aspect-based sentiment analysis (ABSA) aims at extracting opinionated aspect terms in review texts and determining their sentiment polarities, which is widely studied in both academia and industry. As a fine-grained classification task, the annotation cost is extremely high. Domain adaptation is a popular solution to alleviate the data deficiency issue in new domains by transferring common knowledge across domains. Most cross-domain ABSA studies are based on structure correspondence learning (SCL), and use pivot features to construct auxiliary tasks for narrowing down the gap between domains. However, their pivot-based auxiliary tasks can only transfer knowledge of aspect terms but not sentiment, limiting the performance of existing models. In this work, we propose a novel Syntax-guided Domain Adaptation Model, named SDAM, for more effective cross-domain ABSA. SDAM exploits syntactic structure similarities for building pseudo training instances, during which aspect terms of target domain are explicitly related to sentiment polarities. Besides, we propose a syntax-based BERT mask language model for further capturing domain-invariant features. Finally, to alleviate the sentiment inconsistency issue in multi-gram aspect terms, we introduce a span-based joint aspect term and sentiment analysis module into the cross-domain End2End ABSA. Experiments on five benchmark datasets show that our model consistently outperforms the state-of-the-art baselines with respect to Micro-F1 metric for the cross-domain End2End ABSA task.

自我监督的预审查能够为各种视觉文档理解（VDU）任务产生可转移的表示。但是，尚未研究此类表示在测试时间时适应新分配变化的能力。我们提出了Docta，这是一种用于文档的新型测试时间适应方法，该方法通过掩盖的视觉语言建模来利用交叉模式自我观察学习以及伪标签，以适应\ textit {source}域中学习的模型，以使其{source}域中为一个未标记的\ textit {textit {目标}域在测试时间。我们还使用现有的公共数据集介绍了新的基准测试，用于各种VDU任务，包括实体识别，键值提取和文档视觉问题回答任务，其中Doctta将源模型性能提高到1.79 \％（F1分数），3.43 \％（3.43 \％）（F1得分）和17.68 \％（ANLS得分），同时大大降低了目标数据的校准误差。

我们研究了检查问题的事实，旨在识别给定索赔的真实性。具体而言，我们专注于事实提取和验证（发烧）及其伴随数据集的任务。该任务包括从维基百科检索相关文件（和句子）并验证文件中的信息是否支持或驳斥所索赔的索赔。此任务至关重要，可以是假新闻检测和医疗索赔验证等应用程序块。在本文中，我们以通过以结构化和全面的方式呈现文献来更好地了解任务的挑战。我们通过分析不同方法的技术视角并讨论发热数据集的性能结果，描述了所提出的方法，这是最熟悉的和正式结构化的数据集，就是事实提取和验证任务。我们还迄今为止迄今为止确定句子检索组件的有益损失函数的最大实验研究。我们的分析表明，采样负句对于提高性能并降低计算复杂性很重要。最后，我们描述了开放的问题和未来的挑战，我们激励了未来的任务研究。

Deep learning has produced state-of-the-art results for a variety of tasks. While such approaches for supervised learning have performed well, they assume that training and testing data are drawn from the same distribution, which may not always be the case. As a complement to this challenge, single-source unsupervised domain adaptation can handle situations where a network is trained on labeled data from a source domain and unlabeled data from a related but different target domain with the goal of performing well at test-time on the target domain. Many single-source and typically homogeneous unsupervised deep domain adaptation approaches have thus been developed, combining the powerful, hierarchical representations from deep learning with domain adaptation to reduce reliance on potentially-costly target data labels. This survey will compare these approaches by examining alternative methods, the unique and common elements, results, and theoretical insights. We follow this with a look at application areas and open research directions.

As an important fine-grained sentiment analysis problem, aspect-based sentiment analysis (ABSA), aiming to analyze and understand people's opinions at the aspect level, has been attracting considerable interest in the last decade. To handle ABSA in different scenarios, various tasks are introduced for analyzing different sentiment elements and their relations, including the aspect term, aspect category, opinion term, and sentiment polarity. Unlike early ABSA works focusing on a single sentiment element, many compound ABSA tasks involving multiple elements have been studied in recent years for capturing more complete aspect-level sentiment information. However, a systematic review of various ABSA tasks and their corresponding solutions is still lacking, which we aim to fill in this survey. More specifically, we provide a new taxonomy for ABSA which organizes existing studies from the axes of concerned sentiment elements, with an emphasis on recent advances of compound ABSA tasks. From the perspective of solutions, we summarize the utilization of pre-trained language models for ABSA, which improved the performance of ABSA to a new stage. Besides, techniques for building more practical ABSA systems in cross-domain/lingual scenarios are discussed. Finally, we review some emerging topics and discuss some open challenges to outlook potential future directions of ABSA.

We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models (Peters et al., 2018a;Radford et al., 2018), BERT is designed to pretrain deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be finetuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial taskspecific architecture modifications.BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).

Recent advances in NLP are brought by a range of large-scale pretrained language models (PLMs). These PLMs have brought significant performance gains for a range of NLP tasks, circumventing the need to customize complex designs for specific tasks. However, most current work focus on finetuning PLMs on a domain-specific datasets, ignoring the fact that the domain gap can lead to overfitting and even performance drop. Therefore, it is practically important to find an appropriate method to effectively adapt PLMs to a target domain of interest. Recently, a range of methods have been proposed to achieve this purpose. Early surveys on domain adaptation are not suitable for PLMs due to the sophisticated behavior exhibited by PLMs from traditional models trained from scratch and that domain adaptation of PLMs need to be redesigned to take effect. This paper aims to provide a survey on these newly proposed methods and shed light in how to apply traditional machine learning methods to newly evolved and future technologies. By examining the issues of deploying PLMs for downstream tasks, we propose a taxonomy of domain adaptation approaches from a machine learning system view, covering methods for input augmentation, model optimization and personalization. We discuss and compare those methods and suggest promising future research directions.

预先接受的语言模型实现了最先进的导致各种自然语言处理（NLP）任务。 GPT-3表明，缩放预先训练的语言模型可以进一步利用它们的巨大潜力。最近提出了一个名为Ernie 3.0的统一框架，以预先培训大型知识增强型号，并培训了具有10亿参数的模型。 Ernie 3.0在各种NLP任务上表现出最先进的模型。为了探讨缩放的表现，我们培养了百卢比的3.0泰坦参数型号，在PaddlePaddle平台上有高达260亿参数的泰坦。此外，我们设计了一种自我监督的对抗性损失和可控语言建模损失，以使ERNIE 3.0 TITAN产生可信和可控的文本。为了减少计算开销和碳排放，我们向Ernie 3.0泰坦提出了一个在线蒸馏框架，教师模型将同时教授学生和培训。埃塞尼3.0泰坦是迄今为止最大的中国密集预训练模型。经验结果表明，Ernie 3.0泰坦在68个NLP数据集中优于最先进的模型。

自然语言理解（NLU）通过大型基准驱动的大规模进展，与转让学习的研究配对扩大其影响。基准是由一小部分频繁现象的主导，留下了一条长长的不常见现象。在这项工作中，我们反映了问题：转移学习方法足够地解决了长尾的基准训练模型的表现吗？由于基准未列出包括/排除的现象，我们使用宏观级别的宏观尺寸（如经验丰富的类型，主题等）概念化。我们评估通过100个代表性论文转让学习的定性荟萃分析来转移学习研究的趋势nlu。我们的分析问了三个问题：（i）哪个长尾尺寸进行转移学习研究目标？ （ii）哪种特性有助于适应方法改善长尾的性能？ （iii）哪种方法差距对长尾性能有最大的负面影响？我们对这些问题的答案突出了在长尾的转让学习中的未来研究的主要途径。最后，我们展示了一个案例研究，比较了各种适应方法对临床叙事的性能，以表明系统性开展的元实验如何提供能够沿着这些未来的途径取得进展的见解。

Transfer learning aims at improving the performance of target learners on target domains by transferring the knowledge contained in different but related source domains. In this way, the dependence on a large number of target domain data can be reduced for constructing target learners. Due to the wide application prospects, transfer learning has become a popular and promising area in machine learning. Although there are already some valuable and impressive surveys on transfer learning, these surveys introduce approaches in a relatively isolated way and lack the recent advances in transfer learning. Due to the rapid expansion of the transfer learning area, it is both necessary and challenging to comprehensively review the relevant studies. This survey attempts to connect and systematize the existing transfer learning researches, as well as to summarize and interpret the mechanisms and the strategies of transfer learning in a comprehensive way, which may help readers have a better understanding of the current research status and ideas. Unlike previous surveys, this survey paper reviews more than forty representative transfer learning approaches, especially homogeneous transfer learning approaches, from the perspectives of data and model. The applications of transfer learning are also briefly introduced. In order to show the performance of different transfer learning models, over twenty representative transfer learning models are used for experiments. The models are performed on three different datasets, i.e., Amazon Reviews, Reuters-21578, and Office-31. And the experimental results demonstrate the importance of selecting appropriate transfer learning models for different applications in practice.

自动问题应答（QA）系统的目的是以时间有效的方式向用户查询提供答案。通常在数据库（或知识库）或通常被称为语料库的文件集合中找到答案。在过去的几十年里，收购知识的扩散，因此生物医学领域的新科学文章一直是指数增长。因此，即使对于领域专家，也难以跟踪域中的所有信息。随着商业搜索引擎的改进，用户可以在某些情况下键入其查询并获得最相关的一小组文档，以及在某些情况下从文档中的相关片段。但是，手动查找所需信息或答案可能仍然令人疑惑和耗时。这需要开发高效的QA系统，该系统旨在为用户提供精确和精确的答案提供了生物医学领域的自然语言问题。在本文中，我们介绍了用于开发普通域QA系统的基本方法，然后彻底调查生物医学QA系统的不同方面，包括使用结构化数据库和文本集合的基准数据集和几种提出的方​​法。我们还探讨了当前系统的局限性，并探索潜在的途径以获得进一步的进步。

大型语言模型在各种任务上显示出令人印象深刻的几次结果。但是，当知识是此类结果的关键时，就像问题回答和事实检查之类的任务一样，似乎需要存储知识的大量参数计数。众所周知，检索增强模型可以在不需要多个参数的情况下在知识密集的任务上表现出色，但是目前尚不清楚它们是否在几个弹药设置中工作。在这项工作中，我们介绍了地图集，这是一个经过精心设计和预先训练的增强语言模型，能够通过很少的培训示例学习知识密集型任务。我们对包括MMLU，苏格兰短裙和归类等各种任务进行评估，并研究文档索引内容的影响，表明它可以很容易地进行更新。值得注意的是，在自然问题上仅使用64个示例在自然问题上达到超过42 \％的准确性，尽管参数少了50倍，但比540B参数模型的表现优于540b参数模型。

转移学习已通过深度审慎的语言模型广泛用于自然语言处理，例如来自变形金刚和通用句子编码器的双向编码器表示。尽管取得了巨大的成功，但语言模型应用于小型数据集时会过多地适合，并且很容易忘记与分类器进行微调时。为了解决这个忘记将深入的语言模型从一个域转移到另一个领域的问题，现有的努力探索了微调方法，以减少忘记。我们建议DeepeMotex是一种有效的顺序转移学习方法，以检测文本中的情绪。为了避免忘记问题，通过从Twitter收集的大量情绪标记的数据来仪器进行微调步骤。我们使用策划的Twitter数据集和基准数据集进行了一项实验研究。 DeepeMotex模型在测试数据集上实现多级情绪分类的精度超过91％。我们评估了微调DeepeMotex模型在分类Emoint和刺激基准数据集中的情绪时的性能。这些模型在基准数据集中的73％的实例中正确分类了情绪。所提出的DeepeMotex-Bert模型优于BI-LSTM在基准数据集上的BI-LSTM增长23％。我们还研究了微调数据集的大小对模型准确性的影响。我们的评估结果表明，通过大量情绪标记的数据进行微调提高了最终目标任务模型的鲁棒性和有效性。