端到端语音合成模型直接将输入字符转换为音频表示（例如频谱图）。尽管表现令人印象深刻，但此类模型仍很难消除相同拼写单词的发音。为了减轻此问题，可以在合成音频之前将单独的字素至phoneme（G2P）模型转换为音素。本文提出了SoundChoice，这是一种新颖的G2P体系结构，可以处理整个句子而不是在单词级别上操作。所提出的体系结构利用了加权同型损失（改善了歧义），利用课程学习（逐渐从单词级别切换到句子级别的G2P），并整合了Bert的单词嵌入（以进一步提高性能提高）。此外，该模型在语音识别中继承了最佳实践，包括使用Connectionist暂时分类（CTC）的多任务学习和带有嵌入式语言模型的光束搜索。结果，SoundChoice使用LibrisPeech和Wikipedia的数据实现了全句转录的音素错误率（PER），为2.65％。索引术语字素至音量，语音综合，文本传播，语音，发音，歧义。

确保适当的标点符号和字母外壳是朝向应用复杂的自然语言处理算法的关键预处理步骤。这对于缺少标点符号和壳体的文本源，例如自动语音识别系统的原始输出。此外，简短的短信和微博的平台提供不可靠且经常错误的标点符号和套管。本调查概述了历史和最先进的技术，用于恢复标点符号和纠正单词套管。此外，突出了当前的挑战和研究方向。

对于自然语言处理应用可能是有问题的，因为它们的含义不能从其构成词语推断出来。缺乏成功的方法方法和足够大的数据集防止了用于检测成语的机器学习方法的开发，特别是对于在训练集中不发生的表达式。我们提出了一种叫做小鼠的方法，它使用上下文嵌入来实现此目的。我们展示了一个新的多字表达式数据集，具有文字和惯用含义，并使用它根据两个最先进的上下文单词嵌入式培训分类器：Elmo和Bert。我们表明，使用两个嵌入式的深度神经网络比现有方法更好地执行，并且能够检测惯用词使用，即使对于训练集中不存在的表达式。我们展示了开发模型的交叉传输，并分析了所需数据集的大小。

在本文中，我们研究了波斯语的G2P转换的端到端和多模块框架的应用。结果表明，我们提出的多模型G2P系统在准确性和速度方面优于我们的端到端系统。该系统由发音词典作为我们的查找表组成，以及使用GRU和Transformer架构创建的波斯语中的同符，OOV和EZAFE的单独模型。该系统是序列级别而不是单词级别，它使其能够有效地捕获单词（跨字信息）之间的不成文关系，而无需进行任何预处理，而无需进行任何预歧歧义和EZAFE识别。经过评估后，我们的系统达到了94.48％的单词级准确性，表现优于先前的波斯语G2P系统。

We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pretrained on a large text corpus. We show that these representations can be easily added to existing models and significantly improve the state of the art across six challenging NLP problems, including question answering, textual entailment and sentiment analysis. We also present an analysis showing that exposing the deep internals of the pre-trained network is crucial, allowing downstream models to mix different types of semi-supervision signals.

End-to-end text-to-speech synthesis (TTS) can generate highly natural synthetic speech from raw text. However, rendering the correct pitch accents is still a challenging problem for end-to-end TTS. To tackle the challenge of rendering correct pitch accent in Japanese end-to-end TTS, we adopt PnG~BERT, a self-supervised pretrained model in the character and phoneme domain for TTS. We investigate the effects of features captured by PnG~BERT on Japanese TTS by modifying the fine-tuning condition to determine the conditions helpful inferring pitch accents. We manipulate content of PnG~BERT features from being text-oriented to speech-oriented by changing the number of fine-tuned layers during TTS. In addition, we teach PnG~BERT pitch accent information by fine-tuning with tone prediction as an additional downstream task. Our experimental results show that the features of PnG~BERT captured by pretraining contain information helpful inferring pitch accent, and PnG~BERT outperforms baseline Tacotron on accent correctness in a listening test.

来自文本的采矿因果关系是一种复杂的和至关重要的自然语言理解任务，对应于人类认知。其解决方案的现有研究可以分为两种主要类别：基于特征工程和基于神经模型的方法。在本文中，我们发现前者具有不完整的覆盖范围和固有的错误，但提供了先验知识;虽然后者利用上下文信息，但其因果推断不足。为了处理限制，我们提出了一个名为MCDN的新型因果关系检测模型，明确地模拟因果关系，而且，利用两种方法的优势。具体而言，我们采用多头自我关注在Word级别获得语义特征，并在段级别推断出来的SCRN。据我们所知，关于因果关系任务，这是第一次应用关系网络。实验结果表明：1）该方法对因果区检测进行了突出的性能; 2）进一步分析表现出MCDN的有效性和稳健性。

Named Entity Recognition and Intent Classification are among the most important subfields of the field of Natural Language Processing. Recent research has lead to the development of faster, more sophisticated and efficient models to tackle the problems posed by those two tasks. In this work we explore the effectiveness of two separate families of Deep Learning networks for those tasks: Bidirectional Long Short-Term networks and Transformer-based networks. The models were trained and tested on the ATIS benchmark dataset for both English and Greek languages. The purpose of this paper is to present a comparative study of the two groups of networks for both languages and showcase the results of our experiments. The models, being the current state-of-the-art, yielded impressive results and achieved high performance.

数据饥饿的深度神经网络已经将自己作为许多NLP任务的标准建立为包括传统序列标记的标准。尽管他们在高资源语言上表现最先进的表现，但它们仍然落后于低资源场景的统计计数器。一个方法来反击攻击此问题是文本增强，即，从现有数据生成新的合成训练数据点。虽然NLP最近目睹了一种文本增强技术的负载，但该领域仍然缺乏对多种语言和序列标记任务的系统性能分析。为了填补这一差距，我们调查了三类文本增强方法，其在语法（例如，裁剪子句子），令牌（例如，随机字插入）和字符（例如，字符交换）级别上执行更改。我们系统地将它们与语音标记，依赖解析和语义角色标记的分组进行了比较，用于使用各种模型的各种语言系列，包括依赖于诸如MBERT的普赖金的多语言语境化语言模型的架构。增强最显着改善了解析，然后是语音标记和语义角色标记的依赖性解析。我们发现实验技术通常在形态上丰富的语言，而不是越南语等分析语言。我们的研究结果表明，增强技术可以进一步改善基于MBERT的强基线。我们将字符级方法标识为最常见的表演者，而同义词替换和语法增强仪提供不一致的改进。最后，我们讨论了最大依赖于任务，语言对和模型类型的结果。

基于方面的情绪分析（ABSA）是一种文本分析方法，其定义了与特定目标相关的某些方面的意见的极性。 ABSA的大部分研究都是英文，阿拉伯语有少量的工作。最先前的阿拉伯语研究依赖于深度学习模型，主要依赖于独立于上下文的单词嵌入（例如，e.g.word2vec），其中每个单词都有一个独立于其上下文的固定表示。本文探讨了从预先培训的语言模型（如BERT）的上下文嵌入的建模功能，例如BERT，以及在阿拉伯语方面情感极度分类任务中使用句子对输入。特别是，我们开发一个简单但有效的基于伯特的神经基线来处理这项任务。根据三种不同阿拉伯语数据集的实验结果，我们的BERT架构与简单的线性分类层超出了最先进的作品。在Arabic Hotel评论数据库中实现了89.51％的准确性，73％的人类注册书评论数据集和阿拉伯新闻数据集的85.73％。

最近，基于注意的编码器 - 解码器（AED）模型对多个任务的端到端自动语音识别（ASR）显示了高性能。在此类模型中解决了过度控制，本文介绍了轻松关注的概念，这是一种简单地逐渐注入对训练期间对编码器 - 解码器注意重量的统一分配，其易于用两行代码实现。我们调查轻松关注跨不同AED模型架构和两个突出的ASR任务，华尔街日志（WSJ）和LibRisPeech的影响。我们发现，在用外部语言模型解码时，随着宽松的注意力训练的变压器始终如一地始终如一地遵循标准基线模型。在WSJ中，我们为基于变压器的端到端语音识别设置了一个新的基准，以3.65％的单词错误率，最优于13.1％的相对状态，同时仅引入单个HyperParameter。

The rapid advancement of AI technology has made text generation tools like GPT-3 and ChatGPT increasingly accessible, scalable, and effective. This can pose serious threat to the credibility of various forms of media if these technologies are used for plagiarism, including scientific literature and news sources. Despite the development of automated methods for paraphrase identification, detecting this type of plagiarism remains a challenge due to the disparate nature of the datasets on which these methods are trained. In this study, we review traditional and current approaches to paraphrase identification and propose a refined typology of paraphrases. We also investigate how this typology is represented in popular datasets and how under-representation of certain types of paraphrases impacts detection capabilities. Finally, we outline new directions for future research and datasets in the pursuit of more effective paraphrase detection using AI.

命名实体识别是一项信息提取任务，可作为其他自然语言处理任务的预处理步骤，例如机器翻译，信息检索和问题答案。命名实体识别能够识别专有名称以及开放域文本中的时间和数字表达式。对于诸如阿拉伯语，阿姆哈拉语和希伯来语之类的闪族语言，由于这些语言的结构严重变化，指定的实体识别任务更具挑战性。在本文中，我们提出了一个基于双向长期记忆的Amharic命名实体识别系统，并带有条件随机字段层。我们注释了一种新的Amharic命名实体识别数据集（8,070个句子，具有182,691个令牌），并将合成少数群体过度采样技术应用于我们的数据集，以减轻不平衡的分类问题。我们命名的实体识别系统的F_1得分为93％，这是Amharic命名实体识别的新最新结果。

We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models (Peters et al., 2018a;Radford et al., 2018), BERT is designed to pretrain deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be finetuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial taskspecific architecture modifications.BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).

Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject-verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems which represent the current dominant state of the art. In this survey paper, we condense the field into a single article and first outline some of the linguistic challenges of the task, introduce the most popular datasets that are available to researchers (for both English and other languages), and summarise the various methods and techniques that have been developed with a particular focus on artificial error generation. We next describe the many different approaches to evaluation as well as concerns surrounding metric reliability, especially in relation to subjective human judgements, before concluding with an overview of recent progress and suggestions for future work and remaining challenges. We hope that this survey will serve as comprehensive resource for researchers who are new to the field or who want to be kept apprised of recent developments.

数据增强是通过转换为机器学习的人工创建数据的人工创建，是一个跨机器学习学科的研究领域。尽管它对于增加模型的概括功能很有用，但它还可以解决许多其他挑战和问题，从克服有限的培训数据到正规化目标到限制用于保护隐私的数据的数量。基于对数据扩展的目标和应用的精确描述以及现有作品的分类法，该调查涉及用于文本分类的数据增强方法，并旨在为研究人员和从业者提供简洁而全面的概述。我们将100多种方法划分为12种不同的分组，并提供最先进的参考文献来阐述哪种方法可以通过将它们相互关联，从而阐述了哪种方法。最后，提供可能构成未来工作的基础的研究观点。

Short text classification is a crucial and challenging aspect of Natural Language Processing. For this reason, there are numerous highly specialized short text classifiers. However, in recent short text research, State of the Art (SOTA) methods for traditional text classification, particularly the pure use of Transformers, have been unexploited. In this work, we examine the performance of a variety of short text classifiers as well as the top performing traditional text classifier. We further investigate the effects on two new real-world short text datasets in an effort to address the issue of becoming overly dependent on benchmark datasets with a limited number of characteristics. Our experiments unambiguously demonstrate that Transformers achieve SOTA accuracy on short text classification tasks, raising the question of whether specialized short text techniques are necessary.

Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks. Building on recent token-level probing work, we introduce a novel edge probing task design and construct a broad suite of sub-sentence tasks derived from the traditional structured NLP pipeline. We probe word-level contextual representations from four recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena. We find that existing models trained on language modeling and translation produce strong representations for syntactic phenomena, but only offer comparably small improvements on semantic tasks over a non-contextual baseline.

In this modern era of technology with e-commerce developing at a rapid pace, it is very important to understand customer requirements and details from a business conversation. It is very crucial for customer retention and satisfaction. Extracting key insights from these conversations is very important when it comes to developing their product or solving their issue. Understanding customer feedback, responses, and important details of the product are essential and it would be done using Named entity recognition (NER). For extracting the entities we would be converting the conversations to text using the optimal speech-to-text model. The model would be a two-stage network in which the conversation is converted to text. Then, suitable entities are extracted using robust techniques using a NER BERT transformer model. This will aid in the enrichment of customer experience when there is an issue which is faced by them. If a customer faces a problem he will call and register his complaint. The model will then extract the key features from this conversation which will be necessary to look into the problem. These features would include details like the order number, and the exact problem. All these would be extracted directly from the conversation and this would reduce the effort of going through the conversation again.

The word alignment task, despite its prominence in the era of statistical machine translation (SMT), is niche and under-explored today. In this two-part tutorial, we argue for the continued relevance for word alignment. The first part provides a historical background to word alignment as a core component of the traditional SMT pipeline. We zero-in on GIZA++, an unsupervised, statistical word aligner with surprising longevity. Jumping forward to the era of neural machine translation (NMT), we show how insights from word alignment inspired the attention mechanism fundamental to present-day NMT. The second part shifts to a survey approach. We cover neural word aligners, showing the slow but steady progress towards surpassing GIZA++ performance. Finally, we cover the present-day applications of word alignment, from cross-lingual annotation projection, to improving translation.