Geometry problem solving is a well-recognized testbed for evaluating the high-level multi-modal reasoning capability of deep models. In most existing works, two main geometry problems: calculation and proving, are usually treated as two specific tasks, hindering a deep model to unify its reasoning capability on multiple math tasks. However, in essence, these two tasks have similar problem representations and overlapped math knowledge which can improve the understanding and reasoning ability of a deep model on both two tasks. Therefore, we construct a large-scale Unified Geometry problem benchmark, UniGeo, which contains 4,998 calculation problems and 9,543 proving problems. Each proving problem is annotated with a multi-step proof with reasons and mathematical expressions. The proof can be easily reformulated as a proving sequence that shares the same formats with the annotated program sequence for calculation problems. Naturally, we also present a unified multi-task Geometric Transformer framework, Geoformer, to tackle calculation and proving problems simultaneously in the form of sequence generation, which finally shows the reasoning ability can be improved on both two tasks by unifying formulation. Furthermore, we propose a Mathematical Expression Pretraining (MEP) method that aims to predict the mathematical expressions in the problem solution, thus improving the Geoformer model. Experiments on the UniGeo demonstrate that our proposed Geoformer obtains state-of-the-art performance by outperforming task-specific model NGS with over 5.6% and 3.2% accuracies on calculation and proving problems, respectively.

自动数学问题解决最近引起了越来越多的关注作为长期的AI基准。在本文中，我们专注于解决几何问题，这需要全面了解文本描述，视觉图和定理知识。但是，现有方法高度依赖于手工规则，并且仅在小规模数据集上进行评估。因此，我们提出了一个几何问题应答DataSet GeoQA，其中包含4,998个几何问题，其中具有相应的注释程序，其说明了给定问题的解决过程。与另一个公开的数据集GEOS相比，GeoQA是25倍，程序注释可以为未来的明确和解释数值推理提供实际测试平台。此外，我们通过全面解析多媒体信息和产生可解释程序来引入神经几何求解器（NGS）来解决几何问题。我们进一步为NGS添加了多个自我监督的辅助任务，以增强跨模型语义表示。关于GeoQA的广泛实验验证了我们提出的NGS和辅助任务的有效性。然而，结果仍然明显低于人类性能，这为未来的研究留下了大型空间。我们的基准和代码在https://github.com/chen-judge/geoqa发布。

Mathematical reasoning is a fundamental aspect of human intelligence and is applicable in various fields, including science, engineering, finance, and everyday life. The development of artificial intelligence (AI) systems capable of solving math problems and proving theorems has garnered significant interest in the fields of machine learning and natural language processing. For example, mathematics serves as a testbed for aspects of reasoning that are challenging for powerful deep learning models, driving new algorithmic and modeling advances. On the other hand, recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning. In this survey paper, we review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade. We also evaluate existing benchmarks and methods, and discuss future research directions in this domain.

本文旨在通过介绍第一个中国数学预训练的语言模型〜（PLM）来提高机器的数学智能，以有效理解和表示数学问题。与其他标准NLP任务不同，数学文本很难理解，因为它们在问题陈述中涉及数学术语，符号和公式。通常，它需要复杂的数学逻辑和背景知识来解决数学问题。考虑到数学文本的复杂性质，我们设计了一种新的课程预培训方法，用于改善由基本和高级课程组成的数学PLM的学习。特别是，我们首先根据位置偏见的掩盖策略执行令牌级预训练，然后设计基于逻辑的预训练任务，旨在分别恢复改组的句子和公式。最后，我们介绍了一项更加困难的预训练任务，该任务强制执行PLM以检测和纠正其生成的解决方案中的错误。我们对离线评估（包括九个与数学相关的任务）和在线$ A/B $测试进行了广泛的实验。实验结果证明了与许多竞争基线相比，我们的方法的有效性。我们的代码可在：\ textColor {blue} {\ url {https://github.com/rucaibox/jiuzhang}}}中获得。

在过去的几年中，训练前模型的出现将单峰领域（例如计算机视觉（CV）和自然语言处理（NLP））带到了一个新时代。实质性的作品表明它们对下游大学任务有益，并避免从头开始训练新的模型。那么，此类预训练的模型可以应用于多模式任务吗？研究人员探索了这个问题并取得了重大进展。本文调查了视觉预训练（VLP）的最新进展和新的前沿，包括图像文本和视频文本预训练。为了使读者更好地掌握VLP，我们首先从五个方面回顾了其最新进展：功能提取，模型体系结构，培训预训练目标，预训练数据集和下游任务。然后，我们详细概述了特定的VLP模型。最后，我们讨论了VLP中的新边界。据我们所知，这是对VLP的首次调查。我们希望这项调查能够阐明VLP领域的未来研究。

随着图像文本对的大量数据以及视觉和语言（V＆L）任务的多样性，学者在该研究领域引入了大量的深度学习模型。此外，近年来，转移学习还显示出在计算机愿景中的巨大成功，例如图像分类，对象检测等以及在自然语言处理中以进行问答，机器翻译等的自然语言处理。继承转移学习的精神， V＆L的研究工作已经在大规模数据集上设计了多种预训练技术，以增强下游任务的性能。本文的目的是提供当代V＆L预审前模型的全面修订。特别是，我们对预处理的方法进行了分类和描述，以及最先进的视觉和语言预训练模型的摘要。此外，还提供了培训数据集和下游任务的列表，以进一步提高V＆L预处理的观点。最后，我们决定采取进一步的一步，讨论众多未来研究的方向。

最近的语言模型预培训进展取得了巨大的成功，通过利用大规模的非结构化文本数据。然而，由于没有大规模的高质量表格数据，在结构化的表格数据上应用预先培训仍然是一项挑战。在本文中，我们提出了Tapex，以表明通过在合成语料库上学习神经SQL执行程序来实现表预培训，这是通过自动合成可执行的SQL查询和执行输出来获得的。 Tapex通过引导语言模型来模仿SQL执行程序的不同，大规模和高质量的合成语料库来解决数据稀缺性挑战。我们在四个基准数据集中评估Tapex。实验结果表明，Tapex优于以前的表格预训练，并通过大幅度达到了新的最先进的结果。这包括改进弱监管的WikiSQL表示精度为89.5％（+ 2.3％），WikityQuestions表示精度为57.5％（+ 4.8％），SQA表示精度为74.5％（+ 3.5％）和Tabfact精度84.2％（+ 3.2％）。为了我们的知识，这是通过合成可执行程序利用表预培训的第一项工作，并在各种下游任务上实现新的最先进结果。

许多智力努力需要解决数学问题，但这种技能仍然超出了计算机的能力。为了测量机器学习模型中的这种能力，我们介绍了数学，这是一个12,500个挑战性竞争数学问题的新数据集。数学中的每个问题都有一个完整的逐步解决方案，可用于教授模型来生成答案派生和解释。为了促进未来的研究和提高数学准确性，我们还提供了一个大型辅助预制数据集，有助于教导模型数学的基本原则。尽管我们能够提高数学准确性，但我们的结果表明，即使有巨大的变压器模型，即使有巨大的变压器模型也是相对较低的。此外，我们发现，如果缩放趋势持续，则无法增加预算和模型参数计数对于实现强大的数学推理，这将是不切实际的。虽然缩放变压器正在自动解决大多数基于文本的任务，但缩放目前没有解决数学。为了在数学问题上进行更多牵引，我们可能需要更广泛的研究界的新算法进步。

As the complexity of modern software continues to escalate, software engineering has become an increasingly daunting and error-prone endeavor. In recent years, the field of Neural Code Intelligence (NCI) has emerged as a promising solution, leveraging the power of deep learning techniques to tackle analytical tasks on source code with the goal of improving programming efficiency and minimizing human errors within the software industry. Pretrained language models have become a dominant force in NCI research, consistently delivering state-of-the-art results across a wide range of tasks, including code summarization, generation, and translation. In this paper, we present a comprehensive survey of the NCI domain, including a thorough review of pretraining techniques, tasks, datasets, and model architectures. We hope this paper will serve as a bridge between the natural language and programming language communities, offering insights for future research in this rapidly evolving field.

Reasoning, as an essential ability for complex problem-solving, can provide back-end support for various real-world applications, such as medical diagnosis, negotiation, etc. This paper provides a comprehensive survey of cutting-edge research on reasoning with language model prompting. We introduce research works with comparisons and summaries and provide systematic resources to help beginners. We also discuss the potential reasons for emerging such reasoning abilities and highlight future research directions.

Structured tabular data exist across nearly all fields. Reasoning task over these data aims to answer questions or determine the truthiness of hypothesis sentences by understanding the semantic meaning of a table. While previous works have devoted significant efforts to the tabular reasoning task, they always assume there are sufficient labeled data. However, constructing reasoning samples over tables (and related text) is labor-intensive, especially when the reasoning process is complex. When labeled data is insufficient, the performance of models will suffer an unendurable decline. In this paper, we propose a unified framework for unsupervised complex tabular reasoning (UCTR), which generates sufficient and diverse synthetic data with complex logic for tabular reasoning tasks, assuming no human-annotated data at all. We first utilize a random sampling strategy to collect diverse programs of different types and execute them on tables based on a "Program-Executor" module. To bridge the gap between the programs and natural language sentences, we design a powerful "NL-Generator" module to generate natural language sentences with complex logic from these programs. Since a table often occurs with its surrounding texts, we further propose novel "Table-to-Text" and "Text-to-Table" operators to handle joint table-text reasoning scenarios. This way, we can adequately exploit the unlabeled table resources to obtain a well-performed reasoning model under an unsupervised setting. Our experiments cover different tasks (question answering and fact verification) and different domains (general and specific), showing that our unsupervised methods can achieve at most 93% performance compared to supervised models. We also find that it can substantially boost the supervised performance in low-resourced domains as a data augmentation technique. Our code is available at https://github.com/leezythu/UCTR.

语言模型在需要自然语言理解的各种任务上取得了非凡的表现。然而，最先进的模型通常在需要定量推理的任务上挣扎，例如在大学一级解决数学，科学和工程问题。为了帮助缩小这一差距，我们介绍了Minerva，Minerva是一种在一般自然语言数据上鉴定的大型语言模型，并进一步培训了技术内容。该模型在不使用外部工具的情况下实现了技术基准测试的最新性能。我们还评估了我们在需要定量推理的物理学，生物学，化学，经济学和其他科学方面的200多个本科生问题上评估我们的模型，并发现该模型可以正确回答其中几乎三分之一。

视觉问题应答（VQA）任务利用视觉图像和语言分析来回回答图像的文本问题。它是一个流行的研究课题，在过去十年中越来越多的现实应用。本文介绍了我们最近对AliceMind-MMU的研究（阿里巴巴的编码器 - 解码器来自Damo Academy - 多媒体理解的机器智能实验室），其比人类在VQA上获得相似甚至略微更好的结果。这是通过系统地改善VQA流水线来实现的，包括：（1）具有全面的视觉和文本特征表示的预培训; （2）与学习参加的有效跨模型互动; （3）一个新颖的知识挖掘框架，具有专门的专业专家模块，适用于复杂的VQA任务。处理不同类型的视觉问题，需要具有相应的专业知识在提高我们的VQA架构的表现方面发挥着重要作用，这取决于人力水平。进行了广泛的实验和分析，以证明新的研究工作的有效性。

近年来，统一的视觉语言框架已经大大提高，其中大多数采用编码器架构将图像文本任务统一为序列到序列的生成。但是，现有的视频语言（VIDL）模型仍需要在每个任务的模型体系结构和培训目标中进行特定于任务的设计。在这项工作中，我们探索了一个统一的VIDL框架薰衣草，其中蒙版语言建模（MLM）用作所有前训练和下游任务的常见接口。这样的统一导致了简化的模型体系结构，在多模式编码器之上，只需要一个轻巧的MLM头，而不是具有更多参数的解码器。令人惊讶的是，实验结果表明，这个统一的框架在14个VIDL基准测试中实现了竞争性能，涵盖了视频问答，文本到视频检索和视频字幕。广泛的分析进一步证明了薰衣草比现有VIDL方法的优势：（i）在多任务列出时仅使用一组参数值支持所有下游任务； （ii）对各种下游任务的几乎没有概括； （iii）在视频问题回答任务上启用零射门评估。代码可从https://github.com/microsoft/lavender获得。

我们提出了Pangu-Coder，这是一种仅预读的解码器语言模型，该模型采用pangu-alpha架构进行文本到代码生成，即给定自然语言问题描述的编程语言解决方案的合成。我们使用两阶段策略训练Pangu-Coder：第一阶段采用因果语言建模（CLM）来预先培训原始编程语言数据，而第二阶段则使用因果语言建模和掩盖语言建模（MLM）的组合培训目标，专注于文本到代码生成的下游任务，并培训松散的自然语言程序定义和代码功能。最后，我们讨论了pangu-coder-ft，该pander the是通过竞争性编程问题和代码与持续集成测试的结合进行了微调的。我们评估了pangu-coder，重点是它是否生成功能上正确的程序，并证明它在参加较小的上下文窗口和较少的数据培训的同时，它比诸如Codex之类的类似大小的模型（例如Codex）实现等效性或更好的性能。

As an important fine-grained sentiment analysis problem, aspect-based sentiment analysis (ABSA), aiming to analyze and understand people's opinions at the aspect level, has been attracting considerable interest in the last decade. To handle ABSA in different scenarios, various tasks are introduced for analyzing different sentiment elements and their relations, including the aspect term, aspect category, opinion term, and sentiment polarity. Unlike early ABSA works focusing on a single sentiment element, many compound ABSA tasks involving multiple elements have been studied in recent years for capturing more complete aspect-level sentiment information. However, a systematic review of various ABSA tasks and their corresponding solutions is still lacking, which we aim to fill in this survey. More specifically, we provide a new taxonomy for ABSA which organizes existing studies from the axes of concerned sentiment elements, with an emphasis on recent advances of compound ABSA tasks. From the perspective of solutions, we summarize the utilization of pre-trained language models for ABSA, which improved the performance of ABSA to a new stage. Besides, techniques for building more practical ABSA systems in cross-domain/lingual scenarios are discussed. Finally, we review some emerging topics and discuss some open challenges to outlook potential future directions of ABSA.

目前的视觉问题应答（VQA）任务主要考虑回答自然图像的人为注释问题。然而，除了自然图像之外，在视觉理解和推理研究中仍然可以解读具有语义丰富性的抽象图。在这项工作中，我们介绍了ICON问题的新挑战（ICONQA），其目标是在图标图像上下文中回答问题。我们发布了ICONQA，这是一个由107,439个问题和三个子任务组成的大型数据集：多图像选择，多文本选择和填充空白。 ICONQA数据集是由真实世界图中的启发，突出了抽象图理解和综合认知推理的重要性。因此，ICONQA不仅需要对象识别和文本理解等感知技能，而且还需要多种认知推理技能，例如几何推理，致辞推理和算术推理。为了促进潜在的iconqa模型来学习图标图像的语义表示，我们进一步发布了一个图标数据集图标645，其中包含377级上的645,687个彩色图标。我们进行广泛的用户研究和盲目实验，并重现各种先进的VQA方法来基准iconQA任务。此外，我们开发了一个强大的ICONQA基线Patch-TRM，它应用金字塔跨模型变压器，其中包含在图标数据集上预先培训的输入图嵌入式。 iconqa和图标645可在https://iconqa.github.io提供。

Fact verification has attracted a lot of research attention recently, e.g., in journalism, marketing, and policymaking, as misinformation and disinformation online can sway one's opinion and affect one's actions. While fact-checking is a hard task in general, in many cases, false statements can be easily debunked based on analytics over tables with reliable information. Hence, table-based fact verification has recently emerged as an important and growing research area. Yet, progress has been limited due to the lack of datasets that can be used to pre-train language models (LMs) to be aware of common table operations, such as aggregating a column or comparing tuples. To bridge this gap, in this paper we introduce PASTA, a novel state-of-the-art framework for table-based fact verification via pre-training with synthesized sentence-table cloze questions. In particular, we design six types of common sentence-table cloze tasks, including Filter, Aggregation, Superlative, Comparative, Ordinal, and Unique, based on which we synthesize a large corpus consisting of 1.2 million sentence-table pairs from WikiTables. PASTA uses a recent pre-trained LM, DeBERTaV3, and further pretrains it on our corpus. Our experimental results show that PASTA achieves new state-of-the-art performance on two table-based fact verification benchmarks: TabFact and SEM-TAB-FACTS. In particular, on the complex set of TabFact, which contains multiple operations, PASTA largely outperforms the previous state of the art by 4.7 points (85.6% vs. 80.9%), and the gap between PASTA and human performance on the small TabFact test set is narrowed to just 1.5 points (90.6% vs. 92.1%).

图像和语言建模对于视觉前训练（VLP）至关重要，该培训旨在从大规模配对的图像文本数据中学习多模式表示。但是，我们观察到，大多数现有的VLP方法着重于建模图像和文本特征之间的相互作用，同时忽略图像和文本之间的信息差异，从而遭受焦点偏见。为了解决这个问题，我们提出了一个视觉语言掩盖自动编码器框架（VLMAE）。VLMAE采用视觉生成学习，促进该模型获得细粒度和公正的特征。与以前的作品不同，Vlmae注意图像中几乎所有关键的补丁，提供了更全面的理解。广泛的实验表明，VLMAE在各种视觉语言下游任务中取得更好的性能，包括视觉问答，即使有20％的预训练速度，图像文本检索和视觉接地也是如此。

Analogical reasoning is fundamental to human cognition and holds an important place in various fields. However, previous studies mainly focus on single-modal analogical reasoning and ignore taking advantage of structure knowledge. Notably, the research in cognitive psychology has demonstrated that information from multimodal sources always brings more powerful cognitive transfer than single modality sources. To this end, we introduce the new task of multimodal analogical reasoning over knowledge graphs, which requires multimodal reasoning ability with the help of background knowledge. Specifically, we construct a Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained Transformer baselines, illustrating the potential challenges of the proposed task. We further propose a novel model-agnostic Multimodal analogical reasoning framework with Transformer (MarT) motivated by the structure mapping theory, which can obtain better performance.