最近的工作表明，大型审慎的语言模型（LMS）不仅可以在一系列自然语言处理（NLP）任务上表现出色，而且还可以开始改进推理任务，例如算术诱导，象征性操纵，并随着规模的增加而进行常识性推理。模型。但是，目前尚不清楚这些LMS的潜在能力是什么。令人惊讶的是，我们发现这些模型对某些基本的符号操纵任务有局限性，例如复制，反向和加法。当符号总数或重复符号增加时，模型性能会迅速下降。我们研究了这种现象背后的潜在原因，并检查了一组可能的方法，包括明确的位置标记，细粒度的计算步骤以及具有可呼出程序的LMS。实验结果表明，这些技术都无法完全解决最简单的添加感应问题。最后，我们向导师介绍LMS，这展示了每一个教学的步骤。 LMS带有导师的LMS能够在OOD和重复符号的情况下提供100％的精度，从而在诱导中对大型LMS边界产生新的见解。

通用事件边界检测（GEBD）是视频理解中的一项重要但挑战性的任务，该任务旨在检测人类自然感知事件边界的时刻。在本文中，我们为GEBD任务提供了本地上下文建模和全局边界解码方法。提出了局部上下文建模子网络来感知通用事件边界的各种模式，并生成强大的视频表示和可靠的边界信心。基于它们，全局边界解码子网络被利用为从全局视图解码事件边界。我们提出的方法在动力学-GEBD测试集上达到了85.13％的F1得分，与基线方法相比，它实现了22％以上的F1得分增强。该代码可从https://github.com/jackytown/gebd_challenge_cvpr2022获得。

将多模式的知识用于抽象性摘要任务是一个正在进行的研究领域，目前的技术遗传了融合，然后代范式。由于计算机视觉和自然语言处理之间的语义差距，当前方法通常将多个数据点视为单独的对象，并依靠注意机制搜索连接以融合在一起。此外，从许多框架中缺少对跨模式匹配的认识会导致性能降低。为了解决这两个缺点，我们提出了一个迭代对比对准框架（ICAF），该框架使用反复对齐和对比度来捕获图像和文本之间的连贯性。具体而言，我们设计了一个经常性比对（RA）层，以逐步研究图像贴片和文本令牌之间的细粒语义关系。在编码过程中的每个步骤中，跨模式对比度损耗被应用以直接优化嵌入式空间。根据Rouge的说法，相关得分和人类评估，我们的模型表现优于MSMO数据集上最新的基线。还进行了有关我们提出的框架和超参数设置的适用性的实验。

深入学习，核算用于使用精心讲解的神经网络，最近被开发为一种有效而强大的工具，可以解决物理和其他科学中的不同问题。在目前的工作中，我们提出了一种基于混合网络的新型学习方法，其集成了两种不同类型的神经网络：长期内存（LSTM）和深度剩余网络（Reset），以克服数值模拟中遇到困难实际系统的强烈振动动态演变。通过以双倍潜力的浓缩物的动态为例，我们表明我们的新方法是高效的预学习和对整个动态的高保真预测。这种利益来自LSTM和Reset的组合，并且在直接学习的情况下，单个网络是不可能实现的。我们的方法可以应用于借助于辅助频谱分析模拟具有快多频振荡的系统中的复杂协作动态。

Theoretical properties of bilevel problems are well studied when the lower-level problem is strongly convex. In this work, we focus on bilevel optimization problems without the strong-convexity assumption. In these cases, we first show that the common local optimality measures such as KKT condition or regularization can lead to undesired consequences. Then, we aim to identify the mildest conditions that make bilevel problems tractable. We identify two classes of growth conditions on the lower-level objective that leads to continuity. Under these assumptions, we show that the local optimality of the bilevel problem can be defined via the Goldstein stationarity condition of the hyper-objective. We then propose the Inexact Gradient-Free Method (IGFM) to solve the bilevel problem, using an approximate zeroth order oracle that is of independent interest. Our non-asymptotic analysis demonstrates that the proposed method can find a $(\delta, \varepsilon)$ Goldstein stationary point for bilevel problems with a zeroth order oracle complexity that is polynomial in $d, 1/\delta$ and $1/\varepsilon$.

Due to their ability to offer more comprehensive information than data from a single view, multi-view (multi-source, multi-modal, multi-perspective, etc.) data are being used more frequently in remote sensing tasks. However, as the number of views grows, the issue of data quality becomes more apparent, limiting the potential benefits of multi-view data. Although recent deep neural network (DNN) based models can learn the weight of data adaptively, a lack of research on explicitly quantifying the data quality of each view when fusing them renders these models inexplicable, performing unsatisfactorily and inflexible in downstream remote sensing tasks. To fill this gap, in this paper, evidential deep learning is introduced to the task of aerial-ground dual-view remote sensing scene classification to model the credibility of each view. Specifically, the theory of evidence is used to calculate an uncertainty value which describes the decision-making risk of each view. Based on this uncertainty, a novel decision-level fusion strategy is proposed to ensure that the view with lower risk obtains more weight, making the classification more credible. On two well-known, publicly available datasets of aerial-ground dual-view remote sensing images, the proposed approach achieves state-of-the-art results, demonstrating its effectiveness. The code and datasets of this article are available at the following address: https://github.com/gaopiaoliang/Evidential.

Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field. ChatGPT also opens up the possibility of building explanatory AI for stance detection.

Video-language pre-training has advanced the performance of various downstream video-language tasks. However, most previous methods directly inherit or adapt typical image-language pre-training paradigms to video-language pre-training, thus not fully exploiting the unique characteristic of video, i.e., temporal. In this paper, we propose a Hierarchical Temporal-Aware video-language pre-training framework, HiTeA, with two novel pre-training tasks for modeling cross-modal alignment between moments and texts as well as the temporal relations of video-text pairs. Specifically, we propose a cross-modal moment exploration task to explore moments in videos, which results in detailed video moment representation. Besides, the inherent temporal relations are captured by aligning video-text pairs as a whole in different time resolutions with multi-modal temporal relation exploration task. Furthermore, we introduce the shuffling test to evaluate the temporal reliance of datasets and video-language pre-training models. We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks, especially on temporal-oriented datasets (e.g., SSv2-Template and SSv2-Label) with 8.6% and 11.1% improvement respectively. HiTeA also demonstrates strong generalization ability when directly transferred to downstream tasks in a zero-shot manner. Models and demo will be available on ModelScope.

Face manipulation detection has been receiving a lot of attention for the reliability and security of the face images. Recent studies focus on using auxiliary information or prior knowledge to capture robust manipulation traces, which are shown to be promising. As one of the important face features, the face depth map, which has shown to be effective in other areas such as the face recognition or face detection, is unfortunately paid little attention to in literature for detecting the manipulated face images. In this paper, we explore the possibility of incorporating the face depth map as auxiliary information to tackle the problem of face manipulation detection in real world applications. To this end, we first propose a Face Depth Map Transformer (FDMT) to estimate the face depth map patch by patch from a RGB face image, which is able to capture the local depth anomaly created due to manipulation. The estimated face depth map is then considered as auxiliary information to be integrated with the backbone features using a Multi-head Depth Attention (MDA) mechanism that is newly designed. Various experiments demonstrate the advantage of our proposed method for face manipulation detection.

Implicit regularization is an important way to interpret neural networks. Recent theory starts to explain implicit regularization with the model of deep matrix factorization (DMF) and analyze the trajectory of discrete gradient dynamics in the optimization process. These discrete gradient dynamics are relatively small but not infinitesimal, thus fitting well with the practical implementation of neural networks. Currently, discrete gradient dynamics analysis has been successfully applied to shallow networks but encounters the difficulty of complex computation for deep networks. In this work, we introduce another discrete gradient dynamics approach to explain implicit regularization, i.e. landscape analysis. It mainly focuses on gradient regions, such as saddle points and local minima. We theoretically establish the connection between saddle point escaping (SPE) stages and the matrix rank in DMF. We prove that, for a rank-R matrix reconstruction, DMF will converge to a second-order critical point after R stages of SPE. This conclusion is further experimentally verified on a low-rank matrix reconstruction problem. This work provides a new theory to analyze implicit regularization in deep learning.