在本文中，我们从经验上研究了如何充分利用低分辨率框架以进行有效的视频识别。现有方法主要集中于开发紧凑的网络或减轻视频输入的时间冗余以提高效率，而压缩框架分辨率很少被认为是有希望的解决方案。一个主要问题是低分辨率帧的识别准确性不佳。因此，我们首先分析低分辨率帧上性能降解的根本原因。我们的主要发现是，降级的主要原因不是在下采样过程中的信息丢失，而是网络体系结构和输入量表之间的不匹配。通过知识蒸馏（KD）的成功，我们建议通过跨分辨率KD（RESKD）弥合网络和输入大小之间的差距。我们的工作表明，RESKD是一种简单但有效的方法，可以提高低分辨率帧的识别精度。没有铃铛和哨子，RESKD在四个大规模基准数据集（即ActivityNet，FCVID，Mini-Kinetics，sopeings soseings ossings v2）上，就效率和准确性上的所有竞争方法都大大超过了所有竞争方法。此外，我们广泛地展示了其对最先进的体系结构（即3D-CNN和视频变压器）的有效性，以及对超低分辨率帧的可扩展性。结果表明，RESKD可以作为最先进视频识别的一般推理加速方法。我们的代码将在https://github.com/cvmi-lab/reskd上找到。

虽然外源变量对时间序列分析的性能改善有重大影响，但在当前的连续方法中很少考虑这些序列间相关性和时间依赖性。多元时间序列的动力系统可以用复杂的未知偏微分方程（PDE）进行建模，这些方程（PDE）在科学和工程的许多学科中都起着重要作用。在本文中，我们提出了一个任意步骤预测的连续时间模型，以学习多元时间序列中的未知PDE系统，其管理方程是通过自我注意和封闭的复发神经网络参数化的。所提出的模型\下划线{变量及其对目标系列的影响。重要的是，使用特殊设计的正则化指南可以将模型简化为正则化的普通微分方程（ODE）问题，这使得可以触犯的PDE问题以获得数值解决方案，并且可行，以预测目标序列的多个未来值。广泛的实验表明，我们提出的模型可以在强大的基准中实现竞争精度：平均而言，它通过降低RMSE的$ 9.85 \％$和MAE的MAE $ 13.98 \％$的基线表现优于最佳基准，以获得任意步骤预测的MAE $。

数字图像相关性（DIC）已成为一种行业标准，以检索拉伸试验和其他材料表征中的精确位移和应变测量。虽然传统的DIC为一般拉伸检测情况提供了高精度估计，但是在大变形或斑点图案开始撕裂时，预测变得不稳定。此外，传统的DIC需要长的计算时间，并且通常会产生受滤波和散斑图案质量影响的低空间分辨率输出。为了解决这些挑战，我们提出了一种新的深度学习的DIC方法 - 深层DIC，其中两个卷积神经网络，偏移和拉力纳特，旨在共同努力，以实现位移和菌株的端到端预测。 displacementNet预测位移字段并自适应地跟踪感兴趣的区域。 RATEDNET直接从图像输入预测应变场，而不依赖于位移预测，这显着提高了应变预测精度。开发了一种新的数据集生成方法以综合现实和全面的数据集，包括产生散斑图案和具有合成位移场的斑点图像的变形。虽然仅接受了合成数据集的培训，但深度DIC提供了从商业DIC软件获得的真实实验中获得的那些对位移和应变的高度一致和可比的预测，而即使在大型和局部变形和变化的变形和变化的模式质量和变化的模式质量方面，它占商业软件。 。此外，深DIC能够实时预测变形，并将计算时间降至毫秒。

在预先建立的3D环境图中，高精度摄像头重新定位技术是许多任务的基础，例如增强现实，机器人技术和自动驾驶。近几十年来，基于点的视觉重新定位方法已经发达了，但在某些不足的情况下不足。在本文中，我们设计了一条完整的管道，用于使用点和线的相机姿势完善，其中包含创新设计的生产线提取CNN，名为VLSE，线匹配和姿势优化方法。我们采用新颖的线表示，并根据堆叠的沙漏网络自定义混合卷积块，以检测图像上的准确稳定的线路功能。然后，我们采用基于几何的策略，使用表极约束和再投影过滤获得精确的2D-3D线对应关系。构建了以下点线关节成本函数，以通过基于纯点的本地化的初始粗姿势优化相机姿势。在开放数据集（即线框上的线提取器）上进行了足够的实验，在INLOC DUC1和DUC2上的定位性能，以确认我们的点线关节姿势优化方法的有效性。

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.

It has been observed in practice that applying pruning-at-initialization methods to neural networks and training the sparsified networks can not only retain the testing performance of the original dense models, but also sometimes even slightly boost the generalization performance. Theoretical understanding for such experimental observations are yet to be developed. This work makes the first attempt to study how different pruning fractions affect the model's gradient descent dynamics and generalization. Specifically, this work considers a classification task for overparameterized two-layer neural networks, where the network is randomly pruned according to different rates at the initialization. It is shown that as long as the pruning fraction is below a certain threshold, gradient descent can drive the training loss toward zero and the network exhibits good generalization performance. More surprisingly, the generalization bound gets better as the pruning fraction gets larger. To complement this positive result, this work further shows a negative result: there exists a large pruning fraction such that while gradient descent is still able to drive the training loss toward zero (by memorizing noise), the generalization performance is no better than random guessing. This further suggests that pruning can change the feature learning process, which leads to the performance drop of the pruned neural network. Up to our knowledge, this is the \textbf{first} generalization result for pruned neural networks, suggesting that pruning can improve the neural network's generalization.

Time-series anomaly detection is an important task and has been widely applied in the industry. Since manual data annotation is expensive and inefficient, most applications adopt unsupervised anomaly detection methods, but the results are usually sub-optimal and unsatisfactory to end customers. Weak supervision is a promising paradigm for obtaining considerable labels in a low-cost way, which enables the customers to label data by writing heuristic rules rather than annotating each instance individually. However, in the time-series domain, it is hard for people to write reasonable labeling functions as the time-series data is numerically continuous and difficult to be understood. In this paper, we propose a Label-Efficient Interactive Time-Series Anomaly Detection (LEIAD) system, which enables a user to improve the results of unsupervised anomaly detection by performing only a small amount of interactions with the system. To achieve this goal, the system integrates weak supervision and active learning collaboratively while generating labeling functions automatically using only a few labeled data. All of these techniques are complementary and can promote each other in a reinforced manner. We conduct experiments on three time-series anomaly detection datasets, demonstrating that the proposed system is superior to existing solutions in both weak supervision and active learning areas. Also, the system has been tested in a real scenario in industry to show its practicality.

As an important variant of entity alignment (EA), multi-modal entity alignment (MMEA) aims to discover identical entities across different knowledge graphs (KGs) with multiple modalities like images. However, current MMEA algorithms all adopt KG-level modality fusion strategies but ignore modality differences among individual entities, hurting the robustness to potential noise involved in modalities (e.g., unidentifiable images and relations). In this paper we present MEAformer, a multi-modal entity alignment transformer approach for meta modality hybrid, to dynamically predict the mutual correlation coefficients among modalities for instance-level feature fusion. A modal-aware hard entity replay strategy is also proposed for addressing vague entity details. Extensive experimental results show that our model not only achieves SOTA performance on multiple training scenarios including supervised, unsupervised, iterative, and low resource, but also has limited parameters, optimistic speed, and good interpretability. Our code will be available soon.

The task of video prediction and generation is known to be notoriously difficult, with the research in this area largely limited to short-term predictions. Though plagued with noise and stochasticity, videos consist of features that are organised in a spatiotemporal hierarchy, different features possessing different temporal dynamics. In this paper, we introduce Dynamic Latent Hierarchy (DLH) -- a deep hierarchical latent model that represents videos as a hierarchy of latent states that evolve over separate and fluid timescales. Each latent state is a mixture distribution with two components, representing the immediate past and the predicted future, causing the model to learn transitions only between sufficiently dissimilar states, while clustering temporally persistent states closer together. Using this unique property, DLH naturally discovers the spatiotemporal structure of a dataset and learns disentangled representations across its hierarchy. We hypothesise that this simplifies the task of modeling temporal dynamics of a video, improves the learning of long-term dependencies, and reduces error accumulation. As evidence, we demonstrate that DLH outperforms state-of-the-art benchmarks in video prediction, is able to better represent stochasticity, as well as to dynamically adjust its hierarchical and temporal structure. Our paper shows, among other things, how progress in representation learning can translate into progress in prediction tasks.

Implicit regularization is an important way to interpret neural networks. Recent theory starts to explain implicit regularization with the model of deep matrix factorization (DMF) and analyze the trajectory of discrete gradient dynamics in the optimization process. These discrete gradient dynamics are relatively small but not infinitesimal, thus fitting well with the practical implementation of neural networks. Currently, discrete gradient dynamics analysis has been successfully applied to shallow networks but encounters the difficulty of complex computation for deep networks. In this work, we introduce another discrete gradient dynamics approach to explain implicit regularization, i.e. landscape analysis. It mainly focuses on gradient regions, such as saddle points and local minima. We theoretically establish the connection between saddle point escaping (SPE) stages and the matrix rank in DMF. We prove that, for a rank-R matrix reconstruction, DMF will converge to a second-order critical point after R stages of SPE. This conclusion is further experimentally verified on a low-rank matrix reconstruction problem. This work provides a new theory to analyze implicit regularization in deep learning.