During the deployment of deep neural networks (DNNs) on edge devices, many research efforts are devoted to the limited hardware resource. However, little attention is paid to the influence of dynamic power management. As edge devices typically only have a budget of energy with batteries (rather than almost unlimited energy support on servers or workstations), their dynamic power management often changes the execution frequency as in the widely-used dynamic voltage and frequency scaling (DVFS) technique. This leads to highly unstable inference speed performance, especially for computation-intensive DNN models, which can harm user experience and waste hardware resources. We firstly identify this problem and then propose All-in-One, a highly representative pruning framework to work with dynamic power management using DVFS. The framework can use only one set of model weights and soft masks (together with other auxiliary parameters of negligible storage) to represent multiple models of various pruning ratios. By re-configuring the model to the corresponding pruning ratio for a specific execution frequency (and voltage), we are able to achieve stable inference speed, i.e., keeping the difference in speed performance under various execution frequencies as small as possible. Our experiments demonstrate that our method not only achieves high accuracy for multiple models of different pruning ratios, but also reduces their variance of inference latency for various frequencies, with minimal memory consumption of only one model and one soft mask.

早期退出是提高深网推理效率的有效范例。通过构建具有不同资源需求的分类器（出口），此类网络可以在早期出口处输出简单的样本，从而消除了执行更深层的需求。尽管现有作品主要关注多EXIT网络的建筑设计，但此类模型的培训策略在很大程度上没有探索。当前的最新模型在培训期间对所有样品进行了相同的处理。但是，在测试过程中的早期外观行为被忽略了，从而导致训练和测试之间存在差距。在本文中，我们建议通过样品加权来弥合这一差距。从直觉上讲，简单的样品通常在推理期间在网络早期退出，应该为培训早期分类器提供更多贡献。但是，晚期分类器应强调硬样品的培训（主要是从更深层退出）。我们的工作建议采用一个体重预测网络，以加重每个出口处不同训练样本的损失。这个重量预测网络和骨干模型在具有新的优化目标的元学习框架下共同优化。通过将推断期间的适应性行为带入训练阶段，我们表明拟议的加权机制始终提高分类准确性和推理效率之间的权衡。代码可在https://github.com/leaplabthu/l2w-den上找到。

基于深度学习的超分辨率（SR）近年来由于其高图像质量性能和广泛的应用方案而获得了极大的知名度。但是，先前的方法通常会遭受大量计算和巨大的功耗，这会导致实时推断的困难，尤其是在资源有限的平台（例如移动设备）上。为了减轻这种情况，我们建议使用自适应SR块进行深度搜索和每层宽度搜索，以进行深度搜索和每层宽度搜索。推理速度与SR损失一起直接将其带入具有高图像质量的SR模型，同​​时满足实时推理需求。借用了与编译器优化的速度模型在搜索过程中每次迭代中的移动设备上的速度，以预测具有各种宽度配置的SR块的推理潜伏期，以更快地收敛。通过提出的框架，我们在移动平台的GPU/DSP上实现了实时SR推断，以实现具有竞争性SR性能的720p分辨率（三星Galaxy S21）。

准确和高效的行人检测对于关于行人安全和移动性的智能运输系统至关重要，例如先进的驾驶员辅助系统和智能行人人行横道系统。在所有行人检测方法中，基于视觉的检测方法被证明是先前研究中最有效的。然而，现有的基于视觉的行人检测算法仍然有两个限制其实现的限制，那些是实时性能以及对环境因素的影响的阻力，例如，低照明条件。为了解决这些问题，本研究提出了一种轻量级的照明和温度感知多光谱网络（IT-MN），用于准确和高效的行人检测。所提出的IT-Mn是一种有效的一级探测器。为了适应环境因素的影响并增强感测的精度，当视觉图像质量有限时，通过所提出的IT-MN融合了热图像数据，以丰富有用的信息。此外，还开发了一种创新和有效的晚期融合策略来优化图像融合性能。为了使所提出的模型可实现用于边缘计算，应用模型量化以减少模型大小，同时显着缩短推测时间。通过使用由车载摄像机收集的公共数据集进行评估，通过将所提出的算法与所选的最先进的算法进行评估。结果表明，该算法在GPU上以14.19％和0.03秒实现了低的错过率和推理时间。此外，量化的IT-Mn在边缘设备上实现每张映像对的推理时间为0.21秒，这还展示了将所提出的边缘设备上的模型部署为高效的行人检测算法的潜力。

重量修剪是一种有效的模型压缩技术，可以解决在移动设备上实现实时深神经网络（DNN）推断的挑战。然而，由于精度劣化，难以利用硬件加速度，以及某些类型的DNN层的限制，难以降低的应用方案具有有限的应用方案。在本文中，我们提出了一般的细粒度的结构化修剪方案和相应的编译器优化，适用于任何类型的DNN层，同时实现高精度和硬件推理性能。随着使用我们的编译器优化所支持的不同层的灵活性，我们进一步探讨了确定最佳修剪方案的新问题，了解各种修剪方案的不同加速度和精度性能。两个修剪方案映射方法，一个是基于搜索，另一个是基于规则的，建议自动推导出任何给定DNN的每层的最佳修剪规则和块大小。实验结果表明，我们的修剪方案映射方法，以及一般细粒化结构修剪方案，优于最先进的DNN优化框架，最高可达2.48 $ \ times $和1.73 $ \ times $ DNN推理加速在CiFar-10和Imagenet DataSet上没有准确性损失。

只给出了积极的例子和未标记的例子（来自正面和负数），我们可能希望估计准确的正面与负分类器。正式地，该任务分为两个子任务：（i）混合比例估计（MPE） - 确定未标记数据中的正例的分数; （ii）PU-Learning - 鉴于这样的估计，学习所需的正面与负分类器。不幸的是，两个问题的古典方法在高维设置中分解。与此同时，最近提出的启发式缺乏理论一致性，并效力依赖于近双车调谐。在本文中，我们提出了两种简单的技术：最好的箱估计（BBE）（用于MPE）;而有条件的价值忽略风险（CVIR），对PU学习的简单目标。这两种方法都主导了先前的方法，并且对于BBE，我们建立正式保证，每当我们可以培训模型来干净地分离出一小部分积极示例的担保。我们的最终算法（TED）$ ^ N $，两种程序之间交替，显着改善了我们的混合比例估计器和分类器

Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.

Fine-grained visual parsing, including fine-grained part segmentation and fine-grained object recognition, has attracted considerable critical attention due to its importance in many real-world applications, e.g., agriculture, remote sensing, and space technologies. Predominant research efforts tackle these fine-grained sub-tasks following different paradigms, while the inherent relations between these tasks are neglected. Moreover, given most of the research remains fragmented, we conduct an in-depth study of the advanced work from a new perspective of learning the part relationship. In this perspective, we first consolidate recent research and benchmark syntheses with new taxonomies. Based on this consolidation, we revisit the universal challenges in fine-grained part segmentation and recognition tasks and propose new solutions by part relationship learning for these important challenges. Furthermore, we conclude several promising lines of research in fine-grained visual parsing for future research.

Fine-grained visual recognition is to classify objects with visually similar appearances into subcategories, which has made great progress with the development of deep CNNs. However, handling subtle differences between different subcategories still remains a challenge. In this paper, we propose to solve this issue in one unified framework from two aspects, i.e., constructing feature-level interrelationships, and capturing part-level discriminative features. This framework, namely PArt-guided Relational Transformers (PART), is proposed to learn the discriminative part features with an automatic part discovery module, and to explore the intrinsic correlations with a feature transformation module by adapting the Transformer models from the field of natural language processing. The part discovery module efficiently discovers the discriminative regions which are highly-corresponded to the gradient descent procedure. Then the second feature transformation module builds correlations within the global embedding and multiple part embedding, enhancing spatial interactions among semantic pixels. Moreover, our proposed approach does not rely on additional part branches in the inference time and reaches state-of-the-art performance on 3 widely-used fine-grained object recognition benchmarks. Experimental results and explainable visualizations demonstrate the effectiveness of our proposed approach. The code can be found at https://github.com/iCVTEAM/PART.

Despite recent progress towards scaling up multimodal vision-language models, these models are still known to struggle on compositional generalization benchmarks such as Winoground. We find that a critical component lacking from current vision-language models is relation-level alignment: the ability to match directional semantic relations in text (e.g., "mug in grass") with spatial relationships in the image (e.g., the position of the mug relative to the grass). To tackle this problem, we show that relation alignment can be enforced by encouraging the directed language attention from 'mug' to 'grass' (capturing the semantic relation 'in') to match the directed visual attention from the mug to the grass. Tokens and their corresponding objects are softly identified using the cross-modal attention. We prove that this notion of soft relation alignment is equivalent to enforcing congruence between vision and language attention matrices under a 'change of basis' provided by the cross-modal attention matrix. Intuitively, our approach projects visual attention into the language attention space to calculate its divergence from the actual language attention, and vice versa. We apply our Cross-modal Attention Congruence Regularization (CACR) loss to UNITER and improve on the state-of-the-art approach to Winoground.