During the deployment of deep neural networks (DNNs) on edge devices, many research efforts are devoted to the limited hardware resource. However, little attention is paid to the influence of dynamic power management. As edge devices typically only have a budget of energy with batteries (rather than almost unlimited energy support on servers or workstations), their dynamic power management often changes the execution frequency as in the widely-used dynamic voltage and frequency scaling (DVFS) technique. This leads to highly unstable inference speed performance, especially for computation-intensive DNN models, which can harm user experience and waste hardware resources. We firstly identify this problem and then propose All-in-One, a highly representative pruning framework to work with dynamic power management using DVFS. The framework can use only one set of model weights and soft masks (together with other auxiliary parameters of negligible storage) to represent multiple models of various pruning ratios. By re-configuring the model to the corresponding pruning ratio for a specific execution frequency (and voltage), we are able to achieve stable inference speed, i.e., keeping the difference in speed performance under various execution frequencies as small as possible. Our experiments demonstrate that our method not only achieves high accuracy for multiple models of different pruning ratios, but also reduces their variance of inference latency for various frequencies, with minimal memory consumption of only one model and one soft mask.
translated by 谷歌翻译
基于深度学习的超分辨率(SR)近年来由于其高图像质量性能和广泛的应用方案而获得了极大的知名度。但是,先前的方法通常会遭受大量计算和巨大的功耗,这会导致实时推断的困难,尤其是在资源有限的平台(例如移动设备)上。为了减轻这种情况,我们建议使用自适应SR块进行深度搜索和每层宽度搜索,以进行深度搜索和每层宽度搜索。推理速度与SR损失一起直接将其带入具有高图像质量的SR模型,同​​时满足实时推理需求。借用了与编译器优化的速度模型在搜索过程中每次迭代中的移动设备上的速度,以预测具有各种宽度配置的SR块的推理潜伏期,以更快地收敛。通过提出的框架,我们在移动平台的GPU/DSP上实现了实时SR推断,以实现具有竞争性SR性能的720p分辨率(三星Galaxy S21)。
translated by 谷歌翻译
最近,视觉变压器(VIT)在计算机视野中连续建立了新的里程碑,而高计算和内存成本使其在工业生产中的传播困难。修剪是一种用于硬件效率的传统模型压缩范例,已广泛应用于各种DNN结构。尽管如此,它含糊不清,如何在vit结构上进行独家修剪。考虑三个关键点:结构特征,VITS的内部数据模式和相关边缘设备部署,我们利用输入令牌稀疏性并提出了一种计算感知软修剪框架,可以在扁平的vanilla变压器上设置。和CNN型结构,例如基于池的Vit(坑)。更具体地说,我们设计了一种基于动态关注的多头令牌选择器,它是一个轻量级模块,用于自适应实例 - 明智令牌选择。我们进一步引入了一种软修剪技术,它将选择器模块生成的较少的信息令牌集成到将参与后续计算的包令牌,而不是完全丢弃。我们的框架通过我们所提出的计算感知培训策略,我们通过特定边缘设备的准确性和计算限制之间的权衡。实验结果表明,我们的框架显着降低了VIT的计算成本,同时在图像分类上保持了可比性。此外,我们的框架可以保证所识别的模型,以满足移动设备和FPGA的资源规范,甚至在移动平台上实现DEIT-T的实时执行。例如,我们的方法在移动设备上减少了DEIT-T至26毫秒的延迟(26%$ \ SIM 41%的41%),在移动设备上,在0.25%$ \ sim $ 4%的ImageNet上的前1个精度高出4%。我们的代码即将发布。
translated by 谷歌翻译
深度神经网络(DNN)已显示在许多现实生活中提供极好的性能,但它们的大量计算成本和存储要求已阻止它们部署到许多边缘和内部内容(IOT)设备。稀疏的深神经网络,其大多数重量参数是零,可以大大降低模型的计算复杂性和存储器消耗。在实际使用场景中,设备可能遭受不同环境下的可用计算和存储器资源的大波动,并且由于具有大延迟的长尾延长而难以维持服务质量(QoS)。面对现实生活挑战,我们建议培训支持多个稀疏水平的稀疏模型。也就是说,满足权重的分层结构,使得较少稀疏子模型的较少稀疏子模型区域子集的位置和非零参数的位置。以这种方式,可以在推理期间动态地选择适当的稀疏度水平,而存储成本被最小稀疏子模型覆盖。我们已经在各种DNN模型和任务中验证了我们的方法,包括Reset-50,PointNet ++,GNMT和图表注意网络。我们获得稀疏的子模型,平均重量为13.38%,拖鞋14.97%,而准确性也与他们的密集对应物一样好。具有5.38%权重和4.47%的更稀疏的子模型,跨越少量稀疏的跨,只能获得3.25%的相对精度损耗。
translated by 谷歌翻译
已经证明,深度神经网络(DNN)在解决许多现实问题方面是有效的,但其高计算成本禁止将这些模型部署到边缘设备。修剪,作为将零的方法引入模型重量的方法,已显示是在模型精度和计算效率之间提供良好权衡的有效方法,并且是一种生成压缩模型的广泛使用的方法。然而,修剪的粒度使得重要的权衡。在相同的稀疏性水平上,粗粒结构的稀疏图案在传统硬件上更有效,但导致更差的精度,而细粒度的非结构化稀疏模式可以实现更好的精度,但在现有硬件上效率低下。另一方面,一些现代处理器配备了快速的片上刻痕存储器和聚集/散射引擎,用于在这种存储器上执行间接负载和存储操作。在这项工作中,我们提出了一系列新颖的稀疏模式,命名为聚光散射(GS)模式,以利用Scratchpad存储器和收集/散射引擎来加速神经网络推论。相应地,我们呈现了一种紧凑的稀疏格式。提出的稀疏模式,以及一种新颖的修剪方法,解决了负载不平衡问题,并导致质量接近非结构化稀疏模型的型号,以及靠近结构化稀疏型号的计算效率。我们的实验表明,与传统结构稀疏模式相比,GS模式在精度和计算效率之间始终如一地进行折衷。 GS模式可以以相同的精度级别将DNN组件的运行时间减少两到三次。这是在三个不同的深度学习任务和流行模型中确认,即机器翻译的GNMT,用于图像识别的Reset50,以及用于声学语音识别的Japser。
translated by 谷歌翻译
深度神经网络(DNN)在解决许多真实问题方面都有效。较大的DNN模型通常表现出更好的质量(例如,精度,精度),但它们的过度计算会导致长期推理时间。模型稀疏可以降低计算和内存成本,同时保持模型质量。大多数现有的稀疏算法是单向移除的重量,而其他人则随机或贪婪地探索每层进行修剪的小权重子集。这些算法的局限性降低了可实现的稀疏性水平。此外,许多算法仍然需要预先训练的密集模型,因此遭受大的内存占地面积。在本文中,我们提出了一种新颖的预定生长和修剪(间隙)方法,而无需预先培训密集模型。它通过反复生长一个层次的层来解决以前的作品的缺点,然后在一些训练后修剪回到稀疏。实验表明,使用所提出的方法修剪模型匹配或击败高度优化的密集模型的质量,在各种任务中以80%的稀疏度,例如图像分类,客观检测,3D对象分段和翻译。它们还优于模型稀疏的其他最先进的(SOTA)方法。作为一个例子,通过间隙获得的90%不均匀的稀疏resnet-50模型在想象中实现了77.9%的前1个精度,提高了先前的SOTA结果1.5%。所有代码将公开发布。
translated by 谷歌翻译
Artificial Intelligence (AI) has become commonplace to solve routine everyday tasks. Because of the exponential growth in medical imaging data volume and complexity, the workload on radiologists is steadily increasing. We project that the gap between the number of imaging exams and the number of expert radiologist readers required to cover this increase will continue to expand, consequently introducing a demand for AI-based tools that improve the efficiency with which radiologists can comfortably interpret these exams. AI has been shown to improve efficiency in medical-image generation, processing, and interpretation, and a variety of such AI models have been developed across research labs worldwide. However, very few of these, if any, find their way into routine clinical use, a discrepancy that reflects the divide between AI research and successful AI translation. To address the barrier to clinical deployment, we have formed MONAI Consortium, an open-source community which is building standards for AI deployment in healthcare institutions, and developing tools and infrastructure to facilitate their implementation. This report represents several years of weekly discussions and hands-on problem solving experience by groups of industry experts and clinicians in the MONAI Consortium. We identify barriers between AI-model development in research labs and subsequent clinical deployment and propose solutions. Our report provides guidance on processes which take an imaging AI model from development to clinical implementation in a healthcare institution. We discuss various AI integration points in a clinical Radiology workflow. We also present a taxonomy of Radiology AI use-cases. Through this report, we intend to educate the stakeholders in healthcare and AI (AI researchers, radiologists, imaging informaticists, and regulators) about cross-disciplinary challenges and possible solutions.
translated by 谷歌翻译
Smart City applications, such as traffic monitoring and disaster response, often use swarms of intelligent and cooperative drones to efficiently collect sensor data over different areas of interest and time spans. However, when the required sensing becomes spatio-temporally large and varying, a collective arrangement of sensing tasks to a large number of battery-restricted and distributed drones is challenging. To address this problem, we introduce a scalable and energy-aware model for planning and coordination of spatio-temporal sensing. The coordination model is built upon a decentralized multi-agent collective learning algorithm (EPOS) to ensure scalability, resilience, and flexibility that existing approaches lack of. Experimental results illustrate the outstanding performance of the proposed method compared to state-of-the-art methods. Analytical results contribute a deeper understanding of how coordinated mobility of drones influences sensing performance. This novel coordination solution is applied to traffic monitoring using real-world data to demonstrate a $46.45\%$ more accurate and $2.88\%$ more efficient detection of vehicles as the number of drones become a scarce resource.
translated by 谷歌翻译
Unbiased learning to rank (ULTR) studies the problem of mitigating various biases from implicit user feedback data such as clicks, and has been receiving considerable attention recently. A popular ULTR approach for real-world applications uses a two-tower architecture, where click modeling is factorized into a relevance tower with regular input features, and a bias tower with bias-relevant inputs such as the position of a document. A successful factorization will allow the relevance tower to be exempt from biases. In this work, we identify a critical issue that existing ULTR methods ignored - the bias tower can be confounded with the relevance tower via the underlying true relevance. In particular, the positions were determined by the logging policy, i.e., the previous production model, which would possess relevance information. We give both theoretical analysis and empirical results to show the negative effects on relevance tower due to such a correlation. We then propose three methods to mitigate the negative confounding effects by better disentangling relevance and bias. Empirical results on both controlled public datasets and a large-scale industry dataset show the effectiveness of the proposed approaches.
translated by 谷歌翻译
Benefiting from its single-photon sensitivity, single-photon avalanche diode (SPAD) array has been widely applied in various fields such as fluorescence lifetime imaging and quantum computing. However, large-scale high-fidelity single-photon imaging remains a big challenge, due to the complex hardware manufacture craft and heavy noise disturbance of SPAD arrays. In this work, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging over an order of magnitude, with significant enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 $\times$ 32 pixels, 90 scenes, 10 different bit depth, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this real-world physical noise model, we for the first time synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depth, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique on a series of experiments including macroscopic and microscopic imaging, microfluidic inspection, and Fourier ptychography. The experiments validate the technique's state-of-the-art super-resolution SPAD imaging performance, with more than 5 dB superiority on PSNR compared to the existing methods.
translated by 谷歌翻译