边缘设备上卷积神经网络（CNN）的部署受到性能要求和可用处理能力之间的巨大差距的阻碍。尽管最近的研究在开发网络修剪方法以减少CNN的计算开销方面取得了长足的进步，但仍然存在相当大的准确性损失，尤其是在高修剪比率下。质疑为非封闭网络设计的架构可能对修剪网络没有效，我们建议通过定义新的搜索空间和新颖的搜索目标来搜索架构修剪方法。为了改善修剪网络的概括，我们提出了两个新型的原始孔和prunedlinearaare操作。具体而言，这些操作通过正规化修剪网络的目标函数来缓解不稳定梯度的问题。提出的搜索目标使我们能够培训有关修剪权重元素的体系结构参数。定量分析表明，我们的搜索架构优于在CIFAR-10和Imagenet上最先进的修剪网络中使用的体系结构。就硬件效率而言，PR-DARTS将Mobilenet-V2的准确性从73.44％提高到81.35％（+7.91％提高），并且运行3.87 $ \ times $的速度更快。

深度学习技术在各种任务中都表现出了出色的有效性，并且深度学习具有推进多种应用程序（包括在边缘计算中）的潜力，其中将深层模型部署在边缘设备上，以实现即时的数据处理和响应。一个关键的挑战是，虽然深层模型的应用通常会产生大量的内存和计算成本，但Edge设备通常只提供非常有限的存储和计算功能，这些功能可能会在各个设备之间差异很大。这些特征使得难以构建深度学习解决方案，以释放边缘设备的潜力，同时遵守其约束。应对这一挑战的一种有希望的方法是自动化有效的深度学习模型的设计，这些模型轻巧，仅需少量存储，并且仅产生低计算开销。该调查提供了针对边缘计算的深度学习模型设计自动化技术的全面覆盖。它提供了关键指标的概述和比较，这些指标通常用于量化模型在有效性，轻度和计算成本方面的水平。然后，该调查涵盖了深层设计自动化技术的三类最新技术：自动化神经体系结构搜索，自动化模型压缩以及联合自动化设计和压缩。最后，调查涵盖了未来研究的开放问题和方向。

虽然可分辨率的架构搜索（飞镖）已成为神经结构中的主流范例（NAS），因为其简单和效率，最近的作品发现，搜索架构的性能几乎可以随着飞镖的优化程序而增加，以及最终的大小由飞镖获得几乎无法表明运营的重要性。上述观察表明，飞镖中的监督信号可能是架构搜索的穷人或不可靠的指标，鼓励有趣和有趣的方向：我们可以衡量不可分辨率范式下的任何培训的运作重要性吗？我们通过在初始化问题的网络修剪中定制NAS提供肯定的答案。随着最近建议的突触突触效力标准在初始化的网络修剪中，我们寻求在没有任何培训的情况下将候选人行动中的候选人行动的重要性进行评分，并提出了一种名为“免费可分辨的架构搜索}（Freedarts）的小说框架” 。我们表明，没有任何培训，具有不同代理度量的自由路由器可以在不同的搜索空间中优于大多数NAS基线。更重要的是，Freedarts是非常内存的高效和计算效率，因为它放弃了架构搜索阶段的培训，使得能够在更灵活的空间上执行架构搜索并消除架构搜索和评估之间的深度间隙。我们希望我们的工作激励从初始化修剪的角度来激发解决NAS的尝试。

在本文中，我们提出了用于卷积神经网络的可分散的信道稀疏性搜索（DCS）。与需要用户手动设置每个卷积层的紫星比的传统信道修剪算法不同，DCSS自动搜索稀疏的最佳组合。灵感来自可怜的架构搜索（飞镖），我们从连续放松中汲取课程，并利用梯度信息来平衡计算成本和指标。由于直接应用飞镖方案引起形状不匹配和过度的记忆消耗，因此在过滤器内引入一种名为重量共享的新技术。这种技术优雅地消除了具有可忽略额外资源的形状不匹配的问题。我们不仅开展全面的实验，不仅是图像分类，还可以找到包括语义分割和图像超分辨率的粒度任务，以验证DCSS的有效性。与以前的网络修剪方法相比，DCSS实现了图像分类的最先进结果。语义分割和图像超分辨率的实验结果表明，特定于任务特定搜索的性能比转移超薄模型实现了更好的性能，展示了广泛的适用性和高效率的DCSS。

混合精确的深神经网络达到了硬件部署所需的能源效率和吞吐量，尤其是在资源有限的情况下，而无需牺牲准确性。但是，不容易找到保留精度的最佳每层钻头精度，尤其是在创建巨大搜索空间的大量模型，数据集和量化技术中。为了解决这一困难，最近出现了一系列文献，并且已经提出了一些实现有希望的准确性结果的框架。在本文中，我们首先总结了文献中通常使用的量化技术。然后，我们对混合精液框架进行了彻底的调查，该调查是根据其优化技术进行分类的，例如增强学习和量化技术，例如确定性舍入。此外，讨论了每个框架的优势和缺点，我们在其中呈现并列。我们最终为未来的混合精液框架提供了指南。

Neural network pruning offers a promising prospect to facilitate deploying deep neural networks on resourcelimited devices. However, existing methods are still challenged by the training inefficiency and labor cost in pruning designs, due to missing theoretical guidance of non-salient network components. In this paper, we propose a novel filter pruning method by exploring the High Rank of feature maps (HRank). Our HRank is inspired by the discovery that the average rank of multiple feature maps generated by a single filter is always the same, regardless of the number of image batches CNNs receive. Based on HRank, we develop a method that is mathematically formulated to prune filters with low-rank feature maps. The principle behind our pruning is that low-rank feature maps contain less information, and thus pruned results can be easily reproduced. Besides, we experimentally show that weights with high-rank feature maps contain more important information, such that even when a portion is not updated, very little damage would be done to the model performance. Without introducing any additional constraints, HRank leads to significant improvements over the state-of-the-arts in terms of FLOPs and parameters reduction, with similar accuracies. For example, with ResNet-110, we achieve a 58.2%-FLOPs reduction by removing 59.2% of the parameters, with only a small loss of 0.14% in top-1 accuracy on CIFAR-10. With Res-50, we achieve a 43.8%-FLOPs reduction by removing 36.7% of the parameters, with only a loss of 1.17% in the top-1 accuracy on ImageNet. The codes can be available at https://github.com/lmbxmu/HRank.

嵌入式和个人物联网设备由微控制器单元（MCU）供电，其极端资源稀缺是依赖于设备的深度学习推断的应用的主要障碍。与通常需要执行神经网络的内容相比，存储器，内存和计算能力较少的秩序，对网络架构上的严格结构约束并呼叫专业模型压缩方法。在这项工作中，我们为卷积神经网络提出了可分散的结构化网络修剪方法，它集成了模型的MCU特定的资源使用和参数重要性反馈，以获得高度压缩但准确的分类模型。我们的方法（a）提高了高达80倍的模型的关键资源使用; （b）在培训型号的同时迭代地修剪，导致没有开销甚至改善培训时间; （c）与现有MCU的特定方法相比，在比较多的时间内生产具有匹配或改进的资源使用的压缩模型。压缩模型可供下载。

神经结构搜索（NAS）引起了日益增长的兴趣。为了降低搜索成本，最近的工作已经探讨了模型的重量分享，并在单枪NAS进行了重大进展。然而，已经观察到，单次模型精度较高的模型并不一定在独立培训时更好地执行更好。为了解决这个问题，本文提出了搜索空间的逐步自动设计，名为Pad-NAS。与超字幕中的所有层共享相同操作搜索空间的先前方法不同，我们根据操作修剪制定逐行搜索策略，并构建层面操作搜索空间。通过这种方式，Pad-NAS可以自动设计每层的操作，并在搜索空间质量和模型分集之间实现权衡。在搜索过程中，我们还考虑了高效神经网络模型部署的硬件平台约束。关于Imagenet的广泛实验表明我们的方法可以实现最先进的性能。

可区分架构搜索（飞镖）是基于解决双重优化问题的数据驱动神经网络设计的有效方法。尽管在许多体系结构搜索任务中取得了成功，但仍然担心一阶飞镖的准确性和二阶飞镖的效率。在本文中，我们制定了单个级别的替代方案和放松的体系结构搜索（RARTS）方法，该方法通过数据和网络拆分利用整个数据集在体系结构学习中，而无需涉及相应损失功能（如飞镖）的混合第二个衍生物。在我们制定网络拆分的过程中，两个具有不同但相关权重的网络在寻找共享体系结构时进行了合作。 RART比飞镖的优势通过收敛定理和可解析的模型证明是合理的。此外，RART在准确性和搜索效率方面优于飞镖及其变体，如足够的实验结果所示。对于搜索拓扑结构（即边缘和操作）的任务，RART获得了比CIFAR-10上的二阶Darts更高的精度和60 \％的计算成本降低。转移到Imagenet时，RART继续超越表演飞镖，并且与最近的飞镖变体相提并论，尽管我们的创新纯粹是在训练算法上，而无需修改搜索空间。对于搜索宽度的任务，即卷积层中的频道数量，RARTS还优于传统的网络修剪基准。关于公共体系结构搜索基准等NATS BENCH的进一步实验也支持RARTS的优势。

While machine learning is traditionally a resource intensive task, embedded systems, autonomous navigation, and the vision of the Internet of Things fuel the interest in resource-efficient approaches. These approaches aim for a carefully chosen trade-off between performance and resource consumption in terms of computation and energy. The development of such approaches is among the major challenges in current machine learning research and key to ensure a smooth transition of machine learning technology from a scientific environment with virtually unlimited computing resources into everyday's applications. In this article, we provide an overview of the current state of the art of machine learning techniques facilitating these real-world requirements. In particular, we focus on deep neural networks (DNNs), the predominant machine learning models of the past decade. We give a comprehensive overview of the vast literature that can be mainly split into three non-mutually exclusive categories: (i) quantized neural networks, (ii) network pruning, and (iii) structural efficiency. These techniques can be applied during training or as post-processing, and they are widely used to reduce the computational demands in terms of memory footprint, inference speed, and energy efficiency. We also briefly discuss different concepts of embedded hardware for DNNs and their compatibility with machine learning techniques as well as potential for energy and latency reduction. We substantiate our discussion with experiments on well-known benchmark datasets using compression techniques (quantization, pruning) for a set of resource-constrained embedded systems, such as CPUs, GPUs and FPGAs. The obtained results highlight the difficulty of finding good trade-offs between resource efficiency and predictive performance.

The deployment of deep convolutional neural networks (CNNs) in many real world applications is largely hindered by their high computational cost. In this paper, we propose a novel learning scheme for CNNs to simultaneously 1) reduce the model size; 2) decrease the run-time memory footprint; and 3) lower the number of computing operations, without compromising accuracy. This is achieved by enforcing channel-level sparsity in the network in a simple but effective way. Different from many existing approaches, the proposed method directly applies to modern CNN architectures, introduces minimum overhead to the training process, and requires no special software/hardware accelerators for the resulting models. We call our approach network slimming, which takes wide and large networks as input models, but during training insignificant channels are automatically identified and pruned afterwards, yielding thin and compact models with comparable accuracy. We empirically demonstrate the effectiveness of our approach with several state-of-the-art CNN models, including VGGNet, ResNet and DenseNet, on various image classification datasets. For VGGNet, a multi-pass version of network slimming gives a 20× reduction in model size and a 5× reduction in computing operations.

Network pruning is widely used for reducing the heavy inference cost of deep models in low-resource settings. A typical pruning algorithm is a three-stage pipeline, i.e., training (a large model), pruning and fine-tuning. During pruning, according to a certain criterion, redundant weights are pruned and important weights are kept to best preserve the accuracy. In this work, we make several surprising observations which contradict common beliefs. For all state-of-the-art structured pruning algorithms we examined, fine-tuning a pruned model only gives comparable or worse performance than training that model with randomly initialized weights. For pruning algorithms which assume a predefined target network architecture, one can get rid of the full pipeline and directly train the target network from scratch. Our observations are consistent for multiple network architectures, datasets, and tasks, which imply that: 1) training a large, over-parameterized model is often not necessary to obtain an efficient final model, 2) learned "important" weights of the large model are typically not useful for the small pruned model, 3) the pruned architecture itself, rather than a set of inherited "important" weights, is more crucial to the efficiency in the final model, which suggests that in some cases pruning can be useful as an architecture search paradigm. Our results suggest the need for more careful baseline evaluations in future research on structured pruning methods. We also compare with the "Lottery Ticket Hypothesis" (Frankle & Carbin, 2019), and find that with optimal learning rate, the "winning ticket" initialization as used in Frankle & Carbin (2019) does not bring improvement over random initialization. * Equal contribution. † Work done while visiting UC Berkeley.

重量修剪是一种有效的模型压缩技术，可以解决在移动设备上实现实时深神经网络（DNN）推断的挑战。然而，由于精度劣化，难以利用硬件加速度，以及某些类型的DNN层的限制，难以降低的应用方案具有有限的应用方案。在本文中，我们提出了一般的细粒度的结构化修剪方案和相应的编译器优化，适用于任何类型的DNN层，同时实现高精度和硬件推理性能。随着使用我们的编译器优化所支持的不同层的灵活性，我们进一步探讨了确定最佳修剪方案的新问题，了解各种修剪方案的不同加速度和精度性能。两个修剪方案映射方法，一个是基于搜索，另一个是基于规则的，建议自动推导出任何给定DNN的每层的最佳修剪规则和块大小。实验结果表明，我们的修剪方案映射方法，以及一般细粒化结构修剪方案，优于最先进的DNN优化框架，最高可达2.48 $ \ times $和1.73 $ \ times $ DNN推理加速在CiFar-10和Imagenet DataSet上没有准确性损失。

Model quantization enables the deployment of deep neural networks under resource-constrained devices. Vector quantization aims at reducing the model size by indexing model weights with full-precision embeddings, i.e., codewords, while the index needs to be restored to 32-bit during computation. Binary and other low-precision quantization methods can reduce the model size up to 32$\times$, however, at the cost of a considerable accuracy drop. In this paper, we propose an efficient framework for ternary quantization to produce smaller and more accurate compressed models. By integrating hyperspherical learning, pruning and reinitialization, our proposed Hyperspherical Quantization (HQ) method reduces the cosine distance between the full-precision and ternary weights, thus reducing the bias of the straight-through gradient estimator during ternary quantization. Compared with existing work at similar compression levels ($\sim$30$\times$, $\sim$40$\times$), our method significantly improves the test accuracy and reduces the model size.

Slimmable Neural Networks (S-Net) is a novel network which enabled to select one of the predefined proportions of channels (sub-network) dynamically depending on the current computational resource availability. The accuracy of each sub-network on S-Net, however, is inferior to that of individually trained networks of the same size due to its difficulty of simultaneous optimization on different sub-networks. In this paper, we propose Slimmable Pruned Neural Networks (SP-Net), which has sub-network structures learned by pruning instead of adopting structures with the same proportion of channels in each layer (width multiplier) like S-Net, and we also propose new pruning procedures: multi-base pruning instead of one-shot or iterative pruning to realize high accuracy and huge training time saving. We also introduced slimmable channel sorting (scs) to achieve calculation as fast as S-Net and zero padding match (zpm) pruning to prune residual structure in more efficient way. SP-Net can be combined with any kind of channel pruning methods and does not require any complicated processing or time-consuming architecture search like NAS models. Compared with each sub-network of the same FLOPs on S-Net, SP-Net improves accuracy by 1.2-1.5% for ResNet-50, 0.9-4.4% for VGGNet, 1.3-2.7% for MobileNetV1, 1.4-3.1% for MobileNetV2 on ImageNet. Furthermore, our methods outperform other SOTA pruning methods and are on par with various NAS models according to our experimental results on ImageNet. The code is available at https://github.com/hideakikuratsu/SP-Net.

神经网络修剪具有显着性能，可以降低深网络模型的复杂性。最近的网络修剪方法通常集中在网络中删除不重要或冗余过滤器。在本文中，通过探索特征图之间的相似性，我们提出了一种新颖的滤波器修剪方法，中央滤波器（CF），这表明在适当的调整之后滤波器大致等于一组其他滤波器。我们的方法基于发现特征贴图之间的平均相似性的发现，而不管输入图像的数量如何，都会很少变化。基于此发现，我们在特征映射上建立相似性图，并计算每个节点的近密中心以选择中央滤波器。此外，我们设计一种方法，可以在与中央滤波器对应的下一层中直接调整权重，有效地最小化由修剪引起的误差。通过对各种基准网络和数据集的实验，CF产生最先进的性能。例如，对于Reset-56，CF通过去除47.1％的参数来减少约39.7％的絮凝物，甚至在CiFar-10上的精度改善0.33％。通过Googlenet，CF通过去除55.6％的参数来减少大约63.2％的拖鞋，仅在CIFAR-10上的前1个精度下降0.35％的损失。通过resnet-50，CF通过去除36.9％的参数减少约47.9％的拖鞋，仅在Imagenet上的前1个精度下降1.07％。该代码可以在https://github.com/8ubpshlr23/centrter上获得。

在设备训练中，该模型可以通过微调预训练的模型来适应从传感器中收集的新数据。但是，对于具有少量内存资源的物联网设备，训练记忆消耗是过敏的。我们提出了一个算法 - 系统共同设计框架，以便仅使用256KB的内存使设备训练成为可能。在设备训练面临两个独特的挑战：（1）由于比特精确的混合和缺乏归一化而难以优化神经网络的量化图； （2）有限的硬件资源（内存和计算）不允许完整的向后计算。为了应对优化难度，我们提出了量化缩放量表来校准梯度尺度并稳定量化训练。为了减少内存足迹，我们提出稀疏更新，以跳过不太重要的层和子张量的梯度计算。该算法创新是由轻量级训练系统（小型训练引擎）实现的，该系统可修剪向后的计算图，以支持稀疏更新并卸载运行时自动分化以编译时间。我们的框架是第一个实用解决方案，用于在微型IoT设备上进行视觉识别的设备转移学习（例如，仅具有256KB SRAM的微控制器），使用少于1/100的现有框架的存储器，同时匹配云训练的准确性+Tinyml应用程序VWW的边缘部署。我们的研究使IoT设备不仅可以执行推理，还可以不断适应新的数据，以实现终身学习。

Neural architecture search (NAS) has a great impact by automatically designing effective neural network architectures. However, the prohibitive computational demand of conventional NAS algorithms (e.g. 10 4 GPU hours) makes it difficult to directly search the architectures on large-scale tasks (e.g. ImageNet). Differentiable NAS can reduce the cost of GPU hours via a continuous representation of network architecture but suffers from the high GPU memory consumption issue (grow linearly w.r.t. candidate set size). As a result, they need to utilize proxy tasks, such as training on a smaller dataset, or learning with only a few blocks, or training just for a few epochs. These architectures optimized on proxy tasks are not guaranteed to be optimal on the target task. In this paper, we present ProxylessNAS that can directly learn the architectures for large-scale target tasks and target hardware platforms. We address the high memory consumption issue of differentiable NAS and reduce the computational cost (GPU hours and GPU memory) to the same level of regular training while still allowing a large candidate set. Experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of directness and specialization. On CIFAR-10, our model achieves 2.08% test error with only 5.7M parameters, better than the previous state-of-the-art architecture AmoebaNet-B, while using 6× fewer parameters. On ImageNet, our model achieves 3.1% better top-1 accuracy than MobileNetV2, while being 1.2× faster with measured GPU latency. We also apply ProxylessNAS to specialize neural architectures for hardware with direct hardware metrics (e.g. latency) and provide insights for efficient CNN architecture design. 1

现有的可区分通道修剪方法通常将缩放因子或掩模在通道后面的掩盖范围内，以减少重要性的修剪过滤器，并假设输入样品统一贡献以过滤重要性。具体而言，实例复杂性对修剪性能的影响尚未得到充分研究。在本文中，我们提出了一个基于实例复杂性滤波器的重要性得分的简单而有效的可区分网络修剪方法上限。我们通过给硬样品给出更高的权重来定义每个样品的实例复杂性与重量相关的重量，并测量样品特异性软膜的加权总和，以模拟不同输入的非均匀贡献，这鼓励硬样品主导修剪过程和模型性能保存完好。此外，我们还引入了一个新的正规器，以鼓励面具两极分化，以便很容易找到甜蜜的位置以识别要修剪的过滤器。各种网络体系结构和数据集的性能评估表明，CAP在修剪大型网络方面具有优势。例如，CAP在删除65.64％的拖鞋后，CAP在CIFAR-10数据集上的RESNET56的准确性提高了0.33％，而Prunes在ImagEnet数据集上的RESNET50的PRUNES 87.75％，只有0.89％的TOP-1精度损失。

Low-rankness plays an important role in traditional machine learning, but is not so popular in deep learning. Most previous low-rank network compression methods compress the networks by approximating pre-trained models and re-training. However, the optimal solution in the Euclidean space may be quite different from the one in the low-rank manifold. A well-pre-trained model is not a good initialization for the model with low-rank constraints. Thus, the performance of a low-rank compressed network degrades significantly. Compared to other network compression methods such as pruning, low-rank methods attracts less attention in recent years. In this paper, we devise a new training method, low-rank projection with energy transfer (LRPET), that trains low-rank compressed networks from scratch and achieves competitive performance. First, we propose to alternately perform stochastic gradient descent training and projection onto the low-rank manifold. Compared to re-training on the compact model, this enables full utilization of model capacity since solution space is relaxed back to Euclidean space after projection. Second, the matrix energy (the sum of squares of singular values) reduction caused by projection is compensated by energy transfer. We uniformly transfer the energy of the pruned singular values to the remaining ones. We theoretically show that energy transfer eases the trend of gradient vanishing caused by projection. Third, we propose batch normalization (BN) rectification to cut off its effect on the optimal low-rank approximation of the weight matrix, which further improves the performance. Comprehensive experiments on CIFAR-10 and ImageNet have justified that our method is superior to other low-rank compression methods and also outperforms recent state-of-the-art pruning methods. Our code is available at https://github.com/BZQLin/LRPET.