Due to the high activation sparsity and use of accumulates (AC) instead of expensive multiply-and-accumulates (MAC), neuromorphic spiking neural networks (SNNs) have emerged as a promising low-power alternative to traditional DNNs for several computer vision (CV) applications. However, most existing SNNs require multiple time steps for acceptable inference accuracy, hindering real-time deployment and increasing spiking activity and, consequently, energy consumption. Recent works proposed direct encoding that directly feeds the analog pixel values in the first layer of the SNN in order to significantly reduce the number of time steps. Although the overhead for the first layer MACs with direct encoding is negligible for deep SNNs and the CV processing is efficient using SNNs, the data transfer between the image sensors and the downstream processing costs significant bandwidth and may dominate the total energy. To mitigate this concern, we propose an in-sensor computing hardware-software co-design framework for SNNs targeting image recognition tasks. Our approach reduces the bandwidth between sensing and processing by 12-96x and the resulting total energy by 2.32x compared to traditional CV processing, with a 3.8% reduction in accuracy on ImageNet.

Spiking Neural networks (SNN) have emerged as an attractive spatio-temporal computing paradigm for a wide range of low-power vision tasks. However, state-of-the-art (SOTA) SNN models either incur multiple time steps which hinder their deployment in real-time use cases or increase the training complexity significantly. To mitigate this concern, we present a training framework (from scratch) for one-time-step SNNs that uses a novel variant of the recently proposed Hoyer regularizer. We estimate the threshold of each SNN layer as the Hoyer extremum of a clipped version of its activation map, where the clipping threshold is trained using gradient descent with our Hoyer regularizer. This approach not only downscales the value of the trainable threshold, thereby emitting a large number of spikes for weight update with a limited number of iterations (due to only one time step) but also shifts the membrane potential values away from the threshold, thereby mitigating the effect of noise that can degrade the SNN accuracy. Our approach outperforms existing spiking, binary, and adder neural networks in terms of the accuracy-FLOPs trade-off for complex image recognition tasks. Downstream experiments on object detection also demonstrate the efficacy of our approach.

The evaluation of abstractive summarization models typically uses test data that is identically distributed as training data. In real-world practice, documents to be summarized may contain input noise caused by text extraction artifacts or data pipeline bugs. The robustness of model performance under distribution shift caused by such noise is relatively under-studied. We present a large empirical study quantifying the sometimes severe loss in performance (up to 12 ROUGE-1 points) from different types of input noise for a range of datasets and model sizes. We then propose a light-weight method for detecting and removing such noise in the input during model inference without requiring any extra training, auxiliary models, or even prior knowledge of the type of noise. Our proposed approach effectively mitigates the loss in performance, recovering a large fraction of the performance drop, sometimes as large as 11 ROUGE-1 points.

The lack of standardization is a prominent issue in magnetic resonance (MR) imaging. This often causes undesired contrast variations due to differences in hardware and acquisition parameters. In recent years, MR harmonization using image synthesis with disentanglement has been proposed to compensate for the undesired contrast variations. Despite the success of existing methods, we argue that three major improvements can be made. First, most existing methods are built upon the assumption that multi-contrast MR images of the same subject share the same anatomy. This assumption is questionable since different MR contrasts are specialized to highlight different anatomical features. Second, these methods often require a fixed set of MR contrasts for training (e.g., both Tw-weighted and T2-weighted images must be available), which limits their applicability. Third, existing methods generally are sensitive to imaging artifacts. In this paper, we present a novel approach, Harmonization with Attention-based Contrast, Anatomy, and Artifact Awareness (HACA3), to address these three issues. We first propose an anatomy fusion module that enables HACA3 to respect the anatomical differences between MR contrasts. HACA3 is also robust to imaging artifacts and can be trained and applied to any set of MR contrasts. Experiments show that HACA3 achieves state-of-the-art performance under multiple image quality metrics. We also demonstrate the applicability of HACA3 on downstream tasks with diverse MR datasets acquired from 21 sites with different field strengths, scanner platforms, and acquisition protocols.

This paper is a technical overview of DeepMind and Google's recent work on reinforcement learning for controlling commercial cooling systems. Building on expertise that began with cooling Google's data centers more efficiently, we recently conducted live experiments on two real-world facilities in partnership with Trane Technologies, a building management system provider. These live experiments had a variety of challenges in areas such as evaluation, learning from offline data, and constraint satisfaction. Our paper describes these challenges in the hope that awareness of them will benefit future applied RL work. We also describe the way we adapted our RL system to deal with these challenges, resulting in energy savings of approximately 9% and 13% respectively at the two live experiment sites.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

二重优化（BO）可用于解决各种重要的机器学习问题，包括但不限于超参数优化，元学习，持续学习和增强学习。常规的BO方法需要通过与隐式分化的低级优化过程进行区分，这需要与Hessian矩阵相关的昂贵计算。最近，人们一直在寻求BO的一阶方法，但是迄今为止提出的方法对于大规模的深度学习应用程序往往是复杂且不切实际的。在这项工作中，我们提出了一种简单的一阶BO算法，仅取决于一阶梯度信息，不需要隐含的区别，并且对于大规模的非凸函数而言是实用和有效的。我们为提出的方法提供了非注重方法分析非凸目标的固定点，并提出了表明其出色实践绩效的经验结果。

数据驱动的设计和创新是重复使用和提供宝贵和有用信息的过程。但是，现有的设计创新语义网络基于仅限于技术和科学信息的数据源。此外，现有研究仅在统计或语义关系上建立语义网络的边缘，这不太可能充分利用两种类型的关系中的好处，并发现设计创新的隐性知识。因此，我们构建了基于Wikipedia的语义网络Wikilink。 Wikilink引入了概念之间的统计重量和语义权重的合并重量，并开发了四种算法来启发新想法。进行评估实验，结果表明，该网络的特征是术语，关系和学科的高度覆盖范围，这证明了网络的有效性和实用性。然后，演示和案例研究结果表明，Wikilink可以作为概念设计创新的思想生成工具。 Wikilink的源代码和后端数据提供开源，供更多用户探索和构建。

半监督对象检测（SSOD）的最新发展显示了利用未标记数据改善对象检测器的希望。但是，到目前为止，这些方法已经假设未标记的数据不包含分布（OOD）类，这对于较大规模的未标记数据集是不现实的。在本文中，我们考虑了一个更实用但具有挑战性的问题，开放式半监督对象检测（OSSOD）。我们首先发现现有的SSOD方法在开放式条件下获得了较低的性能增长，这是由语义扩展引起的，在该语义扩展中，分散注意力的OOD对象​​被错误预测为半监督训练的分布伪标签。为了解决此问题，我们考虑与SSOD方法集成的在线和离线OOD检测模块。通过广泛的研究，我们发现，基于自我监视的视觉变压器的脱机OOD检测器对在线OOD探测器的表现良好，因为它稳健地对伪标记的干扰。在实验中，我们提出的框架有效地解决了语义扩展问题，并在许多OSSOD基准（包括大规模的可可开放图）上显示出一致的改进。我们还在不同的OSSOD条件下验证框架的有效性，包括不同数量的分布类别，不同程度的监督和不同标记集的组合。

目标条件加固学习（GCRL）具有广泛的潜在现实应用程序，包括机器人技术中的操纵和导航问题。尤其是在这样的机器人技术任务中，对GCRL的样本效率至关重要，因为默认情况下，只有在实现其目标时才会获得奖励。尽管已经提出了几种方法来提高GCRL的样品效率，但一种相对研究的方法是设计神经体系结构以支持样品效率。在这项工作中，我们引入了一种新型的GCRL神经结构，该神经结构比常用的单片网络体系结构可实现的样品效率明显更好。他们的关键见解是，最佳动作值函数q^*（s，a，g）必须在特定意义上满足三角形不平等。此外，我们引入了度量残留网络（MRN），该度量剩余网络（MRN）故意将动作值函数q（s，a，g）分解到否定的度量总和和残留不对称组件中。 MRN可证明近似于任何最佳动作值函数q^*（S，A，G），从而使其成为GCRL的拟合神经结构。我们在GCRL中的12个标准基准环境中进行了全面的实验。经验结果表明，就样本效率而言，MRN均匀地优于其他最先进的GCRL神经体系结构。