Control of light through a microscope objective with a high numerical aperture is a common requirement in applications such as optogenetics, adaptive optics, or laser processing. Light propagation, including polarization effects, can be described under these conditions using the Debye-Wolf diffraction integral. Here, we take advantage of differentiable optimization and machine learning for efficiently optimizing the Debye-Wolf integral for such applications. For light shaping we show that this optimization approach is suitable for engineering arbitrary three-dimensional point spread functions in a two-photon microscope. For differentiable model-based adaptive optics (DAO), the developed method can find aberration corrections with intrinsic image features, for example neurons labeled with genetically encoded calcium indicators, without requiring guide stars. Using computational modeling we further discuss the range of spatial frequencies and magnitudes of aberrations which can be corrected with this approach.

Lensless cameras are a class of imaging devices that shrink the physical dimensions to the very close vicinity of the image sensor by replacing conventional compound lenses with integrated flat optics and computational algorithms. Here we report a diffractive lensless camera with spatially-coded Voronoi-Fresnel phase to achieve superior image quality. We propose a design principle of maximizing the acquired information in optics to facilitate the computational reconstruction. By introducing an easy-to-optimize Fourier domain metric, Modulation Transfer Function volume (MTFv), which is related to the Strehl ratio, we devise an optimization framework to guide the optimization of the diffractive optical element. The resulting Voronoi-Fresnel phase features an irregular array of quasi-Centroidal Voronoi cells containing a base first-order Fresnel phase function. We demonstrate and verify the imaging performance for photography applications with a prototype Voronoi-Fresnel lensless camera on a 1.6-megapixel image sensor in various illumination conditions. Results show that the proposed design outperforms existing lensless cameras, and could benefit the development of compact imaging systems that work in extreme physical conditions.

光学系统的可区分模拟可以与基于深度学习的重建网络结合使用，以通过端到端（E2E）优化光学编码器和深度解码器来实现高性能计算成像。这使成像应用程序（例如3D定位显微镜，深度估计和无透镜摄影）通过优化局部光学编码器。更具挑战性的计算成像应用，例如将3D卷压入单个2D图像的3D快照显微镜，需要高度非本地光学编码器。我们表明，现有的深网解码器具有局部性偏差，可防止这种高度非本地光学编码器的优化。我们使用全球内核傅里叶卷积神经网络（Fouriernets）基于浅神经网络体系结构的解码器来解决此问题。我们表明，在高度非本地分散镜头光学编码器捕获的照片中，傅立叶网络超过了现有的基于网络的解码器。此外，我们表明傅里叶可以对3D快照显微镜的高度非本地光学编码器进行E2E优化。通过将傅立叶网和大规模多GPU可区分的光学模拟相结合，我们能够优化非本地光学编码器170 $ \ times $ \ times $ tos 7372 $ \ times $ \ times $ \ times $比以前的最新状态，并证明了ROI的潜力-type特定的光学编码使用可编程显微镜。

计算光学成像（COI）系统利用其设置中的光学编码元素（CE）在单个或多个快照中编码高维场景，并使用计算算法对其进行解码。 COI系统的性能很大程度上取决于其主要组件的设计：CE模式和用于执行给定任务的计算方法。常规方法依赖于随机模式或分析设计来设置CE的分布。但是，深神经网络（DNNS）的可用数据和算法功能已在CE数据驱动的设计中开辟了新的地平线，该设计共同考虑了光学编码器和计算解码器。具体而言，通过通过完全可区分的图像形成模型对COI测量进行建模，该模型考虑了基于物理的光及其与CES的相互作用，可以在端到端优化定义CE和计算解码器的参数和计算解码器（e2e）方式。此外，通过在同一框架中仅优化CE，可以从纯光学器件中执行推理任务。这项工作调查了CE数据驱动设计的最新进展，并提供了有关如何参数化不同光学元素以将其包括在E2E框架中的指南。由于E2E框架可以通过更改损耗功能和DNN来处理不同的推理应用程序，因此我们提出低级任务，例如光谱成像重建或高级任务，例如使用基于任务的光学光学体系结构来增强隐私的姿势估计，以维护姿势估算。最后，我们说明了使用全镜DNN以光速执行的分类和3D对象识别应用程序。

我们提出了一种依赖工程点扩散功能（PSF）的紧凑型快照单眼估计技术。微观超分辨率成像中使用的传统方法，例如双螺旋PSF（DHPSF），不适合比稀疏的一组点光源更复杂的场景。我们使用cram \'er-rao下限（CRLB）显示，将DHPSF的两个叶分开，从而捕获两个单独的图像导致深度精度的急剧增加。用于生成DHPSF的相掩码的独特属性是，将相掩码分为两个半部分，导致两个裂片的空间分离。我们利用该属性建立一个基于紧凑的极化光学设置，在该设置中，我们将两个正交线性极化器放在DHPSF相位掩码的每一半上，然后使用极化敏感的摄像机捕获所得图像。模拟和实验室原型的结果表明，与包括DHPSF和Tetrapod PSF在内的最新设计相比，我们的技术达到了高达50美元的深度误差，而空间分辨率几乎没有损失。

Ever since the first microscope by Zacharias Janssen in the late 16th century, scientists have been inventing new types of microscopes for various tasks. Inventing a novel architecture demands years, if not decades, worth of scientific experience and creativity. In this work, we introduce Differentiable Microscopy ($\partial\mu$), a deep learning-based design paradigm, to aid scientists design new interpretable microscope architectures. Differentiable microscopy first models a common physics-based optical system however with trainable optical elements at key locations on the optical path. Using pre-acquired data, we then train the model end-to-end for a task of interest. The learnt design proposal can then be simplified by interpreting the learnt optical elements. As a first demonstration, based on the optical 4-$f$ system, we present an all-optical quantitative phase microscope (QPM) design that requires no computational post-reconstruction. A follow-up literature survey suggested that the learnt architecture is similar to the generalized phase contrast method developed two decades ago. Our extensive experiments on multiple datasets that include biological samples show that our learnt all-optical QPM designs consistently outperform existing methods. We experimentally verify the functionality of the optical 4-$f$ system based QPM design using a spatial light modulator. Furthermore, we also demonstrate that similar results can be achieved by an uninterpretable learning based method, namely diffractive deep neural networks (D2NN). The proposed differentiable microscopy framework supplements the creative process of designing new optical systems and would perhaps lead to unconventional but better optical designs.

Multispectral imaging has been used for numerous applications in e.g., environmental monitoring, aerospace, defense, and biomedicine. Here, we present a diffractive optical network-based multispectral imaging system trained using deep learning to create a virtual spectral filter array at the output image field-of-view. This diffractive multispectral imager performs spatially-coherent imaging over a large spectrum, and at the same time, routes a pre-determined set of spectral channels onto an array of pixels at the output plane, converting a monochrome focal plane array or image sensor into a multispectral imaging device without any spectral filters or image recovery algorithms. Furthermore, the spectral responsivity of this diffractive multispectral imager is not sensitive to input polarization states. Through numerical simulations, we present different diffractive network designs that achieve snapshot multispectral imaging with 4, 9 and 16 unique spectral bands within the visible spectrum, based on passive spatially-structured diffractive surfaces, with a compact design that axially spans ~72 times the mean wavelength of the spectral band of interest. Moreover, we experimentally demonstrate a diffractive multispectral imager based on a 3D-printed diffractive network that creates at its output image plane a spatially-repeating virtual spectral filter array with 2x2=4 unique bands at terahertz spectrum. Due to their compact form factor and computation-free, power-efficient and polarization-insensitive forward operation, diffractive multispectral imagers can be transformative for various imaging and sensing applications and be used at different parts of the electromagnetic spectrum where high-density and wide-area multispectral pixel arrays are not widely available.

本书章节介绍了如何利用散射光场中的光谱相关性来进行高精度的飞行时间感测。本章应作为温和的介绍，旨在用于计算成像科学家和新手合成波长成像主题的学生。技术细节（例如检测器或光源规格）将在很大程度上省略。取而代之的是，不同方法之间的相似性将被强调“绘制更大的图景”。

Spatially varying spectral modulation can be implemented using a liquid crystal spatial light modulator (SLM) since it provides an array of liquid crystal cells, each of which can be purposed to act as a programmable spectral filter array. However, such an optical setup suffers from strong optical aberrations due to the unintended phase modulation, precluding spectral modulation at high spatial resolutions. In this work, we propose a novel computational approach for the practical implementation of phase SLMs for implementing spatially varying spectral filters. We provide a careful and systematic analysis of the aberrations arising out of phase SLMs for the purposes of spatially varying spectral modulation. The analysis naturally leads us to a set of "good patterns" that minimize the optical aberrations. We then train a deep network that overcomes any residual aberrations, thereby achieving ideal spectral modulation at high spatial resolution. We show a number of unique operating points with our prototype including dynamic spectral filtering, material classification, and single- and multi-image hyperspectral imaging.

信号处理是几乎任何传感器系统的基本组件，具有不同科学学科的广泛应用。时间序列数据，图像和视频序列包括可以增强和分析信息提取和量化的代表性形式的信号。人工智能和机器学习的最近进步正在转向智能，数据驱动，信号处理的研究。该路线图呈现了最先进的方法和应用程序的关键概述，旨在突出未来的挑战和对下一代测量系统的研究机会。它涵盖了广泛的主题，从基础到工业研究，以简明的主题部分组织，反映了每个研究领域的当前和未来发展的趋势和影响。此外，它为研究人员和资助机构提供了识别新前景的指导。

我们报告了张力层造影差异相位对比度显微镜（T2DPC），这是一种用于同时测量相和各向异性的无定量标签层析成像方法。T2DPC扩展了差异相位对比显微镜（一种定量相成像技术），以突出光的矢量性质。该方法求解了从配备有LED矩阵，圆极偏振器和偏振敏感摄像机的标准显微镜获得的强度测量的各向异性样品的介电常数张量。我们证明了各种验证样品的折射率，双折射和方向的准确体积重建，并证明生物标本的重建极化结构是病理学的预测。

机器学习方法的最新进展以及扫描探针显微镜（SPMS）的可编程接口的新兴可用性使自动化和自动显微镜在科学界的关注方面推向了最前沿。但是，启用自动显微镜需要开发特定于任务的机器学习方法，了解物理发现与机器学习之间的相互作用以及完全定义的发现工作流程。反过来，这需要平衡领域科学家的身体直觉和先验知识与定义实验目标和机器学习算法的奖励，这些算法可以将它们转化为特定的实验协议。在这里，我们讨论了贝叶斯活跃学习的基本原理，并说明了其对SPM的应用。我们从高斯过程作为一种简单的数据驱动方法和对物理模型的贝叶斯推断作为基于物理功能的扩展的贝叶斯推断，再到更复杂的深内核学习方法，结构化的高斯过程和假设学习。这些框架允许使用先验数据，在光谱数据中编码的特定功能以及在实验过程中表现出的物理定律的探索。讨论的框架可以普遍应用于结合成像和光谱，SPM方法，纳米识别，电子显微镜和光谱法以及化学成像方法的所有技术，并且对破坏性或不可逆测量的影响特别影响。

置换矩阵构成了一个重要的计算构建块，这些构建块在各个领域中经常使用，例如通信，信息安全和数据处理。具有相对较大数量的基于功率，快速和紧凑型平台的输入输出互连的置换运算符的光学实现是非常可取的。在这里，我们提出了通过深度学习设计的衍射光学网络，以全面执行置换操作，可以使用被动的传播层在输入和视场之间扩展到数十万个互连，这些互连是在波长规模上单独构造的。 。我们的发现表明，衍射光网络在近似给定置换操作中的容量与系统中衍射层和可训练的传输元件的数量成正比。这种更深的衍射网络设计可以在系统的物理对齐和输出衍射效率方面构成实际挑战。我们通过设计不对对准的衍射设计来解决这些挑战，这些设计可以全面执行任意选择的置换操作，并首次在实验中证明了在频谱的THZ部分运行的衍射排列网络。衍射排列网络可能会在例如安全性，图像加密和数据处理以及电信中找到各种应用程序；尤其是在无线通信中的载波频率接近THZ波段的情况下，提出的衍射置换网络可以潜在地充当无线网络中的通道路由和互连面板。

间接飞行时间（ITOF）相机是一个有希望的深度传感技术。然而，它们容易出现由多路径干扰（MPI）和低信噪比（SNR）引起的错误。传统方法，在去噪后，通过估计编码深度的瞬态图像来减轻MPI。最近，在不使用中间瞬态表示的情况下，共同去噪和减轻MPI的数据驱动方法已经成为最先进的。在本文中，我们建议重新审视瞬态代表。使用数据驱动的Priors，我们将其插入/推断ITOF频率并使用它们来估计瞬态图像。给定直接TOF（DTOF）传感器捕获瞬态图像，我们将我们的方法命名为ITOF2DTOF。瞬态表示是灵活的。它可以集成与基于规则的深度感测算法，对低SNR具有强大，并且可以处理实际上出现的模糊场景（例如，镜面MPI，光学串扰）。我们在真正深度传感方案中展示了先前方法上的ITOF2DTOF的好处。

具有多核光纤（MCF）无透镜微观镜片的定制光的产生广泛用于生物医学。然而，用于这种应用的计算机生成的全息图（CGHS）通常由迭代算法产生，这需要高计算工作，限制在体内光源刺激和光纤细胞操纵中的高级应用。纤维芯的随机和离散分布对CGHS引起了强烈的空间偏大，因此，非常需要一种能够快速生成MCF的量身定制的CGHS的方法。我们展示了一种新型阶段编码器深神经网络（Coreenet），它可以在近视频速率下为MCF产生精确定制的CGHS。模拟表明，与传统的CGH技术相比，CoreNet可以将计算时间加速两个大小，并增加产生的光场的保真度。首次，实时生成的定制CGHS在飞行中加载到仅相位的SLM，用于通过MCF微内窥镜在实验中产生动态光场。这铺设了实时细胞旋转的途径和几种需要在生物医学中实时高保真光传递的几种进一步的应用。

在部署非视线（NLOS）成像系统中，越来越兴趣，以恢复障碍物背后的物体。现有解决方案通常在扫描隐藏对象之前预先校准系统。在封堵器，对象和扫描模式的现场调整需要重新校准。我们提出了一种在线校准技术，直接将所获取的瞬态扫描到LOS和隐藏组件中的所获取的瞬态耦合。我们使用前者直接（RE）在场景/障碍配置，扫描区域和扫描模式的变化时校准系统，而后者通过空间，频率或基于学习的技术恢复后者。我们的技术避免使用辅助校准设备，例如镜子或棋盘，并支持实验室验证和现实世界部署。

A unidirectional imager would only permit image formation along one direction, from an input field-of-view (FOV) A to an output FOV B, and in the reverse path, the image formation would be blocked. Here, we report the first demonstration of unidirectional imagers, presenting polarization-insensitive and broadband unidirectional imaging based on successive diffractive layers that are linear and isotropic. These diffractive layers are optimized using deep learning and consist of hundreds of thousands of diffractive phase features, which collectively modulate the incoming fields and project an intensity image of the input onto an output FOV, while blocking the image formation in the reverse direction. After their deep learning-based training, the resulting diffractive layers are fabricated to form a unidirectional imager. As a reciprocal device, the diffractive unidirectional imager has asymmetric mode processing capabilities in the forward and backward directions, where the optical modes from B to A are selectively guided/scattered to miss the output FOV, whereas for the forward direction such modal losses are minimized, yielding an ideal imaging system between the input and output FOVs. Although trained using monochromatic illumination, the diffractive unidirectional imager maintains its functionality over a large spectral band and works under broadband illumination. We experimentally validated this unidirectional imager using terahertz radiation, very well matching our numerical results. Using the same deep learning-based design strategy, we also created a wavelength-selective unidirectional imager, where two unidirectional imaging operations, in reverse directions, are multiplexed through different illumination wavelengths. Diffractive unidirectional imaging using structured materials will have numerous applications in e.g., security, defense, telecommunications and privacy protection.

浑浊或形成是工业常规用于行业的概念，以解决非织造物和论文的均匀性的偏差。基于图像数据测量浑浊索引是工业质量保证的常见任务。量化云的两个最流行的方式是基于一方面的功率谱或相关功能，另一方面是拉普拉斯金字塔。在这里，我们全面地回顾了第一种方法的数学基础，得出了浑浊指数，并证明了其实际估计。我们证明了拉普拉斯金字塔以及表征云的其他量，如相互作用范围和小角度散射的强度与功率谱非常密切相关。最后，我们表明，功率谱易于分析图像，并且具有比替代方案更多的信息。

We present a novel single-shot interferometric ToF camera targeted for precise 3D measurements of dynamic objects. The camera concept is based on Synthetic Wavelength Interferometry, a technique that allows retrieval of depth maps of objects with optically rough surfaces at submillimeter depth precision. In contrast to conventional ToF cameras, our device uses only off-the-shelf CCD/CMOS detectors and works at their native chip resolution (as of today, theoretically up to 20 Mp and beyond). Moreover, we can obtain a full 3D model of the object in single-shot, meaning that no temporal sequence of exposures or temporal illumination modulation (such as amplitude or frequency modulation) is necessary, which makes our camera robust against object motion. In this paper, we introduce the novel camera concept and show first measurements that demonstrate the capabilities of our system. We present 3D measurements of small (cm-sized) objects with > 2 Mp point cloud resolution (the resolution of our used detector) and up to sub-mm depth precision. We also report a "single-shot 3D video" acquisition and a first single-shot "Non-Line-of-Sight" measurement. Our technique has great potential for high-precision applications with dynamic object movement, e.g., in AR/VR, industrial inspection, medical imaging, and imaging through scattering media like fog or human tissue.

随着Terahertz（THZ）信号产生和辐射方法的最新进展，关节通信和传感应用正在塑造无线系统的未来。为此，预计将在用户设备设备上携带THZ光谱，以识别感兴趣的材料和气态组件。 THZ特异性的信号处理技术应补充这种对THZ感应的重新兴趣，以有效利用THZ频带。在本文中，我们介绍了这些技术的概述，重点是信号预处理（标准的正常差异归一化，最小值 - 最大归一化和Savitzky-Golay滤波），功能提取（主成分分析，部分最小二乘，t，T，T部分，t部分，t部分正方形，T - 分布的随机邻居嵌入和非负矩阵分解）和分类技术（支持向量机器，k-nearest邻居，判别分析和天真的贝叶斯）。我们还通过探索他们在THZ频段的有希望的传感能力来解决深度学习技术的有效性。最后，我们研究了在联合通信和传感的背景下，研究方法的性能和复杂性权衡；我们激励相应的用例，并在该领域提供未来的研究方向。