磁共振成像(MRI)的核心问题是加速度和图像质量之间的折衷。图像重建和超分辨率是磁共振成像(MRI)中的两个重要技术。目前的方法旨在单独执行这些任务,忽略它们之间的相关性。在这项工作中,我们为联合MRI重建和超分辨率提出了一个端到端的任务变压器网络(T $ ^ 2 $ net),它允许在多项任务之间共享表示和特征传输以实现更高质量的,来自高度遮盖率和退化的MRI数据的无序和运动伪影的图像。我们的框架与重建和超分辨率相结合,分为两个子分支,其功能表示为查询和键。具体地,我们鼓励两个任务之间的联合特征学习,从而传输准确的任务信息。我们首先使用两个单独的CNN分支来提取特定于任务的功能。然后,任务变压器模块旨在嵌入和综合两个任务之间的相关性。实验结果表明,我们的多任务模型显着优于高级顺序方法,包括定量和定性。
translated by 谷歌翻译
在相应的辅助对比的指导下,目标对比度的超级分辨磁共振(MR)图像(提供了其他解剖信息)是快速MR成像的新解决方案。但是,当前的多对比超分辨率(SR)方法倾向于直接连接不同的对比度,从而忽略了它们在不同的线索中的关系,例如在高强度和低强度区域中。在这项研究中,我们提出了一个可分离的注意网络(包括高强度的优先注意力和低强度分离注意力),名为SANET。我们的卫生网可以借助辅助对比度探索“正向”和“反向”方向中高强度和低强度区域的区域,同时学习目标对比MR的SR的更清晰的解剖结构和边缘信息图片。 SANET提供了三个吸引人的好处:(1)这是第一个探索可分离的注意机制的模型,该机制使用辅助对比来预测高强度和低强度区域,将更多的注意力转移到精炼这些区域和这些区域之间的任何不确定细节和纠正重建结果中的细小区域。 (2)提出了一个多阶段集成模块,以学习多个阶段的多对比度融合的响应,获得融合表示之间的依赖性,并提高其表示能力。 (3)在FastMRI和Clinical \ textit {in Vivo}数据集上进行了各种最先进的多对比度SR方法的广泛实验,证明了我们模型的优势。
translated by 谷歌翻译
We study on image super-resolution (SR), which aims to recover realistic textures from a low-resolution (LR) image. Recent progress has been made by taking high-resolution images as references (Ref), so that relevant textures can be transferred to LR images. However, existing SR approaches neglect to use attention mechanisms to transfer high-resolution (HR) textures from Ref images, which limits these approaches in challenging cases. In this paper, we propose a novel Texture Transformer Network for Image Super-Resolution (TTSR), in which the LR and Ref images are formulated as queries and keys in a transformer, respectively. TTSR consists of four closely-related modules optimized for image generation tasks, including a learnable texture extractor by DNN, a relevance embedding module, a hard-attention module for texture transfer, and a softattention module for texture synthesis. Such a design encourages joint feature learning across LR and Ref images, in which deep feature correspondences can be discovered by attention, and thus accurate texture features can be transferred. The proposed texture transformer can be further stacked in a cross-scale way, which enables texture recovery from different levels (e.g., from 1× to 4× magnification). Extensive experiments show that TTSR achieves significant improvements over state-of-the-art approaches on both quantitative and qualitative evaluations. The source code can be downloaded at https://github.com/ researchmm/TTSR.
translated by 谷歌翻译
Because of the necessity to obtain high-quality images with minimal radiation doses, such as in low-field magnetic resonance imaging, super-resolution reconstruction in medical imaging has become more popular (MRI). However, due to the complexity and high aesthetic requirements of medical imaging, image super-resolution reconstruction remains a difficult challenge. In this paper, we offer a deep learning-based strategy for reconstructing medical images from low resolutions utilizing Transformer and Generative Adversarial Networks (T-GAN). The integrated system can extract more precise texture information and focus more on important locations through global image matching after successfully inserting Transformer into the generative adversarial network for picture reconstruction. Furthermore, we weighted the combination of content loss, adversarial loss, and adversarial feature loss as the final multi-task loss function during the training of our proposed model T-GAN. In comparison to established measures like PSNR and SSIM, our suggested T-GAN achieves optimal performance and recovers more texture features in super-resolution reconstruction of MRI scanned images of the knees and belly.
translated by 谷歌翻译
缩短采集时间和减少动作伪影是磁共振成像中最重要的两个问题。作为一个有前途的解决方案,已经研究了基于深度学习的高质量MR图像恢复,以产生从缩短采集时间获取的较低分辨率图像的更高分辨率和自由运动伪影图像,而不降低额外的获取时间或修改脉冲序列。然而,仍有许多问题仍然存在,以防止深度学习方法在临床环境中变得实用。具体而言,大多数先前的作品专注于网络模型,但忽略了各种下采样策略对采集时间的影响。此外,长推理时间和高GPU消耗也是瓶颈,以便在诊所部署大部分产品。此外,先验研究采用回顾性运动伪像产生随机运动,导致运动伪影的无法控制的严重程度。更重要的是,医生不确定生成的MR图像是否值得信赖,使诊断困难。为了克服所有这些问题,我们雇用了一个统一的2D深度学习神经网络,用于3D MRI超级分辨率和运动伪影,展示这种框架可以在3D MRI恢复任务中实现更好的性能与最艺术方法的其他状态,并且仍然存在GPU消耗和推理时间明显低,从而更易于部署。我们还基于加速度分析了几种下式采样策略,包括在平面内和穿过平面下采样的多种组合,并开发了一种可控和可量化的运动伪影生成方法。最后,计算并用于估计生成图像的准确性的像素 - 明智的不确定性,提供可靠诊断的附加信息。
translated by 谷歌翻译
Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-ofthe-art single image super-resolution approaches.
translated by 谷歌翻译
具有高分辨率(HR)的磁共振成像(MRI)提供了更详细的信息,以进行准确的诊断和定量图像分析。尽管取得了重大进展,但大多数现有的医学图像重建网络都有两个缺陷:1)所有这些缺陷都是在黑盒原理中设计的,因此缺乏足够的解释性并进一步限制其实际应用。可解释的神经网络模型引起了重大兴趣,因为它们在处理医学图像时增强了临床实践所需的可信赖性。 2)大多数现有的SR重建方法仅使用单个对比度或使用简单的多对比度融合机制,从而忽略了对SR改进至关重要的不同对比度之间的复杂关系。为了解决这些问题,在本文中,提出了一种新颖的模型引导的可解释的深层展开网络(MGDUN),用于医学图像SR重建。模型引导的图像SR重建方法求解手动设计的目标函数以重建HR MRI。我们通过将MRI观察矩阵和显式多对比度关系矩阵考虑到末端到端优化期间,将迭代的MGDUN算法展示为新型模型引导的深层展开网络。多对比度IXI数据集和Brats 2019数据集进行了广泛的实验,证明了我们提出的模型的优势。
translated by 谷歌翻译
近年来,压缩图像超分辨率已引起了极大的关注,其中图像被压缩伪像和低分辨率伪影降解。由于复杂的杂化扭曲变形,因此很难通过简单的超分辨率和压缩伪像消除掉的简单合作来恢复扭曲的图像。在本文中,我们向前迈出了一步,提出了层次的SWIN变压器(HST)网络,以恢复低分辨率压缩图像,该图像共同捕获分层特征表示并分别用SWIN Transformer增强每个尺度表示。此外,我们发现具有超分辨率(SR)任务的预处理对于压缩图像超分辨率至关重要。为了探索不同的SR预审查的影响,我们将常用的SR任务(例如,比科比奇和不同的实际超分辨率仿真)作为我们的预处理任务,并揭示了SR在压缩的图像超分辨率中起不可替代的作用。随着HST和预训练的合作,我们的HST在AIM 2022挑战中获得了低质量压缩图像超分辨率轨道的第五名,PSNR为23.51db。广泛的实验和消融研究已经验证了我们提出的方法的有效性。
translated by 谷歌翻译
基于参考的图像超分辨率(REFSR)旨在利用辅助参考(REF)图像为超溶解的低分辨率(LR)图像。最近,RefSR引起了极大的关注,因为它提供了超越单图SR的替代方法。但是,解决REFSR问题有两个关键的挑战:(i)当它们显着不同时,很难匹配LR和Ref图像之间的对应关系; (ii)如何将相关纹理从参考图像转移以补偿LR图像的细节非常具有挑战性。为了解决RefSR的这些问题,本文提出了一个可变形的注意变压器,即DATSR,具有多个尺度,每个尺度由纹理特征编码器(TFE)模块组成,基于参考的可变形注意(RDA)模块和残差功能聚合(RFA)模块。具体而言,TFE首先提取图像转换(例如,亮度)不敏感的LR和REF图像,RDA可以利用多个相关纹理来补偿更多的LR功能信息,而RFA最终汇总了LR功能和相关纹理,以获得更愉快的宜人的质地结果。广泛的实验表明,我们的DATSR在定量和质量上实现了基准数据集上的最新性能。
translated by 谷歌翻译
Convolutional Neural Network (CNN)-based image super-resolution (SR) has exhibited impressive success on known degraded low-resolution (LR) images. However, this type of approach is hard to hold its performance in practical scenarios when the degradation process is unknown. Despite existing blind SR methods proposed to solve this problem using blur kernel estimation, the perceptual quality and reconstruction accuracy are still unsatisfactory. In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model. We propose a components decomposition and co-optimization network (CDCN) for blind SR. Firstly, CDCN decomposes the input LR image into structure and detail components in feature space. Then, the mutual collaboration block (MCB) is presented to exploit the relationship between both two components. In this way, the detail component can provide informative features to enrich the structural context and the structure component can carry structural context for better detail revealing via a mutual complementary manner. After that, we present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process. Finally, a multi-scale fusion module followed by an upsampling layer is designed to fuse the structure and detail features and perform SR reconstruction. Empowered by such degradation-based components decomposition, collaboration, and mutual optimization, we can bridge the correlation between component learning and degradation modelling for blind SR, thereby producing SR results with more accurate textures. Extensive experiments on both synthetic SR datasets and real-world images show that the proposed method achieves the state-of-the-art performance compared to existing methods.
translated by 谷歌翻译
基于深度学习(DL)磁共振(MR)图像重建的方法已被证明近年来产生卓越的性能。但是,这些方法只利用被采样的数据,或者需要配对的全采样辅助模型来执行多模态重建。因此,现有的方法忽略了探索可以将纹理从引用完全采样数据转移到单个模态内的欠采样数据的注意力机制,这限制了这些方法在具有挑战性的情况下。在本文中,我们提出了一种新颖的纹理变压器模块(TTM),用于加速MRI重建,其中我们将欠采样的数据和参考数据作为验证和键在变压器中装订。 TTM促进了跨越采样和参考数据的联合特征学习,因此可以通过注意,在重建期间可以利用精确的纹理特征来发现特征对应关系。值得注意的是,所提出的TTM可以依赖于先前的MRI重建方法,以进一步提高其性能。广泛的实验表明,TTM可以显着提高几个流行的基于DL的MRI重建方法的性能。
translated by 谷歌翻译
随着深度学习的发展,单图像超分辨率(SISR)取得了重大突破。最近,已经提出了基于全局特征交互的SISR网络性能的方法。但是,需要动态地忽略对上下文的响应的神经元的功能。为了解决这个问题,我们提出了一个轻巧的交叉障碍性推理网络(CFIN),这是一个由卷积神经网络(CNN)和变压器组成的混合网络。具体而言,一种新型的交叉磁场导向变压器(CFGT)旨在通过使用调制卷积内核与局部代表性语义信息结合来自适应修改网络权重。此外,提出了基于CNN的跨尺度信息聚合模块(CIAM),以使模型更好地专注于潜在的实用信息并提高变压器阶段的效率。广泛的实验表明,我们提出的CFIN是一种轻巧有效的SISR模型,可以在计算成本和模型性能之间达到良好的平衡。
translated by 谷歌翻译
Convolutional neural networks have recently demonstrated high-quality reconstruction for single-image superresolution. In this paper, we propose the Laplacian Pyramid Super-Resolution Network (LapSRN) to progressively reconstruct the sub-band residuals of high-resolution images. At each pyramid level, our model takes coarse-resolution feature maps as input, predicts the high-frequency residuals, and uses transposed convolutions for upsampling to the finer level. Our method does not require the bicubic interpolation as the pre-processing step and thus dramatically reduces the computational complexity. We train the proposed LapSRN with deep supervision using a robust Charbonnier loss function and achieve high-quality reconstruction. Furthermore, our network generates multi-scale predictions in one feed-forward pass through the progressive reconstruction, thereby facilitates resource-aware applications. Extensive quantitative and qualitative evaluations on benchmark datasets show that the proposed algorithm performs favorably against the state-of-the-art methods in terms of speed and accuracy.
translated by 谷歌翻译
联合超分辨率和反音调映射(SR-ITM)旨在提高具有分辨率和动态范围具有质量缺陷的视频的视觉质量。当使用4K高动态范围(HDR)电视来观看低分辨率标准动态范围(LR SDR)视频时,就会出现此问题。以前依赖于学习本地信息的方法通常在保留颜色合规性和远程结构相似性方面做得很好,从而导致了不自然的色彩过渡和纹理伪像。为了应对这些挑战,我们建议联合SR-ITM的全球先验指导的调制网络(GPGMNET)。特别是,我们设计了一个全球先验提取模块(GPEM),以提取颜色合规性和结构相似性,分别对ITM和SR任务有益。为了进一步利用全球先验并保留空间信息,我们使用一些用于中间特征调制的参数,设计多个全球先验的指导空间调制块(GSMB),其中调制参数由共享的全局先验和空间特征生成来自空间金字塔卷积块(SPCB)的地图。通过这些精心设计的设计,GPGMNET可以通过较低的计算复杂性实现更高的视觉质量。广泛的实验表明,我们提出的GPGMNET优于最新方法。具体而言,我们提出的模型在PSNR中超过了0.64 dB的最新模型,其中69 $ \%$ $ $较少,3.1 $ \ times $ speedup。该代码将很快发布。
translated by 谷歌翻译
Recently, great progress has been made in single-image super-resolution (SISR) based on deep learning technology. However, the existing methods usually require a large computational cost. Meanwhile, the activation function will cause some features of the intermediate layer to be lost. Therefore, it is a challenge to make the model lightweight while reducing the impact of intermediate feature loss on the reconstruction quality. In this paper, we propose a Feature Interaction Weighted Hybrid Network (FIWHN) to alleviate the above problem. Specifically, FIWHN consists of a series of novel Wide-residual Distillation Interaction Blocks (WDIB) as the backbone, where every third WDIBs form a Feature shuffle Weighted Group (FSWG) by mutual information mixing and fusion. In addition, to mitigate the adverse effects of intermediate feature loss on the reconstruction results, we introduced a well-designed Wide Convolutional Residual Weighting (WCRW) and Wide Identical Residual Weighting (WIRW) units in WDIB, and effectively cross-fused features of different finenesses through a Wide-residual Distillation Connection (WRDC) framework and a Self-Calibrating Fusion (SCF) unit. Finally, to complement the global features lacking in the CNN model, we introduced the Transformer into our model and explored a new way of combining the CNN and Transformer. Extensive quantitative and qualitative experiments on low-level and high-level tasks show that our proposed FIWHN can achieve a good balance between performance and efficiency, and is more conducive to downstream tasks to solve problems in low-pixel scenarios.
translated by 谷歌翻译
Single-image super-resolution (SISR) networks trained with perceptual and adversarial losses provide high-contrast outputs compared to those of networks trained with distortion-oriented losses, such as L1 or L2. However, it has been shown that using a single perceptual loss is insufficient for accurately restoring locally varying diverse shapes in images, often generating undesirable artifacts or unnatural details. For this reason, combinations of various losses, such as perceptual, adversarial, and distortion losses, have been attempted, yet it remains challenging to find optimal combinations. Hence, in this paper, we propose a new SISR framework that applies optimal objectives for each region to generate plausible results in overall areas of high-resolution outputs. Specifically, the framework comprises two models: a predictive model that infers an optimal objective map for a given low-resolution (LR) input and a generative model that applies a target objective map to produce the corresponding SR output. The generative model is trained over our proposed objective trajectory representing a set of essential objectives, which enables the single network to learn various SR results corresponding to combined losses on the trajectory. The predictive model is trained using pairs of LR images and corresponding optimal objective maps searched from the objective trajectory. Experimental results on five benchmarks show that the proposed method outperforms state-of-the-art perception-driven SR methods in LPIPS, DISTS, PSNR, and SSIM metrics. The visual results also demonstrate the superiority of our method in perception-oriented reconstruction. The code and models are available at https://github.com/seungho-snu/SROOE.
translated by 谷歌翻译
面部超分辨率(FSR),也称为面部幻觉,其旨在增强低分辨率(LR)面部图像以产生高分辨率(HR)面部图像的分辨率,是特定于域的图像超分辨率问题。最近,FSR获得了相当大的关注,并目睹了深度学习技术的发展炫目。迄今为止,有很少有基于深入学习的FSR的研究摘要。在本次调查中,我们以系统的方式对基于深度学习的FSR方法进行了全面审查。首先,我们总结了FSR的问题制定,并引入了流行的评估度量和损失功能。其次,我们详细说明了FSR中使用的面部特征和流行数据集。第三,我们根据面部特征的利用大致分类了现有方法。在每个类别中,我们从设计原则的一般描述开始,然后概述代表方法,然后讨论其中的利弊。第四,我们评估了一些最先进的方法的表现。第五,联合FSR和其他任务以及与FSR相关的申请大致介绍。最后,我们设想了这一领域进一步的技术进步的前景。在\ URL {https://github.com/junjun-jiang/face-hallucination-benchmark}上有一个策划的文件和资源的策划文件和资源清单
translated by 谷歌翻译
由于组织和骨骼之间的相似性,在人解剖结构中广泛看到了全球相关性。由于近距离质子密度和T1/T2参数,这些相关性反映在磁共振成像(MRI)扫描中。此外,为了实现加速的MRI,k空间数据的采样不足,从而导致全球混叠伪像。卷积神经网络(CNN)模型被广泛用于加速MRI重建,但是由于卷积操作的固有位置,这些模型在捕获全球相关性方面受到限制。基于自发的变压器模型能够捕获图像特征之间的全局相关性,但是,变压器模型对MRI重建的当前贡献是微小的。现有的贡献主要提供CNN转换器混合解决方案,并且很少利用MRI的物理学。在本文中,我们提出了一种基于物理的独立(无卷积)变压器模型,标题为“多头级联SWIN变压器(MCSTRA),用于加速MRI重建。 MCSTRA将几种相互关联的MRI物理相关概念与变压器网络相结合:它通过移动的窗口自我发场机制利用了全局MR特征;它使用多头设置分别提取属于不同光谱组件的MR特征;它通过级联的网络在中间脱氧和K空间校正之间进行迭代,该网络具有K空间和中间损耗计算中的数据一致性;此外,我们提出了一种新型的位置嵌入生成机制,以使用对应于底面采样掩码的点扩散函数来指导自我发作。我们的模型在视觉上和定量上都大大优于最先进的MRI重建方法,同时描述了改善的分辨率和去除词法。
translated by 谷歌翻译
Convolutional neural network (CNN) depth is of crucial importance for image super-resolution (SR). However, we observe that deeper networks for image SR are more difficult to train. The lowresolution inputs and features contain abundant low-frequency information, which is treated equally across channels, hence hindering the representational ability of CNNs. To solve these problems, we propose the very deep residual channel attention networks (RCAN). Specifically, we propose a residual in residual (RIR) structure to form very deep network, which consists of several residual groups with long skip connections. Each residual group contains some residual blocks with short skip connections. Meanwhile, RIR allows abundant low-frequency information to be bypassed through multiple skip connections, making the main network focus on learning high-frequency information. Furthermore, we propose a channel attention mechanism to adaptively rescale channel-wise features by considering interdependencies among channels. Extensive experiments show that our RCAN achieves better accuracy and visual improvements against state-of-the-art methods.
translated by 谷歌翻译
在临床实践中,由于较短的获取时间和较低的存储成本,通常使用了平面分辨率低的各向异性体积医学图像。然而,粗分辨率可能导致医生或计算机辅助诊断算法的医学诊断困难。基于深度学习的体积超分辨率(SR)方法是改善分辨率的可行方法,其核心是卷积神经网络(CNN)。尽管进展最近,但这些方法受到卷积运算符的固有属性的限制,卷积运算符忽略内容相关性,无法有效地对远程依赖性进行建模。此外,大多数现有方法都使用伪配合的体积进行训练和评估,其中伪低分辨率(LR)体积是通过简单的高分辨率(HR)对应物的简单降解而产生的。但是,伪和现实LR之间的域间隙导致这些方法在实践中的性能不佳。在本文中,我们构建了第一个公共实用数据集RPLHR-CT作为体积SR的基准,并通过重新实现四种基于CNN的最先进的方法来提供基线结果。考虑到CNN的固有缺点,我们还提出了基于注意力机制的变压器体积超分辨率网络(TVSRN),完全与卷积分配。这是首次将纯变压器用于CT体积SR的研究。实验结果表明,TVSRN在PSNR和SSIM上的所有基准都显着胜过。此外,TVSRN方法在图像质量,参数数量和运行时间之间取得了更好的权衡。数据和代码可在https://github.com/smilenaxx/rplhr-ct上找到。
translated by 谷歌翻译