Video super-resolution (VSR) aiming to reconstruct a high-resolution (HR) video from its low-resolution (LR) counterpart has made tremendous progress in recent years. However, it remains challenging to deploy existing VSR methods to real-world data with complex degradations. On the one hand, there are few well-aligned real-world VSR datasets, especially with large super-resolution scale factors, which limits the development of real-world VSR tasks. On the other hand, alignment algorithms in existing VSR methods perform poorly for real-world videos, leading to unsatisfactory results. As an attempt to address the aforementioned issues, we build a real-world 4 VSR dataset, namely MVSR4$\times$, where low- and high-resolution videos are captured with different focal length lenses of a smartphone, respectively. Moreover, we propose an effective alignment method for real-world VSR, namely EAVSR. EAVSR takes the proposed multi-layer adaptive spatial transform network (MultiAdaSTN) to refine the offsets provided by the pre-trained optical flow estimation network. Experimental results on RealVSR and MVSR4$\times$ datasets show the effectiveness and practicality of our method, and we achieve state-of-the-art performance in real-world VSR task. The dataset and code will be publicly available.
translated by 谷歌翻译
Focusing on the complicated pathological features, such as blurred boundaries, severe scale differences between symptoms, background noise interference, etc., in the task of retinal edema lesions joint segmentation from OCT images and enabling the segmentation results more reliable. In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network, which can provide accurate segmentation results with reliability assessment. Specifically, aiming at improving the model's ability to learn the complex pathological features of retinal edema lesions in OCT images, we develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module of our newly designed. Meanwhile, to make the segmentation results more reliable, a novel uncertainty segmentation head based on the subjective logical evidential theory is introduced to generate the final segmentation results with a corresponding overall uncertainty evaluation score map. We conduct comprehensive experiments on the public database of AI-Challenge 2018 for retinal edema lesions segmentation, and the results show that our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches. The code will be released on: https://github.com/LooKing9218/ReliableRESeg.
translated by 谷歌翻译
开放设定的半监督学习(OSSL)引起了人们日益增长的兴趣,该学习调查了一个更实用的情况,在该情况下,仅在未标记的数据中包含了分布式(OOD)样本。现有的OSSL方法(例如OpenMatch)学习一个OOD检测器以识别离群值,该检测器通常会更新所有模态参数(即完整的微调),以从标记的数据传播类信息到未标记的数据。当前,已经开发了及时的学习来弥合预训练和微调之间的差距,这在几个下游任务中显示出较高的计算效率。在本文中,我们提出了一个迅速驱动的有效OSSL框架,称为OpenPrompt,该框架可以将类别的类信息传播到标记到未标记数据的类信息,只有少数可训练的参数。我们提出了一种迅速驱动的关节空间学习机制来检测OOD数据,通过在未标记的数据中最大化ID和OOD样本之间的分布差距,从而使我们的方法可以以新的方式检测到异常值。三个公共数据集的实验结果表明,OpenPrompt优于不到1%可训练参数的最先进方法。更重要的是,OpenPrompt在CIFAR10上完全监督模型的AUROC检测方面取得了4%的改善。
translated by 谷歌翻译
联合学习(FL)可用于通过使多个机构协作,改善磁共振(MR)图像重建的数据隐私和效率,而无需聚合本地数据。然而,由不同MR成像协议引起的域移位可以显着降低FL模型的性能。最近的流程倾向于通过增强全局模型的概括来解决这一点,但它们忽略了特定于域的特征,这可能包含有关设备属性的重要信息,并且对本地重建有用。在本文中,我们提出了一种针对MR图像重建(FEDMRI)的特异性保存流算法。核心思想是将MR重建模型划分为两个部分:全局共享编码器,以在全局级别获取概括的表示,以及客户特定的解码器,以保留每个客户端的特定于域的属性,这对于协作很重要当客户具有独特的分发时重建。此外,为了进一步提高全局共享编码器的收敛,当存在域移位时,引入加权对比正规化以在优化期间直接校正客户端和服务器之间的任何偏差。广泛的实验表明,我们的Fedmri的重建结果是最接近多机构数据的地面真理,并且它优于最先进的FL方法。
translated by 谷歌翻译
在相应的辅助对比的指导下,目标对比度的超级分辨磁共振(MR)图像(提供了其他解剖信息)是快速MR成像的新解决方案。但是,当前的多对比超分辨率(SR)方法倾向于直接连接不同的对比度,从而忽略了它们在不同的线索中的关系,例如在高强度和低强度区域中。在这项研究中,我们提出了一个可分离的注意网络(包括高强度的优先注意力和低强度分离注意力),名为SANET。我们的卫生网可以借助辅助对比度探索“正向”和“反向”方向中高强度和低强度区域的区域,同时学习目标对比MR的SR的更清晰的解剖结构和边缘信息图片。 SANET提供了三个吸引人的好处:(1)这是第一个探索可分离的注意机制的模型,该机制使用辅助对比来预测高强度和低强度区域,将更多的注意力转移到精炼这些区域和这些区域之间的任何不确定细节和纠正重建结果中的细小区域。 (2)提出了一个多阶段集成模块,以学习多个阶段的多对比度融合的响应,获得融合表示之间的依赖性,并提高其表示能力。 (3)在FastMRI和Clinical \ textit {in Vivo}数据集上进行了各种最先进的多对比度SR方法的广泛实验,证明了我们模型的优势。
translated by 谷歌翻译
磁共振成像(MRI)的核心问题是加速度和图像质量之间的折衷。图像重建和超分辨率是磁共振成像(MRI)中的两个重要技术。目前的方法旨在单独执行这些任务,忽略它们之间的相关性。在这项工作中,我们为联合MRI重建和超分辨率提出了一个端到端的任务变压器网络(T $ ^ 2 $ net),它允许在多项任务之间共享表示和特征传输以实现更高质量的,来自高度遮盖率和退化的MRI数据的无序和运动伪影的图像。我们的框架与重建和超分辨率相结合,分为两个子分支,其功能表示为查询和键。具体地,我们鼓励两个任务之间的联合特征学习,从而传输准确的任务信息。我们首先使用两个单独的CNN分支来提取特定于任务的功能。然后,任务变压器模块旨在嵌入和综合两个任务之间的相关性。实验结果表明,我们的多任务模型显着优于高级顺序方法,包括定量和定性。
translated by 谷歌翻译
Increasing research interests focus on sequential recommender systems, aiming to model dynamic sequence representation precisely. However, the most commonly used loss function in state-of-the-art sequential recommendation models has essential limitations. To name a few, Bayesian Personalized Ranking (BPR) loss suffers the vanishing gradient problem from numerous negative sampling and predictionbiases; Binary Cross-Entropy (BCE) loss subjects to negative sampling numbers, thereby it is likely to ignore valuable negative examples and reduce the training efficiency; Cross-Entropy (CE) loss only focuses on the last timestamp of the training sequence, which causes low utilization of sequence information and results in inferior user sequence representation. To avoid these limitations, in this paper, we propose to calculate Cumulative Cross-Entropy (CCE) loss over the sequence. CCE is simple and direct, which enjoys the virtues of painless deployment, no negative sampling, and effective and efficient training. We conduct extensive experiments on five benchmark datasets to demonstrate the effectiveness and efficiency of CCE. The results show that employing CCE loss on three state-of-the-art models GRU4Rec, SASRec, and S3-Rec can reach 125.63%, 69.90%, and 33.24% average improvement of full ranking NDCG@5, respectively. Using CCE, the performance curve of the models on the test data increases rapidly with the wall clock time, and is superior to that of other loss functions in almost the whole process of model training.
translated by 谷歌翻译
In the scenario of black-box adversarial attack, the target model's parameters are unknown, and the attacker aims to find a successful adversarial perturbation based on query feedback under a query budget. Due to the limited feedback information, existing query-based black-box attack methods often require many queries for attacking each benign example. To reduce query cost, we propose to utilize the feedback information across historical attacks, dubbed example-level adversarial transferability. Specifically, by treating the attack on each benign example as one task, we develop a meta-learning framework by training a meta-generator to produce perturbations conditioned on benign examples. When attacking a new benign example, the meta generator can be quickly fine-tuned based on the feedback information of the new task as well as a few historical attacks to produce effective perturbations. Moreover, since the meta-train procedure consumes many queries to learn a generalizable generator, we utilize model-level adversarial transferability to train the meta-generator on a white-box surrogate model, then transfer it to help the attack against the target model. The proposed framework with the two types of adversarial transferability can be naturally combined with any off-the-shelf query-based attack methods to boost their performance, which is verified by extensive experiments.
translated by 谷歌翻译
Supervised Deep-Learning (DL)-based reconstruction algorithms have shown state-of-the-art results for highly-undersampled dynamic Magnetic Resonance Imaging (MRI) reconstruction. However, the requirement of excessive high-quality ground-truth data hinders their applications due to the generalization problem. Recently, Implicit Neural Representation (INR) has appeared as a powerful DL-based tool for solving the inverse problem by characterizing the attributes of a signal as a continuous function of corresponding coordinates in an unsupervised manner. In this work, we proposed an INR-based method to improve dynamic MRI reconstruction from highly undersampled k-space data, which only takes spatiotemporal coordinates as inputs. Specifically, the proposed INR represents the dynamic MRI images as an implicit function and encodes them into neural networks. The weights of the network are learned from sparsely-acquired (k, t)-space data itself only, without external training datasets or prior images. Benefiting from the strong implicit continuity regularization of INR together with explicit regularization for low-rankness and sparsity, our proposed method outperforms the compared scan-specific methods at various acceleration factors. E.g., experiments on retrospective cardiac cine datasets show an improvement of 5.5 ~ 7.1 dB in PSNR for extremely high accelerations (up to 41.6-fold). The high-quality and inner continuity of the images provided by INR has great potential to further improve the spatiotemporal resolution of dynamic MRI, without the need of any training data.
translated by 谷歌翻译
Recent studies have shown that using an external Language Model (LM) benefits the end-to-end Automatic Speech Recognition (ASR). However, predicting tokens that appear less frequently in the training set is still quite challenging. The long-tail prediction problems have been widely studied in many applications, but only been addressed by a few studies for ASR and LMs. In this paper, we propose a new memory augmented lookup dictionary based Transformer architecture for LM. The newly introduced lookup dictionary incorporates rich contextual information in training set, which is vital to correctly predict long-tail tokens. With intensive experiments on Chinese and English data sets, our proposed method is proved to outperform the baseline Transformer LM by a great margin on both word/character error rate and tail tokens error rate. This is achieved without impact on the decoding efficiency. Overall, we demonstrate the effectiveness of our proposed method in boosting the ASR decoding performance, especially for long-tail tokens.
translated by 谷歌翻译