There are multiple scales of abstraction from which we can describe the same image, depending on whether we are focusing on fine-grained details or a more global attribute of the image. In brain mapping, learning to automatically parse images to build representations of both small-scale features (e.g., the presence of cells or blood vessels) and global properties of an image (e.g., which brain region the image comes from) is a crucial and open challenge. However, most existing datasets and benchmarks for neuroanatomy consider only a single downstream task at a time. To bridge this gap, we introduce a new dataset, annotations, and multiple downstream tasks that provide diverse ways to readout information about brain structure and architecture from the same image. Our multi-task neuroimaging benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large thalamocortical section of mouse brain, encompassing multiple cortical and subcortical regions. We generated a number of different prediction challenges and evaluated several supervised and self-supervised models for brain-region prediction and pixel-level semantic segmentation of microstructures. Our experiments not only highlight the rich heterogeneity of this dataset, but also provide insights into how self-supervised approaches can be used to learn representations that capture multiple attributes of a single image and perform well on a variety of downstream tasks. Datasets, code, and pre-trained baseline models are provided at: https://mtneuro.github.io/ .
translated by 谷歌翻译
The lack of efficient segmentation methods and fully-labeled datasets limits the comprehensive assessment of optical coherence tomography angiography (OCTA) microstructures like retinal vessel network (RVN) and foveal avascular zone (FAZ), which are of great value in ophthalmic and systematic diseases evaluation. Here, we introduce an innovative OCTA microstructure segmentation network (OMSN) by combining an encoder-decoder-based architecture with multi-scale skip connections and the split-attention-based residual network ResNeSt, paying specific attention to OCTA microstructural features while facilitating better model convergence and feature representations. The proposed OMSN achieves excellent single/multi-task performances for RVN or/and FAZ segmentation. Especially, the evaluation metrics on multi-task models outperform single-task models on the same dataset. On this basis, a fully annotated retinal OCTA segmentation (FAROS) dataset is constructed semi-automatically, filling the vacancy of a pixel-level fully-labeled OCTA dataset. OMSN multi-task segmentation model retrained with FAROS further certifies its outstanding accuracy for simultaneous RVN and FAZ segmentation.
translated by 谷歌翻译
Image manipulation localization aims at distinguishing forged regions from the whole test image. Although many outstanding prior arts have been proposed for this task, there are still two issues that need to be further studied: 1) how to fuse diverse types of features with forgery clues; 2) how to progressively integrate multistage features for better localization performance. In this paper, we propose a tripartite progressive integration network (TriPINet) for end-to-end image manipulation localization. First, we extract both visual perception information, e.g., RGB input images, and visual imperceptible features, e.g., frequency and noise traces for forensic feature learning. Second, we develop a guided cross-modality dual-attention (gCMDA) module to fuse different types of forged clues. Third, we design a set of progressive integration squeeze-and-excitation (PI-SE) modules to improve localization performance by appropriately incorporating multiscale features in the decoder. Extensive experiments are conducted to compare our method with state-of-the-art image forensics approaches. The proposed TriPINet obtains competitive results on several benchmark datasets.
translated by 谷歌翻译
We propose a sparse end-to-end multi-person pose regression framework, termed QueryPose, which can directly predict multi-person keypoint sequences from the input image. The existing end-to-end methods rely on dense representations to preserve the spatial detail and structure for precise keypoint localization. However, the dense paradigm introduces complex and redundant post-processes during inference. In our framework, each human instance is encoded by several learnable spatial-aware part-level queries associated with an instance-level query. First, we propose the Spatial Part Embedding Generation Module (SPEGM) that considers the local spatial attention mechanism to generate several spatial-sensitive part embeddings, which contain spatial details and structural information for enhancing the part-level queries. Second, we introduce the Selective Iteration Module (SIM) to adaptively update the sparse part-level queries via the generated spatial-sensitive part embeddings stage-by-stage. Based on the two proposed modules, the part-level queries are able to fully encode the spatial details and structural information for precise keypoint regression. With the bipartite matching, QueryPose avoids the hand-designed post-processes and surpasses the existing dense end-to-end methods with 73.6 AP on MS COCO mini-val set and 72.7 AP on CrowdPose test set. Code is available at https://github.com/buptxyb666/QueryPose.
translated by 谷歌翻译
Hoist scheduling has become a bottleneck in electroplating industry applications with the development of autonomous devices. Although there are a few approaches proposed to target at the challenging problem, they generally cannot scale to large-scale scheduling problems. In this paper, we formulate the hoist scheduling problem as a new temporal planning problem in the form of adapted PDDL, and propose a novel hierarchical temporal planning approach to efficiently solve the scheduling problem. Additionally, we provide a collection of real-life benchmark instances that can be used to evaluate solution methods for the problem. We exhibit that the proposed approach is able to efficiently find solutions of high quality for large-scale real-life benchmark instances, with comparison to state-of-the-art baselines.
translated by 谷歌翻译
无人驾驶飞机(UAV)通过低成本,大型覆盖,实时和高分辨率数据采集能力而广泛应用于检查,搜索和救援行动的目的。在这些过程中产生了大量航空视频,在这些过程中,正常事件通常占压倒性的比例。本地化和提取异常事件非常困难,这些事件包含手动从长视频流中的潜在有价值的信息。因此,我们致力于开发用于解决此问题的异常检测方法。在本文中,我们创建了一个新的数据集,名为Droneanomaly,用于空中视频中的异常检测。该数据集提供了37个培训视频序列和22个测试视频序列,这些视频序列来自7个不同的现实场景,其中包括各种异常事件。有87,488个彩色视频框架(训练51,635,测试35,853),每秒30帧的尺寸为640美元\ times 640美元。基于此数据集,我们评估现有方法并为此任务提供基准。此外,我们提出了一种新的基线模型,即变压器(ANDT)的异常检测,该模型将连续的视频帧视为一系列小管,它利用变压器编码器从序列中学习特征表示,并利用解码器来预测下一帧。我们的网络模型在训练阶段模型正常,并确定了具有不可预测的时间动力学的事件,作为测试阶段的异常。此外,为了全面评估我们提出的方法的性能,我们不仅使用无人机 - 异常数据集,而且使用另一个数据集。我们将使我们的数据集和代码公开可用。可以在https://youtu.be/ancczyryoby上获得演示视频。我们使数据集和代码公开可用。
translated by 谷歌翻译
由于其低成本和快速移动性,无人驾驶汽车(UAV)现在已广泛应用于数据获取。随着航空视频量的增加,对这些视频自动解析的需求正在激增。为了实现这一目标,当前的研究主要集中于在空间和时间维度沿着卷积的整体特征提取整体特征。但是,这些方法受到小时接收场的限制,无法充分捕获长期的时间依赖性,这对于描述复杂动力学很重要。在本文中,我们提出了一个新颖的深神经网络,称为futh-net,不仅为整体特征建模,而且还模拟了空中视频分类的时间关系。此外,在新型融合模块中,多尺度的时间关系可以完善整体特征,以产生更具歧视性的视频表示。更特别地,FUTH-NET采用了两条道路架构:(1)学习框架外观和短期时间变化的一般特征的整体代表途径,以及(2)捕获跨任意跨越任意时间关系的时间关系途径框架,提供长期的时间依赖性。之后,提出了一个新型的融合模块,以时空整合从这两种途径中学到的两个特征。我们的模型对两个航空视频分类数据集进行了评估,即ERA和无人机操作,并实现了最新结果。这表明了其在不同识别任务(事件分类和人类行动识别)之间的有效性和良好的概括能力。为了促进进一步的研究,我们在https://gitlab.lrz.de/ai4eo/reasoning/futh-net上发布该代码。
translated by 谷歌翻译
Stack Overflow是最受欢迎的编程社区之一,开发人员可以为他们遇到的问题寻求帮助。然而,如果没有经验的开发人员无法清楚地描述他们的问题,那么他们很难吸引足够的关注并获得预期的答案。我们提出了M $ _3 $ NSCT5,这是一种自动从给定代码片段生成多个帖子标题的新颖方法。开发人员可以使用生成的标题查找密切相关的帖子并完成其问题描述。 M $ _3 $ NSCT5使用Codet5骨干,这是一种具有出色语言理解和发电能力的预训练的变压器模型。为了减轻歧义问题,即在不同背景下可以将相同的代码片段与不同的标题保持一致,我们提出了最大的边缘多元核抽样策略,以一次产生多个高质量和不同的标题候选者,以便开发人员选择。我们构建了一个大规模数据集,其中包含890,000个问题帖子,其中涵盖了八种编程语言,以验证M $ _3 $ NSCT5的有效性。 BLEU和胭脂指标的自动评估结果表明,M $ _3 $ NSCT5的优势比六个最先进的基线模型。此外,具有值得信赖结果的人类评估也证明了我们对现实世界应用方法的巨大潜力。
translated by 谷歌翻译
OD区域对之间的原点污染(OD)矩阵记录定向流数据。矩阵中复杂的时空依赖性使OD矩阵预测(ODMF)问题不仅可以棘手,而且是非平凡的。但是,大多数相关方法都是为在特定的应用程序方案中预测非常短的序列时间序列而设计的,在特定的应用程序场景中,该方法无法满足方案和预测实用应用长度的差异要求。为了解决这些问题,我们提出了一个名为Odformer的类似变压器的模型,具有两个显着特征:(i)新型的OD注意机制,该机制捕获了相同起源(目的地)之间的特殊空间依赖性,可大大提高与捕获OD区域之间空间依赖关系的2D-GCN结合后,预测交叉应用方案的模型。 (ii)一个时期的自我注意力,可以有效地预测长序列OD矩阵序列,同时适应不同情况下的周期性差异。在三个应用程序背景(即运输流量,IP骨干网络流量,人群流)中进行的慷慨实验表明,我们的方法的表现优于最新方法。
translated by 谷歌翻译
随着移动摄影技术的迅速发展,主要的手机制造商正在争先恐后地提高设备的拍摄能力和软件的照片美化算法。但是,智能设备和算法的改进不能取代人类的主观摄影技术。在本文中,我们提出了图像的美学语言指导(ALG)。我们根据指导规则是基于摄影模板还是指导图像,将ALG分为ALG-T和ALG-I。无论是ALG-T还是ALG-I,我们都会从三个颜色,照明和图像组成的属性中指导摄影。输入图像和摄影模板或指导图像之间的三个属性的差异用自然语言描述,即美学自然语言指导(ALG)。另外,由于景观图像和肖像图像之间的照明和组成差异,我们将输入图像分为景观图像和肖像图像。 ALG-T和ALG-I分别针对两种类型的输入图像(景观图像和肖像图像)进行美学指导。
translated by 谷歌翻译