过去,现实世界中社交网络的图表错过了两个重要元素:连接的多重性和表示时间。为此,在本文中,我们为社交网络提供了一个新的动态异质图表示,其中包括图形的每个组件中的时间,即节点和边缘,每种捕获异质性的不同类型。我们通过提出四个与时间有关的查询和深度学习问题来说明这种表示的力量,这些查询和深度学习问题无法轻易在常规的均匀图表中处理。作为概念的证明,我们介绍了新的社交媒体平台(Steemit)的详细表示,我们用它来说明动态查询功能以及使用图形神经网络(GNNS)的预测任务。结果说明了动态异质图表示对社交网络的模型的力量。鉴于这是一个相对研究的领域,我们还说明了在查询优化方面的未来工作以及异质图结构的新动态预测任务的机会。
translated by 谷歌翻译
Entity matching in Customer 360 is the task of determining if multiple records represent the same real world entity. Entities are typically people, organizations, locations, and events represented as attributed nodes in a graph, though they can also be represented as records in relational data. While probabilistic matching engines and artificial neural network models exist for this task, explaining entity matching has received less attention. In this demo, we present our Explainable Entity Matching (xEM) system and discuss the different AI/ML considerations that went into its implementation.
translated by 谷歌翻译
The vision community has explored numerous pose guided human editing methods due to their extensive practical applications. Most of these methods still use an image-to-image formulation in which a single image is given as input to produce an edited image as output. However, the problem is ill-defined in cases when the target pose is significantly different from the input pose. Existing methods then resort to in-painting or style transfer to handle occlusions and preserve content. In this paper, we explore the utilization of multiple views to minimize the issue of missing information and generate an accurate representation of the underlying human model. To fuse the knowledge from multiple viewpoints, we design a selector network that takes the pose keypoints and texture from images and generates an interpretable per-pixel selection map. After that, the encodings from a separate network (trained on a single image human reposing task) are merged in the latent space. This enables us to generate accurate, precise, and visually coherent images for different editing tasks. We show the application of our network on 2 newly proposed tasks - Multi-view human reposing, and Mix-and-match human image generation. Additionally, we study the limitations of single-view editing and scenarios in which multi-view provides a much better alternative.
translated by 谷歌翻译
Explainability has been widely stated as a cornerstone of the responsible and trustworthy use of machine learning models. With the ubiquitous use of Deep Neural Network (DNN) models expanding to risk-sensitive and safety-critical domains, many methods have been proposed to explain the decisions of these models. Recent years have also seen concerted efforts that have shown how such explanations can be distorted (attacked) by minor input perturbations. While there have been many surveys that review explainability methods themselves, there has been no effort hitherto to assimilate the different methods and metrics proposed to study the robustness of explanations of DNN models. In this work, we present a comprehensive survey of methods that study, understand, attack, and defend explanations of DNN models. We also present a detailed review of different metrics used to evaluate explanation methods, as well as describe attributional attack and defense methods. We conclude with lessons and take-aways for the community towards ensuring robust explanations of DNN model predictions.
translated by 谷歌翻译
Large-scale diffusion-based generative models have led to breakthroughs in text-conditioned high-resolution image synthesis. Starting from random noise, such text-to-image diffusion models gradually synthesize images in an iterative fashion while conditioning on text prompts. We find that their synthesis behavior qualitatively changes throughout this process: Early in sampling, generation strongly relies on the text prompt to generate text-aligned content, while later, the text conditioning is almost entirely ignored. This suggests that sharing model parameters throughout the entire generation process may not be ideal. Therefore, in contrast to existing works, we propose to train an ensemble of text-to-image diffusion models specialized for different synthesis stages. To maintain training efficiency, we initially train a single model, which is then split into specialized models that are trained for the specific stages of the iterative generation process. Our ensemble of diffusion models, called eDiff-I, results in improved text alignment while maintaining the same inference computation cost and preserving high visual quality, outperforming previous large-scale text-to-image diffusion models on the standard benchmark. In addition, we train our model to exploit a variety of embeddings for conditioning, including the T5 text, CLIP text, and CLIP image embeddings. We show that these different embeddings lead to different behaviors. Notably, the CLIP image embedding allows an intuitive way of transferring the style of a reference image to the target text-to-image output. Lastly, we show a technique that enables eDiff-I's "paint-with-words" capability. A user can select the word in the input text and paint it in a canvas to control the output, which is very handy for crafting the desired image in mind. The project page is available at https://deepimagination.cc/eDiff-I/
translated by 谷歌翻译
在包括搜索在内的各种应用程序中,积极消费数字文档的研究范围为研究范围。传统上,文档中的搜索是作为文本匹配的问题施放的,忽略了结构化文档,表格等中常见的丰富布局和视觉提示。为此,我们提出了一个大多数未探索的问题:“我们可以搜索其他类似的snippets在目标文档页面中存在给定文档摘要的单个查询实例吗?”。我们建议单体将其作为单拍的摘要检测任务解决。单体融合了摘要和文档的视觉,文本和空间方式的上下文,以在目标文档中找到查询片段。我们进行了广泛的消融和实验,显示单体从一击对象检测(BHRL),模板匹配和文档理解(Layoutlmv3)中优于几个基线。由于目前的任务缺乏相关数据,因此我们对单体进行了编程生成的数据训练,该数据具有许多视觉上相似的查询片段和来自两个数据集的目标文档对 - Flamingo表单和PublayNet。我们还进行人类研究以验证生成的数据。
translated by 谷歌翻译
由于肿胀和病态增大,人体组织中组织的异常发育被称为肿瘤。它们主要被归类为良性和恶性。大脑中的肿瘤可能是致命的,因为它可能是癌性的,因此可以以附近的健康细胞为食并不断增加大小。这可能会影响大脑中软组织,神经细胞和小血管。因此,有必要以最高的精度在早期阶段检测和分类。脑肿瘤的大小和位置不同,这使得很难理解其性质。由于附近的健康细胞与肿瘤之间的相似性,即使使用先进的MRI(磁共振成像)技术,脑肿瘤的检测和分类过程也可能是一项繁重的任务。在本文中,我们使用Keras和Tensorflow来实施最先进的卷积神经网络(CNN)架构,例如EdgitionNetB0,Resnet50,Xpection,MobilenetV2和VGG16,使用转移学习来检测和分类三种类型的大脑肿瘤,即神经胶质瘤,脑膜瘤和垂体。我们使用的数据集由3264个2-D磁共振图像和4个类组成。由于数据集的尺寸较小,因此使用各种数据增强技术来增加数据集的大小。我们提出的方法不仅包括数据增强,而且还包括各种图像降级技术,头骨剥离,裁剪和偏置校正。在我们提出的工作效率NETB0体系结构中,最佳准确性为97.61%。本文的目的是区分正常和异常像素,并以更好的准确性对它们进行分类。
translated by 谷歌翻译
建模是什么使广告有说服力的原因,即引起消费者的所需响应,对于宣传,社会心理学和营销的研究至关重要。尽管其重要性,但计算机视觉中说服力的计算建模仍处于起步阶段,这主要是由于缺乏可以提供与ADS相关的说服力标签的基准数据集。由社会心理学和市场营销中的说服文学的激励,我们引入了广泛的说服策略词汇,并建立了用说服策略注释的第一个AD图像语料库。然后,我们通过多模式学习制定说服策略预测的任务,在该任务中,我们设计了一个多任务注意融合模型,该模型可以利用其他广告理解的任务来预测说服策略。此外,我们对30家财富500家公司的1600个广告活动进行了真实的案例研究,我们使用模型的预测来分析哪些策略与不同的人口统计学(年龄和性别)一起使用。该数据集还提供图像分割掩码,该蒙版在测试拆分上标记了相应的AD图像中的说服力策略。我们公开发布代码和数据集https://midas-research.github.io/persuasion-avertisements/。
translated by 谷歌翻译
有效的视觉在延迟预算下的精度最大化。这些作品一次评估脱机准确性,一次是一张图像。但是,诸如自动驾驶之类的实时视觉应用在流媒体设置中运行,在这些设置中,地面真相在推理开始和终点之间会发生变化。这会导致明显的准确性下降。因此,最近提出的一项旨在最大程度地提高流媒体设置准确性的工作。在本文中,我们建议在每个环境环境中最大化流的准确性。我们认为场景难度会影响初始(离线)精度差异,而场景中的障碍物位移会影响后续的准确性降解。我们的方法章鱼使用这些方案属性来选择在测试时最大化流量准确性的配置。我们的方法将跟踪性能(S-MOTA)提高了7.4%,而常规静态方法则提高了。此外,使用我们的方法提高性能,而不是离线准确性的进步,而不是代替而不是进步。
translated by 谷歌翻译
大型基于变压器的预训练的语言模型在各种知识密集的任务上取得了令人印象深刻的表现,并可以在其参数中捕获事实知识。我们认为,考虑到不断增长的知识和资源需求,在模型参数中存储大量知识是亚最佳选择。我们认为,更有效的替代方法是向模型提供对上下文相关的结构化知识的明确访问,并训练它以使用该知识。我们提出了LM核 - 实现这一目标的一般框架 - 允许从外部知识源对语言模型培训的\ textit {解耦},并允许后者更新而不会影响已经训练的模型。实验结果表明,LM核心获得外部知识,在知识探索任务上的最先进的知识增强语言模型中实现了重要而强大的优于性能。可以有效处理知识更新;并在两个下游任务上表现良好。我们还提出了一个彻底的错误分析,突出了LM核的成功和失败。
translated by 谷歌翻译