在视频中利用时空冗余的自适应抽样对于在有限的计算机和电池资源的可穿戴设备上始终进行动作识别至关重要。常用的固定采样策略不是上下文感知的,并且可能会在视觉内容下进行样本,从而对计算效率和准确性产生不利影响。受到人类视觉感知机制的动脉视觉和动力前处理的概念的启发,我们引入了一种新型的自适应时空抽样方案,以进行有效的动作识别。我们的系统以低分辨率为扫描前扫视全球场景上下文,并决定跳过或要求在显着区域的高分辨率功能进行进一步处理。我们在Epic-Kitchens和UCF-101数据集上验证该系统以进行动作识别,并表明我们所提出的方法可以大大加快与最先进基线相比的准确性丧失的推断。
translated by 谷歌翻译
在过去的两年中,从2020年到2021年,Covid-19在包括越南在内的许多国家 /地区都破坏了预防疾病措施,并对人类生活和社会社区的各个方面产生了负面影响。此外,社区中的误导性信息和有关大流行的虚假新闻也是严重的情况。因此,我们提出了第一个基于越南社区的问题答复数据集,用于开发COVID-19的问题答案系统,称为UIT-VICOV19QA。该数据集包括从可信赖的医疗来源收集的4,500对提问,至少有一个答案,每个问题最多有四个独特的解释答案。除数据集外,我们还建立了各种深度学习模型作为基线,以评估数据集的质量,并通过BLEU,Meteor和Rouge-l等常用指标来进一步研究基准结果,以进行进一步的研究。我们还说明了对这些模型进行多个解释答案的积极影响,尤其是在变压器上 - 研究领域的主要结构。
translated by 谷歌翻译
ICECUBE是一种用于检测1 GEV和1 PEV之间大气和天体中微子的光学传感器的立方公斤阵列,该阵列已部署1.45 km至2.45 km的南极的冰盖表面以下1.45 km至2.45 km。来自ICE探测器的事件的分类和重建在ICeCube数据分析中起着核心作用。重建和分类事件是一个挑战,这是由于探测器的几何形状,不均匀的散射和冰中光的吸收,并且低于100 GEV的光,每个事件产生的信号光子数量相对较少。为了应对这一挑战,可以将ICECUBE事件表示为点云图形,并将图形神经网络(GNN)作为分类和重建方法。 GNN能够将中微子事件与宇宙射线背景区分开,对不同的中微子事件类型进行分类,并重建沉积的能量,方向和相互作用顶点。基于仿真,我们提供了1-100 GEV能量范围的比较与当前ICECUBE分析中使用的当前最新最大似然技术,包括已知系统不确定性的影响。对于中微子事件分类,与当前的IceCube方法相比,GNN以固定的假阳性速率(FPR)提高了信号效率的18%。另外,GNN在固定信号效率下将FPR的降低超过8(低于半百分比)。对于能源,方向和相互作用顶点的重建,与当前最大似然技术相比,分辨率平均提高了13%-20%。当在GPU上运行时,GNN能够以几乎是2.7 kHz的中位数ICECUBE触发速率的速率处理ICECUBE事件,这打开了在在线搜索瞬态事件中使用低能量中微子的可能性。
translated by 谷歌翻译
来自RGB视频的多人姿势理解包括三个复杂的任务:姿势估计,跟踪和运动预测。在这三个任务中,姿势估计和跟踪是相关的,跟踪对于运动预测至关重要。大多数现有作品要么专注于单个任务,要么采用级联方法来分别解决每个任务。在本文中,我们提出了狙击手,这是一个框架,以同时进行单个推断,同时进行多人3D姿势估计,跟踪和运动预测。具体而言,我们首先提出了一种可变形的注意机制,以从视频片段中汇总时空信息。基于这种可变形的注意力,学会了视觉变压器来编码从多框架图像中的时空特征,并解码信息性姿势功能以更新多人姿势查询。最后,对这些查询进行了回归,以预测一个正向传球中的多人姿势轨迹和未来动作。在实验中,我们显示了狙击手对三个具有挑战性的公共数据集的有效性,在该数据集中,通用模型竞争对手专门的姿势估计,跟踪和预测的最先进基线。代码可在\ href {https://github.com/jimmyzou/snipper} {https://github.com/jimmyzou/snipper}中获得。
translated by 谷歌翻译
基于坐标的体积表示有可能从图像中生成光真实的虚拟化身。但是,即使是可能未观察到的新姿势,虚拟化身也需要控制。传统技术(例如LBS)提供了这样的功能;但是,通常需要手工设计的车身模板,3D扫描数据和有限的外观模型。另一方面,神经表示在表示视觉细节方面具有强大的作用,但在变形的动态铰接式参与者方面受到了探索。在本文中,我们提出了TAVA,这是一种基于神经表示形式创建无象光动画体积参与者的方法。我们仅依靠多视图数据和跟踪的骨骼来创建演员的体积模型,该模型可以在给定的新颖姿势的测试时间中进行动画。由于塔瓦不需要身体模板,因此它适用于人类以及其他动物(例如动物)。此外,Tava的设计使其可以恢复准确的密集对应关系,从而使其适合于内容创建和编辑任务。通过广泛的实验,我们证明了所提出的方法可以很好地推广到新颖的姿势以及看不见的观点和展示基本的编辑功能。
translated by 谷歌翻译
铰接式3D形状重建的事先工作通常依赖于专用传感器(例如,同步的多摄像机系统)或预先构建的3D可变形模型(例如,Smal或SMPL)。这些方法无法在野外扩展到不同的各种物体。我们呈现Banmo,这是一种需要专用传感器的方法,也不需要预定义的模板形状。 Banmo在可怜的渲染框架中从许多单眼休闲视频中建立高保真,铰接式的3D模型(包括形状和动画皮肤的重量)。虽然许多视频的使用提供了更多的相机视图和对象关节的覆盖范围,但它们在建立不同背景,照明条件等方面建立了重大挑战。我们的主要洞察力是合并三所思想学校; (1)使用铰接骨骼和混合皮肤的经典可变形形状模型,(2)可容纳基于梯度的优化,(3)在像素之间产生对应关系的规范嵌入物模型。我们介绍了神经混合皮肤模型,可允许可微分和可逆的铰接变形。与规范嵌入式结合时,这些模型允许我们在跨越可通过循环一致性自我监督的视频中建立密集的对应。在真实和合成的数据集上,Banmo显示比人类和动物的先前工作更高保真3D重建,具有从新颖的观点和姿势的现实图像。项目网页:Banmo-www.github.io。
translated by 谷歌翻译
Accurate determination of a small molecule candidate (ligand) binding pose in its target protein pocket is important for computer-aided drug discovery. Typical rigid-body docking methods ignore the pocket flexibility of protein, while the more accurate pose generation using molecular dynamics is hindered by slow protein dynamics. We develop a tiered tensor transform (3T) algorithm to rapidly generate diverse protein-ligand complex conformations for both pose and affinity estimation in drug screening, requiring neither machine learning training nor lengthy dynamics computation, while maintaining both coarse-grain-like coordinated protein dynamics and atomistic-level details of the complex pocket. The 3T conformation structures we generate are closer to experimental co-crystal structures than those generated by docking software, and more importantly achieve significantly higher accuracy in active ligand classification than traditional ensemble docking using hundreds of experimental protein conformations. 3T structure transformation is decoupled from the system physics, making future usage in other computational scientific domains possible.
translated by 谷歌翻译
Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations. Existing approaches to posterior collapse often attribute it to the use of neural networks or optimization issues due to variational approximation. In this paper, we consider posterior collapse as a problem of latent variable non-identifiability. We prove that the posterior collapses if and only if the latent variables are non-identifiable in the generative model. This fact implies that posterior collapse is not a phenomenon specific to the use of flexible distributions or approximate inference. Rather, it can occur in classical probabilistic models even with exact inference, which we also demonstrate. Based on these results, we propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. This model class resolves the problem of latent variable non-identifiability by leveraging bijective Brenier maps and parameterizing them with input convex neural networks, without special variational inference objectives or optimization tricks. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.
translated by 谷歌翻译
We propose a new causal inference framework to learn causal effects from multiple, decentralized data sources in a federated setting. We introduce an adaptive transfer algorithm that learns the similarities among the data sources by utilizing Random Fourier Features to disentangle the loss function into multiple components, each of which is associated with a data source. The data sources may have different distributions; the causal effects are independently and systematically incorporated. The proposed method estimates the similarities among the sources through transfer coefficients, and hence requiring no prior information about the similarity measures. The heterogeneous causal effects can be estimated with no sharing of the raw training data among the sources, thus minimizing the risk of privacy leak. We also provide minimax lower bounds to assess the quality of the parameters learned from the disparate sources. The proposed method is empirically shown to outperform the baselines on decentralized data sources with dissimilar distributions.
translated by 谷歌翻译
Differentiable Architecture Search (DARTS) has attracted considerable attention as a gradient-based Neural Architecture Search (NAS) method. Since the introduction of DARTS, there has been little work done on adapting the action space based on state-of-art architecture design principles for CNNs. In this work, we aim to address this gap by incrementally augmenting the DARTS search space with micro-design changes inspired by ConvNeXt and studying the trade-off between accuracy, evaluation layer count, and computational cost. To this end, we introduce the Pseudo-Inverted Bottleneck conv block intending to reduce the computational footprint of the inverted bottleneck block proposed in ConvNeXt. Our proposed architecture is much less sensitive to evaluation layer count and outperforms a DARTS network with similar size significantly, at layer counts as small as 2. Furthermore, with less layers, not only does it achieve higher accuracy with lower GMACs and parameter count, GradCAM comparisons show that our network is able to better detect distinctive features of target objects compared to DARTS.
translated by 谷歌翻译