我们描述了我们使用对CAD表示的深度学习来推断机械组件中交配部分之间的自由度的工作。我们使用由CAD零件和配偶将它们组成的大型实际机械组件的大型数据集训练我们的模型。我们提出了重新定义这些伴侣的方法，以使它们更好地反映组件的运动，并缩小可能的运动轴。我们还进行了一项用户研究，以创建具有更可靠标签的运动声音测试集。

我们提出了一种框架插值算法，该算法从两个输入图像中综合了具有大型内部运动的两个输入图像。最近的方法使用多个网络来估计光流或深度以及专用于框架合成的单独网络。这通常是复杂的，需要稀缺的光流或深度地面真相。在这项工作中，我们提出了一个单一的统一网络，该网络以多尺度的特征提取器为特色，该特征提取器在各个尺度上共享权重，并且可以单独从框架中训练。为了综合酥脆和令人愉悦的框架，我们建议使用革兰氏矩阵损失来优化我们的网络，从而衡量特征地图之间的相关差异。我们的方法优于XIPH大型运动基准的最先进方法。与使用感知损失的方法相比，我们还可以在Vimeo-90K，Middlebury和UCF101上获得更高的分数。我们研究了体重共享和培训的效果，该数据集的运动范围不断增加。最后，我们证明了模型在综合高质量和临时连贯的视频中的有效性，该视频在具有挑战性的近乎修复的照片数据集中。代码和预训练模型可在https://film-net.github.io上找到。

我们介绍了一个自由视的渲染方法 - Humannerf - 这对人类进行了复杂的身体运动的给定单曲视频工作，例如，来自YouTube的视频。我们的方法可以在任何帧中暂停视频，并从任意新相机视点呈现对象，甚至是该特定帧和身体姿势的完整360度摄像机路径。这项任务特别具有挑战性，因为它需要合成身体的光电型细节，如从输入视频中可能不存在的各种相机角度所见，以及合成布折叠和面部外观的细细节。我们的方法优化了在规范T型姿势中的人的体积表示，同时通过运动场，该运动场通过向后的警报将估计的规范表示映射到视频的每个帧。运动场分解成骨骼刚性和非刚性运动，由深网络产生。我们对现有工作显示出显着的性能改进，以及从移动人类的单眼视频的令人尖锐的观点渲染的阐释示例，以挑战不受控制的捕获场景。

每次坐在电视或监视器前，都会通过时变的光模式积极照亮。本文建议使用这种时断照明，以与任何新的照明条件进行脸部的合成复兴。在这样做时，我们从Debevec等人的轻型阶段工作中获取灵感。谁首先展示了在受控照明环境中捕获的人的能力。然而，现有的光级需要昂贵的房间级球形捕获龙门，并且在世界上只有一些实验室存在，我们演示了如何从普通电视或台式机监视器获取有用的数据。我们不对用户感到不舒服的快速闪烁光图案，而不是在用户观看YouTube视频或其他标准内容的用户的图像上运行。我们在图像上培训一个深网络以及给定用户的监视器模式，并学会在任何目标照明（监视器模式）下预测该用户的图像。实验评估表明，我们的方法产生了现实的发感结果。视频结果可在http://grail.cs.washington.edu/projects/light_stage_on_every_desk/上获得。

Remote sensing imagery provides comprehensive views of the Earth, where different sensors collect complementary data at different spatial scales. Large, pretrained models are commonly finetuned with imagery that is heavily augmented to mimic different conditions and scales, with the resulting models used for various tasks with imagery from a range of spatial scales. Such models overlook scale-specific information in the data. In this paper, we present Scale-MAE, a pretraining method that explicitly learns relationships between data at different, known scales throughout the pretraining process. Scale-MAE pretrains a network by masking an input image at a known input scale, where the area of the Earth covered by the image determines the scale of the ViT positional encoding, not the image resolution. Scale-MAE encodes the masked image with a standard ViT backbone, and then decodes the masked image through a bandpass filter to reconstruct low/high frequency images at lower/higher scales. We find that tasking the network with reconstructing both low/high frequency images leads to robust multiscale representations for remote sensing imagery. Scale-MAE achieves an average of a $5.0\%$ non-parametric kNN classification improvement across eight remote sensing datasets compared to current state-of-the-art and obtains a $0.9$ mIoU to $3.8$ mIoU improvement on the SpaceNet building segmentation transfer task for a range of evaluation scales.

Traditional screening practices for anxiety and depression pose an impediment to monitoring and treating these conditions effectively. However, recent advances in NLP and speech modelling allow textual, acoustic, and hand-crafted language-based features to jointly form the basis of future mental health screening and condition detection. Speech is a rich and readily available source of insight into an individual's cognitive state and by leveraging different aspects of speech, we can develop new digital biomarkers for depression and anxiety. To this end, we propose a multi-modal system for the screening of depression and anxiety from self-administered speech tasks. The proposed model integrates deep-learned features from audio and text, as well as hand-crafted features that are informed by clinically-validated domain knowledge. We find that augmenting hand-crafted features with deep-learned features improves our overall classification F1 score comparing to a baseline of hand-crafted features alone from 0.58 to 0.63 for depression and from 0.54 to 0.57 for anxiety. The findings of our work suggest that speech-based biomarkers for depression and anxiety hold significant promise in the future of digital health.

This paper addresses the kinodynamic motion planning for non-holonomic robots in dynamic environments with both static and dynamic obstacles -- a challenging problem that lacks a universal solution yet. One of the promising approaches to solve it is decomposing the problem into the smaller sub problems and combining the local solutions into the global one. The crux of any planning method for non-holonomic robots is the generation of motion primitives that generates solutions to local planning sub-problems. In this work we introduce a novel learnable steering function (policy), which takes into account kinodynamic constraints of the robot and both static and dynamic obstacles. This policy is efficiently trained via the policy optimization. Empirically, we show that our steering function generalizes well to unseen problems. We then plug in the trained policy into the sampling-based and lattice-based planners, and evaluate the resultant POLAMP algorithm (Policy Optimization that Learns Adaptive Motion Primitives) in a range of challenging setups that involve a car-like robot operating in the obstacle-rich parking-lot environments. We show that POLAMP is able to plan collision-free kinodynamic trajectories with success rates higher than 92%, when 50 simultaneously moving obstacles populate the environment showing better performance than the state-of-the-art competitors.

Most deep-learning-based continuous sign language recognition (CSLR) models share a similar backbone consisting of a visual module, a sequential module, and an alignment module. However, due to limited training samples, a connectionist temporal classification loss may not train such CSLR backbones sufficiently. In this work, we propose three auxiliary tasks to enhance the CSLR backbones. The first task enhances the visual module, which is sensitive to the insufficient training problem, from the perspective of consistency. Specifically, since the information of sign languages is mainly included in signers' facial expressions and hand movements, a keypoint-guided spatial attention module is developed to enforce the visual module to focus on informative regions, i.e., spatial attention consistency. Second, noticing that both the output features of the visual and sequential modules represent the same sentence, to better exploit the backbone's power, a sentence embedding consistency constraint is imposed between the visual and sequential modules to enhance the representation power of both features. We name the CSLR model trained with the above auxiliary tasks as consistency-enhanced CSLR, which performs well on signer-dependent datasets in which all signers appear during both training and testing. To make it more robust for the signer-independent setting, a signer removal module based on feature disentanglement is further proposed to remove signer information from the backbone. Extensive ablation studies are conducted to validate the effectiveness of these auxiliary tasks. More remarkably, with a transformer-based backbone, our model achieves state-of-the-art or competitive performance on five benchmarks, PHOENIX-2014, PHOENIX-2014-T, PHOENIX-2014-SI, CSL, and CSL-Daily.

The xView2 competition and xBD dataset spurred significant advancements in overhead building damage detection, but the competition's pixel level scoring can lead to reduced solution performance in areas with tight clusters of buildings or uninformative context. We seek to advance automatic building damage assessment for disaster relief by proposing an auxiliary challenge to the original xView2 competition. This new challenge involves a new dataset and metrics indicating solution performance when damage is more local and limited than in xBD. Our challenge measures a network's ability to identify individual buildings and their damage level without excessive reliance on the buildings' surroundings. Methods that succeed on this challenge will provide more fine-grained, precise damage information than original xView2 solutions. The best-performing xView2 networks' performances dropped noticeably in our new limited/local damage detection task. The common causes of failure observed are that (1) building objects and their classifications are not separated well, and (2) when they are, the classification is strongly biased by surrounding buildings and other damage context. Thus, we release our augmented version of the dataset with additional object-level scoring metrics https://gitlab.kitware.com/dennis.melamed/xfbd to test independence and separability of building objects, alongside the pixel-level performance metrics of the original competition. We also experiment with new baseline models which improve independence and separability of building damage predictions. Our results indicate that building damage detection is not a fully-solved problem, and we invite others to use and build on our dataset augmentations and metrics.

We investigate how humans perform the task of dubbing video content from one language into another, leveraging a novel corpus of 319.57 hours of video from 54 professionally produced titles. This is the first such large-scale study we are aware of. The results challenge a number of assumptions commonly made in both qualitative literature on human dubbing and machine-learning literature on automatic dubbing, arguing for the importance of vocal naturalness and translation quality over commonly emphasized isometric (character length) and lip-sync constraints, and for a more qualified view of the importance of isochronic (timing) constraints. We also find substantial influence of the source-side audio on human dubs through channels other than the words of the translation, pointing to the need for research on ways to preserve speech characteristics, as well as semantic transfer such as emphasis/emotion, in automatic dubbing systems.