Modern autonomous driving system is characterized as modular tasks in sequential order, i.e., perception, prediction and planning. As sensors and hardware get improved, there is trending popularity to devise a system that can perform a wide diversity of tasks to fulfill higher-level intelligence. Contemporary approaches resort to either deploying standalone models for individual tasks, or designing a multi-task paradigm with separate heads. These might suffer from accumulative error or negative transfer effect. Instead, we argue that a favorable algorithm framework should be devised and optimized in pursuit of the ultimate goal, i.e. planning of the self-driving-car. Oriented at this goal, we revisit the key components within perception and prediction. We analyze each module and prioritize the tasks hierarchically, such that all these tasks contribute to planning (the goal). To this end, we introduce Unified Autonomous Driving (UniAD), the first comprehensive framework up-to-date that incorporates full-stack driving tasks in one network. It is exquisitely devised to leverage advantages of each module, and provide complementary feature abstractions for agent interaction from a global perspective. Tasks are communicated with unified query design to facilitate each other toward planning. We instantiate UniAD on the challenging nuScenes benchmark. With extensive ablations, the effectiveness of using such a philosophy is proven to surpass previous state-of-the-arts by a large margin in all aspects. The full suite of codebase and models would be available to facilitate future research in the community.

在本报告中，我们在CVPR 2022的Waymo Open数据集挑战中介绍了解决方案和流程预测挑战，该挑战在排行榜上排名第一。我们已经开发了一个新型的层次空间时间网络，该网络具有时空编码器，一个富含潜在变量的多尺度聚合器以及一个递归层次结构3D解码器。我们使用多种损失，包括局灶性损失和修改的流量损失来有效指导训练过程。我们的方法达到了一个占地0.8389的流动占用AUC，并且优于排行榜上所有其他团队。

如最近的研究所示，支持机器智能的系统容易受到对抗性操纵或自然分配变化产生的测试案例的影响。这引起了人们对现实应用程序部署机器学习算法的极大关注，尤其是在自动驾驶（AD）等安全性领域中。另一方面，由于自然主义场景的传统广告测试需要数亿英里，这是由于现实世界中安全关键方案的高度和稀有性。结果，已经探索了几种自动驾驶评估方法，但是，这些方法通常是基于不同的仿真平台，安全性 - 关键的情况的类型，场景生成算法和驾驶路线变化的方法。因此，尽管在自动驾驶测试方面进行了大量努力，但在相似条件下，比较和了解不同测试场景产生算法和测试机制的有效性和效率仍然是一项挑战。在本文中，我们旨在提供第一个统一的平台Safebench，以整合不同类型的安全性测试方案，场景生成算法以及其他变体，例如驾驶路线和环境。同时，我们实施了4种基于深入学习的AD算法，具有4种类型的输入（例如，鸟类视图，相机，相机），以对SafeBench进行公平的比较。我们发现，我们的生成的测试场景确实更具挑战性，并观察到在良性和关键安全测试方案下的广告代理的性能之间的权衡。我们认为，我们的统一平台安全基地用于大规模和有效的自动驾驶测试，将激发新的测试场景生成和安全AD算法的开发。 SafeBench可从https://safebench.github.io获得。

从点云的3D检测中有两条流：单级方法和两级方法。虽然前者更加计算高效，但后者通常提供更好的检测精度。通过仔细检查两级方法，我们发现如果设计，第一阶段可以产生准确的盒子回归。在这种情况下，第二阶段主要重新分配盒子，使得具有更好的本地化的盒子得到选择。从这个观察开始，我们设计了一个可以满足这些要求的单级锚定网络。该网络名为AFDETV2，通过在骨干网中包含一个自校准的卷积块，键盘辅助监控和多任务头中的IOU预测分支来扩展了先前的工作。结果，检测精度在单阶段中大大提升。为了评估我们的方法，我们在Waymo Open DataSet和Nuscenes DataSet上进行了广泛的实验。我们观察到我们的AFDETv2在这两个数据集上实现了最先进的结果，优于所有现有技术，包括单级和两级SE3D探测器。 AFDETv2在Waymo Open DataSet挑战的实时3D检测中获得了第1位的第1位，我们的模型AFDetv2基地的变体题为挑战赞助商的“最有效的模型”，呈现出卓越的计算效率。为了证明这种单级方法的一般性，我们还将其应用于两级网络的第一阶段。毫无例外，结果表明，利用加强的骨干和救护方法，不再需要第二阶段细化。

Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes. While neural networks have achieved competitive performance, the resulting policies are often over-parameterized black boxes that are difficult to interpret and deploy efficiently. More recent symbolic RL frameworks have shown that high-level domain-specific programming logic can be designed to handle both policy learning and symbolic planning. However, these approaches rely on coded primitives with little feature learning, and when applied to high-dimensional visual scenes, they can suffer from scalability issues and perform poorly when images have complex object interactions. To address these challenges, we propose \textit{Differentiable Symbolic Expression Search} (DiffSES), a novel symbolic learning approach that discovers discrete symbolic policies using partially differentiable optimization. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions, while also incorporating the strengths of neural networks for feature learning and optimization. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more and scalable than state-of-the-art symbolic RL methods, with a reduced amount of symbolic prior knowledge.

Denoising Diffusion Probabilistic Models (DDPMs) are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples. However, their iterative refinement process in high-dimensional data space results in slow inference speed, which restricts their application in real-time systems. Previous works have explored speeding up by minimizing the number of inference steps but at the cost of sample quality. In this work, to improve the inference speed for DDPM-based TTS model while achieving high sample quality, we propose ResGrad, a lightweight diffusion model which learns to refine the output spectrogram of an existing TTS model (e.g., FastSpeech 2) by predicting the residual between the model output and the corresponding ground-truth speech. ResGrad has several advantages: 1) Compare with other acceleration methods for DDPM which need to synthesize speech from scratch, ResGrad reduces the complexity of task by changing the generation target from ground-truth mel-spectrogram to the residual, resulting into a more lightweight model and thus a smaller real-time factor. 2) ResGrad is employed in the inference process of the existing TTS model in a plug-and-play way, without re-training this model. We verify ResGrad on the single-speaker dataset LJSpeech and two more challenging datasets with multiple speakers (LibriTTS) and high sampling rate (VCTK). Experimental results show that in comparison with other speed-up methods of DDPMs: 1) ResGrad achieves better sample quality with the same inference speed measured by real-time factor; 2) with similar speech quality, ResGrad synthesizes speech faster than baseline methods by more than 10 times. Audio samples are available at https://resgrad1.github.io/.

Accurate and smooth global navigation satellite system (GNSS) positioning for pedestrians in urban canyons is still a challenge due to the multipath effects and the non-light-of-sight (NLOS) receptions caused by the reflections from surrounding buildings. The recently developed factor graph optimization (FGO) based GNSS positioning method opened a new window for improving urban GNSS positioning by effectively exploiting the measurement redundancy from the historical information to resist the outlier measurements. Unfortunately, the FGO-based GNSS standalone positioning is still challenged in highly urbanized areas. As an extension of the previous FGO-based GNSS positioning method, this paper exploits the potential of the pedestrian dead reckoning (PDR) model in FGO to improve the GNSS standalone positioning performance in urban canyons. Specifically, the relative motion of the pedestrian is estimated based on the raw acceleration measurements from the onboard smartphone inertial measurement unit (IMU) via the PDR algorithm. Then the raw GNSS pseudorange, Doppler measurements, and relative motion from PDR are integrated using the FGO. Given the context of pedestrian navigation with a small acceleration most of the time, a novel soft motion model is proposed to smooth the states involved in the factor graph model. The effectiveness of the proposed method is verified step-by-step through two datasets collected in dense urban canyons of Hong Kong using smartphone-level GNSS receivers. The comparison between the conventional extended Kalman filter, several existing methods, and FGO-based integration is presented. The results reveal that the existing FGO-based GNSS standalone positioning is highly complementary to the PDR's relative motion estimation. Both improved positioning accuracy and trajectory smoothness are obtained with the help of the proposed method.

A lot of theoretical and empirical evidence shows that the flatter local minima tend to improve generalization. Adversarial Weight Perturbation (AWP) is an emerging technique to efficiently and effectively find such minima. In AWP we minimize the loss w.r.t. a bounded worst-case perturbation of the model parameters thereby favoring local minima with a small loss in a neighborhood around them. The benefits of AWP, and more generally the connections between flatness and generalization, have been extensively studied for i.i.d. data such as images. In this paper, we extensively study this phenomenon for graph data. Along the way, we first derive a generalization bound for non-i.i.d. node classification tasks. Then we identify a vanishing-gradient issue with all existing formulations of AWP and we propose a new Weighted Truncated AWP (WT-AWP) to alleviate this issue. We show that regularizing graph neural networks with WT-AWP consistently improves both natural and robust generalization across many different graph learning tasks and models.

This paper presents the TransBoat, a novel omnidirectional unmanned surface vehicle (USV) with a magnetbased docking system for overwater construction with wave disturbances. This is the first such USV that can build overwater structures by transporting modules. The TransBoat incorporates two features designed to reject wave disturbances. First, the TransBoat's expandable body structure can actively transform from a mono-hull into a multi-hull for stabilization in turbulent environments by extending its four outrigger hulls. Second, a real-time nonlinear model predictive control (NMPC) scheme is proposed for all shapes of the TransBoat to enhance its maneuverability and resist disturbance to its movement, based on a nonlinear dynamic model. An experimental approach is proposed to identify the parameters of the dynamic model, and a subsequent trajectory tracking test validates the dynamics, NMPC controller and system mobility. Further, docking experiments identify improved performance in the expanded form of the TransBoat compared with the contracted form, including an increased success rate (of ~ 10%) and reduced docking time (of ~ 40 s on average). Finally, a bridge construction test verifies our system design and the NMPC control method.

Video dubbing aims to translate the original speech in a film or television program into the speech in a target language, which can be achieved with a cascaded system consisting of speech recognition, machine translation and speech synthesis. To ensure the translated speech to be well aligned with the corresponding video, the length/duration of the translated speech should be as close as possible to that of the original speech, which requires strict length control. Previous works usually control the number of words or characters generated by the machine translation model to be similar to the source sentence, without considering the isochronicity of speech as the speech duration of words/characters in different languages varies. In this paper, we propose a machine translation system tailored for the task of video dubbing, which directly considers the speech duration of each token in translation, to match the length of source and target speech. Specifically, we control the speech length of generated sentence by guiding the prediction of each word with the duration information, including the speech duration of itself as well as how much duration is left for the remaining words. We design experiments on four language directions (German -> English, Spanish -> English, Chinese <-> English), and the results show that the proposed method achieves better length control ability on the generated speech than baseline methods. To make up the lack of real-world datasets, we also construct a real-world test set collected from films to provide comprehensive evaluations on the video dubbing task.