Recent CLIP-guided 3D optimization methods, e.g., DreamFields and PureCLIPNeRF achieve great success in zero-shot text-guided 3D synthesis. However, due to the scratch training and random initialization without any prior knowledge, these methods usually fail to generate accurate and faithful 3D structures that conform to the corresponding text. In this paper, we make the first attempt to introduce the explicit 3D shape prior to CLIP-guided 3D optimization methods. Specifically, we first generate a high-quality 3D shape from input texts in the text-to-shape stage as the 3D shape prior. We then utilize it as the initialization of a neural radiance field and then optimize it with the full prompt. For the text-to-shape generation, we present a simple yet effective approach that directly bridges the text and image modalities with a powerful text-to-image diffusion model. To narrow the style domain gap between images synthesized by the text-to-image model and shape renderings used to train the image-to-shape generator, we further propose to jointly optimize a learnable text prompt and fine-tune the text-to-image diffusion model for rendering-style image generation. Our method, namely, Dream3D, is capable of generating imaginative 3D content with better visual quality and shape accuracy than state-of-the-art methods.

Three-dimensional (3D) ultrasound imaging technique has been applied for scoliosis assessment, but current assessment method only uses coronal projection image and cannot illustrate the 3D deformity and vertebra rotation. The vertebra detection is essential to reveal 3D spine information, but the detection task is challenging due to complex data and limited annotations. We propose VertMatch, a two-step framework to detect vertebral structures in 3D ultrasound volume by utilizing unlabeled data in semi-supervised manner. The first step is to detect the possible positions of structures on transverse slice globally, and then the local patches are cropped based on detected positions. The second step is to distinguish whether the patches contain real vertebral structures and screen the predicted positions from the first step. VertMatch develops three novel components for semi-supervised learning: for position detection in the first step, (1) anatomical prior is used to screen pseudo labels generated from confidence threshold method; (2) multi-slice consistency is used to utilize more unlabeled data by inputting multiple adjacent slices; (3) for patch identification in the second step, the categories are rebalanced in each batch to solve imbalance problem. Experimental results demonstrate that VertMatch can detect vertebra accurately in ultrasound volume and outperforms state-of-the-art methods. VertMatch is also validated in clinical application on forty ultrasound scans, and it can be a promising approach for 3D assessment of scoliosis.

Lifelong person re-identification (LReID) is in significant demand for real-world development as a large amount of ReID data is captured from diverse locations over time and cannot be accessed at once inherently. However, a key challenge for LReID is how to incrementally preserve old knowledge and gradually add new capabilities to the system. Unlike most existing LReID methods, which mainly focus on dealing with catastrophic forgetting, our focus is on a more challenging problem, which is, not only trying to reduce the forgetting on old tasks but also aiming to improve the model performance on both new and old tasks during the lifelong learning process. Inspired by the biological process of human cognition where the somatosensory neocortex and the hippocampus work together in memory consolidation, we formulated a model called Knowledge Refreshing and Consolidation (KRC) that achieves both positive forward and backward transfer. More specifically, a knowledge refreshing scheme is incorporated with the knowledge rehearsal mechanism to enable bi-directional knowledge transfer by introducing a dynamic memory model and an adaptive working model. Moreover, a knowledge consolidation scheme operating on the dual space further improves model stability over the long term. Extensive evaluations show KRC's superiority over the state-of-the-art LReID methods on challenging pedestrian benchmarks.

We represent the ResNeRF, a novel geometry-guided two-stage framework for indoor scene novel view synthesis. Be aware of that a good geometry would greatly boost the performance of novel view synthesis, and to avoid the geometry ambiguity issue, we propose to characterize the density distribution of the scene based on a base density estimated from scene geometry and a residual density parameterized by the geometry. In the first stage, we focus on geometry reconstruction based on SDF representation, which would lead to a good geometry surface of the scene and also a sharp density. In the second stage, the residual density is learned based on the SDF learned in the first stage for encoding more details about the appearance. In this way, our method can better learn the density distribution with the geometry prior for high-fidelity novel view synthesis while preserving the 3D structures. Experiments on large-scale indoor scenes with many less-observed and textureless areas show that with the good 3D surface, our method achieves state-of-the-art performance for novel view synthesis.

在规范空间中对人体进行建模是捕捉和动画的常见实践。但是，当涉及神经辐射场（NERF）时，在规范空间中学习静态NERF是不够的，因为即使人体移动时，即使场景照明是恒定的，身体的照明也会变化。以前的方法通过学习人均嵌入来减轻照明的不一致，但是此操作并不能推广到看不见的姿势。鉴于照明条件在世界空间中是静态的，而人体在规范空间中是一致的，我们提出了一个双空间的nerf，该nerf在场景照明和人体中对两个单独空间的两个MLP进行建模。为了弥合这两个空间，以前的方法主要依赖于线性混合剥皮（LBS）算法。但是，动态神经场的LB的混合重量很难棘手，因此通常用另一个MLP记住，这不会推广到新型姿势。尽管可以借用参数网格（例如SMPL）的混合权重，但插值操作会引入更多的伪像。在本文中，我们建议使用Barycentric映射，该映射可以直接概括为看不见的姿势并出奇地取得了比具有神经混合重量的LB的优势。人类36M和ZJU-MOCAP数据集的定量和定性结果显示了我们方法的有效性。

我们提出了联合隐式功能（UNIF），这是一种基于原始扫描和骨骼作为输入的人类重建和动画的零件方法。先前的基于部分的人重建方法依赖于SMPL的地面零件标签，因此仅限于最小衣服。相比之下，我们的方法学会了将部分与身体运动分开，而不是部分监督，因此可以扩展到穿衣服的人类和其他铰接的物体。我们的分区从动作进行分区是通过以骨骼为中心的初始化，骨限度损失和正常损失来实现的，即使训练姿势受到限制，也可以确保稳定的零件分裂。我们还为SDF提供了最小的周边损失，以抑制额外的表面和部分重叠。我们方法的另一个核心是一种相邻的部分接缝算法，该算法会产生非刚性变形，以维持显着缓解基于部分伪像的部分之间的连接。在该算法下，我们进一步提出了“竞争部分”，该方法通过点对骨骼而不是绝对位置的相对位置定义了重量，从而避免了神经隐式函数的概括性问题（线性混合皮肤）。我们通过在CAPE和ClothSeq数据集上穿衣服的人体重建和动画来证明我们方法的有效性。

我们提出了一个小说嵌入字段\ emph {pref}作为促进神经信号建模和重建任务的紧凑表示。基于纯的多层感知器（MLP）神经技术偏向低频信号，并依赖于深层或傅立叶编码以避免丢失细节。取而代之的是，基于傅立叶嵌入空间的相拟合公式，PREF采用了紧凑且物理上解释的编码场。我们进行全面的实验，以证明PERF比最新的空间嵌入技术的优势。然后，我们使用近似的逆傅里叶变换方案以及新型的parseval正常器来开发高效的频率学习框架。广泛的实验表明，我们的高效和紧凑的基于频率的神经信号处理技术与2D图像完成，3D SDF表面回归和5D辐射场现场重建相同，甚至比最新的。

在本文中，我们提出了一种新的序列验证任务，该任务旨在区分从具有阶梯级变换的负面的正视频对，但仍然进行相同的任务。这种具有挑战性的任务驻留在没有先前操作检测或需要事件级别甚至帧级注释的分段的开放式设置。为此，我们仔细重新组成了具有步骤过程任务结构的两个公开的动作相关的数据集。为了充分调查任何方法的有效性，我们收集了统计化学实验中各种步进变换的脚本视频数据集。此外，引入了一种新的评估度量加权距离比以确保评估期间不同的步进级变换等效。最后，基于具有新序列对准损耗的变压器的简单但有效的基线被引入到更好地表征步骤之间的长期依赖性，这优于其他动作识别方法。将发布代码和数据。

在本文中，我们提出了一种添加在生成的对抗网络（GaN）中不可替代的约束的方法（GaN）的任意大小原始拜耳图像生成。理论上，通过使用GaN培训中的转换数据来说，它能够改善原始数据分布的学习，由于在可逆性和可微分的变换下的两个分布之间的Jensen-Shannon（JS）发散。受益于所提出的方法，可以通过将变换配置为Demosaicing来生成原始拜耳图案图像。结果表明，通过添加另一个变换，所提出的方法能够合成具有任意尺寸的高质量未加工拜耳图像。实验结果表明，所提出的方法生成的图像优于FR \'Echet Inception距离（FID）得分中的现有方法，峰值信号到噪声比（PSNR），以及平均结构相似度（MSSIM），训练过程更多稳定的。为了提出作者的最佳知识，未加工拜耳域中没有开源，大型图像数据集，这对于研究工程至关重要，旨在探索计算机视觉任务的图像信号处理（ISP）管道设计。将现有的常用彩色图像数据集转换为相应的博客版本，所提出的方法可以是对原始图像数据集问题的有希望的解决方案。我们还在实验中显示，通过使用合成的原始拜耳图像训练对象检测框架，可以以端到端的方式（从原始图像到视觉任务）使用，具有可忽略的性能下降。

大多数行人轨迹预测方法都取决于大量的轨迹注释，这是耗时且昂贵的。此外，训练有素的模型可能无法有效地推广到另一台相机捕获的新场景。因此，希望将在注释源域上训练的模型调整到目标域。为了实现轨迹预测的域适应性，我们提出了跨域轨迹预测网络（CTP-NET）。在此框架中，在两个域中使用编码器来编码观察到的轨迹，然后它们的特征由跨域特征鉴别器对齐。此外，考虑到观察到的轨迹和预测轨迹之间的一致性，目标域偏移判别器被用来对抗对未来的轨迹预测进行对流规范，以与观察到的轨迹相符。广泛的实验证明了我们方法对行人轨迹预测的域适应性的有效性。