估计3D人类姿势已被证明是一个具有挑战性的任务，主要是因为人体关节，闭塞和照明条件的可变性的复杂性。在本文中，我们介绍了一个高阶图卷积框架，具有初始剩余连接，用于2D-3D姿势估计。使用多跳邻域进行节点特征聚合，我们的模型能够捕获身体关节之间的远程依赖性。此外，我们的方法利用了通过设计中的设计集成的剩余连接，确保学习的特征表示从输入层的初始特征中保留重要信息，因为网络深度增加。在两个标准基准上进行的实验和消融研究表明了我们模型的有效性，实现了对3D人类姿态估算的强大基线方法的卓越性能。

3D human pose estimation is a difficult task, due to challenges such as occluded body parts and ambiguous poses. Graph convolutional networks encode the structural information of the human skeleton in the form of an adjacency matrix, which is beneficial for better pose prediction. We propose one such graph convolutional network named PoseGraphNet for 3D human pose regression from 2D poses. Our network uses an adaptive adjacency matrix and kernels specific to neighbor groups. We evaluate our model on the Human3.6M dataset which is a standard dataset for 3D pose estimation. Our model's performance is close to the state-of-the-art, but with much fewer parameters. The model learns interesting adjacency relations between joints that have no physical connections, but are behaviorally similar.

最近的2D-3D人类姿势估计工作倾向于利用人体骨架的拓扑形成的图形结构。但是，我们认为这种骨架拓扑太稀疏，无法反映身体结构并遭受严重的2D-3D模糊问题。为了克服这些弱点，我们提出了一种新颖的图表卷积网络架构，层次图形网络（HGN）。它基于我们的多尺度图结构建筑策略产生的密度图形拓扑，从而提供更精细的几何信息。所提出的架构包含三个并行组织的稀疏微小表示子网，其中通过新颖的特征融合策略处理多尺度图形结构特征，并通过新颖的特征融合策略进行交换信息，导致丰富的分层表示。我们还介绍了3D粗网格约束，以进一步提高与细节相关的特征学习。广泛的实验表明，我们的HGN通过减少的网络参数实现了最先进的性能

现代的多层感知器（MLP）模型在不自我注意力的情况下学习视觉表现方面显示了竞争成果。但是，现有的MLP模型不擅长捕获本地细节，并且缺乏人类配置的先验知识，这限制了其骨骼表示学习的模型能力。为了解决这些问题，我们提出了一个名为GraphMLP的简单而有效的图形增强的MLP样结构，该体系结构将MLP和图形卷积网络（GCN）组合在3D人类姿势估计的全球 - 局部 - 单位图形统一体系中。GraphMLP将人体的图结构结合到MLP模型中，以满足域特异性需求，同时允许局部和全局空间相互作用。广泛的实验表明，所提出的GraphMLP在两个数据集（即Human3.6M和MPI-INF-3DHP）上实现了最先进的性能。我们的源代码和预估计的模型将公开可用。

Spatial-temporal graphs have been widely used by skeleton-based action recognition algorithms to model human action dynamics. To capture robust movement patterns from these graphs, long-range and multi-scale context aggregation and spatial-temporal dependency modeling are critical aspects of a powerful feature extractor. However, existing methods have limitations in achieving (1) unbiased long-range joint relationship modeling under multiscale operators and (2) unobstructed cross-spacetime information flow for capturing complex spatial-temporal dependencies. In this work, we present (1) a simple method to disentangle multi-scale graph convolutions and (2) a unified spatial-temporal graph convolutional operator named G3D. The proposed multi-scale aggregation scheme disentangles the importance of nodes in different neighborhoods for effective long-range modeling. The proposed G3D module leverages dense cross-spacetime edges as skip connections for direct information propagation across the spatial-temporal graph. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model 1 outperforms previous state-of-the-art methods on three large-scale datasets: NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400.

我们为图形结构数据（名为Kog-Transformer）和一个名为GASE-NET的3D姿势对形状估计网络提出了一个新颖的基于注意力的2到3D姿势估计网络，并提出了一个名为KOG-Transformer的数据。先前的3D姿势估计方法集中在对图卷积内核的各种修改上，例如放弃重量共享或增加接受场。其中一些方法采用基于注意力的非本地模块作为辅助模块。为了更好地模拟图形结构数据中的节点之间的关系并以差异化的方式融合不同邻居节点的信息，我们对注意模块进行了针对性的修改，并提出了设计用于图形结构数据的两个模块，图形相对位置编码多头自我注意事项（GR-MSA）和K级面向图形的多头自我注意力（KOG-MSA）。通过堆叠GR-MSA和KOG-MSA，我们提出了一个新型的网络KOG转换器，以进行2到3D姿势估计。此外，我们提出了一个在手数据上进行形状估计的网络，称为Graistention形状估计网络（GASE-NET），该网络以3D姿势为输入，并逐渐将手的形状从稀疏到密集建模。我们通过广泛的实验从经验上证明了KOG转化器的优势。实验结果表明，KOG转换器在基准数据集Human36M上的先前最新方法显着优于先前的最新方法。我们评估了GASE-NET对两个公共可用手数据集的影响Obman和Interhand 2.6M。 GASE-NET可以预测具有强泛化能力的输入姿势的相应形状。

Thanks to the development of 2D keypoint detectors, monocular 3D human pose estimation (HPE) via 2D-to-3D uplifting approaches have achieved remarkable improvements. Still, monocular 3D HPE is a challenging problem due to the inherent depth ambiguities and occlusions. To handle this problem, many previous works exploit temporal information to mitigate such difficulties. However, there are many real-world applications where frame sequences are not accessible. This paper focuses on reconstructing a 3D pose from a single 2D keypoint detection. Rather than exploiting temporal information, we alleviate the depth ambiguity by generating multiple 3D pose candidates which can be mapped to an identical 2D keypoint. We build a novel diffusion-based framework to effectively sample diverse 3D poses from an off-the-shelf 2D detector. By considering the correlation between human joints by replacing the conventional denoising U-Net with graph convolutional network, our approach accomplishes further performance improvements. We evaluate our method on the widely adopted Human3.6M and HumanEva-I datasets. Comprehensive experiments are conducted to prove the efficacy of the proposed method, and they confirm that our model outperforms state-of-the-art multi-hypothesis 3D HPE methods.

估计来自图像的3D人形和姿势的能力在许多环境中都可以是有用的。最近的方法探索了使用图形卷积网络并取得了有希望的结果。 3D形状由网格表示的事实是一个无向图形，使得图形卷积网络自然适合该问题。但是，图形卷积网络具有有限的表示功率。从图中的节点中的信息传递给连接的邻居，并且信息的传播需要连续的图形卷积。为了克服这种限制，我们提出了一种双尺度图形方法。我们使用从密集图中衍生的粗糙图来估计人类的3D姿势，以及密集图来估计3D形状。与密集图相比，粗糙图中的信息可以在更长的距离上传播。此外，有关姿势的信息可以指导恢复本地形状细节，反之亦然。我们认识到，粗糙和密集之间的连接本身是图形，并引入图形融合块以在具有不同尺度之间的图形之间交换信息。我们培训我们的模型端到端，并表明我们可以为几个评估数据集实现最先进的结果。

尽管来自视频的3D人类姿势估算的巨大进展，但是充分利用冗余2D姿势序列来学习用于生成一个3D姿势的代表表示的开放问题。为此，我们提出了一种改进的基于变压器的架构，称为冲压变压器，简单地有效地将长期的2D联合位置升高到单个3D姿势。具体地，采用Vanilla变压器编码器（VTE）来模拟2D姿势序列的远程依赖性。为了减少序列的冗余，vte的前馈网络中的完全连接的层被冲击卷积替换，以逐步缩小序列长度并从本地上下文聚合信息。修改的VTE称为STRIVEIVERCHER ENCODER（STE），其构建在VTE的输出时。 STE不仅有效地将远程信息聚集到分层全球和本地时尚的单载体表示，而且显着降低了计算成本。此外，全序列和单个目标帧尺度都设计了全序，分别适用于VTE和ST的输出。该方案与单个目标帧监督结合施加额外的时间平滑度约束，因此有助于产生更平滑和更准确的3D姿势。所提出的轮廓变压器在两个具有挑战性的基准数据集，Human3.6M和HumanVa-I中进行评估，并通过更少的参数实现最先进的结果。代码和模型可用于\ url {https://github.com/vegetebird/stridedtransformer-pose3d}。

估计单眼视频的3D人类姿势是由于深度模糊和自动阻塞的具有挑战性的任务。大多数现有的作品试图通过利用空间和时间关系来解决这两个问题。然而，这些作品忽略了它是存在多种可行解决方案（即假设）的逆问题。为了减轻这种限制，我们提出了一种多假设变压器（MHFormer），其学习多个合理的姿势假设的时空表示。为了有效地模拟多假设依赖性并构建跨假设特征的强烈关系，任务分解为三个阶段：（i）生成多个初始假设表示; （ii）模型自立通信，将多个假设合并到单个融合表示中，然后将其分组成几个分歧假设; （iii）学习横向假设通信并汇总多假设特征以合成最终的3D姿势。通过上述过程，最终表示增强，合成的姿势更准确。广泛的实验表明，MHFORMER在两个具有挑战性的数据集上实现最先进的结果：Humanet3.6M和MPI-INF-3DHP。没有钟声和吹口哨，其性能超过了以人3.6M的大幅度为3％的最佳结果。代码和模型可在https://github.com/vegetebird/mhformer中找到。

单眼3D人姿势估计技术有可能大大增加人类运动数据的可用性。单位图2D-3D提升使用图卷积网络（GCN）的表现最佳模型，通常需要一些手动输入来定义不同的身体关节之间的关系。我们提出了一种基于变压器的新型方法，该方法使用更广泛的自我发场机制来学习代表关节的代币序列。我们发现，使用中间监督以及堆叠编码器福利性能之间的剩余连接。我们还建议，将错误预测作为多任务学习框架的一部分，可以通过允许网络弥补其置信度来改善性能。我们进行广泛的消融研究，以表明我们的每项贡献都会提高性能。此外，我们表明我们的方法的表现超过了最新的单帧3D人类姿势估计的最新技术状态。我们的代码和训练有素的模型可在GitHub上公开提供。

Following the success of deep convolutional networks, state-of-the-art methods for 3d human pose estimation have focused on deep end-to-end systems that predict 3d joint locations given raw image pixels. Despite their excellent performance, it is often not easy to understand whether their remaining error stems from a limited 2d pose (visual) understanding, or from a failure to map 2d poses into 3dimensional positions.With the goal of understanding these sources of error, we set out to build a system that given 2d joint locations predicts 3d positions. Much to our surprise, we have found that, with current technology, "lifting" ground truth 2d joint locations to 3d space is a task that can be solved with a remarkably low error rate: a relatively simple deep feedforward network outperforms the best reported result by about 30% on Human3.6M, the largest publicly available 3d pose estimation benchmark. Furthermore, training our system on the output of an off-the-shelf state-of-the-art 2d detector (i.e., using images as input) yields state of the art results -this includes an array of systems that have been trained end-to-end specifically for this task. Our results indicate that a large portion of the error of modern deep 3d pose estimation systems stems from their visual analysis, and suggests directions to further advance the state of the art in 3d human pose estimation.

Action recognition with skeleton data has recently attracted much attention in computer vision. Previous studies are mostly based on fixed skeleton graphs, only capturing local physical dependencies among joints, which may miss implicit joint correlations. To capture richer dependencies, we introduce an encoder-decoder structure, called A-link inference module, to capture action-specific latent dependencies, i.e. actional links, directly from actions. We also extend the existing skeleton graphs to represent higherorder dependencies, i.e. structural links. Combing the two types of links into a generalized skeleton graph, we further propose the actional-structural graph convolution network (AS-GCN), which stacks actional-structural graph convolution and temporal convolution as a basic building block, to learn both spatial and temporal features for action recognition. A future pose prediction head is added in parallel to the recognition head to help capture more detailed action patterns through self-supervision. We validate AS-GCN in action recognition using two skeleton data sets, NTU-RGB+D and Kinetics. The proposed AS-GCN achieves consistently large improvement compared to the state-of-the-art methods. As a side product, AS-GCN also shows promising results for future pose prediction. Our code is available at https://github.com/limaosen0/AS-GCN . 1

基于深度学习的人网格重建方法具有构建更大网络的趋势，以实现更高的准确性。尽管是人网格重建模型的实际使用的关键特征，但往往忽略了计算复杂性和模型大小（例如，虚拟试用系统）。在本文中，我们呈现GTR，这是一种基于轻量级的姿势的方法，可以从2D人类姿势重建人网。我们提出了一种姿势分析模块，它使用曲线图形是利用结构化和隐式的关节相关性，以及将提取的姿势特征与网格模板组合以重建最终人体网格的网格回归模块。我们通过对人类3.6M和3DPW数据集进行广泛的评估，展示了GTR的效率和泛化。特别是，GTRS比SOTA姿势的方法POSE2MESH实现了更好的精度，同时仅使用10.2％的参数（PARAMS）和2.5％的跨越式3DPW数据集。代码将公开。

Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Recently, GCNs and subsequent variants have shown superior performance in various application areas on real-world datasets. Despite their success, most of the current GCN models are shallow, due to the over-smoothing problem.In this paper, we study the problem of designing and analyzing deep graph convolutional networks. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques: Initial residual and Identity mapping. We provide theoretical and empirical evidence that the two techniques effectively relieves the problem of over-smoothing. Our experiments show that the deep GCNII model outperforms the state-of-the-art methods on various semi-and fullsupervised tasks. Code is available at https: //github.com/chennnM/GCNII.

In this work, we demonstrate that 3D poses in video can be effectively estimated with a fully convolutional model based on dilated temporal convolutions over 2D keypoints. We also introduce back-projection, a simple and effective semi-supervised training method that leverages unlabeled video data. We start with predicted 2D keypoints for unlabeled video, then estimate 3D poses and finally back-project to the input 2D keypoints. In the supervised setting, our fully-convolutional model outperforms the previous best result from the literature by 6 mm mean per-joint position error on Human3.6M, corresponding to an error reduction of 11%, and the model also shows significant improvements on HumanEva-I. Moreover, experiments with back-projection show that it comfortably outperforms previous state-of-the-art results in semisupervised settings where labeled data is scarce. Code and models are available at https://github.com/ facebookresearch/VideoPose3D

3D hand pose estimation from RGB images suffers from the difficulty of obtaining the depth information. Therefore, a great deal of attention has been spent on estimating 3D hand pose from 2D hand joints. In this paper, we leverage the advantage of spatial-temporal Graph Convolutional Neural Networks and propose LG-Hand, a powerful method for 3D hand pose estimation. Our method incorporates both spatial and temporal dependencies into a single process. We argue that kinematic information plays an important role, contributing to the performance of 3D hand pose estimation. We thereby introduce two new objective functions, Angle and Direction loss, to take the hand structure into account. While Angle loss covers locally kinematic information, Direction loss handles globally kinematic one. Our LG-Hand achieves promising results on the First-Person Hand Action Benchmark (FPHAB) dataset. We also perform an ablation study to show the efficacy of the two proposed objective functions.

Existing popular methods for semi-supervised learning with Graph Neural Networks (such as the Graph Convolutional Network) provably cannot learn a general class of neighborhood mixing relationships. To address this weakness, we propose a new model, MixHop, that can learn these relationships, including difference operators, by repeatedly mixing feature representations of neighbors at various distances. MixHop requires no additional memory or computational complexity, and outperforms on challenging baselines. In addition, we propose sparsity regularization that allows us to visualize how the network prioritizes neighborhood information across different graph datasets. Our analysis of the learned architectures reveals that neighborhood mixing varies per datasets. 1 We use "like", as graph edges are not axis-aligned.

人类运动预测是许多计算机视觉应用领域中的重要且挑战性的任务。最近的工作专注于利用经常性神经网络（RNN）的定时处理能力，实现短期预测的光滑且可靠的结果。但是，正如以前的工作所证明的那样，RNNS遭受错误累积，导致结果不可靠。在本文中，我们提出了一种简单的前馈深神经网络，用于运动预测，这考虑了人体关节之间的时间平滑度和空间依赖性。我们设计了一个多尺度的时空图卷积网络（GCNS），以隐式地建立人类运动过程中的时空依赖，其中在训练期间动态融合的不同尺度。整个模型适用于所有操作，然后遵循编码器解码器的框架。编码器由时间GCN组成，用于捕获帧和半自主学习空间GCN之间的运动特征，以提取关节轨迹之间的空间结构。解码器使用时间卷积网络（TCN）来维持其广泛的能力。广泛的实验表明，我们的方法优于人类3.6M和CMU Mocap的数据集上的SOTA方法，同时只需要更大的参数。代码将在https://github.com/yzg9353/dmsgcn上获得。

建模各种时空依赖项是识别骨架序列中人类动作的关键。大多数现有方法过度依赖于遍历规则或图形拓扑的设计，以利用动态关节的依赖性，这是反映远处但重要的关节的关系不足。此外，由于本地采用的操作，因此在现有的工作中探索了重要的远程时间信息。为了解决这个问题，在这项工作中，我们提出了LSTA-Net：一种新型长期短期时空聚合网络，可以以时空的方式有效地捕获长/短距离依赖性。我们将我们的模型设计成纯粹的分解体系结构，可以交替执行空间特征聚合和时间特征聚合。为了改善特征聚合效果，还设计和采用了一种通道明智的注意机制。在三个公共基准数据集中进行了广泛的实验，结果表明，我们的方法可以在空间和时域中捕获长短短程依赖性，从而产生比其他最先进的方法更高的结果。代码可在https://github.com/tailin1009/lsta-net。