This work introduces a novel convolutional network architecture for the task of human pose estimation. Features are processed across all scales and consolidated to best capture the various spatial relationships associated with the body. We show how repeated bottom-up, top-down processing used in conjunction with intermediate supervision is critical to improving the performance of the network. We refer to the architecture as a "stacked hourglass" network based on the successive steps of pooling and upsampling that are done to produce a final set of predictions. State-of-the-art results are achieved on the FLIC and MPII benchmarks outcompeting all recent methods.

我们提出Bapose，一种新颖的自下而上的方法，实现了多人姿态估计的最先进结果。我们的最终培训框架利用了解开的多尺度瀑布架构，并将自适应卷曲融合在拥挤的场景中更准确地推断出闭塞的关键点。由BAPOSE中的解开瀑布模块获得的多尺度表示，利用级联架构中进行逐行滤波的效率，同时保持与空间金字塔配置的多尺度视图相当。我们对挑战性的Coco和Crowdose数据集的结果表明，Bapose是多人姿态估计的高效且稳健的框架，实现了最先进的准确性的显着改善。

这项研究礼物活动融合的理念引入现有的姿势估计架构，以提高他们的预测能力。这是由上升现代机器学习架构发现更高层次的概念，并相信活动上下文的信息姿态估计的问题，一个有用的片段动机。要分析这个概念，我们利用现有的深度学习建筑和一个额外的1x1卷积保险丝活动信息到模型中增加它。我们在一个共同的姿势估计数据集进行评价和比较，并表现出性能改善了我们的基础模型，特别是在罕见的姿势和上通常难以关节。此外，我们执行烧蚀分析表明，性能的提升事实上确实从活动信息绘制。

In this paper, we are interested in the human pose estimation problem with a focus on learning reliable highresolution representations. Most existing methods recover high-resolution representations from low-resolution representations produced by a high-to-low resolution network. Instead, our proposed network maintains high-resolution representations through the whole process.We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutliresolution subnetworks in parallel. We conduct repeated multi-scale fusions such that each of the high-to-low resolution representations receives information from other parallel representations over and over, leading to rich highresolution representations. As a result, the predicted keypoint heatmap is potentially more accurate and spatially more precise. We empirically demonstrate the effectiveness of our network through the superior pose estimation results over two benchmark datasets: the COCO keypoint detection dataset and the MPII Human Pose dataset. In addition, we show the superiority of our network in pose tracking on the PoseTrack dataset. The code and models have been publicly available at https://github.com/leoxiaobin/ deep-high-resolution-net.pytorch.

Pose Machines provide a sequential prediction framework for learning rich implicit spatial models. In this work we show a systematic design for how convolutional networks can be incorporated into the pose machine framework for learning image features and image-dependent spatial models for the task of pose estimation. The contribution of this paper is to implicitly model long-range dependencies between variables in structured prediction tasks such as articulated pose estimation. We achieve this by designing a sequential architecture composed of convolutional networks that directly operate on belief maps from previous stages, producing increasingly refined estimates for part locations, without the need for explicit graphical model-style inference. Our approach addresses the characteristic difficulty of vanishing gradients during training by providing a natural learning objective function that enforces intermediate supervision, thereby replenishing back-propagated gradients and conditioning the learning procedure. We demonstrate state-of-the-art performance and outperform competing methods on standard benchmarks including the MPII, LSP, and FLIC datasets.

Recent state-of-the-art performance on human-body pose estimation has been achieved with Deep Convolutional Networks (ConvNets). Traditional ConvNet architectures include pooling and sub-sampling layers which reduce computational requirements, introduce invariance and prevent over-training. These benefits of pooling come at the cost of reduced localization accuracy. We introduce a novel architecture which includes an efficient 'position refinement' model that is trained to estimate the joint offset location within a small region of the image. This refinement model is jointly trained in cascade with a state-of-the-art ConvNet model [21] to achieve improved accuracy in human joint location estimation. We show that the variance of our detector approaches the variance of human annotations on the FLIC [20] dataset and outperforms all existing approaches on the MPII-human-pose dataset [1].

Figure 1. Besides extreme variability in articulations, many of the joints are barely visible. We can guess the location of the right arm in the left image only because we see the rest of the pose and anticipate the motion or activity of the person. Similarly, the left body half of the person on the right is not visible at all. These are examples of the need for holistic reasoning. We believe that DNNs can naturally provide such type of reasoning.

大多数实时人类姿势估计方法都基于检测接头位置。使用检测到的关节位置，可以计算偏差和肢体的俯仰。然而，由于这种旋转轴仍然不观察，因此不能计算沿着肢体沿着肢体至关重要的曲折，这对于诸如体育分析和计算机动画至关重要。在本文中，我们引入了方向关键点，一种用于估计骨骼关节的全位置和旋转的新方法，仅使用单帧RGB图像。灵感来自Motion-Capture Systems如何使用一组点标记来估计全骨骼旋转，我们的方法使用虚拟标记来生成足够的信息，以便准确地推断使用简单的后处理。旋转预测改善了接头角度最佳报告的平均误差48％，并且在15个骨骼旋转中实现了93％的精度。该方法还通过MPJPE在原理数据集上测量，通过MPJPE测量，该方法还改善了当前的最新结果14％，并概括为野外数据集。

Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this paper, we present AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime. To this end, we propose several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly pose estimation and tracking. During training, we resort to Part-Guided Proposal Generator (PGPG) and multi-domain knowledge distillation to further improve the accuracy. Our method is able to localize whole-body keypoints accurately and tracks humans simultaneously given inaccurate bounding boxes and redundant detections. We show a significant improvement over current state-of-the-art methods in both speed and accuracy on COCO-wholebody, COCO, PoseTrack, and our proposed Halpe-FullBody pose estimation dataset. Our model, source codes and dataset are made publicly available at https://github.com/MVIG-SJTU/AlphaPose.

Realtime multi-person 2D pose estimation is a key component in enabling machines to have an understanding of people in images and videos. In this work, we present a realtime approach to detect the 2D pose of multiple people in an image. The proposed method uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. This bottom-up system achieves high accuracy and realtime performance, regardless of the number of people in the image. In previous work, PAFs and body part location estimation were refined simultaneously across training stages. We demonstrate that a PAF-only refinement rather than both PAF and body part location refinement results in a substantial increase in both runtime performance and accuracy. We also present the first combined body and foot keypoint detector, based on an internal annotated foot dataset that we have publicly released. We show that the combined detector not only reduces the inference time compared to running them sequentially, but also maintains the accuracy of each component individually. This work has culminated in the release of OpenPose, the first open-source realtime system for multi-person 2D pose detection, including body, foot, hand, and facial keypoints.

We propose a CNN-based approach for 3D human body pose estimation from single RGB images that addresses the issue of limited generalizability of models trained solely on the starkly limited publicly available 3D pose data. Using only the existing 3D pose data and 2D pose data, we show state-of-the-art performance on established benchmarks through transfer of learned features, while also generalizing to in-the-wild scenes. We further introduce a new training set for human body pose estimation from monocular images of real humans that has the ground truth captured with a multi-camera marker-less motion capture system. It complements existing corpora with greater diversity in pose, human appearance, clothing, occlusion, and viewpoints, and enables an increased scope of augmentation. We also contribute a new benchmark that covers outdoor and indoor scenes, and demonstrate that our 3D pose dataset shows better in-the-wild performance than existing annotated data, which is further improved in conjunction with transfer learning from 2D pose data. All in all, we argue that the use of transfer learning of representations in tandem with algorithmic and data contributions is crucial for general 3D body pose estimation.

The topic of multi-person pose estimation has been largely improved recently, especially with the development of convolutional neural network. However, there still exist a lot of challenging cases, such as occluded keypoints, invisible keypoints and complex background, which cannot be well addressed. In this paper, we present a novel network structure called Cascaded Pyramid Network (CPN) which targets to relieve the problem from these "hard" keypoints. More specifically, our algorithm includes two stages: Glob-alNet and RefineNet. GlobalNet is a feature pyramid network which can successfully localize the "simple" keypoints like eyes and hands but may fail to precisely recognize the occluded or invisible keypoints. Our RefineNet tries explicitly handling the "hard" keypoints by integrating all levels of feature representations from the Global-Net together with an online hard keypoint mining loss. In general, to address the multi-person pose estimation problem, a top-down pipeline is adopted to first generate a set of human bounding boxes based on a detector, followed by our CPN for keypoint localization in each human bounding box. Based on the proposed algorithm, we achieve stateof-art results on the COCO keypoint benchmark, with average precision at 73.0 on the COCO test-dev dataset and 72.1 on the COCO test-challenge dataset, which is a 19% relative improvement compared with 60.5 from the COCO 2016 keypoint challenge. Code 1 and the detection results are publicly available for further research.

本文调查了2D全身人类姿势估计的任务，该任务旨在将整个人体（包括身体，脚，脸部和手）局部定位在整个人体上。我们提出了一种称为Zoomnet的单网络方法，以考虑到完整人体的层次结构，并解决不同身体部位的规模变化。我们进一步提出了一个称为Zoomnas的神经体系结构搜索框架，以促进全身姿势估计的准确性和效率。Zoomnas共同搜索模型体系结构和不同子模块之间的连接，并自动为搜索的子模块分配计算复杂性。为了训练和评估Zoomnas，我们介绍了第一个大型2D人类全身数据集，即可可叶全体V1.0，它注释了133个用于野外图像的关键点。广泛的实验证明了Zoomnas的有效性和可可叶v1.0的重要性。

Following the success of deep convolutional networks, state-of-the-art methods for 3d human pose estimation have focused on deep end-to-end systems that predict 3d joint locations given raw image pixels. Despite their excellent performance, it is often not easy to understand whether their remaining error stems from a limited 2d pose (visual) understanding, or from a failure to map 2d poses into 3dimensional positions.With the goal of understanding these sources of error, we set out to build a system that given 2d joint locations predicts 3d positions. Much to our surprise, we have found that, with current technology, "lifting" ground truth 2d joint locations to 3d space is a task that can be solved with a remarkably low error rate: a relatively simple deep feedforward network outperforms the best reported result by about 30% on Human3.6M, the largest publicly available 3d pose estimation benchmark. Furthermore, training our system on the output of an off-the-shelf state-of-the-art 2d detector (i.e., using images as input) yields state of the art results -this includes an array of systems that have been trained end-to-end specifically for this task. Our results indicate that a large portion of the error of modern deep 3d pose estimation systems stems from their visual analysis, and suggests directions to further advance the state of the art in 3d human pose estimation.

在本文中，我们考虑了同时找到和从单个2D图像中恢复多手的具有挑战性的任务。先前的研究要么关注单手重建，要么以多阶段的方式解决此问题。此外，常规的两阶段管道首先检测到手部区域，然后估计每个裁剪贴片的3D手姿势。为了减少预处理和特征提取中的计算冗余，我们提出了一条简洁但有效的单阶段管道。具体而言，我们为多手重建设计了多头自动编码器结构，每个HEAD网络分别共享相同的功能图并分别输出手动中心，姿势和纹理。此外，我们采用了一个弱监督的计划来减轻昂贵的3D现实世界数据注释的负担。为此，我们提出了一系列通过舞台训练方案优化的损失，其中根据公开可用的单手数据集生成具有2D注释的多手数据集。为了进一步提高弱监督模型的准确性，我们在单手和多个手设置中采用了几个功能一致性约束。具体而言，从本地功能估算的每只手的关键点应与全局功能预测的重新投影点一致。在包括Freihand，HO3D，Interhand 2.6M和RHD在内的公共基准测试的广泛实验表明，我们的方法在弱监督和完全监督的举止中优于基于最先进的模型方法。代码和模型可在{\ url {https://github.com/zijinxuxu/smhr}}上获得。

在自然谈话和互动中，我们的手经常重叠或彼此接触。由于双手的均匀外观，这使得估计从图像互动的3D姿势困难。在本文中，我们证明了自我相似性，以及将像素观测分配给各自的手和它们的部分的产生的歧义是最终3D姿势错误的主要原因。通过这种洞察力，我们提出了数字，一种估计来自单眼图像的两个交互手的3D姿势的新方法。该方法包括两个交织分支，该分支处理输入图像到每个像素语义部分分段掩模和视觉特征卷。与事先工作相比，我们不会从姿势估计阶段解耦分割，而是直接利用每个像素概率直接在下游姿势估计任务中。为此，零件概率与视觉功能合并并通过全卷积层处理。我们通过实验表明，该方法在Interhand2.6M数据集中实现了新的最先进的性能。我们提供详细的消融研究，以证明我们方法的功效，并提供对像素所有权建模如何影响3D手姿势估计的见解。

We propose CornerNet, a new approach to object detection where we detect an object bounding box as a pair of keypoints, the top-left corner and the bottom-right corner, using a single convolution neural network. By detecting objects as paired keypoints, we eliminate the need for designing a set of anchor boxes commonly used in prior single-stage detectors. In addition to our novel formulation, we introduce corner pooling, a new type of pooling layer that helps the network better localize corners. Experiments show that Corner-Net achieves a 42.2% AP on MS COCO, outperforming all existing one-stage detectors.

闭塞对单眼多人3D人体姿势估计构成了极大的威胁，这是由于封闭器的形状，外观和位置方面的差异很大。尽管现有的方法试图用姿势先验/约束，数据增强或隐性推理处理遮挡，但它们仍然无法概括地看不见姿势或遮挡案例，并且在出现多人时可能会犯大错误。受到人类从可见线索推断关节的显着能力的启发，我们开发了一种方法来显式建模该过程，该过程可以显着改善有或没有遮挡的情况下，可以显着改善自下而上的多人姿势估计。首先，我们将任务分为两个子任务：可见的关键点检测和遮挡的关键点推理，并提出了深入监督的编码器蒸馏（DSED）网络以求解第二个网络。为了训练我们的模型，我们提出了一种骨骼引导的人形拟合（SSF）方法，以在现有数据集上生成伪遮挡标签，从而实现明确的遮挡推理。实验表明，从遮挡中明确学习可以改善人类姿势估计。此外，利用可见关节的特征级信息使我们可以更准确地推理遮挡关节。我们的方法的表现优于几个基准的最新自上而下和自下而上的方法。

Bottom-up human pose estimation methods have difficulties in predicting the correct pose for small persons due to challenges in scale variation. In this paper, we present HigherHRNet: a novel bottom-up human pose estimation method for learning scale-aware representations using high-resolution feature pyramids. Equipped with multi-resolution supervision for training and multiresolution aggregation for inference, the proposed approach is able to solve the scale variation challenge in bottom-up multi-person pose estimation and localize keypoints more precisely, especially for small person. The feature pyramid in HigherHRNet consists of feature map outputs from HRNet and upsampled higher-resolution outputs through a transposed convolution. HigherHR-Net outperforms the previous best bottom-up method by 2.5% AP for medium person on COCO test-dev, showing its effectiveness in handling scale variation. Furthermore, HigherHRNet achieves new state-of-the-art result on COCO test-dev (70.5% AP) without using refinement or other post-processing techniques, surpassing all existing bottom-up methods. HigherHRNet even surpasses all topdown methods on CrowdPose test (67.6% AP), suggesting its robustness in crowded scene. The code and models are available at https://github.com/HRNet/ Higher-HRNet-Human-Pose-Estimation.

内部的姿势估计显示出在医院患者监测，睡眠研究和智能家居等领域的价值。在本文中，我们探讨了借助现有的姿势估计器，从高度模棱两可的压力数据中检测身体姿势的不同策略。我们通过直接使用或通过在两个压力数据集上对其进行重新训练来检查预训练的姿势估计器的性能。我们还利用可学习的预处理域适应步骤探索了其他策略，该步骤将模糊的压力图转换为更接近共同目的姿势估计模块的预期输入空间的表示。因此，我们使用了具有多个尺度的完全卷积网络，以向预训练的姿势估计模块提供压力图的姿势特异性特征。我们对不同方法的完整分析表明，在压力数据上，可学习的预处理模块的组合以及重新训练基于图像的姿势估计器能够克服诸如高度模糊的压力点之类的问题，以实现很高的姿势估计准确性。