Recently, human pose estimation mainly focuses on how to design a more effective and better deep network structure as human features extractor, and most designed feature extraction networks only introduce the position of each anatomical keypoint to guide their training process. However, we found that some human anatomical keypoints kept their topology invariance, which can help to localize them more accurately when detecting the keypoints on the feature map. But to the best of our knowledge, there is no literature that has specifically studied it. Thus, in this paper, we present a novel 2D human pose estimation method with explicit anatomical keypoints structure constraints, which introduces the topology constraint term that consisting of the differences between the distance and direction of the keypoint-to-keypoint and their groundtruth in the loss object. More importantly, our proposed model can be plugged in the most existing bottom-up or top-down human pose estimation methods and improve their performance. The extensive experiments on the benchmark dataset: COCO keypoint dataset, show that our methods perform favorably against the most existing bottom-up and top-down human pose estimation methods, especially for Lite-HRNet, when our model is plugged into it, its AP scores separately raise by 2.9\% and 3.3\% on COCO val2017 and test-dev2017 datasets.

Document images are a ubiquitous source of data where the text is organized in a complex hierarchical structure ranging from fine granularity (e.g., words), medium granularity (e.g., regions such as paragraphs or figures), to coarse granularity (e.g., the whole page). The spatial hierarchical relationships between content at different levels of granularity are crucial for document image understanding tasks. Existing methods learn features from either word-level or region-level but fail to consider both simultaneously. Word-level models are restricted by the fact that they originate from pure-text language models, which only encode the word-level context. In contrast, region-level models attempt to encode regions corresponding to paragraphs or text blocks into a single embedding, but they perform worse with additional word-level features. To deal with these issues, we propose MGDoc, a new multi-modal multi-granular pre-training framework that encodes page-level, region-level, and word-level information at the same time. MGDoc uses a unified text-visual encoder to obtain multi-modal features across different granularities, which makes it possible to project the multi-granular features into the same hyperspace. To model the region-word correlation, we design a cross-granular attention mechanism and specific pre-training tasks for our model to reinforce the model of learning the hierarchy between regions and words. Experiments demonstrate that our proposed model can learn better features that perform well across granularities and lead to improvements in downstream tasks.

Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 meters. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.

高密度物体（例如金属植入物和牙科填充物）的存在可以在计算机断层扫描（CT）图像中引入严重的条纹样伪像，从而极大地限制了随后的诊断。尽管已经提出了用于减少金属伪像的各种基于神经网络的方法（MAR），但由于对正式域中的全球环境的利用有限，图像域引入的次生伪像，它们的性能通常不佳，并且需要精确的次要伪像。金属面具。为了解决这些问题，本文探讨了在辛图和图像域中在MAR中的快速傅立叶卷积，并提出了MAR的傅立叶双域网络，称为FD-MAR。具体而言，我们首先提出了一个傅立叶曲调恢复网络，该网络可以利用辛克图范围内的接受环境来填充来自未腐败区域的金属腐败区域，因此对金属痕迹是可靠的。其次，我们在图像域中提出了一个傅立叶细化网络，该网络可以通过探索整个图像范围的上下文信息以局部到全球的方式来完善重建的图像。结果，拟议的FD-MAR可以探索MAR的正式和图像范围的接收场。通过通过复合损失函数优化FD-MAR，广泛的实验结果证明了拟议的FD-MAR在定量指标和视觉比较方面的优越性优于最先进的MAR方法。值得注意的是，FD-MAR不需要精确的金属口罩，这在临床常规中非常重要。

Twitter机器人检测已成为打击错误信息，促进社交媒体节制并保持在线话语的完整性的越来越重要的任务。最先进的机器人检测方法通常利用Twitter网络的图形结构，在面对传统方法无法检测到的新型Twitter机器人时，它们表现出令人鼓舞的性能。但是，现有的Twitter机器人检测数据集很少是基于图形的，即使这些基于图形的数据集也遭受有限的数据集量表，不完整的图形结构以及低注释质量。实际上，缺乏解决这些问题的大规模基于图的Twitter机器人检测基准，严重阻碍了基于图形的机器人检测方法的开发和评估。在本文中，我们提出了Twibot-22，这是一个综合基于图的Twitter机器人检测基准，它显示了迄今为止最大的数据集，在Twitter网络上提供了多元化的实体和关系，并且与现有数据集相比具有更好的注释质量。此外，我们重新实施35代表性的Twitter机器人检测基线，并在包括Twibot-22在内的9个数据集上进行评估，以促进对模型性能和对研究进度的整体了解的公平比较。为了促进进一步的研究，我们将所有实施的代码和数据集巩固到Twibot-22评估框架中，研究人员可以在其中始终如一地评估新的模型和数据集。 Twibot-22 Twitter机器人检测基准和评估框架可在https://twibot22.github.io/上公开获得。

近年来，Experts（MOE）的混合物已成为一种有前途的深度学习技术，可以将模型能力扩展为万亿多个参数，同时通过稀疏计算降低计算成本。虽然MoE开设了一个非常大的模型的新领域，但由于MOE的动态性质与系统的静态平行性/管道层之间的不匹配，因此其数以千计的GPU的实现受到限制。我们提出了Tutel，这是一种具有动态自适应并行性和管道的高度可扩展的堆栈设计和实现。 TUTEL在运行时提供自适应并行性切换和自适应管道，分别达到1.74倍和2.00倍的单MOE层加速度。我们还提出了一种用于MOE通信速度的新颖的二维层次结构算法，该算法的表现超过了2,048 GPU的先前最先前的最新时间。 Tutel汇总了所有技术，最终在16 GPU和2,048 GPU上分别提供了4.96倍和5.75倍的加速度，分别通过Fairseq：Meta的Facebook AI AI研究序列到序列工具Kit（Tutel（Tutel）（Tutel）（Tutel）（现在由Fairseq部分采用）。 Tutel源代码可在公共场所获得：https：//github.com/microsoft/tutel。我们的评估表明，Tutel有效，有效地运行了一个基于现实的MOE模型，名为Swinv2-Moe，建立在Swin Transformer V2上，这是一种最先进的计算机视觉体系结构。在效率方面，Tutel加速了Swinv2-MoE，在FairSeq的训练和推理中分别达到1.55倍和2.11倍的速度。关于有效性，SWINV2-MOE模型在预训练和下游计算机视觉任务（例如可可对象检测）方面都比对应的密度密度模型都达到了卓越的精度，这表明Tutel准备对端到端现实世界模型训练的准备就绪和推理。 Swinv2-Moe在https://github.com/microsoft/swin-transformer中开放。

Contextual information is vital in visual understanding problems, such as semantic segmentation and object detection. We propose a Criss-Cross Network (CCNet) for obtaining full-image contextual information in a very effective and efficient way. Concretely, for each pixel, a novel criss-cross attention module harvests the contextual information of all the pixels on its criss-cross path. By taking a further recurrent operation, each pixel can finally capture the full-image dependencies. Besides, a category consistent loss is proposed to enforce the criss-cross attention module to produce more discriminative features. Overall, CCNet is with the following merits: 1) GPU memory friendly. Compared with the non-local block, the proposed recurrent criss-cross attention module requires 11× less GPU memory usage. 2) High computational efficiency. The recurrent criss-cross attention significantly reduces FLOPs by about 85% of the non-local block. 3) The state-of-the-art performance. We conduct extensive experiments on semantic segmentation benchmarks including Cityscapes, ADE20K, human parsing benchmark LIP, instance segmentation benchmark COCO, video segmentation benchmark CamVid. In particular, our CCNet achieves the mIoU scores of 81.9%, 45.76% and 55.47% on the Cityscapes test set, the ADE20K validation set and the LIP validation set respectively, which are the new state-of-the-art results. The source codes are available at https://github.com/speedinghzl/CCNet.

Prior works on Information Extraction (IE) typically predict different tasks and instances (e.g., event triggers, entities, roles, relations) independently, while neglecting their interactions and leading to model inefficiency. In this work, we introduce a joint IE framework, HighIE, that learns and predicts multiple IE tasks by integrating high-order cross-task and cross-instance dependencies. Specifically, we design two categories of high-order factors: homogeneous factors and heterogeneous factors. Then, these factors are utilized to jointly predict labels of all instances. To address the intractability problem of exact high-order inference, we incorporate a high-order neural decoder that is unfolded from a mean-field variational inference method. The experimental results show that our approach achieves consistent improvements on three IE tasks compared with our baseline and prior work.

We study a challenging task, conditional human motion generation, which produces plausible human motion sequences according to various conditional inputs, such as action classes or textual descriptors. Since human motions are highly diverse and have a property of quite different distribution from conditional modalities, such as textual descriptors in natural languages, it is hard to learn a probabilistic mapping from the desired conditional modality to the human motion sequences. Besides, the raw motion data from the motion capture system might be redundant in sequences and contain noises; directly modeling the joint distribution over the raw motion sequences and conditional modalities would need a heavy computational overhead and might result in artifacts introduced by the captured noises. To learn a better representation of the various human motion sequences, we first design a powerful Variational AutoEncoder (VAE) and arrive at a representative and low-dimensional latent code for a human motion sequence. Then, instead of using a diffusion model to establish the connections between the raw motion sequences and the conditional inputs, we perform a diffusion process on the motion latent space. Our proposed Motion Latent-based Diffusion model (MLD) could produce vivid motion sequences conforming to the given conditional inputs and substantially reduce the computational overhead in both the training and inference stages. Extensive experiments on various human motion generation tasks demonstrate that our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks, with two orders of magnitude faster than previous diffusion models on raw motion sequences.

语义Web技术已成功促进了许多具有丰富数据表示方法的RDF模型。它还具有代表和存储多模式知识库（例如多模式场景图）的潜在能力。但是，大多数现有的查询语言，尤其是SPARQL，几乎没有探索隐式多模式关系，例如语义相似性，空间关系等。我们首先通过在RDF图数据库中组织一个大型场景图（即视觉基因组）来探索此问题。基于建议的RDF存储的多模式场景图，我们扩展了SPARQL查询，以回答包含有关颜色，空间等关系推理的问题。进一步的演示（即VGStore）显示了定制查询和显示多模式数据的有效性。