Panoptic Part Segmentation (PPS) unifies panoptic segmentation and part segmentation into one task. Previous works utilize separated approaches to handle thing, stuff, and part predictions without shared computation and task association. We aim to unify these tasks at the architectural level, designing the first end-to-end unified framework named Panoptic-PartFormer. Moreover, we find the previous metric PartPQ biases to PQ. To handle both issues, we make the following contributions: Firstly, we design a meta-architecture that decouples part feature and things/stuff feature, respectively. We model things, stuff, and parts as object queries and directly learn to optimize all three forms of prediction as a unified mask prediction and classification problem. We term our model as Panoptic-PartFormer. Secondly, we propose a new metric Part-Whole Quality (PWQ) to better measure such task from both pixel-region and part-whole perspectives. It can also decouple the error for part segmentation and panoptic segmentation. Thirdly, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross attention scheme to further boost part segmentation qualities. We design a new part-whole interaction method using masked cross attention. Finally, the extensive ablation studies and analysis demonstrate the effectiveness of both Panoptic-PartFormer and Panoptic-PartFormer++. Compared with previous Panoptic-PartFormer, our Panoptic-PartFormer++ achieves 2% PartPQ and 3% PWQ improvements on the Cityscapes PPS dataset and 5% PartPQ on the Pascal Context PPS dataset. On both datasets, Panoptic-PartFormer++ achieves new state-of-the-art results with a significant cost drop of 70% on GFlops and 50% on parameters. Our models can serve as a strong baseline and aid future research in PPS. Code will be available.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model. We introduce a 3D tokenizer to quantize a video into spatial-temporal visual tokens and propose an embedding method for masked video token modeling to facilitate multi-task learning. We conduct extensive experiments to demonstrate the quality, efficiency, and flexibility of MAGVIT. Our experiments show that (i) MAGVIT performs favorably against state-of-the-art approaches and establishes the best-published FVD on three video generation benchmarks, including the challenging Kinetics-600. (ii) MAGVIT outperforms existing methods in inference time by two orders of magnitude against diffusion models and by 60x against autoregressive models. (iii) A single MAGVIT model supports ten diverse generation tasks and generalizes across videos from different visual domains. The source code and trained models will be released to the public at https://magvit.cs.cmu.edu.

Weakly supervised semantic segmentation is typically inspired by class activation maps, which serve as pseudo masks with class-discriminative regions highlighted. Although tremendous efforts have been made to recall precise and complete locations for each class, existing methods still commonly suffer from the unsolicited Out-of-Candidate (OC) error predictions that not belongs to the label candidates, which could be avoidable since the contradiction with image-level class tags is easy to be detected. In this paper, we develop a group ranking-based Out-of-Candidate Rectification (OCR) mechanism in a plug-and-play fashion. Firstly, we adaptively split the semantic categories into In-Candidate (IC) and OC groups for each OC pixel according to their prior annotation correlation and posterior prediction correlation. Then, we derive a differentiable rectification loss to force OC pixels to shift to the IC group. Incorporating our OCR with seminal baselines (e.g., AffinityNet, SEAM, MCTformer), we can achieve remarkable performance gains on both Pascal VOC (+3.2%, +3.3%, +0.8% mIoU) and MS COCO (+1.0%, +1.3%, +0.5% mIoU) datasets with negligible extra training overhead, which justifies the effectiveness and generality of our OCR.

尽管基于3D点云表示的基于自我监督的对比度学习模型最近取得了成功，但此类预训练模型的对抗性鲁棒性引起了人们的关注。对抗性对比学习（ACL）被认为是改善预训练模型的鲁棒性的有效方法。相比之下，投影仪被认为是在对比度预处理过程中删除不必要的特征信息的有效组成部分，并且大多数ACL作品还使用对比度损失，与预测的功能表示形式相比损失，在预处理中产生对抗性示例，而“未转移”的功能表征用于发电的对抗性输入。在推理期间。由于投影和“未投影”功能之间的分布差距，其模型受到限制，以获取下游任务的可靠特征表示。我们介绍了一种新方法，通过利用虚拟对抗性损失在对比度学习框架中使用“未重新注射”功能表示，以生成高质量的3D对抗示例，以进行对抗训练。我们介绍了强大的意识损失功能，以对抗自我监督对比度学习框架。此外，我们发现选择具有正常操作员（DON）操作员差异的高差异作为对抗性自学对比度学习的附加输入，可以显着提高预训练模型的对抗性鲁棒性。我们在下游任务上验证我们的方法，包括3D分类和使用多个数据集的3D分割。它在最先进的对抗性学习方法上获得了可比的鲁棒精度。

扩散模型（DMS）显示出高质量图像合成的巨大潜力。但是，当涉及到具有复杂场景的图像时，如何正确描述图像全局结构和对象细节仍然是一项具有挑战性的任务。在本文中，我们提出了弗里多（Frido），这是一种特征金字塔扩散模型，该模型执行了图像合成的多尺度粗到1个降解过程。我们的模型将输入图像分解为依赖比例的矢量量化特征，然后是用于产生图像输出的粗到细门。在上述多尺度表示阶段，可以进一步利用文本，场景图或图像布局等其他输入条件。因此，还可以将弗里多应用于条件或跨模式图像合成。我们对各种无条件和有条件的图像生成任务进行了广泛的实验，从文本到图像综合，布局到图像，场景环形图像到标签形象。更具体地说，我们在五个基准测试中获得了最先进的FID分数，即可可和开阔图像的布局到图像，可可和视觉基因组的场景环形图像以及可可的标签对图像图像。 。代码可在https://github.com/davidhalladay/frido上找到。

在过去的十年中，多任务学习方法在解决全景驱动感知问题方面取得了令人鼓舞的结果，提供了高精度和高效效率。在为实时自动驾驶系统设计网络时，它已成为流行的范式，在该系统中，计算资源受到限制。本文提出了一个有效，有效的多任务学习网络，以同时执行交通对象检测，可驱动的道路区域细分和车道检测的任务。我们的模型以挑战性的BDD100K数据集的准确性和速度来实现新的最先进（SOTA）性能。特别是，与先前的SOTA模型相比，推理时间减少了一半。代码将在不久的将来发布。

本文回顾了AIM 2022上压缩图像和视频超级分辨率的挑战。这项挑战包括两条曲目。轨道1的目标是压缩图像的超分辨率，轨迹〜2靶向压缩视频的超分辨率。在轨道1中，我们使用流行的数据集DIV2K作为培训，验证和测试集。在轨道2中，我们提出了LDV 3.0数据集，其中包含365个视频，包括LDV 2.0数据集（335个视频）和30个其他视频。在这一挑战中，有12支球队和2支球队分别提交了赛道1和赛道2的最终结果。所提出的方法和解决方案衡量了压缩图像和视频上超分辨率的最先进。提出的LDV 3.0数据集可在https://github.com/renyang-home/ldv_dataset上找到。此挑战的首页是在https://github.com/renyang-home/aim22_compresssr。

拓扑不平衡是由标记节点的不均匀拓扑位置引起的一个特异性不平衡问题，它大大损害了GNN的性能。什么拓扑不平衡意味着如何衡量其对图形学习的影响。在本文中，从全球视图中，我们对监督信息分布的全球视图提供了对拓扑 - 不平衡的新理解，从不足和过度划分的角度来看，这激发了两个定量指标作为测量。鉴于我们的分析，我们提出了一个新颖的位置感知的图形结构学习框架，该框架名为柔和，该框架直接优化了信息传播路径并解决了本质上解决拓扑 - 不平衡问题。我们的关键见解是增强同一类中节点的连接性，以获取更多的监督信息，从而减轻不足和过度的现象。具体而言，我们设计了一个基于锚的位置编码机制，该机制可以更好地结合相对拓扑位置并通过最大化标签影响来增强类内部电感偏置。我们进一步提出了作为边缘权重的阶级冲突度量，这有利于不同节点类别的分离。广泛的实验表明，在不同的数据注释方案中增强GNNS的功率方面，柔和的能力具有较高的潜力和适应性。

如果不正确地进行，无监督的自我锻炼练习和体育训练可能会造成严重伤害。我们介绍了一个基于学习的框架，该框架可以识别用户犯的错误，并提出纠正措施，以更轻松，更安全的个人培训。我们的框架不依赖于硬编码的启发式规则。取而代之的是，它从数据中学习，这有助于其适应特定用户需求。为此，我们使用作用于用户姿势序列的图形卷积网络（GCN）体系结构来模拟身体关节轨迹之间的关系。为了评估我们的方法，我们介绍了一个具有3种不同体育锻炼的数据集。我们的方法产生了90.9％的错误识别准确性，并成功纠正了94.2％的错误。