给定图像和参考字幕，图像标题编辑任务旨在纠正未对准错误并生成精制的字幕。但是，所有现有的字幕编辑作品都是隐式模型，即它们直接生成精制字幕而无需与参考标题明确连接。在本文中，我们介绍了一项新任务：显式字幕编辑（ECE）。 ECE模型明确生成了一系列编辑操作，此编辑操作序列可以将参考字幕转换为精制的字幕。与隐式编辑相比，ECE具有多个优点：1）可解释：它可以追踪整个编辑路径。 2）编辑有效：它只需要修改几个单词。 3）像人类一样：它类似于人类执行字幕编辑的方式，并试图保持原始句子结构。为了解决这项新任务，我们提出了第一个ECE模型：Tiger。 Tiger是一种非自动回形变压器的模型，由三个模块组成：Tagger_del，Tagger_Add和Inserter。具体而言，Tagger_del决定是否应该保留每个单词，Tagger_add决定添加新单词的位置，而Inserster预测了添加的特定单词。为了进一步促进ECE研究，我们分别重新组织了两个现有数据集，分别为Coco-EE和FlickR30K-EE，提出了两个新的ECE基准。两个基准上的大量消融都证明了老虎的有效性。

半监督学习（SSL）从根本上是一个缺失的标签问题，与广泛的随机假设完全既贴心又无标记的标签完全失踪，而不是随机（mnar）问题（mnar）问题更现实和挑战数据共享相同的类分布。与现有的SSL解决方案不同，这些解决方案忽略了“类”在引起非随机性中的作用，例如，用户更有可能将流行类标记为“类别”，我们将“类”明确地纳入SSL。我们的方法是三倍：1）我们建议使用偏置标记的数据来利用未标记的数据来利用未标记的数据来训练改进的分类器。 2）鼓励罕见的课堂培训，其模型是低回调但高精度，丢弃了太多的伪标记的数据，我们提出了类动态降低（或增加）伪标签分配阈值的class感知插补（CAI）稀有（或频繁）的课程。 3）总体而言，我们将CAP和CAI集成到训练无偏的SSL模型的双重稳健估计器中。在各种MNAR设置和消融中，我们的方法不仅显着优于现有基线，而且超过了其他标签偏置删除SSL方法。请通过以下方式查看我们的代码：https：//github.com/joyhuyy1412/cadr-fixmatch。

了解多媒体内容中描述或显示的事件彼此相关是开发可用于真实世界媒体的强大人工智能系统的关键组成部分。尽管许多研究专门用于文本，图像和视频域中的事件理解，但没有一个研究探索事件跨域中经历的复杂关系。例如，新闻文章可能会描述“抗议”事件，而视频显示“逮捕”事件。认识到视觉“逮捕”事件是更广泛的“抗议”事件的一个子事件，这是一个具有挑战性但重要的问题，但前面的工作尚未探讨。在本文中，我们提出了多模式事件关系关系的新任务，以识别这种跨模式事件关系。我们贡献了一个大规模数据集，该数据集由100K视频新文章对组成，以及密集注释的数据的基准。我们还提出了一种弱监督的多模式方法，该方法将来自外部知识库（KB）的常识性知识整合在一起，以预测丰富的多模式事件层次结构。实验表明，我们的模型在我们提出的基准上优于许多竞争基线。我们还对模型的性能进行了详细的分析，并建议未来研究的方向。

我们在现有的长尾分类方法中解决了被忽视的无偏见：我们发现它们的整体改善主要归因于尾部过度的偏置偏好，因为假设测试分配是平衡的;但是，当测试与长尾训练数据一样不平衡 - 让测试尊重ZIPF的自然定律 - 尾巴偏差不再有益，因为它伤害了大多数人。在本文中，我们提出了跨域经验风险最小化（XIM）来训练一个非偏见模型，以实现对两个测试分布的强大性能，经验证明Xerm通过学习更好的特征表示而不是头部与头部来改善分类。游戏。基于因果关系，我们进一步理论上解释了Xerm实现了非偏见的原因：通过调整不平衡域和平衡但不合形的结构域的经验风险来消除由域选择引起的偏差。代码可在https://github.com/beierzhu/xerm获得。

今天的VIDSGG模型是基于建议的方法，即，它们首先生成众多配对的主题对象片段作为提案，然后对每个提案进行谓词分类。在本文中，我们认为这种普遍的基于建议的框架有三个固有的缺点：1）建议的地面真理谓词标签部分是正确的。 2）他们打破了相同主题对象对的不同谓词实例之间的高阶关系。 3）Vidsgg性能是由提案质量的大约。为此，我们向Vidsgg提出了一个新的分类 - 然后接地框架，可以避免所有三个被忽视的缺点。同时，在此框架下，我们将视频场景图形为临时二分形图形，其中实体和谓词是具有时隙的两种类型的节点，并且边缘在这些节点之间表示不同的语义角色。此配方充分利用了我们的新框架。因此，我们进一步提出了一种基于新的二分曲线图的SGG模型：大。具体而言，大由两部分组成：分类阶段和接地阶段，前者旨在对所有节点和边缘的类别进行分类，并且后者试图本地化每个关系实例的时间位置。两个Vidsgg数据集上的广泛消融已证明我们框架和大的有效性。

问题答案（QA）模型是众所周知的，用于利用数据偏差，例如在Visual QA之前的语言和阅读理解中的位置偏差。最近的脱叠方法实现了良好的分配（OOD）概括性，具有相当大的牺牲，对分销（ID）性能。因此，它们仅适用于预先已知测试分配的域。在本文中，我们提出了一种称为内省蒸馏的新型脱达方法（介绍），以充分为QA的世界。我们的主要技术贡献是通过省略培训样本是否适合事实ID世界或反事实_一种策略来融合OOD和ID的归纳偏差。在Visual QA Datasets VQA V2，VQA-CP和阅读理解数据集小队的实验表明，与其他脱叠方法相比，我们的提议介绍了竞争性的ood性能，同时与非脱叠相比牺牲很少甚至实现更好的ID性能。

Today's scene graph generation (SGG) task is still far from practical, mainly due to the severe training bias, e.g., collapsing diverse human walk on/ sit on/lay on beach into human on beach. Given such SGG, the down-stream tasks such as VQA can hardly infer better scene structures than merely a bag of objects. However, debiasing in SGG is not trivial because traditional debiasing methods cannot distinguish between the good and bad bias, e.g., good context prior (e.g., person read book rather than eat) and bad long-tailed bias (e.g., near dominating behind/in front of). In this paper, we present a novel SGG framework based on causal inference but not the conventional likelihood. We first build a causal graph for SGG, and perform traditional biased training with the graph. Then, we propose to draw the counterfactual causality from the trained graph to infer the effect from the bad bias, which should be removed. In particular, we use Total Direct Effect as the proposed final predicate score for unbiased SGG. Note that our framework is agnostic to any SGG model and thus can be widely applied in the community who seeks unbiased predictions. By using the proposed Scene Graph Diagnosis toolkit 1 on the SGG benchmark Visual Genome and several prevailing models, we observed significant improvements over the previous state-of-the-art methods.

Reinforcement learning (RL) is one of the most important branches of AI. Due to its capacity for self-adaption and decision-making in dynamic environments, reinforcement learning has been widely applied in multiple areas, such as healthcare, data markets, autonomous driving, and robotics. However, some of these applications and systems have been shown to be vulnerable to security or privacy attacks, resulting in unreliable or unstable services. A large number of studies have focused on these security and privacy problems in reinforcement learning. However, few surveys have provided a systematic review and comparison of existing problems and state-of-the-art solutions to keep up with the pace of emerging threats. Accordingly, we herein present such a comprehensive review to explain and summarize the challenges associated with security and privacy in reinforcement learning from a new perspective, namely that of the Markov Decision Process (MDP). In this survey, we first introduce the key concepts related to this area. Next, we cover the security and privacy issues linked to the state, action, environment, and reward function of the MDP process, respectively. We further highlight the special characteristics of security and privacy methodologies related to reinforcement learning. Finally, we discuss the possible future research directions within this area.

Pure transformers have shown great potential for vision tasks recently. However, their accuracy in small or medium datasets is not satisfactory. Although some existing methods introduce a CNN as a teacher to guide the training process by distillation, the gap between teacher and student networks would lead to sub-optimal performance. In this work, we propose a new One-shot Vision transformer search framework with Online distillation, namely OVO. OVO samples sub-nets for both teacher and student networks for better distillation results. Benefiting from the online distillation, thousands of subnets in the supernet are well-trained without extra finetuning or retraining. In experiments, OVO-Ti achieves 73.32% top-1 accuracy on ImageNet and 75.2% on CIFAR-100, respectively.

We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and quantify the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analysis are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.