对比度学习通常用作一种自我监督学习的方法，“锚”和“正”是给定输入图像的两个随机增强，而“负”是所有其他图像的集合。但是，对大批量和记忆库的需求使训练变得困难和缓慢。这促使有监督的对比方法的崛起通过使用带注释的数据来克服这些问题。我们希望通过基于其相似性进行排名，并观察人类偏见（以排名形式）对学习表示的影响，以进一步改善受监督的对比学习。我们认为这是一个重要的问题，因为学习良好的功能嵌入是在计算机视觉中长期以来一直追求的问题。

对比学习通常应用于自学的学习，并且已被证明超过了传统方法，例如三胞胎损失和n对损失。但是，对大批量和记忆库的需求使训练变得困难和缓慢。最近，已经开发出有监督的对比方法来克服这些问题。他们更多地专注于分别或在各个班级之间为每个班级学习一个良好的表示。在这项工作中，我们尝试使用用户定义的排名来基于相似性对类进行排名，以了解所有类之间的有效表示。我们观察到如何将人类偏见纳入学习过程可以改善参数空间中的学习表征。我们表明，我们的结果可与受监督的对比度学习用于图像分类和对象检测，并讨论其在OOD检测中的缺点

Recent work has shown that large language models are capable of generating natural language reasoning steps or Chains-of-Thoughts (CoT) to answer a multi-step question when prompted to do so. This is insufficient, however, when the necessary knowledge is not available or up-to-date within a model's parameters. A straightforward approach to address this is to retrieve text from an external knowledge source using the question as a query and prepend it as context to the model's input. This, however, is also insufficient for multi-step QA where \textit{what to retrieve} depends on \textit{what has already been derived}. To address this issue we propose IRCoT, a new approach that interleaves retrieval with CoT for multi-step QA, guiding the retrieval with CoT and in turn using retrieved results to improve CoT. Our experiments with GPT3 show substantial improvements in retrieval (up to 22 points) and downstream QA (up to 16 points) over the baselines on four datasets: HotpotQA, 2WikiMultihopQA, MuSiQue, and IIRC. Notably, our method also works well for much smaller models such as T5-Flan-large (0.7B) without any additional training.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Knowledge about outcomes is critical for complex event understanding but is hard to acquire. We show that by pre-identifying a participant in a complex event, crowd workers are able to (1) infer the collective impact of salient events that make up the situation, (2) annotate the volitional engagement of participants in causing the situation, and (3) ground the outcome of the situation in state changes of the participants. By creating a multi-step interface and a careful quality control strategy, we collect a high quality annotated dataset of 8K short newswire narratives and ROCStories with high inter-annotator agreement (0.74-0.96 weighted Fleiss Kappa). Our dataset, POQue (Participant Outcome Questions), enables the exploration and development of models that address multiple aspects of semantic understanding. Experimentally, we show that current language models lag behind human performance in subtle ways through our task formulations that target abstract and specific comprehension of a complex event, its outcome, and a participant's influence over the event culmination.

Explainability has been widely stated as a cornerstone of the responsible and trustworthy use of machine learning models. With the ubiquitous use of Deep Neural Network (DNN) models expanding to risk-sensitive and safety-critical domains, many methods have been proposed to explain the decisions of these models. Recent years have also seen concerted efforts that have shown how such explanations can be distorted (attacked) by minor input perturbations. While there have been many surveys that review explainability methods themselves, there has been no effort hitherto to assimilate the different methods and metrics proposed to study the robustness of explanations of DNN models. In this work, we present a comprehensive survey of methods that study, understand, attack, and defend explanations of DNN models. We also present a detailed review of different metrics used to evaluate explanation methods, as well as describe attributional attack and defense methods. We conclude with lessons and take-aways for the community towards ensuring robust explanations of DNN model predictions.

Estimating treatment effects from observational data is a central problem in causal inference. Methods to solve this problem exploit inductive biases and heuristics from causal inference to design multi-head neural network architectures and regularizers. In this work, we propose to use neurosymbolic program synthesis, a data-efficient, and interpretable technique, to solve the treatment effect estimation problem. We theoretically show that neurosymbolic programming can solve the treatment effect estimation problem. By designing a Domain Specific Language (DSL) for treatment effect estimation problem based on the inductive biases used in literature, we argue that neurosymbolic programming is a better alternative to treatment effect estimation than traditional methods. Our empirical study reveals that our method, which implicitly encodes inductive biases in a DSL, achieves better performance on benchmark datasets than the state-of-the-art methods.

A machine learning model, under the influence of observed or unobserved confounders in the training data, can learn spurious correlations and fail to generalize when deployed. For image classifiers, augmenting a training dataset using counterfactual examples has been empirically shown to break spurious correlations. However, the counterfactual generation task itself becomes more difficult as the level of confounding increases. Existing methods for counterfactual generation under confounding consider a fixed set of interventions (e.g., texture, rotation) and are not flexible enough to capture diverse data-generating processes. Given a causal generative process, we formally characterize the adverse effects of confounding on any downstream tasks and show that the correlation between generative factors (attributes) can be used to quantitatively measure confounding between generative factors. To minimize such correlation, we propose a counterfactual generation method that learns to modify the value of any attribute in an image and generate new images given a set of observed attributes, even when the dataset is highly confounded. These counterfactual images are then used to regularize the downstream classifier such that the learned representations are the same across various generative factors conditioned on the class label. Our method is computationally efficient, simple to implement, and works well for any number of generative factors and confounding variables. Our experimental results on both synthetic (MNIST variants) and real-world (CelebA) datasets show the usefulness of our approach.

从有限的资源中获得最大收益可以进步自然语言处理（NLP）研究和实践，同时保守资源。这些资源可能是数据，时间，存储或能源。NLP的最新工作从缩放率产生了有趣的结果。但是，仅使用比例来改善结果意味着资源消耗也会扩展。这种关系激发了对有效方法的研究，这些方法需要更少的资源才能获得相似的结果。这项调查涉及NLP效率的方法和发现，旨在指导该领域的新研究人员并激发新方法的发展。

本文提出了一个基于混合融合的多模式情感识别系统，该系统将语音话语和相应图像描绘的情绪分类为离散类。已经开发了一种新的可解释性技术，以确定重要的语音和图像特征，从而预测特定的情感类别。拟议的系统的体系结构是通过大量消融研究确定的。它融合了语音和图像特征，然后结合了语音，图像和中间融合输出。提出的可解释性技术结合了划分和征服方法，以计算表示每个语音和图像特征的重要性的刻薄值。我们还构建了一个大规模数据集（IIT-R较小的数据集），包括语音话语，相应的图像和班级标签，即“愤怒”，“快乐”，“仇恨”和“悲伤”。拟议的系统已达到83.29％的情绪识别精度。提出的系统的增强性能提倡利用多种模式中的互补信息来识别情绪的重要性。