Objectives: Discussions of fairness in criminal justice risk assessments typically lack conceptual precision. Rhetoric too often substitutes for careful analysis. In this paper, we seek to clarify the tradeoffs between different kinds of fairness and between fairness and accuracy.Methods: We draw on the existing literatures in criminology, computer science and statistics to provide an integrated examination of fairness and accuracy in criminal justice risk assessments. We also provide an empirical illustration using data from arraignments.Results: We show that there are at least six kinds of fairness, some of which are incompatible with one another and with accuracy.Conclusions: Except in trivial cases, it is impossible to maximize accuracy and fairness at the same time, and impossible simultaneously to satisfy all kinds of fairness. In practice, a major complication is different base rates across different legally protected groups. There is a need to consider challenging tradeoffs.

公平性是确保机器学习（ML）预测系统不会歧视特定个人或整个子人群（尤其是少数族裔）的重要要求。鉴于观察公平概念的固有主观性，文献中已经引入了几种公平概念。本文是一项调查，说明了通过大量示例和场景之间的公平概念之间的微妙之处。此外，与文献中的其他调查不同，它解决了以下问题：哪种公平概念最适合给定的现实世界情景，为什么？我们试图回答这个问题的尝试包括（1）确定手头现实世界情景的一组与公平相关的特征，（2）分析每个公平概念的行为，然后（3）适合这两个元素以推荐每个特定设置中最合适的公平概念。结果总结在决策图中可以由从业者和政策制定者使用，以导航相对较大的ML目录。

Advocates of algorithmic techniques like data mining argue that these techniques eliminate human biases from the decision-making process. But an algorithm is only as good as the data it works with. Data is frequently imperfect in ways that allow these algorithms to inherit the prejudices of prior decision makers. In other cases, data may simply reflect the widespread biases that persist in society at large. In still others, data mining can discover surprisingly useful regularities that are really just preexisting patterns of exclusion and inequality. Unthinking reliance on data mining can deny historically disadvantaged and vulnerable groups full participation in society. Worse still, because the resulting discrimination is almost always an unintentional emergent property of the algorithm's use rather than a conscious choice by its programmers, it can be unusually hard to identify the source of the problem or to explain it to a court. This Essay examines these concerns through the lens of American antidiscrimination law-more particularly, through Title

目的：我们研究使用机器学习（ML）模型的可解释的累入预测，并在预测能力，稀疏性和公平性方面分析性能。与以前的作品不同，本研究列举了输出概率而不是二进制预测的可解释模型，并使用定量公平定义来评估模型。本研究还研究了模型是否可以横跨地理位置概括。方法：我们在佛罗里达州和肯塔基州的两个不同的刑事核查数据集上生成了黑盒和可解释的ML模型。我们将这些模型的预测性能和公平与目前用于司法系统中使用的两种方法进行了比较，以预测审前常规率：Arnold PSA和Compas。我们评估了所有模型的预测性能，可以在两次跨越两次预测六种不同类型犯罪的模型。结果：几种可解释的ML模型可以预测常规和黑盒ML模型，比Compas或Arnold PSA更准确。这些模型在实践中可能有用。类似于Arnold PSA，这些可解释模型中的一些可以作为一个简单的表格写入。其他可以使用一组可视化显示。我们的地理分析表明ML模型应分开培训，以便单独的位置并随时间更新。我们还为可​​解释模型提供了公平分析。结论：可解释的机器学习模型可以在预测准确性和公平性方面表现，也可以表现，也可以表现，也可以执行不可解释的方法和目前使用的风险评估尺度。机器学习模型对于单独培训，可以更准确地进行不同的位置，并保持最新。

测量黑匣子预测算法中变量重要性的最流行方法是利用合成输入，这些输入结合了来自多个受试者的预测变量。这些输入可能是不可能的，身体上不可能的，甚至在逻辑上是不可能的。结果，对这种情况的预测可以基于数据，这与对黑匣子的训练非常不同。我们认为，当解释使用此类值时，用户不能相信预测算法的决定的解释。取而代之的是，我们主张一种称为同类沙普利的方法，该方法基于经济游戏理论，与大多数其他游戏理论方法不同，它仅使用实际观察到的数据来量化可变重要性。莎普利队的同伙通过缩小判断的主题的缩小，被认为与一个或多个功能上的目标主题相似。如果使用它来缩小队列对队列平均值有很大的不同，则功能很重要。我们在算法公平问题上进行了说明，其中必须将重要性归因于未经训练模型的保护变量。对于每个主题和每个预测变量，我们可以计算该预测因子对受试者的预测响应或对其实际响应的重要性。这些值可以汇总，例如在所有黑色受试者上，我们提出了一个贝叶斯引导程序来量化个人和骨料莎普利值的不确定性。

近年来，解决机器学习公平性（ML）和自动决策的问题引起了处理人工智能的科学社区的大量关注。已经提出了ML中的公平定义的一种不同的定义，认为不同概念是影响人口中个人的“公平决定”的不同概念。这些概念之间的精确差异，含义和“正交性”尚未在文献中完全分析。在这项工作中，我们试图在这个解释中汲取一些订单。

Algorithms are now regularly used to decide whether defendants awaiting trial are too dangerous to be released back into the community. In some cases, black defendants are substantially more likely than white defendants to be incorrectly classi ed as high risk. To mitigate such disparities, several techniques have recently been proposed to achieve algorithmic fairness. Here we reformulate algorithmic fairness as constrained optimization: the objective is to maximize public safety while satisfying formal fairness constraints designed to reduce racial disparities. We show that for several past de nitions of fairness, the optimal algorithms that result require detaining defendants above race-speci c risk thresholds. We further show that the optimal unconstrained algorithm requires applying a single, uniform threshold to all defendants. e unconstrained algorithm thus maximizes public safety while also satisfying one important understanding of equality: that all individuals are held to the same standard, irrespective of race. Because the optimal constrained and unconstrained algorithms generally di er, there is tension between improving public safety and satisfying prevailing notions of algorithmic fairness. By examining data from Broward County, Florida, we show that this trade-o can be large in practice. We focus on algorithms for pretrial release decisions, but the principles we discuss apply to other domains, and also to human decision makers carrying out structured decision rules.

基于AI和机器学习的决策系统已在各种现实世界中都使用，包括医疗保健，执法，教育和金融。不再是牵强的，即设想一个未来，自治系统将推动整个业务决策，并且更广泛地支持大规模决策基础设施以解决社会最具挑战性的问题。当人类做出决定时，不公平和歧视的问题普遍存在，并且当使用几乎没有透明度，问责制和公平性的机器做出决定时（或可能会放大）。在本文中，我们介绍了\ textit {Causal公平分析}的框架，目的是填补此差距，即理解，建模，并可能解决决策设置中的公平性问题。我们方法的主要见解是将观察到数据中存在的差异的量化与基本且通常是未观察到的因果机制收集的因果机制的收集，这些机制首先会产生差异，挑战我们称之为因果公平的基本问题分析（FPCFA）。为了解决FPCFA，我们研究了分解差异和公平性的经验度量的问题，将这种变化归因于结构机制和人群的不同单位。我们的努力最终达到了公平地图，这是组织和解释文献中不同标准之间关系的首次系统尝试。最后，我们研究了进行因果公平分析并提出一本公平食谱的最低因果假设，该假设使数据科学家能够评估不同影响和不同治疗的存在。

We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy.In line with other studies, our notion is oblivious: it depends only on the joint statistics of the predictor, the target and the protected attribute, but not on interpretation of individual features. We study the inherent limits of defining and identifying biases based on such oblivious measures, outlining what can and cannot be inferred from different oblivious tests.We illustrate our notion using a case study of FICO credit scores.

We study critical systems that allocate scarce resources to satisfy basic needs, such as homeless services that provide housing. These systems often support communities disproportionately affected by systemic racial, gender, or other injustices, so it is crucial to design these systems with fairness considerations in mind. To address this problem, we propose a framework for evaluating fairness in contextual resource allocation systems that is inspired by fairness metrics in machine learning. This framework can be applied to evaluate the fairness properties of a historical policy, as well as to impose constraints in the design of new (counterfactual) allocation policies. Our work culminates with a set of incompatibility results that investigate the interplay between the different fairness metrics we propose. Notably, we demonstrate that: 1) fairness in allocation and fairness in outcomes are usually incompatible; 2) policies that prioritize based on a vulnerability score will usually result in unequal outcomes across groups, even if the score is perfectly calibrated; 3) policies using contextual information beyond what is needed to characterize baseline risk and treatment effects can be fairer in their outcomes than those using just baseline risk and treatment effects; and 4) policies using group status in addition to baseline risk and treatment effects are as fair as possible given all available information. Our framework can help guide the discussion among stakeholders in deciding which fairness metrics to impose when allocating scarce resources.

“算法公平性”的新兴领域提供了一种用于推理算法预测和决策的公平的一组新颖的方法。甚至作为算法公平已经成为提高域名在此类公共政策中平等的努力的突出成分，它也面临着显着的限制和批评。最基本的问题是称为“公平性不可能”的数学结果（公平的数学定义之间的不相容性）。此外，满足公平标准的许多算法实际上加剧了压迫。这两个问题呼吁质疑算法公平是否可以在追求平等中发挥富有成效的作用。在本文中，我将这些问题诊断为算法公平方法的乘积，并提出了该领域的替代路径。 “正式算法公平”的主导方法遭受了基本限制：它依赖于狭窄的分析框架，这些分析框架仅限于特定决策过程，孤立于这些决定的背景。鉴于这种缺点，我借鉴了法律和哲学的实质性平等的理论，提出了一种替代方法：“实质性算法公平。”实质性算法公平性采用更广泛的范围来分析公平性，超出特定决策点，以考虑社会等级，以及算法促进的决策的影响。因此，实质性算法公平表明，改革，使压迫压迫和逃避公平的不可能性。此外，实质性算法公平呈现出算法公平领域的新方向：远离“公平性”的正式数学模型，并朝着算法促进平等的实质性评估。

Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.

在本文中，我们对机器学习文献中统计公平性辩论的两个标准进行道德分析：1）组之间的校准和2）组之间的假阳性和假负率的平等。在我们的论文中，我们专注于支持任何一种措施的道德论点。群体校准与假阳性和假负率平等之间的冲突是有关从业者团体公平定义的辩论中的核心问题之一。对于任何彻底的道德分析，必须明确并正确定义公平性的含义。对于我们的论文，我们将公平等同于（非）歧视，这是关于群体公平的讨论中的合理理解。更具体地说，我们将其等同于表面上的错误歧视，从某种意义上说，这用于Lippert-Rasmussen教授对此定义的处理。在本文中，我们认为违反群体校准的行为在某些情况下可能是不公平的，但在其他情况下并不公平。这与文献中已经提出的主张一致，即应以对上下文敏感的方式定义算法公平性。最重要的实际含义是基于基于示例的论点，即公平性需要组间校准或假阳性/假阴性率的平等性，并没有概括。因为在一种情况下，组校准可能是公平的要求，而不是另一种情况。

业务分析（BA）的广泛采用带来了财务收益和提高效率。但是，当BA以公正的影响为决定时，这些进步同时引起了人们对法律和道德挑战的不断增加。作为对这些关注的回应，对算法公平性的新兴研究涉及算法输出，这些算法可能会导致不同的结果或其他形式的对人群亚组的不公正现象，尤其是那些在历史上被边缘化的人。公平性是根据法律合规，社会责任和效用是相关的；如果不充分和系统地解决，不公平的BA系统可能会导致社会危害，也可能威胁到组织自己的生存，其竞争力和整体绩效。本文提供了有关算法公平的前瞻性，注重BA的评论。我们首先回顾有关偏见来源和措施的最新研究以及偏见缓解算法。然后，我们对公用事业关系的详细讨论进行了详细的讨论，强调经常假设这两种构造之间经常是错误的或短视的。最后，我们通过确定企业学者解决有效和负责任的BA的关键的有影响力的公开挑战的机会来绘制前进的道路。

Recommender systems can strongly influence which information we see online, e.g., on social media, and thus impact our beliefs, decisions, and actions. At the same time, these systems can create substantial business value for different stakeholders. Given the growing potential impact of such AI-based systems on individuals, organizations, and society, questions of fairness have gained increased attention in recent years. However, research on fairness in recommender systems is still a developing area. In this survey, we first review the fundamental concepts and notions of fairness that were put forward in the area in the recent past. Afterward, through a review of more than 150 scholarly publications, we present an overview of how research in this field is currently operationalized, e.g., in terms of general research methodology, fairness measures, and algorithmic approaches. Overall, our analysis of recent works points to specific research gaps. In particular, we find that in many research works in computer science, very abstract problem operationalizations are prevalent, and questions of the underlying normative claims and what represents a fair recommendation in the context of a given application are often not discussed in depth. These observations call for more interdisciplinary research to address fairness in recommendation in a more comprehensive and impactful manner.

算法公平吸引了机器学习社区越来越多的关注。文献中提出了各种定义，但是它们之间的差异和联系并未清楚地解决。在本文中，我们回顾并反思了机器学习文献中先前提出的各种公平概念，并试图与道德和政治哲学，尤其是正义理论的论点建立联系。我们还从动态的角度考虑了公平的询问，并进一步考虑了当前预测和决策引起的长期影响。鉴于特征公平性的差异，我们提出了一个流程图，该流程图包括对数据生成过程，预测结果和诱导的影响的不同类型的公平询问的隐式假设和预期结果。本文展示了与任务相匹配的重要性（人们希望执行哪种公平性）和实现预期目的的手段（公平分析的范围是什么，什么是适当的分析计划）。

作为一种预测模型的评分系统具有可解释性和透明度的显着优势，并有助于快速决策。因此，评分系统已广泛用于各种行业，如医疗保健和刑事司法。然而，这些模型中的公平问题长期以来一直受到批评，并且使用大数据和机器学习算法在评分系统的构建中提高了这个问题。在本文中，我们提出了一般框架来创建公平知识，数据驱动评分系统。首先，我们开发一个社会福利功能，融入了效率和群体公平。然后，我们将社会福利最大化问题转换为机器学习中的风险最小化任务，并在混合整数编程的帮助下导出了公平感知评分系统。最后，导出了几种理论界限用于提供参数选择建议。我们拟议的框架提供了适当的解决方案，以解决进程中的分组公平问题。它使政策制定者能够设置和定制其所需的公平要求以及其他特定于应用程序的约束。我们用几个经验数据集测试所提出的算法。实验证据支持拟议的评分制度在实现利益攸关方的最佳福利以及平衡可解释性，公平性和效率的需求方面的有效性。

应用标准机器学习方法可以在不同的人口组中产生不等的结果。当在现实世界中使用时，这些不公平可能会产生负面影响。这激发了近年来通过机器学习模型公平分类的各种方法的发展。在本文中，我们考虑修改黑箱机器学习分类器的预测的问题，以便在多种多组设置中实现公平性。为实现这一目标，我们在Hardt等人中扩展了“后处理”方法。 2016年，侧重于二进制分类的公平，以实现公平的多种式分类。我们探讨我们的方法通过系统合成实验产生公平和准确的预测，并在几个公开的现实世界应用数据集中评估歧视 - 公平权衡。我们发现整体而言，当数据集中的个体的数量相对于类和受保护组的数量很高时，我们的方法可以精确地产生轻微的滴度并强制执行公平性。

Machine learning can impact people with legal or ethical consequences when it is used to automate decisions in areas such as insurance, lending, hiring, and predictive policing. In many of these scenarios, previous decisions have been made that are unfairly biased against certain subpopulations, for example those of a particular race, gender, or sexual orientation. Since this past data may be biased, machine learning predictors must account for this to avoid perpetuating or creating discriminatory practices. In this paper, we develop a framework for modeling fairness using tools from causal inference. Our definition of counterfactual fairness captures the intuition that a decision is fair towards an individual if it is the same in (a) the actual world and (b) a counterfactual world where the individual belonged to a different demographic group. We demonstrate our framework on a real-world problem of fair prediction of success in law school. * Equal contribution. This work was done while JL was a Research Fellow at the Alan Turing Institute. 2 https://obamawhitehouse.archives.gov/blog/2016/05/04/big-risks-big-opportunities-intersection-big-dataand-civil-rights 31st Conference on Neural Information Processing Systems (NIPS 2017),

What does it mean for an algorithm to be biased? In U.S. law, unintentional bias is encoded via disparate impact, which occurs when a selection process has widely different outcomes for different groups, even as it appears to be neutral. This legal determination hinges on a definition of a protected class (ethnicity, gender) and an explicit description of the process.When computers are involved, determining disparate impact (and hence bias) is harder. It might not be possible to disclose the process. In addition, even if the process is open, it might be hard to elucidate in a legal setting how the algorithm makes its decisions. Instead of requiring access to the process, we propose making inferences based on the data it uses.We present four contributions. First, we link disparate impact to a measure of classification accuracy that while known, has received relatively little attention. Second, we propose a test for disparate impact based on how well the protected class can be predicted from the other attributes. Third, we describe methods by which data might be made unbiased. Finally, we present empirical evidence supporting the effectiveness of our test for disparate impact and our approach for both masking bias and preserving relevant information in the data. Interestingly, our approach resembles some actual selection practices that have recently received legal scrutiny.