机器学习(ML)可解释性技术可以揭示数据中的不良模式,这些模型模型开发以做出预测 - 一旦部署就会​​造成危害。但是,如何采取行动解决这些模式并不总是很清楚。在ML与人类计算机互动研究人员,医师和数据科学家之间的合作中,我们开发了GAM Changer,这是第一个互动系统,可帮助域专家和数据科学家轻松,负责任地编辑通用的添加剂模型(GAM)和修复有问题的模式。借助新颖的交互技术,我们的工具将可解释性置于行动中 - 使用户能够分析,验证和使模型行为与知识和价值相结合。医师已经开始使用我们的工具来调查和修复肺炎和败血症的风险预测模型,以及在不同领域工作的7位数据科学家的评估突出显示我们的工具易于使用,满足他们的模型编辑需求,并适合他们当前的工作流程。我们的工具以现代网络技术为基础,在用户的网络浏览器或计算笔记本电脑中本地运行,从而降低了使用的障碍。 GAM Changer可在以下公共演示链接中获得:https://interpret.ml/gam-changer。
translated by 谷歌翻译
数据对于机器学习(ML)模型的开发和评估至关重要。但是,在部署所得模型时,使用有问题或不适当的数据集可能会造成危害。为了通过对数据集进行更故意的反思和创建过程的透明度来鼓励负责任的练习,研究人员和从业人员已开始倡导增加数据文档,并提出了几个数据文档框架。但是,几乎没有研究这些数据文档框架是否满足创建和消费数据集的ML从业者的需求。为了解决这一差距,我们着手了解ML从业人员的数据文档感知,需求,挑战和Desiderata,目的是推导设计要求,以便为将来的数据文档框架提供信息。我们对一家大型国际技术公司的14名ML从业者进行了一系列半结构化访谈。我们让他们回答从数据集的数据表中提取的问题列表(Gebru,2021)。我们的发现表明,目前的数据文档方法在很大程度上是临时的,而且本质上是近视的。参与者表达了对数据文档框架的需求,可以适应其上下文,并将其集成到现有的工具和工作流程中,并尽可能自动化。尽管事实上,数据文档框架通常是从负责人的AI的角度出发的,但参与者并未在他们被要求回答的问题与负责的AI含义之间建立联系。此外,参与者通常会在数据集消费者的需求中优先考虑,并提供了不熟悉其数据集可能需要知道的信息。基于这些发现,我们为将来的数据文档框架得出了七个设计要求。
translated by 谷歌翻译
最近在可解释的机器学习中的进展(ML)研究表明,模型利用数据中的不良模式来进行预测,这可能导致部署危害。但是,尚不清楚我们如何解决这些模型。我们介绍了我们正在进行的工作,游戏改变者,一个开源交互式系统,以帮助数据科学家和领域专家轻松且负责任地编辑其广义添加剂模型(Gams)。通过新颖的可视化技术,我们的工具将可解释性投入到行动 - 使人类用户能够分析,验证和对齐模型行为与他们的知识和价值。使用现代Web技术建造,我们的工具在用户的计算笔记本或Web浏览器中在本地运行,而无需额外计算资源,降低屏障以创建更负责的ML模型。Gam更换器可在https://interpret.ml/gam-changer中获得。
translated by 谷歌翻译
We present a novel corpus for French dialect identification comprising 413,522 French text samples collected from public news websites in Belgium, Canada, France and Switzerland. To ensure an accurate estimation of the dialect identification performance of models, we designed the corpus to eliminate potential biases related to topic, writing style, and publication source. More precisely, the training, validation and test splits are collected from different news websites, while searching for different keywords (topics). This leads to a French cross-domain (FreCDo) dialect identification task. We conduct experiments with four competitive baselines, a fine-tuned CamemBERT model, an XGBoost based on fine-tuned CamemBERT features, a Support Vector Machines (SVM) classifier based on fine-tuned CamemBERT features, and an SVM based on word n-grams. Aside from presenting quantitative results, we also make an analysis of the most discriminative features learned by CamemBERT. Our corpus is available at https://github.com/MihaelaGaman/FreCDo.
translated by 谷歌翻译
Causal deep learning (CDL) is a new and important research area in the larger field of machine learning. With CDL, researchers aim to structure and encode causal knowledge in the extremely flexible representation space of deep learning models. Doing so will lead to more informed, robust, and general predictions and inference -- which is important! However, CDL is still in its infancy. For example, it is not clear how we ought to compare different methods as they are so different in their output, the way they encode causal knowledge, or even how they represent this knowledge. This is a living paper that categorises methods in causal deep learning beyond Pearl's ladder of causation. We refine the rungs in Pearl's ladder, while also adding a separate dimension that categorises the parametric assumptions of both input and representation, arriving at the map of causal deep learning. Our map covers machine learning disciplines such as supervised learning, reinforcement learning, generative modelling and beyond. Our paradigm is a tool which helps researchers to: find benchmarks, compare methods, and most importantly: identify research gaps. With this work we aim to structure the avalanche of papers being published on causal deep learning. While papers on the topic are being published daily, our map remains fixed. We open-source our map for others to use as they see fit: perhaps to offer guidance in a related works section, or to better highlight the contribution of their paper.
translated by 谷歌翻译
It is important to guarantee that machine learning algorithms deployed in the real world do not result in unfairness or unintended social consequences. Fair ML has largely focused on the protection of single attributes in the simpler setting where both attributes and target outcomes are binary. However, the practical application in many a real-world problem entails the simultaneous protection of multiple sensitive attributes, which are often not simply binary, but continuous or categorical. To address this more challenging task, we introduce FairCOCCO, a fairness measure built on cross-covariance operators on reproducing kernel Hilbert Spaces. This leads to two practical tools: first, the FairCOCCO Score, a normalised metric that can quantify fairness in settings with single or multiple sensitive attributes of arbitrary type; and second, a subsequent regularisation term that can be incorporated into arbitrary learning objectives to obtain fair predictors. These contributions address crucial gaps in the algorithmic fairness literature, and we empirically demonstrate consistent improvements against state-of-the-art techniques in balancing predictive power and fairness on real-world datasets.
translated by 谷歌翻译
While there have been a number of remarkable breakthroughs in machine learning (ML), much of the focus has been placed on model development. However, to truly realize the potential of machine learning in real-world settings, additional aspects must be considered across the ML pipeline. Data-centric AI is emerging as a unifying paradigm that could enable such reliable end-to-end pipelines. However, this remains a nascent area with no standardized framework to guide practitioners to the necessary data-centric considerations or to communicate the design of data-centric driven ML systems. To address this gap, we propose DC-Check, an actionable checklist-style framework to elicit data-centric considerations at different stages of the ML pipeline: Data, Training, Testing, and Deployment. This data-centric lens on development aims to promote thoughtfulness and transparency prior to system development. Additionally, we highlight specific data-centric AI challenges and research opportunities. DC-Check is aimed at both practitioners and researchers to guide day-to-day development. As such, to easily engage with and use DC-Check and associated resources, we provide a DC-Check companion website (https://www.vanderschaar-lab.com/dc-check/). The website will also serve as an updated resource as methods and tooling evolve over time.
translated by 谷歌翻译
在许多情况下,更简单的模型比更复杂的模型更可取,并且该模型复杂性的控制是机器学习中许多方法的目标,例如正则化,高参数调整和体系结构设计。在深度学习中,很难理解复杂性控制的潜在机制,因为许多传统措施并不适合深度神经网络。在这里,我们开发了几何复杂性的概念,该概念是使用离散的dirichlet能量计算的模型函数变异性的量度。使用理论论据和经验结果的结合,我们表明,许多常见的训练启发式方法,例如参数规范正规化,光谱规范正则化,平稳性正则化,隐式梯度正则化,噪声正则化和参数初始化的选择,都可以控制几何学复杂性,并提供一个统一的框架,以表征深度学习模型的行为。
translated by 谷歌翻译
我们研究并介绍了复杂和双色复合物环境中的新梯度运算符,这是受自适应线性神经元(Adaline)在1960年发明的著名的最少均等(LMS)算法的启发。这些梯度运算符将用于制定最小二平方(BLM)算法的新学习规则。这种方法既扩展了经典的真实和复杂的LMS算法。
translated by 谷歌翻译
基于概念的解释允许通过用户指定的概念镜头来了解深神经网络(DNN)的预测。现有方法假设说明概念的示例是在DNN潜在空间的固定方向上映射的。当这种情况下,该概念可以用指向该方向的概念激活向量(CAV)表示。在这项工作中,我们建议通过允许概念示例散布在DNN潜在空间中的不同群集中来放松这一假设。然后,每个概念都由DNN潜在空间的区域表示,该区域包括这些簇,我们称为概念激活区域(CAR)。为了使这个想法形式化,我们介绍了基于内核技巧和支持向量分类器的CAV形式主义的扩展。这种汽车形式主义产生了基于全球概念的解释和基于本地概念的特征重要性。我们证明,用径向核建造的汽车解释在潜在空间等法下是不变的。这样,汽车将相同的解释分配给具有相同几何形状的潜在空间。我们进一步证明汽车提供(1)更准确地描述了概念如何散布在DNN的潜在空间中; (2)更接近人类概念注释和(3)基于概念的特征的重要性重要性的全球解释,这些特征的重要性是有意义地相互关联的。最后,我们使用汽车表明DNN可以自主重新发现已知的科学概念,例如前列腺癌分级系统。
translated by 谷歌翻译