Integrative analysis of data from multiple sources is critical to making generalizable discoveries. Associations that are consistently observed across multiple source populations are more likely to be generalized to target populations with possible distributional shifts. In this paper, we model the heterogeneous multi-source data with multiple high-dimensional regressions and make inferences for the maximin effect (Meinshausen, B{\"u}hlmann, AoS, 43(4), 1801--1830). The maximin effect provides a measure of stable associations across multi-source data. A significant maximin effect indicates that a variable has commonly shared effects across multiple source populations, and these shared effects may be generalized to a broader set of target populations. There are challenges associated with inferring maximin effects because its point estimator can have a non-standard limiting distribution. We devise a novel sampling method to construct valid confidence intervals for maximin effects. The proposed confidence interval attains a parametric length. This sampling procedure and the related theoretical analysis are of independent interest for solving other non-standard inference problems. Using genetic data on yeast growth in multiple environments, we demonstrate that the genetic variants with significant maximin effects have generalizable effects under new environments.
translated by 谷歌翻译
套索是一种高维回归的方法,当时,当协变量$ p $的订单数量或大于观测值$ n $时,通常使用它。由于两个基本原因,经典的渐近态性理论不适用于该模型:$(1)$正规风险是非平滑的; $(2)$估算器$ \ wideHat {\ boldsymbol {\ theta}} $与true参数vector $ \ boldsymbol {\ theta}^*$无法忽略。结果,标准的扰动论点是渐近正态性的传统基础。另一方面,套索估计器可以精确地以$ n $和$ p $大,$ n/p $的订单为一。这种表征首先是在使用I.I.D的高斯设计的情况下获得的。协变量:在这里,我们将其推广到具有非偏差协方差结构的高斯相关设计。这是根据更简单的``固定设计''模型表示的。我们在两个模型中各种数量的分布之间的距离上建立了非反应界限,它们在合适的稀疏类别中均匀地固定在信号上$ \ boldsymbol {\ theta}^*$。作为应用程序,我们研究了借助拉索的分布,并表明需要校正程度对于计算有效的置信区间是必要的。
translated by 谷歌翻译
个性化决定规则(IDR)是一个决定函数,可根据他/她观察到的特征分配给定的治疗。文献中的大多数现有工作考虑使用二进制或有限的许多治疗方案的设置。在本文中,我们专注于连续治疗设定,并提出跳跃间隔 - 学习,开发一个最大化预期结果的个性化间隔值决定规则(I2DR)。与推荐单一治疗的IDRS不同,所提出的I2DR为每个人产生了一系列治疗方案,使其在实践中实施更加灵活。为了获得最佳I2DR,我们的跳跃间隔学习方法估计通过跳转惩罚回归给予治疗和协变量的结果的条件平均值,并基于估计的结果回归函数来衍生相应的最佳I2DR。允许回归线是用于清晰的解释或深神经网络的线性,以模拟复杂的处理 - 协调会相互作用。为了实现跳跃间隔学习,我们开发了一种基于动态编程的搜索算法,其有效计算结果回归函数。当结果回归函数是处理空间的分段或连续功能时,建立所得I2DR的统计特性。我们进一步制定了一个程序,以推断(估计)最佳政策下的平均结果。进行广泛的模拟和对华法林研究的真实数据应用,以证明所提出的I2DR的经验有效性。
translated by 谷歌翻译
预测一组结果 - 而不是独特的结果 - 是统计学习中不确定性定量的有前途的解决方案。尽管有关于构建具有统计保证的预测集的丰富文献,但适应未知的协变量转变(实践中普遍存在的问题)还是一个严重的未解决的挑战。在本文中,我们表明具有有限样本覆盖范围保证的预测集是非信息性的,并提出了一种新型的无灵活分配方法PredSet-1Step,以有效地构建了在未知协方差转移下具有渐近覆盖范围保证的预测集。我们正式表明我们的方法是\ textIt {渐近上可能是近似正确},对大型样本的置信度有很好的覆盖误差。我们说明,在南非队列研究中,它在许多实验和有关HIV风险预测的数据集中实现了名义覆盖范围。我们的理论取决于基于一般渐近线性估计器的WALD置信区间覆盖范围的融合率的新结合。
translated by 谷歌翻译
Classical asymptotic theory for statistical inference usually involves calibrating a statistic by fixing the dimension $d$ while letting the sample size $n$ increase to infinity. Recently, much effort has been dedicated towards understanding how these methods behave in high-dimensional settings, where $d$ and $n$ both increase to infinity together. This often leads to different inference procedures, depending on the assumptions about the dimensionality, leaving the practitioner in a bind: given a dataset with 100 samples in 20 dimensions, should they calibrate by assuming $n \gg d$, or $d/n \approx 0.2$? This paper considers the goal of dimension-agnostic inference; developing methods whose validity does not depend on any assumption on $d$ versus $n$. We introduce an approach that uses variational representations of existing test statistics along with sample splitting and self-normalization to produce a new test statistic with a Gaussian limiting distribution, regardless of how $d$ scales with $n$. The resulting statistic can be viewed as a careful modification of degenerate U-statistics, dropping diagonal blocks and retaining off-diagonal blocks. We exemplify our technique for some classical problems including one-sample mean and covariance testing, and show that our tests have minimax rate-optimal power against appropriate local alternatives. In most settings, our cross U-statistic matches the high-dimensional power of the corresponding (degenerate) U-statistic up to a $\sqrt{2}$ factor.
translated by 谷歌翻译
本文提出了在多阶段实验的背景下的异质治疗效应的置信区间结构,以$ N $样品和高维,$ D $,混淆。我们的重点是$ d \ gg n $的情况,但获得的结果也适用于低维病例。我们展示了正则化估计的偏差,在高维变焦空间中不可避免,具有简单的双重稳固分数。通过这种方式,不需要额外的偏差,并且我们获得root $ N $推理结果,同时允许治疗和协变量的多级相互依赖性。记忆财产也没有假设;治疗可能取决于所有先前的治疗作业以及以前的所有多阶段混淆。我们的结果依赖于潜在依赖的某些稀疏假设。我们发现具有动态处理的强大推理所需的新产品率条件。
translated by 谷歌翻译
我们考虑在估计涉及依赖参数的高维滋扰的估计方程中估计一个低维参数。一个中心示例是因果推理中(局部)分位数处理效应((L)QTE)的有效估计方程,涉及在分位数以估计的分位数评估的协方差累积分布函数。借记机学习(DML)是一种使用灵活的机器学习方法估算高维滋扰的数据分解方法,但是将其应用于参数依赖性滋扰的问题是不切实际的。对于(L)QTE,DML要求我们学习整个协变量累积分布函数。相反,我们提出了局部偏见的机器学习(LDML),该学习避免了这一繁重的步骤,并且只需要对参数进行一次初始粗糙猜测而估算烦恼。对于(L)QTE,LDML仅涉及学习两个回归功能,这是机器学习方法的标准任务。我们证明,在松弛速率条件下,我们的估计量与使用未知的真实滋扰的不可行的估计器具有相同的有利渐近行为。因此,LDML值得注意的是,当我们必须控制许多协变量和/或灵活的关系时,如(l)QTES在((l)QTES)中,实际上可以有效地估算重要数量,例如(l)QTES。
translated by 谷歌翻译
This paper provides estimation and inference methods for a conditional average treatment effects (CATE) characterized by a high-dimensional parameter in both homogeneous cross-sectional and unit-heterogeneous dynamic panel data settings. In our leading example, we model CATE by interacting the base treatment variable with explanatory variables. The first step of our procedure is orthogonalization, where we partial out the controls and unit effects from the outcome and the base treatment and take the cross-fitted residuals. This step uses a novel generic cross-fitting method we design for weakly dependent time series and panel data. This method "leaves out the neighbors" when fitting nuisance components, and we theoretically power it by using Strassen's coupling. As a result, we can rely on any modern machine learning method in the first step, provided it learns the residuals well enough. Second, we construct an orthogonal (or residual) learner of CATE -- the Lasso CATE -- that regresses the outcome residual on the vector of interactions of the residualized treatment with explanatory variables. If the complexity of CATE function is simpler than that of the first-stage regression, the orthogonal learner converges faster than the single-stage regression-based learner. Third, we perform simultaneous inference on parameters of the CATE function using debiasing. We also can use ordinary least squares in the last two steps when CATE is low-dimensional. In heterogeneous panel data settings, we model the unobserved unit heterogeneity as a weakly sparse deviation from Mundlak (1978)'s model of correlated unit effects as a linear function of time-invariant covariates and make use of L1-penalization to estimate these models. We demonstrate our methods by estimating price elasticities of groceries based on scanner data. We note that our results are new even for the cross-sectional (i.i.d) case.
translated by 谷歌翻译
Testing the significance of a variable or group of variables $X$ for predicting a response $Y$, given additional covariates $Z$, is a ubiquitous task in statistics. A simple but common approach is to specify a linear model, and then test whether the regression coefficient for $X$ is non-zero. However, when the model is misspecified, the test may have poor power, for example when $X$ is involved in complex interactions, or lead to many false rejections. In this work we study the problem of testing the model-free null of conditional mean independence, i.e. that the conditional mean of $Y$ given $X$ and $Z$ does not depend on $X$. We propose a simple and general framework that can leverage flexible nonparametric or machine learning methods, such as additive models or random forests, to yield both robust error control and high power. The procedure involves using these methods to perform regressions, first to estimate a form of projection of $Y$ on $X$ and $Z$ using one half of the data, and then to estimate the expected conditional covariance between this projection and $Y$ on the remaining half of the data. While the approach is general, we show that a version of our procedure using spline regression achieves what we show is the minimax optimal rate in this nonparametric testing problem. Numerical experiments demonstrate the effectiveness of our approach both in terms of maintaining Type I error control, and power, compared to several existing approaches.
translated by 谷歌翻译
High-dimensional data can often display heterogeneity due to heteroscedastic variance or inhomogeneous covariate effects. Penalized quantile and expectile regression methods offer useful tools to detect heteroscedasticity in high-dimensional data. The former is computationally challenging due to the non-smooth nature of the check loss, and the latter is sensitive to heavy-tailed error distributions. In this paper, we propose and study (penalized) robust expectile regression (retire), with a focus on iteratively reweighted $\ell_1$-penalization which reduces the estimation bias from $\ell_1$-penalization and leads to oracle properties. Theoretically, we establish the statistical properties of the retire estimator under two regimes: (i) low-dimensional regime in which $d \ll n$; (ii) high-dimensional regime in which $s\ll n\ll d$ with $s$ denoting the number of significant predictors. In the high-dimensional setting, we carefully characterize the solution path of the iteratively reweighted $\ell_1$-penalized retire estimation, adapted from the local linear approximation algorithm for folded-concave regularization. Under a mild minimum signal strength condition, we show that after as many as $\log(\log d)$ iterations the final iterate enjoys the oracle convergence rate. At each iteration, the weighted $\ell_1$-penalized convex program can be efficiently solved by a semismooth Newton coordinate descent algorithm. Numerical studies demonstrate the competitive performance of the proposed procedure compared with either non-robust or quantile regression based alternatives.
translated by 谷歌翻译
由于在数据稀缺的设置中,交叉验证的性能不佳,我们提出了一个新颖的估计器,以估计数据驱动的优化策略的样本外部性能。我们的方法利用优化问题的灵敏度分析来估计梯度关于数据中噪声量的最佳客观值,并利用估计的梯度将策略的样本中的表现为依据。与交叉验证技术不同,我们的方法避免了为测试集牺牲数据,在训练和因此非常适合数据稀缺的设置时使用所有数据。我们证明了我们估计量的偏见和方差范围,这些问题与不确定的线性目标优化问题,但已知的,可能是非凸的,可行的区域。对于更专业的优化问题,从某种意义上说,可行区域“弱耦合”,我们证明结果更强。具体而言,我们在估算器的错误上提供明确的高概率界限,该估计器在策略类别上均匀地保持,并取决于问题的维度和策略类的复杂性。我们的边界表明,在轻度条件下,随着优化问题的尺寸的增长,我们的估计器的误差也会消失,即使可用数据的量仍然很小且恒定。说不同的是,我们证明我们的估计量在小型数据中的大规模政权中表现良好。最后,我们通过数值将我们提出的方法与最先进的方法进行比较,通过使用真实数据调度紧急医疗响应服务的案例研究。我们的方法提供了更准确的样本外部性能估计,并学习了表现更好的政策。
translated by 谷歌翻译
我们讨论了具有未知IV有效性的线性仪器变量(IV)模型中识别的基本问题。我们重新审视了流行的多数和多元化规则,并表明通常没有识别条件是“且仅在总体上”。假设“最稀少的规则”,该规则等同于多数规则,但在计算算法中变得运作,我们研究并证明了基于两步选择的其他IV估计器的非convex惩罚方法的优势,就两步选择而言选择一致性和单独弱IV的适应性。此外,我们提出了一种与识别条件保持一致的替代较低的惩罚,并同时提供甲骨文稀疏结构。与先前的文献相比,针对静脉强度较弱的估计仪得出了理想的理论特性。使用模拟证明了有限样本特性,并且选择和估计方法应用于有关贸易对经济增长的影响的经验研究。
translated by 谷歌翻译
统计推断中的主要范式取决于I.I.D.的结构。来自假设的无限人群的数据。尽管它取得了成功,但在复杂的数据结构下,即使在清楚无限人口所代表的内容的情况下,该框架在复杂的数据结构下仍然不灵活。在本文中,我们探讨了一个替代框架,在该框架中,推断只是对模型误差的不变性假设,例如交换性或符号对称性。作为解决这个不变推理问题的一般方法,我们提出了一个基于随机的过程。我们证明了该过程的渐近有效性的一般条件,并在许多数据结构中说明了,包括单向和双向布局中的群集误差。我们发现,通过残差随机化的不变推断具有三个吸引人的属性:(1)在弱且可解释的条件下是有效的,可以解决重型数据,有限聚类甚至一些高维设置的问题。 (2)它在有限样品中是可靠的,因为它不依赖经典渐近学所需的规律性条件。 (3)它以适应数据结构的统一方式解决了推断问题。另一方面,诸如OLS或Bootstrap之类的经典程序以I.I.D.为前提。结构,只要实际问题结构不同,就需要修改。经典框架中的这种不匹配导致了多种可靠的误差技术和自举变体,这些变体经常混淆应用研究。我们通过广泛的经验评估证实了这些发现。残留随机化对许多替代方案的表现有利,包括可靠的误差方法,自举变体和分层模型。
translated by 谷歌翻译
即使是最精确的经济数据集也具有嘈杂,丢失,离散化或私有化的变量。实证研究的标准工作流程涉及数据清理,然后是数据分析,通常忽略数据清洁的偏差和方差后果。我们制定了具有损坏数据的因果推理的半造型模型,以包括数据清洁和数据分析。我们提出了一种新的数据清洁,估计和推理的新的端到端程序,以及数据清洁调整的置信区间。通过有限的示例参数,我们证明了因果关系参数的估算器的一致性,高斯近似和半游戏效率。 Gaussian近似的速率为N ^ { - 1/2} $,如平均治疗效果,如平均治疗效果,并且优雅地为当地参数劣化,例如特定人口统计的异构治疗效果。我们的关键假设是真正的协变量是较低的等级。在我们的分析中,我们为矩阵完成,统计学习和半统计统计提供了非对症的理论贡献。我们验证了数据清洁调整的置信区间隔的覆盖范围校准,以类似于2020年美国人口普查中实施的差异隐私。
translated by 谷歌翻译
我们解决了如何在没有严格缩放条件的情况下实现分布式分数回归中最佳推断的问题。由于分位数回归(QR)损失函数的非平滑性质,这是具有挑战性的,这使现有方法的使用无效。难度通过应用于本地(每个数据源)和全局目标函数的双光滑方法解决。尽管依赖局部和全球平滑参数的精致组合,但分位数回归模型是完全参数的,从而促进了解释。在低维度中,我们为顺序定义的分布式QR估计器建立了有限样本的理论框架。这揭示了通信成本和统计错误之间的权衡。我们进一步讨论并比较了基于WALD和得分型测试和重采样技术的反转的几种替代置信集结构,并详细介绍了对更极端分数系数有效的改进。在高维度中,采用了一个稀疏的框架,其中提出的双滑目标功能与$ \ ell_1 $ -penalty相辅相成。我们表明,相应的分布式QR估计器在近乎恒定的通信回合之后达到了全球收敛率。一项彻底的模拟研究进一步阐明了我们的发现。
translated by 谷歌翻译
We develop a general framework for distribution-free predictive inference in regression, using conformal inference. The proposed methodology allows for the construction of a prediction band for the response variable using any estimator of the regression function. The resulting prediction band preserves the consistency properties of the original estimator under standard assumptions, while guaranteeing finite-sample marginal coverage even when these assumptions do not hold. We analyze and compare, both empirically and theoretically, the two major variants of our conformal framework: full conformal inference and split conformal inference, along with a related jackknife method. These methods offer different tradeoffs between statistical accuracy (length of resulting prediction intervals) and computational efficiency. As extensions, we develop a method for constructing valid in-sample prediction intervals called rank-one-out conformal inference, which has essentially the same computational efficiency as split conformal inference. We also describe an extension of our procedures for producing prediction bands with locally varying length, in order to adapt to heteroskedascity in the data. Finally, we propose a model-free notion of variable importance, called leave-one-covariate-out or LOCO inference. Accompanying this paper is an R package conformalInference that implements all of the proposals we have introduced. In the spirit of reproducibility, all of our empirical results can also be easily (re)generated using this package.
translated by 谷歌翻译
我们研究了称为“乐观速率”(Panchenko 2002; Srebro等,2010)的统一收敛概念,用于与高斯数据的线性回归。我们的精致分析避免了现有结果中的隐藏常量和对数因子,这已知在高维设置中至关重要,特别是用于了解插值学习。作为一个特殊情况,我们的分析恢复了Koehler等人的保证。(2021年),在良性过度的过度条件下,严格地表征了低规范内插器的人口风险。但是,我们的乐观速度绑定还分析了具有任意训练错误的预测因子。这使我们能够在随机设计下恢复脊和套索回归的一些经典统计保障,并有助于我们在过度参数化制度中获得精确了解近端器的过度风险。
translated by 谷歌翻译
在因果推理和强盗文献中,基于观察数据的线性功能估算线性功能的问题是规范的。我们分析了首先估计治疗效果函数的广泛的两阶段程序,然后使用该数量来估计线性功能。我们证明了此类过程的均方误差上的非反应性上限:这些边界表明,为了获得非反应性最佳程序,应在特定加权$ l^2 $中最大程度地估算治疗效果的误差。 -规范。我们根据该加权规范的约束回归分析了两阶段的程序,并通过匹配非轴突局部局部最小值下限,在有限样品中建立了实例依赖性最优性。这些结果表明,除了取决于渐近效率方差之外,最佳的非质子风险除了取决于样本量支持的最富有函数类别的真实结果函数与其近似类别之间的加权规范距离。
translated by 谷歌翻译
离线政策评估(OPE)被认为是强化学习(RL)的基本且具有挑战性的问题。本文重点介绍了基于从无限 - 马尔可夫决策过程的框架下从可能不同策略生成的预收集的数据的目标策略的价值估计。由RL最近开发的边际重要性采样方法和因果推理中的协变量平衡思想的动机,我们提出了一个新颖的估计器,具有大约投影的国家行动平衡权重,以进行策略价值估计。我们获得了这些权重的收敛速率,并表明拟议的值估计量在技术条件下是半参数有效的。就渐近学而言,我们的结果比例均以每个轨迹的轨迹数量和决策点的数量进行扩展。因此,当决策点数量分歧时,仍然可以使用有限的受试者实现一致性。此外,我们开发了一个必要且充分的条件,以建立贝尔曼操作员在政策环境中的适当性,这表征了OPE的困难,并且可能具有独立的利益。数值实验证明了我们提出的估计量的有希望的性能。
translated by 谷歌翻译
将回归系数融合到均匀组中可以揭示在每个组内共享共同值的系数。这种扩展均匀性降低了参数空间的内在尺寸,并释放统计学精度。我们提出并调查了一个名为$ l_0 $ -fusion的新的组合分组方法,这些方法可用于混合整数优化(MIO)。在统计方面,我们识别称为分组灵敏度的基本量,该基本量为恢复真实组的难度。我们展示$ l_0 $ -fusion在分组灵敏度的最弱需求下实现了分组一致性:如果违反了这一要求,则小组拼写的最低风险将无法收敛到零。此外,我们展示了在高维制度中,可以使用无需任何必要的统计效率损失的确保筛选特征,同时降低计算成本的校正特征耦合耦合的$ L_0 $ -Fusion。在算法方面,我们为$ l_0 $ -fusion提供了一个mio配方,以及温暖的开始策略。仿真和实际数据分析表明,在分组准确性方面,$ L_0 $ -FUSUS展示其竞争对手的优势。
translated by 谷歌翻译