我们讨论了具有未知IV有效性的线性仪器变量（IV）模型中识别的基本问题。我们重新审视了流行的多数和多元化规则，并表明通常没有识别条件是“且仅在总体上”。假设“最稀少的规则”，该规则等同于多数规则，但在计算算法中变得运作，我们研究并证明了基于两步选择的其他IV估计器的非convex惩罚方法的优势，就两步选择而言选择一致性和单独弱IV的适应性。此外，我们提出了一种与识别条件保持一致的替代较低的惩罚，并同时提供甲骨文稀疏结构。与先前的文献相比，针对静脉强度较弱的估计仪得出了理想的理论特性。使用模拟证明了有限样本特性，并且选择和估计方法应用于有关贸易对经济增长的影响的经验研究。

High-dimensional data can often display heterogeneity due to heteroscedastic variance or inhomogeneous covariate effects. Penalized quantile and expectile regression methods offer useful tools to detect heteroscedasticity in high-dimensional data. The former is computationally challenging due to the non-smooth nature of the check loss, and the latter is sensitive to heavy-tailed error distributions. In this paper, we propose and study (penalized) robust expectile regression (retire), with a focus on iteratively reweighted $\ell_1$-penalization which reduces the estimation bias from $\ell_1$-penalization and leads to oracle properties. Theoretically, we establish the statistical properties of the retire estimator under two regimes: (i) low-dimensional regime in which $d \ll n$; (ii) high-dimensional regime in which $s\ll n\ll d$ with $s$ denoting the number of significant predictors. In the high-dimensional setting, we carefully characterize the solution path of the iteratively reweighted $\ell_1$-penalized retire estimation, adapted from the local linear approximation algorithm for folded-concave regularization. Under a mild minimum signal strength condition, we show that after as many as $\log(\log d)$ iterations the final iterate enjoys the oracle convergence rate. At each iteration, the weighted $\ell_1$-penalized convex program can be efficiently solved by a semismooth Newton coordinate descent algorithm. Numerical studies demonstrate the competitive performance of the proposed procedure compared with either non-robust or quantile regression based alternatives.

套索是一种高维回归的方法，当时，当协变量$ p $的订单数量或大于观测值$ n $时，通常使用它。由于两个基本原因，经典的渐近态性理论不适用于该模型：$（1）$正规风险是非平滑的； $（2）$估算器$ \ wideHat {\ boldsymbol {\ theta}} $与true参数vector $ \ boldsymbol {\ theta}^*$无法忽略。结果，标准的扰动论点是渐近正态性的传统基础。另一方面，套索估计器可以精确地以$ n $和$ p $大，$ n/p $的订单为一。这种表征首先是在使用I.I.D的高斯设计的情况下获得的。协变量：在这里，我们将其推广到具有非偏差协方差结构的高斯相关设计。这是根据更简单的``固定设计''模型表示的。我们在两个模型中各种数量的分布之间的距离上建立了非反应界限，它们在合适的稀疏类别中均匀地固定在信号上$ \ boldsymbol {\ theta}^*$。作为应用程序，我们研究了借助拉索的分布，并表明需要校正程度对于计算有效的置信区间是必要的。

我们研究了对识别的非唯一麻烦的线性功能的通用推断，该功能定义为未识别条件矩限制的解决方案。这个问题出现在各种应用中，包括非参数仪器变量模型，未衡量的混杂性下的近端因果推断以及带有阴影变量的丢失 - 与随机数据。尽管感兴趣的线性功能（例如平均治疗效应）在适当的条件下是可以识别出的，但令人讨厌的非独家性对统计推断构成了严重的挑战，因为在这种情况下，常见的滋扰估计器可能是不稳定的，并且缺乏固定限制。在本文中，我们提出了对滋扰功能的受惩罚的最小估计器，并表明它们在这种挑战性的环境中有效推断。提出的滋扰估计器可以适应灵活的功能类别，重要的是，无论滋扰是否是唯一的，它们都可以融合到由惩罚确定的固定限制。我们使用受惩罚的滋扰估计器来形成有关感兴趣的线性功能的依据估计量，并在通用高级条件下证明其渐近正态性，这提供了渐近有效的置信区间。

本文提出了在多阶段实验的背景下的异质治疗效应的置信区间结构，以$ N $样品和高维，$ D $，混淆。我们的重点是$ d \ gg n $的情况，但获得的结果也适用于低维病例。我们展示了正则化估计的偏差，在高维变焦空间中不可避免，具有简单的双重稳固分数。通过这种方式，不需要额外的偏差，并且我们获得root $ N $推理结果，同时允许治疗和协变量的多级相互依赖性。记忆财产也没有假设;治疗可能取决于所有先前的治疗作业以及以前的所有多阶段混淆。我们的结果依赖于潜在依赖的某些稀疏假设。我们发现具有动态处理的强大推理所需的新产品率条件。

We develop a general framework for distribution-free predictive inference in regression, using conformal inference. The proposed methodology allows for the construction of a prediction band for the response variable using any estimator of the regression function. The resulting prediction band preserves the consistency properties of the original estimator under standard assumptions, while guaranteeing finite-sample marginal coverage even when these assumptions do not hold. We analyze and compare, both empirically and theoretically, the two major variants of our conformal framework: full conformal inference and split conformal inference, along with a related jackknife method. These methods offer different tradeoffs between statistical accuracy (length of resulting prediction intervals) and computational efficiency. As extensions, we develop a method for constructing valid in-sample prediction intervals called rank-one-out conformal inference, which has essentially the same computational efficiency as split conformal inference. We also describe an extension of our procedures for producing prediction bands with locally varying length, in order to adapt to heteroskedascity in the data. Finally, we propose a model-free notion of variable importance, called leave-one-covariate-out or LOCO inference. Accompanying this paper is an R package conformalInference that implements all of the proposals we have introduced. In the spirit of reproducibility, all of our empirical results can also be easily (re)generated using this package.

在本文中，我们利用过度参数化来设计高维单索索引模型的无规矩算法，并为诱导的隐式正则化现象提供理论保证。具体而言，我们研究了链路功能是非线性且未知的矢量和矩阵单索引模型，信号参数是稀疏向量或低秩对称矩阵，并且响应变量可以是重尾的。为了更好地理解隐含正规化的角色而没有过度的技术性，我们假设协变量的分布是先验的。对于载体和矩阵设置，我们通过采用分数函数变换和专为重尾数据的强大截断步骤来构造过度参数化最小二乘损耗功能。我们建议通过将无规则化的梯度下降应用于损耗函数来估计真实参数。当初始化接近原点并且步骤中足够小时，我们证明了所获得的解决方案在载体和矩阵案件中实现了最小的收敛统计速率。此外，我们的实验结果支持我们的理论调查结果，并表明我们的方法在$ \ ell_2 $ -staticatisticated率和变量选择一致性方面具有明确的正则化的经验卓越。

Testing the significance of a variable or group of variables $X$ for predicting a response $Y$, given additional covariates $Z$, is a ubiquitous task in statistics. A simple but common approach is to specify a linear model, and then test whether the regression coefficient for $X$ is non-zero. However, when the model is misspecified, the test may have poor power, for example when $X$ is involved in complex interactions, or lead to many false rejections. In this work we study the problem of testing the model-free null of conditional mean independence, i.e. that the conditional mean of $Y$ given $X$ and $Z$ does not depend on $X$. We propose a simple and general framework that can leverage flexible nonparametric or machine learning methods, such as additive models or random forests, to yield both robust error control and high power. The procedure involves using these methods to perform regressions, first to estimate a form of projection of $Y$ on $X$ and $Z$ using one half of the data, and then to estimate the expected conditional covariance between this projection and $Y$ on the remaining half of the data. While the approach is general, we show that a version of our procedure using spline regression achieves what we show is the minimax optimal rate in this nonparametric testing problem. Numerical experiments demonstrate the effectiveness of our approach both in terms of maintaining Type I error control, and power, compared to several existing approaches.

我们提出了一个程序，该程序将层次聚类与从一组大型IV中选择有效仪器变量（IV）的限制的测试结合在一起。其中一些可能是无效的，因为它们未通过排除限制。我们表明，如果最大的IV组有效，我们的方法就可以实现Oracle属性。与现有技术不同，我们的工作涉及多个内源回归器，较弱的仪器，异质效应和几乎有效性。在模拟中，我们的过程优于硬阈值和置信区间方法。该方法适用于估计移民对工资和教育回报的影响。

预测一组结果 - 而不是独特的结果 - 是统计学习中不确定性定量的有前途的解决方案。尽管有关于构建具有统计保证的预测集的丰富文献，但适应未知的协变量转变（实践中普遍存在的问题）还是一个严重的未解决的挑战。在本文中，我们表明具有有限样本覆盖范围保证的预测集是非信息性的，并提出了一种新型的无灵活分配方法PredSet-1Step，以有效地构建了在未知协方差转移下具有渐近覆盖范围保证的预测集。我们正式表明我们的方法是\ textIt {渐近上可能是近似正确}，对大型样本的置信度有很好的覆盖误差。我们说明，在南非队列研究中，它在许多实验和有关HIV风险预测的数据集中实现了名义覆盖范围。我们的理论取决于基于一般渐近线性估计器的WALD置信区间覆盖范围的融合率的新结合。

组选择的最佳子集（BSG）是选择一小部分非重叠组以在响应变量上获得最佳解释性的过程。它吸引了越来越多的关注，并且在实践中具有深远的应用。但是，由于BSG在高维环境中的计算棘手性，开发用于解决BSGS的有效算法仍然是研究热点。在本文中，我们提出了一种划分的算法，该算法迭代地检测相关组并排除了无关的组。此外，再加上新的组信息标准，我们开发了一种自适应算法来确定最佳模型大小。在轻度条件下，我们的算法可以在多项式时间内以高概率确定组的最佳子集是可以证明的。最后，我们通过将它们与合成数据集和现实世界中的几种最新算法进行比较来证明我们的方法的效率和准确性。

个性化决定规则（IDR）是一个决定函数，可根据他/她观察到的特征分配给定的治疗。文献中的大多数现有工作考虑使用二进制或有限的许多治疗方案的设置。在本文中，我们专注于连续治疗设定，并提出跳跃间隔 - 学习，开发一个最大化预期结果的个性化间隔值决定规则（I2DR）。与推荐单一治疗的IDRS不同，所提出的I2DR为每个人产生了一系列治疗方案，使其在实践中实施更加灵活。为了获得最佳I2DR，我们的跳跃间隔学习方法估计通过跳转惩罚回归给予治疗和协变量的结果的条件平均值，并基于估计的结果回归函数来衍生相应的最佳I2DR。允许回归线是用于清晰的解释或深神经网络的线性，以模拟复杂的处理 - 协调会相互作用。为了实现跳跃间隔学习，我们开发了一种基于动态编程的搜索算法，其有效计算结果回归函数。当结果回归函数是处理空间的分段或连续功能时，建立所得I2DR的统计特性。我们进一步制定了一个程序，以推断（估计）最佳政策下的平均结果。进行广泛的模拟和对华法林研究的真实数据应用，以证明所提出的I2DR的经验有效性。

This paper provides estimation and inference methods for a conditional average treatment effects (CATE) characterized by a high-dimensional parameter in both homogeneous cross-sectional and unit-heterogeneous dynamic panel data settings. In our leading example, we model CATE by interacting the base treatment variable with explanatory variables. The first step of our procedure is orthogonalization, where we partial out the controls and unit effects from the outcome and the base treatment and take the cross-fitted residuals. This step uses a novel generic cross-fitting method we design for weakly dependent time series and panel data. This method "leaves out the neighbors" when fitting nuisance components, and we theoretically power it by using Strassen's coupling. As a result, we can rely on any modern machine learning method in the first step, provided it learns the residuals well enough. Second, we construct an orthogonal (or residual) learner of CATE -- the Lasso CATE -- that regresses the outcome residual on the vector of interactions of the residualized treatment with explanatory variables. If the complexity of CATE function is simpler than that of the first-stage regression, the orthogonal learner converges faster than the single-stage regression-based learner. Third, we perform simultaneous inference on parameters of the CATE function using debiasing. We also can use ordinary least squares in the last two steps when CATE is low-dimensional. In heterogeneous panel data settings, we model the unobserved unit heterogeneity as a weakly sparse deviation from Mundlak (1978)'s model of correlated unit effects as a linear function of time-invariant covariates and make use of L1-penalization to estimate these models. We demonstrate our methods by estimating price elasticities of groceries based on scanner data. We note that our results are new even for the cross-sectional (i.i.d) case.

Integrative analysis of data from multiple sources is critical to making generalizable discoveries. Associations that are consistently observed across multiple source populations are more likely to be generalized to target populations with possible distributional shifts. In this paper, we model the heterogeneous multi-source data with multiple high-dimensional regressions and make inferences for the maximin effect (Meinshausen, B{\"u}hlmann, AoS, 43(4), 1801--1830). The maximin effect provides a measure of stable associations across multi-source data. A significant maximin effect indicates that a variable has commonly shared effects across multiple source populations, and these shared effects may be generalized to a broader set of target populations. There are challenges associated with inferring maximin effects because its point estimator can have a non-standard limiting distribution. We devise a novel sampling method to construct valid confidence intervals for maximin effects. The proposed confidence interval attains a parametric length. This sampling procedure and the related theoretical analysis are of independent interest for solving other non-standard inference problems. Using genetic data on yeast growth in multiple environments, we demonstrate that the genetic variants with significant maximin effects have generalizable effects under new environments.

统计推断中的主要范式取决于I.I.D.的结构。来自假设的无限人群的数据。尽管它取得了成功，但在复杂的数据结构下，即使在清楚无限人口所代表的内容的情况下，该框架在复杂的数据结构下仍然不灵活。在本文中，我们探讨了一个替代框架，在该框架中，推断只是对模型误差的不变性假设，例如交换性或符号对称性。作为解决这个不变推理问题的一般方法，我们提出了一个基于随机的过程。我们证明了该过程的渐近有效性的一般条件，并在许多数据结构中说明了，包括单向和双向布局中的群集误差。我们发现，通过残差随机化的不变推断具有三个吸引人的属性：（1）在弱且可解释的条件下是有效的，可以解决重型数据，有限聚类甚至一些高维设置的问题。 （2）它在有限样品中是可靠的，因为它不依赖经典渐近学所需的规律性条件。 （3）它以适应数据结构的统一方式解决了推断问题。另一方面，诸如OLS或Bootstrap之类的经典程序以I.I.D.为前提。结构，只要实际问题结构不同，就需要修改。经典框架中的这种不匹配导致了多种可靠的误差技术和自举变体，这些变体经常混淆应用研究。我们通过广泛的经验评估证实了这些发现。残留随机化对许多替代方案的表现有利，包括可靠的误差方法，自举变体和分层模型。

在本文中，我们的目标是提供对半监督（SS）因果推理的一般性和完全理解治疗效果。具体而言，我们考虑两个这样的估计值：（a）平均治疗效果和（b）定量处理效果，作为原型案例，在SS设置中，其特征在于两个可用的数据集：（i）标记的数据集大小$ N $，为响应和一组高维协变量以及二元治疗指标提供观察。 （ii）一个未标记的数据集，大小超过$ n $，但未观察到的响应。使用这两个数据集，我们开发了一个SS估计系列，该系列是：（1）更强大，并且（2）比其监督对应力更高的基于标记的数据集。除了通过监督方法可以实现的“标准”双重稳健结果（在一致性方面），我们还在正确指定模型中的倾向得分，我们进一步建立了我们SS估计的根本-N一致性和渐近常态。没有需要涉及的特定形式的滋扰职能。这种改善的鲁棒性来自使用大规模未标记的数据，因此通常不能在纯粹监督的环境中获得。此外，只要正确指定所有滋扰函数，我们的估计值都显示为半参数效率。此外，作为滋扰估计器的说明，我们考虑逆概率加权型核平滑估计，涉及未知的协变量转换机制，并在高维情景新颖的情况下建立其统一的收敛速率，这应该是独立的兴趣。两种模拟和实际数据的数值结果验证了我们对其监督对应物的优势，了解鲁棒性和效率。

Sparse reduced rank regression is an essential statistical learning method. In the contemporary literature, estimation is typically formulated as a nonconvex optimization that often yields to a local optimum in numerical computation. Yet, their theoretical analysis is always centered on the global optimum, resulting in a discrepancy between the statistical guarantee and the numerical computation. In this research, we offer a new algorithm to address the problem and establish an almost optimal rate for the algorithmic solution. We also demonstrate that the algorithm achieves the estimation with a polynomial number of iterations. In addition, we present a generalized information criterion to simultaneously ensure the consistency of support set recovery and rank estimation. Under the proposed criterion, we show that our algorithm can achieve the oracle reduced rank estimation with a significant probability. The numerical studies and an application in the ovarian cancer genetic data demonstrate the effectiveness and scalability of our approach.

我们考虑在估计涉及依赖参数的高维滋扰的估计方程中估计一个低维参数。一个中心示例是因果推理中（局部）分位数处理效应（（L）QTE）的有效估计方程，涉及在分位数以估计的分位数评估的协方差累积分布函数。借记机学习（DML）是一种使用灵活的机器学习方法估算高维滋扰的数据分解方法，但是将其应用于参数依赖性滋扰的问题是不切实际的。对于（L）QTE，DML要求我们学习整个协变量累积分布函数。相反，我们提出了局部偏见的机器学习（LDML），该学习避免了这一繁重的步骤，并且只需要对参数进行一次初始粗糙猜测而估算烦恼。对于（L）QTE，LDML仅涉及学习两个回归功能，这是机器学习方法的标准任务。我们证明，在松弛速率条件下，我们的估计量与使用未知的真实滋扰的不可行的估计器具有相同的有利渐近行为。因此，LDML值得注意的是，当我们必须控制许多协变量和/或灵活的关系时，如（l）QTES在（（l）QTES）中，实际上可以有效地估算重要数量，例如（l）QTES。

Classical asymptotic theory for statistical inference usually involves calibrating a statistic by fixing the dimension $d$ while letting the sample size $n$ increase to infinity. Recently, much effort has been dedicated towards understanding how these methods behave in high-dimensional settings, where $d$ and $n$ both increase to infinity together. This often leads to different inference procedures, depending on the assumptions about the dimensionality, leaving the practitioner in a bind: given a dataset with 100 samples in 20 dimensions, should they calibrate by assuming $n \gg d$, or $d/n \approx 0.2$? This paper considers the goal of dimension-agnostic inference; developing methods whose validity does not depend on any assumption on $d$ versus $n$. We introduce an approach that uses variational representations of existing test statistics along with sample splitting and self-normalization to produce a new test statistic with a Gaussian limiting distribution, regardless of how $d$ scales with $n$. The resulting statistic can be viewed as a careful modification of degenerate U-statistics, dropping diagonal blocks and retaining off-diagonal blocks. We exemplify our technique for some classical problems including one-sample mean and covariance testing, and show that our tests have minimax rate-optimal power against appropriate local alternatives. In most settings, our cross U-statistic matches the high-dimensional power of the corresponding (degenerate) U-statistic up to a $\sqrt{2}$ factor.

当并非观察到所有混杂因子并获得负面对照时，我们研究因果参数的估计。最近的工作表明，这些方法如何通过两个所谓的桥梁函数来实现识别和有效估计。在本文中，我们使用阴性对照来应对因果推断的主要挑战：这些桥梁功能的识别和估计。先前的工作依赖于这些功能的完整性条件，以识别因果参数并在估计中需要进行独特性假设，并且还集中于桥梁函数的参数估计。相反，我们提供了一种新的识别策略，以避免完整性条件。而且，我们根据最小学习公式为这些功能提供新的估计量。这些估计值适合通用功能类别，例如重现Hilbert空间和神经网络。我们研究了有限样本收敛的结果，既可以估计桥梁功能本身，又要在各种假设组合下对因果参数进行最终估计。我们尽可能避免桥梁上的独特条件。