合奏方法（例如随机森林）由于其高预测精度而在应用中很受欢迎。现有文献将随机的森林预测视为无限顺序不完整的U统计量，以量化其不确定性。但是，这些方法集中在每棵树的小次采样大小上，这在理论上是有效但实际上有限的。本文基于不完整的U统计数据，开发了公正的方差估计器，该估计量可以与整体样本量相当，从而使统计推断在更广泛的实际应用中成为可能。仿真结果表明，我们的估计量没有额外的计算成本，估计器的偏见和更准确的覆盖率。我们还提出了一项局部平滑过程，以减少估计器的变化，当树木数量相对较小时，该过程显示出改善的数值性能。此外，我们研究了在特定方案下提出的方差估计器的比率一致性。特别是，我们开发了一种新的“双U统计”公式，以分析估算器差异的HOFFING分解。

随机森林仍然是最受欢迎的现成监督学习算法之一。尽管他们记录了良好的经验成功，但直到最近，很少有很少的理论结果来描述他们的表现和行为。在这项工作中，我们通过建立随机森林和其他受监督学习集合的融合率来推动最近的一致性和渐近正常的工作。我们培养了广义U形统计的概念，并显示在此框架内，随机森林预测可能对比以前建立的较大的子样本尺寸可能保持渐近正常。我们还提供Berry-esseen的界限，以量化这种收敛的速度，使得分列大小的角色和确定随机森林预测分布的树木的角色。

Classical asymptotic theory for statistical inference usually involves calibrating a statistic by fixing the dimension $d$ while letting the sample size $n$ increase to infinity. Recently, much effort has been dedicated towards understanding how these methods behave in high-dimensional settings, where $d$ and $n$ both increase to infinity together. This often leads to different inference procedures, depending on the assumptions about the dimensionality, leaving the practitioner in a bind: given a dataset with 100 samples in 20 dimensions, should they calibrate by assuming $n \gg d$, or $d/n \approx 0.2$? This paper considers the goal of dimension-agnostic inference; developing methods whose validity does not depend on any assumption on $d$ versus $n$. We introduce an approach that uses variational representations of existing test statistics along with sample splitting and self-normalization to produce a new test statistic with a Gaussian limiting distribution, regardless of how $d$ scales with $n$. The resulting statistic can be viewed as a careful modification of degenerate U-statistics, dropping diagonal blocks and retaining off-diagonal blocks. We exemplify our technique for some classical problems including one-sample mean and covariance testing, and show that our tests have minimax rate-optimal power against appropriate local alternatives. In most settings, our cross U-statistic matches the high-dimensional power of the corresponding (degenerate) U-statistic up to a $\sqrt{2}$ factor.

Many scientific and engineering challenges-ranging from personalized medicine to customized marketing recommendations-require an understanding of treatment effect heterogeneity. In this paper, we develop a non-parametric causal forest for estimating heterogeneous treatment effects that extends Breiman's widely used random forest algorithm. In the potential outcomes framework with unconfoundedness, we show that causal forests are pointwise consistent for the true treatment effect, and have an asymptotically Gaussian and centered sampling distribution. We also discuss a practical method for constructing asymptotic confidence intervals for the true treatment effect that are centered at the causal forest estimates. Our theoretical results rely on a generic Gaussian theory for a large family of random forest algorithms. To our knowledge, this is the first set of results that allows any type of random forest, including classification and regression forests, to be used for provably valid statistical inference. In experiments, we find causal forests to be substantially more powerful than classical methods based on nearest-neighbor matching, especially in the presence of irrelevant covariates.

We study a double robust Bayesian inference procedure on the average treatment effect (ATE) under unconfoundedness. Our Bayesian approach involves a correction term for prior distributions adjusted by the propensity score. We prove asymptotic equivalence of our Bayesian estimator and efficient frequentist estimators by establishing a new semiparametric Bernstein-von Mises theorem under double robustness; i.e., the lack of smoothness of conditional mean functions can be compensated by high regularity of the propensity score and vice versa. Consequently, the resulting Bayesian point estimator internalizes the bias correction as the frequentist-type doubly robust estimator, and the Bayesian credible sets form confidence intervals with asymptotically exact coverage probability. In simulations, we find that this corrected Bayesian procedure leads to significant bias reduction of point estimation and accurate coverage of confidence intervals, especially when the dimensionality of covariates is large relative to the sample size and the underlying functions become complex. We illustrate our method in an application to the National Supported Work Demonstration.

加权最近的邻居（WNN）估计量通常用作平均回归估计的灵活且易于实现的非参数工具。袋装技术是一种优雅的方式，可以自动生成最近邻居的重量的WNN估计器；我们将最终的估计量命名为分布最近的邻居（DNN），以便于参考。然而，这种估计器缺乏分布结果，从而将其应用于统计推断。此外，当平均回归函数具有高阶平滑度时，DNN无法达到最佳的非参数收敛率，这主要是由于偏差问题。在这项工作中，我们对DNN提供了深入的技术分析，我们建议通过线性将两个DNN估计量与不同的子采样量表进行线性相结合，从而提出了DNN估计量的偏差方法，从而导致新型的两尺度DNN（TDNN（TDNN） ）估计器。两尺度的DNN估计量具有等效的WNN表示，重量承认明确形式，有些则是负面的。我们证明，由于使用负权重，两尺度DNN估计器在四阶平滑度条件下估算回归函数时享有最佳的非参数收敛速率。我们进一步超出了估计，并确定DNN和两个规模的DNN均无渐进地正常，因为亚次采样量表和样本量差异到无穷大。对于实际实施，我们还使用二尺度DNN的Jacknife和Bootstrap技术提供方差估计器和分配估计器。可以利用这些估计器来构建有效的置信区间，以用于回归函数的非参数推断。建议的两尺度DNN方法的理论结果和吸引人的有限样本性能用几个数值示例说明了。

Testing the significance of a variable or group of variables $X$ for predicting a response $Y$, given additional covariates $Z$, is a ubiquitous task in statistics. A simple but common approach is to specify a linear model, and then test whether the regression coefficient for $X$ is non-zero. However, when the model is misspecified, the test may have poor power, for example when $X$ is involved in complex interactions, or lead to many false rejections. In this work we study the problem of testing the model-free null of conditional mean independence, i.e. that the conditional mean of $Y$ given $X$ and $Z$ does not depend on $X$. We propose a simple and general framework that can leverage flexible nonparametric or machine learning methods, such as additive models or random forests, to yield both robust error control and high power. The procedure involves using these methods to perform regressions, first to estimate a form of projection of $Y$ on $X$ and $Z$ using one half of the data, and then to estimate the expected conditional covariance between this projection and $Y$ on the remaining half of the data. While the approach is general, we show that a version of our procedure using spline regression achieves what we show is the minimax optimal rate in this nonparametric testing problem. Numerical experiments demonstrate the effectiveness of our approach both in terms of maintaining Type I error control, and power, compared to several existing approaches.

This paper provides estimation and inference methods for an identified set's boundary (i.e., support function) where the selection among a very large number of covariates is based on modern regularized tools. I characterize the boundary using a semiparametric moment equation. Combining Neyman-orthogonality and sample splitting ideas, I construct a root-N consistent, uniformly asymptotically Gaussian estimator of the boundary and propose a multiplier bootstrap procedure to conduct inference. I apply this result to the partially linear model, the partially linear IV model and the average partial derivative with an interval-valued outcome.

In nonparametric independence testing, we observe i.i.d.\ data $\{(X_i,Y_i)\}_{i=1}^n$, where $X \in \mathcal{X}, Y \in \mathcal{Y}$ lie in any general spaces, and we wish to test the null that $X$ is independent of $Y$. Modern test statistics such as the kernel Hilbert-Schmidt Independence Criterion (HSIC) and Distance Covariance (dCov) have intractable null distributions due to the degeneracy of the underlying U-statistics. Thus, in practice, one often resorts to using permutation testing, which provides a nonasymptotic guarantee at the expense of recalculating the quadratic-time statistics (say) a few hundred times. This paper provides a simple but nontrivial modification of HSIC and dCov (called xHSIC and xdCov, pronounced ``cross'' HSIC/dCov) so that they have a limiting Gaussian distribution under the null, and thus do not require permutations. This requires building on the newly developed theory of cross U-statistics by Kim and Ramdas (2020), and in particular developing several nontrivial extensions of the theory in Shekhar et al. (2022), which developed an analogous permutation-free kernel two-sample test. We show that our new tests, like the originals, are consistent against fixed alternatives, and minimax rate optimal against smooth local alternatives. Numerical simulations demonstrate that compared to the full dCov or HSIC, our variants have the same power up to a $\sqrt 2$ factor, giving practitioners a new option for large problems or data-analysis pipelines where computation, not sample size, could be the bottleneck.

We study estimation and testing in the Poisson regression model with noisy high dimensional covariates, which has wide applications in analyzing noisy big data. Correcting for the estimation bias due to the covariate noise leads to a non-convex target function to minimize. Treating the high dimensional issue further leads us to augment an amenable penalty term to the target function. We propose to estimate the regression parameter through minimizing the penalized target function. We derive the L1 and L2 convergence rates of the estimator and prove the variable selection consistency. We further establish the asymptotic normality of any subset of the parameters, where the subset can have infinitely many components as long as its cardinality grows sufficiently slow. We develop Wald and score tests based on the asymptotic normality of the estimator, which permits testing of linear functions of the members if the subset. We examine the finite sample performance of the proposed tests by extensive simulation. Finally, the proposed method is successfully applied to the Alzheimer's Disease Neuroimaging Initiative study, which motivated this work initially.

作为一种特殊的无限级矢量自回旋（VAR）模型，矢量自回归移动平均值（VARMA）模型比广泛使用的有限级var模型可以捕获更丰富的时间模式。然而，长期以来，其实用性一直受到其不可识别性，计算疾病性和解释相对难度的阻碍。本文介绍了一种新颖的无限级VAR模型，该模型不仅避免了VARMA模型的缺点，而且继承了其有利的时间模式。作为另一个有吸引力的特征，可以单独解释该模型的时间和横截面依赖性结构，因为它们的特征是不同的参数集。对于高维时间序列，这种分离激发了我们对确定横截面依赖性的参数施加稀疏性。结果，可以在不牺牲任何时间信息的情况下实现更高的统计效率和可解释性。我们为提出的模型引入了一个$ \ ell_1 $调查估计量，并得出相应的非反应误差边界。开发了有效的块坐标下降算法和一致的模型顺序选择方法。拟议方法的优点得到了模拟研究和现实世界的宏观经济数据分析的支持。

主题模型为学习，提取和发现大型文本语料库中的潜在结构提供了有用的文本挖掘工具。尽管已经为主题建模提出了大量方法，但文献缺乏是对潜在主题估计的统计识别性和准确性的正式理论研究。在本文中，我们提出了一个基于特定的集成可能性的潜在主题的最大似然估计量（MLE），该主题自然地与该概念相连，在计算几何学中，体积最小化。我们的理论介绍了主题模型可识别性的一组新几何条件，这些条件比常规的可分离性条件弱，这些条件通常依赖于纯主题文档或锚定词的存在。较弱的条件允许更广泛的调查，因此可能会更加富有成果的研究。我们对拟议的估计器进行有限样本误差分析，并讨论我们的结果与先前研究的结果之间的联系。我们以使用模拟和真实数据集的实证研究结论。

我们研究了张量张量的回归，其中的目标是将张量的响应与张量协变量与塔克等级参数张量/矩阵连接起来，而没有其内在等级的先验知识。我们提出了Riemannian梯度下降（RGD）和Riemannian Gauss-Newton（RGN）方法，并通过研究等级过度参数化的影响来应对未知等级的挑战。我们通过表明RGD和RGN分别线性地和四边形地收敛到两个等级的统计最佳估计值，从而为一般的张量调节回归提供了第一个收敛保证。我们的理论揭示了一种有趣的现象：Riemannian优化方法自然地适应了过度参数化，而无需修改其实施。我们还为低度多项式框架下的标量调整回归中的统计计算差距提供了第一个严格的证据。我们的理论证明了``统计计算差距的祝福''现象：在张张量的张量回归中，对于三个或更高的张紧器，在张张量的张量回归中，计算所需的样本量与中等级别相匹配的计算量相匹配。在考虑计算可行的估计器时，虽然矩阵设置没有此类好处。这表明中等等级的过度参数化本质上是``在张量调整的样本量三分或更高的样本大小上，三分或更高的样本量。最后，我们进行仿真研究以显示我们提出的方法的优势并证实我们的理论发现。

The estimation of cumulative distribution functions (CDFs) is an important learning task with a great variety of downstream applications, such as risk assessments in predictions and decision making. In this paper, we study functional regression of contextual CDFs where each data point is sampled from a linear combination of context dependent CDF basis functions. We propose functional ridge-regression-based estimation methods that estimate CDFs accurately everywhere. In particular, given $n$ samples with $d$ basis functions, we show estimation error upper bounds of $\widetilde{O}(\sqrt{d/n})$ for fixed design, random design, and adversarial context cases. We also derive matching information theoretic lower bounds, establishing minimax optimality for CDF functional regression. Furthermore, we remove the burn-in time in the random design setting using an alternative penalized estimator. Then, we consider agnostic settings where there is a mismatch in the data generation process. We characterize the error of the proposed estimators in terms of the mismatched error, and show that the estimators are well-behaved under model mismatch. Finally, to complete our study, we formalize infinite dimensional models where the parameter space is an infinite dimensional Hilbert space, and establish self-normalized estimation error upper bounds for this setting.

由于其出色的经验表现，随机森林是过去十年中使用的机器学习方法之一。然而，由于其黑框的性质，在许多大数据应用中很难解释随机森林的结果。量化各个特征在随机森林中的实用性可以大大增强其解释性。现有的研究表明，一些普遍使用的特征对随机森林的重要性措施遭受了偏见问题。此外，对于大多数现有方法，缺乏全面的规模和功率分析。在本文中，我们通过假设检验解决了问题，并提出了一个自由化特征 - 弥散性相关测试（事实）的框架，以评估具有偏见性属性的随机森林模型中给定特征的重要性，我们零假设涉及该特征是否与所有其他特征有条件地独立于响应。关于高维随机森林一致性的一些最新发展，对随机森林推断的这种努力得到了赋予的能力。在存在功能依赖性的情况下，我们的事实测试的香草版可能会遇到偏见问题。我们利用偏置校正的不平衡和调节技术。我们通过增强功率的功能转换将合奏的想法进一步纳入事实统计范围。在相当普遍的具有依赖特征的高维非参数模型设置下，我们正式确定事实可以提供理论上合理的随机森林具有P值，并通过非催化分析享受吸引人的力量。新建议的方法的理论结果和有限样本优势通过几个模拟示例和与Covid-19的经济预测应用进行了说明。

In a mixed generalized linear model, the objective is to learn multiple signals from unlabeled observations: each sample comes from exactly one signal, but it is not known which one. We consider the prototypical problem of estimating two statistically independent signals in a mixed generalized linear model with Gaussian covariates. Spectral methods are a popular class of estimators which output the top two eigenvectors of a suitable data-dependent matrix. However, despite the wide applicability, their design is still obtained via heuristic considerations, and the number of samples $n$ needed to guarantee recovery is super-linear in the signal dimension $d$. In this paper, we develop exact asymptotics on spectral methods in the challenging proportional regime in which $n, d$ grow large and their ratio converges to a finite constant. By doing so, we are able to optimize the design of the spectral method, and combine it with a simple linear estimator, in order to minimize the estimation error. Our characterization exploits a mix of tools from random matrices, free probability and the theory of approximate message passing algorithms. Numerical simulations for mixed linear regression and phase retrieval display the advantage enabled by our analysis over existing designs of spectral methods.

基于中央限制定理（CLT）的置信区间是经典统计的基石。尽管仅渐近地有效，但它们是无处不在的，因为它们允许在非常弱的假设下进行统计推断，即使不可能进行非反应性推断，通常也可以应用于问题。本文引入了这种渐近置信区间的时间均匀类似物。为了详细说明，我们的方法采用置信序列（CS）的形式 - 随着时间的推移均匀有效的置信区间序列。 CSS在任意停止时间时提供有效的推断，与需要预先确定样本量的经典置信区间不同，因此没有受到“窥视”数据的惩罚。文献中现有的CSS是非肿瘤的，因此不享受上述渐近置信区间的广泛适用性。我们的工作通过给出“渐近CSS”的定义来弥合差距，并得出仅需要类似CLT的假设的通用渐近CS。虽然CLT在固定样本量下近似于高斯的样本平均值的分布，但我们使用强大的不变性原理（来自Komlos，Major和Tusnady的1970年代的开创性工作），按照整个样品平均过程均匀地近似于整个样品平均过程。隐性的高斯过程。我们通过在观察性研究中基于双重稳健的估计量来得出非参数渐近级别的CSS来证明它们的实用性，即使在固定的时间方案中，也可能不存在非催化方法（由于混淆偏见）。这些使双重强大的因果推断可以连续监测并自适应地停止。

我们在分布式框架中得出最小值测试错误，其中数据被分成多个机器，并且它们与中央机器的通信仅限于$ b $位。我们研究了高斯白噪声下的$ d $ - 和无限维信号检测问题。我们还得出达到理论下限的分布式测试算法。我们的结果表明，分布式测试受到从根本上不同的现象，这些现象在分布式估计中未观察到。在我们的发现中，我们表明，可以访问共享随机性的测试协议在某些制度中的性能比不进行的测试协议可以更好地表现。我们还观察到，即使仅使用单个本地计算机上可用的信息，一致的非参数分布式测试始终是可能的，即使只有$ 1 $的通信和相应的测试优于最佳本地测试。此外，我们还得出了自适应非参数分布测试策略和相应的理论下限。

套索是一种高维回归的方法，当时，当协变量$ p $的订单数量或大于观测值$ n $时，通常使用它。由于两个基本原因，经典的渐近态性理论不适用于该模型：$（1）$正规风险是非平滑的； $（2）$估算器$ \ wideHat {\ boldsymbol {\ theta}} $与true参数vector $ \ boldsymbol {\ theta}^*$无法忽略。结果，标准的扰动论点是渐近正态性的传统基础。另一方面，套索估计器可以精确地以$ n $和$ p $大，$ n/p $的订单为一。这种表征首先是在使用I.I.D的高斯设计的情况下获得的。协变量：在这里，我们将其推广到具有非偏差协方差结构的高斯相关设计。这是根据更简单的``固定设计''模型表示的。我们在两个模型中各种数量的分布之间的距离上建立了非反应界限，它们在合适的稀疏类别中均匀地固定在信号上$ \ boldsymbol {\ theta}^*$。作为应用程序，我们研究了借助拉索的分布，并表明需要校正程度对于计算有效的置信区间是必要的。

We consider the problem of estimating a multivariate function $f_0$ of bounded variation (BV), from noisy observations $y_i = f_0(x_i) + z_i$ made at random design points $x_i \in \mathbb{R}^d$, $i=1,\ldots,n$. We study an estimator that forms the Voronoi diagram of the design points, and then solves an optimization problem that regularizes according to a certain discrete notion of total variation (TV): the sum of weighted absolute differences of parameters $\theta_i,\theta_j$ (which estimate the function values $f_0(x_i),f_0(x_j)$) at all neighboring cells $i,j$ in the Voronoi diagram. This is seen to be equivalent to a variational optimization problem that regularizes according to the usual continuum (measure-theoretic) notion of TV, once we restrict the domain to functions that are piecewise constant over the Voronoi diagram. The regression estimator under consideration hence performs (shrunken) local averaging over adaptively formed unions of Voronoi cells, and we refer to it as the Voronoigram, following the ideas in Koenker (2005), and drawing inspiration from Tukey's regressogram (Tukey, 1961). Our contributions in this paper span both the conceptual and theoretical frontiers: we discuss some of the unique properties of the Voronoigram in comparison to TV-regularized estimators that use other graph-based discretizations; we derive the asymptotic limit of the Voronoi TV functional; and we prove that the Voronoigram is minimax rate optimal (up to log factors) for estimating BV functions that are essentially bounded.