Maximum Inner Product Search (MIPS) is a popular problem in the machine learning literature due to its applicability in a wide array of applications, such as recommender systems. In high-dimensional settings, however, MIPS queries can become computationally expensive as most existing solutions do not scale well with data dimensionality. In this work, we present a state-of-the-art algorithm for the MIPS problem in high dimensions, dubbed BanditMIPS. BanditMIPS is a randomized algorithm that borrows techniques from multi-armed bandits to reduce the MIPS problem to a best-arm identification problem. BanditMIPS reduces the complexity of state-of-the-art algorithms from $O(\sqrt{d})$ to $O(\text{log}d)$, where $d$ is the dimension of the problem data vectors. On high-dimensional real-world datasets, BanditMIPS runs approximately 12 times faster than existing approaches and returns the same solution. BanditMIPS requires no preprocessing of the data and includes a hyperparameter that practitioners may use to trade off accuracy and runtime. We also propose a variant of our algorithm, named BanditMIPS-$\alpha$, which employs non-uniform sampling across the data dimensions to provide further speedups.

Data depth, introduced by Tukey (1975), is an important tool in data science, robust statistics, and computational geometry. One chief barrier to its broader practical utility is that many common measures of depth are computationally intensive, requiring on the order of $n^d$ operations to exactly compute the depth of a single point within a data set of $n$ points in $d$-dimensional space. Often however, we are not directly interested in the absolute depths of the points, but rather in their \textit{relative ordering}. For example, we may want to find the most central point in a data set (a generalized median), or to identify and remove all outliers (points on the fringe of the data set with low depth). With this observation, we develop a novel and instance-adaptive algorithm for adaptive data depth computation by reducing the problem of exactly computing $n$ depths to an $n$-armed stochastic multi-armed bandit problem which we can efficiently solve. We focus our exposition on simplicial depth, developed by \citet{liu1990notion}, which has emerged as a promising notion of depth due to its interpretability and asymptotic properties. We provide general instance-dependent theoretical guarantees for our proposed algorithms, which readily extend to many other common measures of data depth including majority depth, Oja depth, and likelihood depth. When specialized to the case where the gaps in the data follow a power law distribution with parameter $\alpha<2$, we show that we can reduce the complexity of identifying the deepest point in the data set (the simplicial median) from $O(n^d)$ to $\tilde{O}(n^{d-(d-1)\alpha/2})$, where $\tilde{O}$ suppresses logarithmic factors. We corroborate our theoretical results with numerical experiments on synthetic data, showing the practical utility of our proposed methods.

Random forests are some of the most widely used machine learning models today, especially in domains that necessitate interpretability. We present an algorithm that accelerates the training of random forests and other popular tree-based learning methods. At the core of our algorithm is a novel node-splitting subroutine, dubbed MABSplit, used to efficiently find split points when constructing decision trees. Our algorithm borrows techniques from the multi-armed bandit literature to judiciously determine how to allocate samples and computational power across candidate split points. We provide theoretical guarantees that MABSplit improves the sample complexity of each node split from linear to logarithmic in the number of data points. In some settings, MABSplit leads to 100x faster training (an 99% reduction in training time) without any decrease in generalization performance. We demonstrate similar speedups when MABSplit is used across a variety of forest-based variants, such as Extremely Random Forests and Random Patches. We also show our algorithm can be used in both classification and regression tasks. Finally, we show that MABSplit outperforms existing methods in generalization performance and feature importance calculations under a fixed computational budget. All of our experimental results are reproducible via a one-line script at https://github.com/ThrunGroup/FastForest.

我们提出了两种线性土匪算法，具有每步复杂性sublerear的武器$ k $。该算法专为手臂集非常大且缓慢变化的应用而设计。我们的关键意识到，选择手臂还原为最大的内部产品搜索（MIPS）问题，该问题可以大约解决，而无需打破后悔保证。现有的近似MIPS求解器以均匀时间运行。我们扩展了这些求解器，并为在线学习问题提供理论保证，在线学习问题（即，以后的步骤取决于上一步中的反馈）成为一个独特的挑战。然后，我们明确表征了每步复杂性与遗憾之间的权衡。对于足够大的$ k $，我们的算法具有sublinear每步复杂性和$ \ tilde o（\ sqrt {t}）$遗憾。从经验上讲，我们在合成环境和现实世界中的电影推荐问题中评估了我们提出的算法。与线性时间基线相比，我们提出的算法可以提供超过72倍的速度，同时保留了类似的遗憾。

我们以已知的奖励和未知的约束来研究顺序决策，这是由约束代表昂贵评估人类偏好（例如安全舒适的驾驶行为）的情况所激发的。我们将互动学习这些约束作为新的线性匪徒问题的挑战正式化，我们称之为约束的线性最佳臂识别。为了解决这个问题，我们提出了自适应约束学习（ACOL）算法。我们为约束线性最佳臂识别提供了一个依赖实例的下限，并表明Acol的样品复杂性与最坏情况下的下限匹配。在平均情况下，ACOL的样品复杂性结合仍然比简单方法的边界更紧密。在合成实验中，ACOL与Oracle溶液相同，并且表现优于一系列基准。作为应用程序，我们考虑学习限制，以代表驾驶模拟中的人类偏好。对于此应用，ACOL比替代方案要高得多。此外，我们发现学习偏好作为约束对驾驶场景的变化比直接编码奖励函数中的偏好更强大。

积极的学习方法在减少学习所需的样本数量方面表现出了巨大的希望。随着自动化学习系统被采用到实时的现实世界决策管道中，越来越重要的是，这种算法的设计考虑到了安全性。在这项工作中，我们研究了在互动环境中学习最佳安全决定的复杂性。我们将这个问题减少到约束的线性匪徒问题，我们的目标是找到满足某些（未知）安全限制的最佳手臂。我们提出了一种基于自适应的实验性设计算法，在显示ARM的难度与次优的难度之间，我们表现出了有效的交易。据我们所知，我们的结果是具有安全限制的线性匪徒最佳武器识别。实际上，我们证明了这种方法在合成和现实世界数据集上的表现很好。

级别设置估计问题旨在查找域$ {\ cal x} $的所有点，其中一个未知函数$ f：{\ cal x} \ lightarrow \ mathbb {r} $超过阈值$ \ alpha $ 。估计基于可以在$ {\ cal x} $中顺序和自适应地选择的位置获取的嘈杂函数评估。阈值$ \ alpha $可以是\弹性{显式}，并提供先验，或\ \ ich {隐式}，相对于最佳函数值定义，即$ \ alpha =（1- \ epsilon）f（x_ \ AST）$关于给定$ \ epsilon> 0 $ why $ f（x_ \ ist）$是最大函数值，并且未知。在这项工作中，我们通过将其与最近的自适应实验设计方法相关联，为近期自适应实验设计方法提供了一种新的再现内核盗窃空间（RKHS）设置。我们假设可以通过RKHS中的函数近似于未知的拼写，并为此设置中隐含和显式案件提供新的算法，具有很强的理论保证。此外，在线性（内核）设置中，我们表明我们的界限几乎是最佳的，即，我们的上限与阈值线性匪徒的现有下限匹配。据我们所知，这项工作提供了第一个实例依赖性非渐近的上限，就匹配信息理论下限的水平设定估计的样本复杂性。

本文调查$ \纺织品{污染} $随机多臂爆炸中最佳臂识别问题。在此设置中，从任何臂获得的奖励由来自概率$ \ varepsilon $的对抗性模型的样本所取代。考虑了固定的置信度（无限地平线）设置，其中学习者的目标是识别最大的平均值。由于奖励的对抗污染，每个ARM的平均值仅部分可识别。本文提出了两种算法，基于连续消除的基于间隙的算法和一个，以便在亚高斯匪徒中最佳臂识别。这些算法涉及平均估计，从渐近估计的估计值达到真实均值的偏差上实现最佳误差保证。此外，这些算法渐近地实现了最佳的样本复杂性。具体地，对于基于差距的算法，样本复杂性呈渐近最佳到恒定因子，而对于基于连续的基于算法，​​它是最佳的对数因子。最后，提供了数值实验以说明与现有基线相比的算法的增益。

Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation and early-stopping. We formulate hyperparameter optimization as a pure-exploration nonstochastic infinite-armed bandit problem where a predefined resource like iterations, data samples, or features is allocated to randomly sampled configurations. We introduce a novel algorithm, Hyperband, for this framework and analyze its theoretical properties, providing several desirable guarantees. Furthermore, we compare Hyperband with popular Bayesian optimization methods on a suite of hyperparameter optimization problems. We observe that Hyperband can provide over an order-of-magnitude speedup over our competitor set on a variety of deep-learning and kernel-based learning problems.

推荐系统正面临审查，因为它们对我们可以获得的机会的影响越来越大。目前对公平的审计仅限于敏感群体水平的粗粒度评估。我们建议审核嫉妒 - 狂喜，一个与个别偏好对齐的更精细的标准：每个用户都应该更喜欢他们的建议给其他用户的建议。由于审计要求估计用户超出现有建议的用户的偏好，因此我们将审计作为多武装匪徒的新纯粹探索问题。我们提出了一种采样的效率算法，具有理论上的保证，它不会恶化用户体验。我们还研究了现实世界推荐数据集实现的权衡。

我们介绍了一个多臂强盗模型，其中奖励是多个随机变量的总和，每个动作只会改变其中的分布。每次动作之后，代理都会观察所有变量的实现。该模型是由营销活动和推荐系统激励的，在该系统中，变量代表单个客户的结果，例如点击。我们提出了UCB风格的算法，以估计基线上的动作的提升。我们研究了问题的多种变体，包括何时未知基线和受影响的变量，并证明所有这些变量均具有sublrinear后悔界限。我们还提供了较低的界限，以证明我们的建模假设的必要性是合理的。关于合成和现实世界数据集的实验显示了估计不使用这种结构的策略的振奋方法的好处。

We introduce a new setting, optimize-and-estimate structured bandits. Here, a policy must select a batch of arms, each characterized by its own context, that would allow it to both maximize reward and maintain an accurate (ideally unbiased) population estimate of the reward. This setting is inherent to many public and private sector applications and often requires handling delayed feedback, small data, and distribution shifts. We demonstrate its importance on real data from the United States Internal Revenue Service (IRS). The IRS performs yearly audits of the tax base. Two of its most important objectives are to identify suspected misreporting and to estimate the "tax gap" -- the global difference between the amount paid and true amount owed. Based on a unique collaboration with the IRS, we cast these two processes as a unified optimize-and-estimate structured bandit. We analyze optimize-and-estimate approaches to the IRS problem and propose a novel mechanism for unbiased population estimation that achieves rewards comparable to baseline approaches. This approach has the potential to improve audit efficacy, while maintaining policy-relevant estimates of the tax gap. This has important social consequences given that the current tax gap is estimated at nearly half a trillion dollars. We suggest that this problem setting is fertile ground for further research and we highlight its interesting challenges. The results of this and related research are currently being incorporated into the continual improvement of the IRS audit selection methods.

我们提出了置信度序列 - 置信区间序列，其均匀地随时间均匀 - 用于基于I.I.D的流的完整，完全有序集中的任何分布的量级。观察。我们提供用于跟踪固定定量的方法并同时跟踪所有定量。具体而言，我们提供具有小常数的明确表达式，其宽度以尽可能快的$ \ SQRT {t} \ log \ log t} $率，以及实证分布函数的非渐近浓度不等式以相同的速率均匀地持续持续。后者加强了Smirnov迭代对数的实证过程法，延长了DVORETZKY-KIEFER-WOLFOITZ不等式以均匀地保持一段时间。我们提供了一种新的算法和样本复杂性，用于在多武装强盗框架中选择具有大约最佳定量的臂。在仿真中，我们的方法需要比现有方法更少五到五十的样品。

我们为随机线性匪徒问题提出了一种新的基于自举的在线算法。关键的想法是采用残留的自举勘探，在该探索中，代理商通过重新采样平均奖励估算的残差来估算下一步奖励。我们的算法，随机线性匪徒（\ texttt {linreboot}）的残留bootstrap探索，从其重新采样分布中估算了线性奖励，并以最高的奖励估计拉动了手臂。特别是，我们为理论框架做出了一个理论框架，以使基于自举的探索机制在随机线性匪徒问题中脱颖而出。关键见解是，Bootstrap探索的强度基于在线学习模型和残差的重新采样分布之间的乐观情绪。这样的观察使我们能够证明所提出的\ texttt {linreboot}确保了高概率$ \ tilde {o}（d \ sqrt {n}）$ sub-linear在温和条件下的遗憾。我们的实验支持\ texttt {重新启动}原理在线性匪徒问题的各种公式中的简易概括性，并显示了\ texttt {linreboot}的显着计算效率。

我们考虑多臂绷带（MAB）中最好的臂识别（Bai）问题的变体，其中有两组臂（源头和目标），目的是确定最佳目标臂，同时仅拉动源臂。在本文中，我们研究了设置的时候，尽管是未知的手段，但源和目标MAB实例之间存在已知的附加关系。我们展示了我们的框架如何涵盖一系列以前研究的纯粹探索问题，并且还捕获了新的问题。我们提出并理论上分析了LUCB风格的算法，以识别具有高概率的$ \ epsilon $ -optimal目标手臂。我们的理论分析强调了在典型的BAI设置中不会出现的这种转移学习问题的方面，但恢复了单个域Bai的Lucb算法作为特殊情况。

随机梯度马尔可夫链Monte Carlo（SGMCMC）是一种流行的可扩展贝叶斯推断算法。然而，这些算法包括诸如步进尺寸或批量尺寸，这些算法基于所获得的后样品影响估计器的准确性。因此，必须由从业者调整这些超级参数，目前没有具体的和自动化方式来调整它们存在。基于接受率的标准MCMC调整方法不能用于SGMCMC，从而需要替代工具和诊断。我们提出了一种基于新的基于强盗的算法，通过最小化真正的后后部和蒙特卡罗近似之间的斯坦坦差异来调谐SGMCMC近似度。我们提供支持这种方法的理论结果，并评估各种基于Stein的差异。我们通过对模拟和实际数据集的实验支持我们的结果，并发现该方法对于各种应用程序实用。

推荐系统在市场中使用时发挥了双重作用：它们可以帮助用户从大型游泳池中选择最需要的物品，并有助于将有限数量的物品分配给最想要它们的用户。尽管在许多现实世界中的推荐设置中，能力限制的流行率普遍存在，但缺乏将它们纳入这些系统设计的原则性方式。在此激励的情况下，我们提出了一个交互式框架，系统提供商可以通过机会主义探索分配来提高向用户的建议质量，从而最大程度地利用用户奖励并使用适当的定价机制尊重容量约束。我们将问题建模为低排名组合的多臂匪徒问题的实例，并在手臂上进行了选择约束。我们采用一种集成方法，使用协作过滤，组合匪徒和最佳资源分配中的技术，以提供一种算法，可证明可以实现次线性遗憾，即$ \ tilde {\ mathcal {\ sqrt {o}}（\ sqrt {\ sqrt {n+m（n+m）{n+m（n+m） ）rt}）$ in $ t $ rounds，用于$ n $用户，$ m $项目和排名$ r $ ney奖励矩阵的问题。关于合成和现实世界数据的实证研究也证明了我们方法的有效性和性能。

We consider the stochastic linear contextual bandit problem with high-dimensional features. We analyze the Thompson sampling (TS) algorithm, using special classes of sparsity-inducing priors (e.g. spike-and-slab) to model the unknown parameter, and provide a nearly optimal upper bound on the expected cumulative regret. To the best of our knowledge, this is the first work that provides theoretical guarantees of Thompson sampling in high dimensional and sparse contextual bandits. For faster computation, we use spike-and-slab prior to model the unknown parameter and variational inference instead of MCMC to approximate the posterior distribution. Extensive simulations demonstrate improved performance of our proposed algorithm over existing ones.

We study the best-arm identification problem in multi-armed bandits with stochastic, potentially private rewards, when the goal is to identify the arm with the highest quantile at a fixed, prescribed level. First, we propose a (non-private) successive elimination algorithm for strictly optimal best-arm identification, we show that our algorithm is $\delta$-PAC and we characterize its sample complexity. Further, we provide a lower bound on the expected number of pulls, showing that the proposed algorithm is essentially optimal up to logarithmic factors. Both upper and lower complexity bounds depend on a special definition of the associated suboptimality gap, designed in particular for the quantile bandit problem, as we show when the gap approaches zero, best-arm identification is impossible. Second, motivated by applications where the rewards are private, we provide a differentially private successive elimination algorithm whose sample complexity is finite even for distributions with infinite support-size, and we characterize its sample complexity. Our algorithms do not require prior knowledge of either the suboptimality gap or other statistical information related to the bandit problem at hand.

在本文中，我们制定了在内核强盗问题（COPE-KB）中的协作纯探索，它为在有限的通信和一般奖励函数下提供了一种用于多智能组件多任务决策的新型模型，并且适用于许多在线学习任务，例如，推荐系统和网络调度。我们考虑两个COPE-KB，即固定信道（FC）和固定预算（FB）的设置，以及设计两个最佳算法COOPKERNECC（FC）和Coopkerhelfb（FB）。我们的算法配备了创新和高效的核化估计，同时实现了计算和通信效率。建立统计和通信度量标准下的上限和下限以证明我们算法的最优性。理论界限成功地量化了任务相似性对学习加速度的影响，并且只取决于内核特征空间的有效维度。我们的分析技术，包括数据尺寸分解，线性结构化实例转换和（通信）圆形加速感应，是新颖的，适用于其他强盗问题。提供了实证评估以验证我们的理论结果，并展示我们算法的性能优势。