In this paper, we propose a novel technique to accelerate Ising machines hyperparameter tuning. Firstly, we define Ising machine performance and explain the goal of hyperparameter tuning in regard to this performance definition. Secondly, we compare well-known hyperparameter tuning techniques, namely random sampling and Tree-structured Parzen Estimator (TPE) on different combinatorial optimization problems. Thirdly, we propose a new convergence acceleration method for TPE which we call "FastConvergence".It aims at limiting the number of required TPE trials to reach best performing hyperparameter values combination. We compare FastConvergence to previously mentioned well-known hyperparameter tuning techniques to show its effectiveness. For experiments, well-known Travel Salesman Problem (TSP) and Quadratic Assignment Problem (QAP) instances are used as input. The Ising machine used is Fujitsu's third generation Digital Annealer (DA). Results show, in most cases, FastConvergence can reach similar results to TPE alone within less than half the number of trials.

In more recent years, there has been increasing research interest in exploiting the use of application specific hardware for solving optimisation problems. Examples of solvers that use specialised hardware are IBM's Quantum System One and D-wave's Quantum Annealer (QA) and Fujitsu's Digital Annealer (DA). These solvers have been developed to optimise problems faster than traditional meta-heuristics implemented on general purpose machines. Previous research has shown that these solvers (can optimise many problems much quicker than exact solvers such as GUROBI and CPLEX. Such conclusions have not been made when comparing hardware solvers with classical evolutionary algorithms. Making a fair comparison between traditional evolutionary algorithms, such as Genetic Algorithm (GA), and the DA (or other similar solvers) is challenging because the later benefits from the use of application specific hardware while evolutionary algorithms are often implemented on general-purpose machines. Moreover, quantum or quantum-inspired solvers are limited to solving problems in a specific format. A common formulation used is Quadratic Unconstrained Binary Optimisation (QUBO). Many optimisation problems are however constrained and have natural representations that are non-binary. Converting such problems to QUBO can lead to more problem difficulty and/or larger search space. The question addressed in this paper is whether quantum or quantum-inspired solvers can optimise QUBO transformations of combinatorial optimisation problems faster than classical evolutionary algorithms applied to the same problems in their natural representations. We show that the DA often present better average objective function values than GA on Travelling Salesman, Quadratic Assignment and Multi-dimensional Knapsack Problem instances.

近年来，设计用于量子计算机或其他专业硬件的优化算法引起了研究的兴趣。这些求解器中的许多只能优化二进制和二次形式的问题。因此，二次不受约束的二进制优化（QUBO）是这些求解器使用的常见公式。有许多组合优化问题自然表示为排列，例如旅行推销员问题。但是，使用二进制变量编码置换问题，但是提出了一些挑战。许多QUBO求解器是单个翻转求解器，因此可以生成无法解码为有效置换的解决方案。为了产生产生可行解决方案的偏见，我们使用惩罚权重。为各种类型问题设定静态罚重的过程并不是微不足道的。这是因为太小的值会导致求解器返回不可行的解决方案，而太大的值可能会导致收敛速度较慢。在这项研究中，我们探讨了在QUBO配方中设置惩罚权重的一些方法。我们提出了新的静态方法来计算惩罚权重，这比现有方法更有希望的结果。

算法配置（AC）与对参数化算法最合适的参数配置的自动搜索有关。目前，文献中提出了各种各样的交流问题变体和方法。现有评论没有考虑到AC问题的所有衍生物，也没有提供完整的分类计划。为此，我们引入分类法以分别描述配置方法的交流问题和特征。我们回顾了分类法的镜头中现有的AC文献，概述相关的配置方法的设计选择，对比方法和问题变体相互对立，并描述行业中的AC状态。最后，我们的评论为研究人员和从业人员提供了AC领域的未来研究方向。

由于强烈的非线性系统行为和多个竞争目标，能源系统优化问题很复杂，例如，经济增益与环境影响。此外，大量输入变量和不同的变量类型，例如，连续和分类，是现实世界应用中常见的挑战。在某些情况下，提出的最佳解决方案需要遵守与物理性质或安全关键操作条件相关的显式输入限制。本文提出了一种新的数据驱动策略，使用树集合用于对黑匣子问题的约束多目标优化，与模型或未知的基础系统动态太复杂的异构变量空间。在由合成基准和相关能源应用组成的广泛案例研究中，我们展示了与其他最先进的工具相比，所提出的算法的竞争性能和采样效率，使其成为一个有用的全能解决方案 - 世界申请有限评价预算。

我们解决了与行业相关的尺度上的机器人轨迹计划问题。我们的端到端解决方案将高度通用的随机键算法与模型堆叠和集成技术集成在一起，以及用于溶液细化的路径重新链接。核心优化模块由偏置的随机基遗传算法组成。通过与问题依赖性和问题相关模块的独特分离，我们通过约束的天然编码实现了有效的问题表示。我们表明，对替代算法范式（例如模拟退火）的概括是直接的。我们为行业规模的数据集提供数值基准结果。发现我们的方法始终超过贪婪的基线结果。为了评估当今量子硬件的功能，我们使用Amazon Braket上的QBSOLV在量子退火硬件上获得的经典方法进行了补充。最后，我们展示了如何将后者集成到我们的较大管道中，从而为问题提供了量子准备的混合解决方案。

Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation and early-stopping. We formulate hyperparameter optimization as a pure-exploration nonstochastic infinite-armed bandit problem where a predefined resource like iterations, data samples, or features is allocated to randomly sampled configurations. We introduce a novel algorithm, Hyperband, for this framework and analyze its theoretical properties, providing several desirable guarantees. Furthermore, we compare Hyperband with popular Bayesian optimization methods on a suite of hyperparameter optimization problems. We observe that Hyperband can provide over an order-of-magnitude speedup over our competitor set on a variety of deep-learning and kernel-based learning problems.

近年来，在平衡（超级）图分配算法的设计和评估中取得了重大进展。我们调查了过去十年的实用算法的趋势，用于平衡（超级）图形分区以及未来的研究方向。我们的工作是对先前有关该主题的调查的更新。特别是，该调查还通过涵盖了超图形分区和流算法来扩展先前的调查，并额外关注并行算法。

Two-stage robust optimization problems constitute one of the hardest optimization problem classes. One of the solution approaches to this class of problems is K-adaptability. This approach simultaneously seeks the best partitioning of the uncertainty set of scenarios into K subsets, and optimizes decisions corresponding to each of these subsets. In general case, it is solved using the K-adaptability branch-and-bound algorithm, which requires exploration of exponentially-growing solution trees. To accelerate finding high-quality solutions in such trees, we propose a machine learning-based node selection strategy. In particular, we construct a feature engineering scheme based on general two-stage robust optimization insights that allows us to train our machine learning tool on a database of resolved B&B trees, and to apply it as-is to problems of different sizes and/or types. We experimentally show that using our learned node selection strategy outperforms a vanilla, random node selection strategy when tested on problems of the same type as the training problems, also in case the K-value or the problem size differs from the training ones.

Algorithms that involve both forecasting and optimization are at the core of solutions to many difficult real-world problems, such as in supply chains (inventory optimization), traffic, and in the transition towards carbon-free energy generation in battery/load/production scheduling in sustainable energy systems. Typically, in these scenarios we want to solve an optimization problem that depends on unknown future values, which therefore need to be forecast. As both forecasting and optimization are difficult problems in their own right, relatively few research has been done in this area. This paper presents the findings of the ``IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling," held in 2021. We present a comparison and evaluation of the seven highest-ranked solutions in the competition, to provide researchers with a benchmark problem and to establish the state of the art for this benchmark, with the aim to foster and facilitate research in this area. The competition used data from the Monash Microgrid, as well as weather data and energy market data. It then focused on two main challenges: forecasting renewable energy production and demand, and obtaining an optimal schedule for the activities (lectures) and on-site batteries that lead to the lowest cost of energy. The most accurate forecasts were obtained by gradient-boosted tree and random forest models, and optimization was mostly performed using mixed integer linear and quadratic programming. The winning method predicted different scenarios and optimized over all scenarios jointly using a sample average approximation method.

自动化机器学习（AutomL）的基本任务是在给定数据集中自动查找流量的流水线的问题。此问题已通过贝叶斯优化，语法族遗传算法和树搜索算法等复杂的黑盒优化技术来解决了这个问题。大多数当前方法都是通过假设优化分离的管道的组分可以产生次优效果。我们展示了天真的Automl，这一方法确实如此：它可以在隔离中优化预定义的流水线方案的不同算法。最后返回的管道通过仅采用每个插槽的最佳算法获得。孤立的优化导致大幅减少的搜索空间，并且令人惊讶地，这种方法产生比目前最先进的优化器的相当且有时更好的性能。

In the field of derivative-free optimization, both of its main branches, the deterministic and nature-inspired techniques, experienced in recent years substantial advancement. In this paper, we provide an extensive computational comparison of selected methods from each of these branches. The chosen representatives were either standard and well-utilized methods, or the best-performing methods from recent numerical comparisons. The computational comparison was performed on five different benchmark sets and the results were analyzed in terms of performance, time complexity, and convergence properties of the selected methods. The results showed that, when dealing with situations where the objective function evaluations are relatively cheap, the nature-inspired methods have a significantly better performance than their deterministic counterparts. However, in situations when the function evaluations are costly or otherwise prohibited, the deterministic methods might provide more consistent and overall better results.

表达性和计算便宜的两分图神经网络（GNN）已被证明是基于深度学习的混合成分线性程序（MILP）求解器的重要组成部分。最近的工作证明了此类GNN在分支结合（B＆B）求解器中取代分支（可变选择）启发式方面的有效性。这些GNN经过训练，离线和集合，以模仿一个非常好但计算昂贵的分支启发式，强大的分支。鉴于B＆B会导致子隔间树，我们问（a）目标启发式启发式在B＆B树的邻近节点之间是否存在很强的依赖性，并且（b）如果是这样，我们是否可以将它们合并到我们的培训程序。具体来说，我们发现，有了强大的分支启发式，孩子节点的最佳选择通常是父母的第二好的选择。我们将其称为“回顾”现象。令人惊讶的是，Gasse等人的典型分支GNN。 （2019年）经常错过这个简单的“答案”。为了通过将回顾现象纳入GNN来更紧密地模仿目标行为，我们提出了两种方法：（a）标准跨凝性损失函数的目标平滑，（b）添加父级（PAT）target（PAT）回顾量学期。最后，我们提出了一个模型选择框架，以结合更难构建的目标，例如在最终模型中解决时间。通过对标准基准实例进行广泛的实验，我们表明我们的提案导致B＆B树大小的22％减少，并且在解决时间的解决方案中提高了15％。

黑盒优化在许多应用中具有潜力，例如在实验设计中的机器学习和优化中的超参数优化。 ISING机器对二进制优化问题很有用，因为变量可以由Ising机器的单个二进制变量表示。但是，使用ISING机器的常规方法无法处理具有非二进制值的黑框优化问题。为了克服这一限制，我们通过与三种不同的整数编码方法合作，通过使用ISING/退火计算机和分解计算机来提出一种用于整数变量的黑盒优化问题的方法。使用不同的编码方法，使用一个简单的问题来计算最稳定状态下的氢分子能量，以不同的编码方法进行数值评估。提出的方法可以使用任何整数编码方法来计算能量。但是，单次编码对于小尺寸的问题很有用。

自动化封路计优化（HPO）已经获得了很大的普及，并且是大多数自动化机器学习框架的重要成分。然而，设计HPO算法的过程仍然是一个不系统和手动的过程：确定了现有工作的限制，提出的改进是 - 即使是专家知识的指导 - 仍然是一定任意的。这很少允许对哪些算法分量的驾驶性能进行全面了解，并且承载忽略良好算法设计选择的风险。我们提出了一个原理的方法来实现应用于多倍性HPO（MF-HPO）的自动基准驱动算法设计的原则方法：首先，我们正式化包括的MF-HPO候选的丰富空间，但不限于普通的HPO算法，然后呈现可配置的框架覆盖此空间。要自动和系统地查找最佳候选者，我们遵循通过优化方法，并通过贝叶斯优化搜索算法候选的空间。我们挑战是否必须通过执行消融分析来挑战所发现的设计选择或可以通过更加天真和更简单的设计。我们观察到使用相对简单的配置，在某些方式中比建立的方法更简单，只要某些关键配置参数具有正确的值，就可以很好地执行得很好。

通过边界估计可以显着简化求解约束优化问题（COP），即提供成本函数的紧密边界。通过使用由已知边界的数据组成的数据以及COMPS提取的特征来馈送监督机器学习（ML）模型，可以训练模型以估计新COP实例的边界。在本文中，我们首先概述了来自问题实例的约束编程（CP）的ML的现有知识体系。其次，我们介绍了应用于支持CP解算器的工具的边界估计框架。在该框架内，讨论并评估了不同的ML模型，并评估其对边界估计的适用性，并避免避免求解器找到最佳解决方案的不可行估计的对策。第三，我们在七个警察中提出了一种实验研究，与不同的CP溶剂。我们的结果表明，可以仅限于这些警察的近似最佳边界。这些估计的边界将客观域大小减少60-88％，可以帮助求解器在搜索期间提前找到近乎最佳解决方案。

在确定性优化中，通常假定问题的所有参数都是固定和已知的。但是，实际上，某些参数可能是未知的先验参数，但可以从历史数据中估算。典型的预测 - 优化方法将预测和优化分为两个阶段。最近，端到端的预测到优化已成为有吸引力的替代方法。在这项工作中，我们介绍了PYEPO软件包，这是一个基于Pytorch的端到端预测，然后在Python中进行了优化的库。据我们所知，PYEPO（发音为“带有静音” n“”的“菠萝”）是线性和整数编程的第一个通用工具，具有预测的目标函数系数。它提供了两种基本算法：第一种基于Elmachtoub＆Grigas（2021）的开创性工作的凸替代损失函数，第二个基于Vlastelica等人的可区分黑盒求解器方法。 （2019）。 PYEPO提供了一个简单的接口，用于定义新的优化问题，最先进的预测 - 优化训练算法，自定义神经网络体系结构的使用以及端到端方法与端到端方法与与端到端方法的比较两阶段的方法。 PYEPO使我们能够进行一系列全面的实验，以比较沿轴上的多种端到端和两阶段方法，例如预测准确性，决策质量和运行时间，例如最短路径，多个背包和旅行等问题销售人员问题。我们讨论了这些实验中的一些经验见解，这些见解可以指导未来的研究。 PYEPO及其文档可在https://github.com/khalil-research/pyepo上找到。

Several recent advances to the state of the art in image classification benchmarks have come from better configurations of existing techniques rather than novel approaches to feature learning. Traditionally, hyper-parameter optimization has been the job of humans because they can be very efficient in regimes where only a few trials are possible. Presently, computer clusters and GPU processors make it possible to run more trials and we show that algorithmic approaches can find better results. We present hyper-parameter optimization results on tasks of training neural networks and deep belief networks (DBNs). We optimize hyper-parameters using random search and two new greedy sequential methods based on the expected improvement criterion. Random search has been shown to be sufficiently efficient for learning neural networks for several datasets, but we show it is unreliable for training DBNs. The sequential algorithms are applied to the most difficult DBN learning problems from [1] and find significantly better results than the best previously reported. This work contributes novel techniques for making response surface models P (y|x) in which many elements of hyper-parameter assignment (x) are known to be irrelevant given particular values of other elements.

为混合整数线性编程问题（MILLS）找到高质量解决方案对于许多实际应用非常重要。在这方面，提出了精炼启发式局部分支（LB）来生产改进解决方案，并且对MILP中的本地搜索方法的开发产生了高度影响力。该算法迭代地探索由所谓的本地分支约束定义的一系列解决方案邻域，即，限制与参考解决方案的距离的线性不等式。对于LB算法，邻域大小的选择对于性能至关重要。虽然它是由原始LB方案中的保守值初始化的，但我们的新观察是最佳规模强烈依赖于特定的MILP实例。在这项工作中，我们调查搜索附近的大小与底层LB算法的行为之间的关系，我们设计了一种基于倾斜的框架，用于引导LB启发式的邻居搜索。该框架由两阶段战略组成。对于第一阶段，训练缩放的回归模型以通过回归任务在第一迭代中预测LB邻域的大小。在第二阶段，我们利用加强学习和设计加强的邻域搜索策略，以动态调整随后的迭代处的大小。我们计算地表明，确实可以学习邻域大小，导致改进的性能，并且整个算法在实例大小相对于实例大小概括，并且显着地跨越实例概括。

决策树学习是机器学习中广泛使用的方法，在需要简洁明了的模型的应用中受到青睐。传统上，启发式方法用于快速生产具有相当高准确性的模型。然而，一个普遍的批评是，从精度和大小方面，所产生的树可能不一定是数据的最佳表示。近年来，这激发了最佳分类树算法的发展，这些算法与执行一系列本地最佳决策的启发式方法相比，在全球范围内优化决策树。我们遵循这一工作线，并提供了一种基于动态编程和搜索的最佳分类树的新颖算法。我们的算法支持对树的深度和节点数量的约束。我们方法的成功归因于一系列专门技术，这些技术利用了分类树独有的属性。传统上，最佳分类树的算法受到了高运行时的困扰和有限的可伸缩性，但我们在一项详细的实验研究中表明，我们的方法仅使用最先进的时间所需的时间，并且可以处理数十个数据集的数据集在数千个实例中，提供了几个数量级的改进，并特别有助于实现最佳决策树的实现。