Evolutionary algorithms (EAs) are a kind of nature-inspired general-purpose optimization algorithm, and have shown empirically good performance in solving various real-word optimization problems. During the past two decades, promising results on the running time analysis (one essential theoretical aspect) of EAs have been obtained, while most of them focused on isolated combinatorial optimization problems, which do not reflect the general-purpose nature of EAs. To provide a general theoretical explanation of the behavior of EAs, it is desirable to study their performance on general classes of combinatorial optimization problems. To the best of our knowledge, the only result towards this direction is the provably good approximation guarantees of EAs for the problem class of maximizing monotone submodular functions with matroid constraints. The aim of this work is to contribute to this line of research. Considering that many combinatorial optimization problems involve non-monotone or non-submodular objective functions, we study the general problem classes, maximizing submodular functions with/without a size constraint and maximizing monotone approximately submodular functions with a size constraint. We prove that a simple multi-objective EA called GSEMO-C can generally achieve good approximation guarantees in polynomial expected running time.

Clustering is a fundamental problem in many areas, which aims to partition a given data set into groups based on some distance measure, such that the data points in the same group are similar while that in different groups are dissimilar. Due to its importance and NP-hardness, a lot of methods have been proposed, among which evolutionary algorithms are a class of popular ones. Evolutionary clustering has found many successful applications, but all the results are empirical, lacking theoretical support. This paper fills this gap by proving that the approximation performance of the GSEMO (a simple multi-objective evolutionary algorithm) for solving the three popular formulations of clustering, i.e., $k$-center, $k$-median and $k$-means, can be theoretically guaranteed. Furthermore, we prove that evolutionary clustering can have theoretical guarantees even when considering fairness, which tries to avoid algorithmic bias, and has recently been an important research topic in machine learning.

Evolutionary algorithms (EAs) are general-purpose optimization algorithms, inspired by natural evolution. Recent theoretical studies have shown that EAs can achieve good approximation guarantees for solving the problem classes of submodular optimization, which have a wide range of applications, such as maximum coverage, sparse regression, influence maximization, document summarization and sensor placement, just to name a few. Though they have provided some theoretical explanation for the general-purpose nature of EAs, the considered submodular objective functions are defined only over sets or multisets. To complement this line of research, this paper studies the problem class of maximizing monotone submodular functions over sequences, where the objective function depends on the order of items. We prove that for each kind of previously studied monotone submodular objective functions over sequences, i.e., prefix monotone submodular functions, weakly monotone and strongly submodular functions, and DAG monotone submodular functions, a simple multi-objective EA, i.e., GSEMO, can always reach or improve the best known approximation guarantee after running polynomial time in expectation. Note that these best-known approximation guarantees can be obtained only by different greedy-style algorithms before. Empirical studies on various applications, e.g., accomplishing tasks, maximizing information gain, search-and-tracking and recommender systems, show the excellent performance of the GSEMO.

Evolutionary algorithms (EAs) have found many successful real-world applications, where the optimization problems are often subject to a wide range of uncertainties. To understand the practical behaviors of EAs theoretically, there are a series of efforts devoted to analyzing the running time of EAs for optimization under uncertainties. Existing studies mainly focus on noisy and dynamic optimization, while another common type of uncertain optimization, i.e., robust optimization, has been rarely touched. In this paper, we analyze the expected running time of the (1+1)-EA solving robust linear optimization problems (i.e., linear problems under robust scenarios) with a cardinality constraint $k$. Two common robust scenarios, i.e., deletion-robust and worst-case, are considered. Particularly, we derive tight ranges of the robust parameter $d$ or budget $k$ allowing the (1+1)-EA to find an optimal solution in polynomial running time, which disclose the potential of EAs for robust optimization.

机会受到限制的优化问题允许建模问题，其中涉及随机组件的约束仅应以较小的概率侵犯。进化算法已应用于这种情况，并证明可以实现高质量的结果。在本文中，我们有助于对进化算法的理论理解，以进行偶然的优化。我们研究独立且正态分布的随机组件的场景。考虑到简单的单对象（1+1）〜EA，我们表明，施加额外的统一约束已经导致局部最佳选择，对于非常有限的场景和指数优化时间。因此，我们引入了问题的多目标公式，该公式可以摆脱预期成本及其差异。我们表明，在使用此公式时，多目标进化算法是非常有效的，并获得一组解决方案，该解决方案包含最佳解决方案，以适用于施加在约束上的任何可能的置信度。此外，我们证明这种方法还可以用于计算一组最佳解决方案，以限制最小跨越树问题。为了在多目标配方中呈指数指数的折衷，我们提出并分析了改进的凸多目标方法。关于NP-固定随机最小重量占主导地位问题的实例的实验研究证实了多目标和改进的凸多目标方法的益处。

In many real-world optimization problems, the objective function evaluation is subject to noise, and we cannot obtain the exact objective value. Evolutionary algorithms (EAs), a type of general-purpose randomized optimization algorithm, have been shown to be able to solve noisy optimization problems well. However, previous theoretical analyses of EAs mainly focused on noise-free optimization, which makes the theoretical understanding largely insufficient for the noisy case. Meanwhile, the few existing theoretical studies under noise often considered the one-bit noise model, which flips a randomly chosen bit of a solution before evaluation; while in many realistic applications, several bits of a solution can be changed simultaneously. In this paper, we study a natural extension of one-bit noise, the bit-wise noise model, which independently flips each bit of a solution with some probability. We analyze the running time of the (1+1)-EA solving OneMax and LeadingOnes under bit-wise noise for the first time, and derive the ranges of the noise level for polynomial and super-polynomial running time bounds. The analysis on LeadingOnes under bit-wise noise can be easily transferred to one-bit noise, and improves the previously known results. Since our analysis discloses that the (1+1)-EA can be efficient only under low noise levels, we also study whether the sampling strategy can bring robustness to noise. We prove that using sampling can significantly increase the largest noise level allowing a polynomial running time, that is, sampling is robust to noise.

In noisy evolutionary optimization, sampling is a common strategy to deal with noise. By the sampling strategy, the fitness of a solution is evaluated multiple times (called \emph{sample size}) independently, and its true fitness is then approximated by the average of these evaluations. Most previous studies on sampling are empirical, and the few theoretical studies mainly showed the effectiveness of sampling with a sufficiently large sample size. In this paper, we theoretically examine what strategies can work when sampling with any fixed sample size fails. By constructing a family of artificial noisy examples, we prove that sampling is always ineffective, while using parent or offspring populations can be helpful on some examples. We also construct an artificial noisy example to show that when using neither sampling nor populations is effective, a tailored adaptive sampling (i.e., sampling with an adaptive sample size) strategy can work. These findings may enhance our understanding of sampling to some extent, but future work is required to validate them in natural situations.

设置子模块目标函数的优化问题具有许多现实世界应用。在离散场景中，在可以选择同一项目的情况下，域通过设置到有界整数格的2元素概括。在这项工作中，我们考虑最大化界限整数晶格上的单调子模块功能的问题，受到基数约束。特别是，我们专注于最大化D​​R-SubsoDular函数，即在整数格中定义的函数，该函数展示递减返回属性。给定任何epsilon> 0，我们介绍了一种随机算法的概率保证o（1 - 1 / e-epsilon）近似，使用由Mirzasoleiman等人开发的随机贪婪算法启发的框架。然后，我们表明，在合成DR-IMODOOMULAL功能上，在整数晶格上应用我们的建议算法比替代方案快，包括将目标问题还原到集合域，然后应用于最快的已知的集合子态最大化算法。

我们研究动态算法，以便在$ N $插入和删除流中最大化单调子模块功能的问题。我们显示任何维护$（0.5+ epsilon）$ - 在基数约束下的近似解决方案的算法，对于任何常数$ \ epsilon> 0 $，必须具有$ \ mathit {polynomial} $的摊销查询复杂性$ n $。此外，需要线性摊销查询复杂性，以维持0.584美元 - 批量的解决方案。这与近期[LMNF + 20，MON20]的最近动态算法相比，达到$（0.5- \ epsilon）$ - 近似值，与$ \ mathsf {poly} \ log（n）$摊销查询复杂性。在正面，当流是仅插入的时候，我们在基数约束下的问题和近似的Matroid约束下提供有效的算法，近似保证$ 1-1 / e-\ epsilon $和摊销查询复杂性$ \ smash {o （\ log（k / \ epsilon）/ \ epsilon ^ 2）} $和$ \ smash {k ^ {\ tilde {o}（1 / \ epsilon ^ 2）} \ log n} $，其中$ k $表示基数参数或Matroid的等级。

非主导的分类遗传算法II（NSGA-II）是现实应用中最强烈使用的多目标进化算法（MOEA）。然而，与几个通过数学手段分析的几个简单的MOES相反，到目前为止，NSGA-II也不存在这种研究。在这项工作中，我们表明，数学运行时分析也可用于NSGA-II。结果，我们证明，由于持续因素大于帕累托前方大小的人口大小，具有两个经典突变算子的NSGA-II和三种不同的选择父母的方式满足与Semo和GSEMO相同的渐近运行时保证基本ineminmax和Lotz基准函数的算法。但是，如果人口大小仅等于帕累托前面的大小，那么NSGA-II就无法有效地计算完整的帕累托前部（对于指数迭代，人口总是错过帕累托前部的恒定分数） 。我们的实验证实了上述研究结果。

The Makespan Scheduling problem is an extensively studied NP-hard problem, and its simplest version looks for an allocation approach for a set of jobs with deterministic processing times to two identical machines such that the makespan is minimized. However, in real life scenarios, the actual processing time of each job may be stochastic around the expected value with a variance, under the influence of external factors, and the actual processing times of these jobs may be correlated with covariances. Thus within this paper, we propose a chance-constrained version of the Makespan Scheduling problem and investigate the theoretical performance of the classical Randomized Local Search and (1+1) EA for it. More specifically, we first study two variants of the Chance-constrained Makespan Scheduling problem and their computational complexities, then separately analyze the expected runtime of the two algorithms to obtain an optimal solution or almost optimal solution to the instances of the two variants. In addition, we investigate the experimental performance of the two algorithms for the two variants.

In model selection problems for machine learning, the desire for a well-performing model with meaningful structure is typically expressed through a regularized optimization problem. In many scenarios, however, the meaningful structure is specified in some discrete space, leading to difficult nonconvex optimization problems. In this paper, we connect the model selection problem with structure-promoting regularizers to submodular function minimization with continuous and discrete arguments. In particular, we leverage the theory of submodular functions to identify a class of these problems that can be solved exactly and efficiently with an agnostic combination of discrete and continuous optimization routines. We show how simple continuous or discrete constraints can also be handled for certain problem classes and extend these ideas to a robust optimization framework. We also show how some problems outside of this class can be embedded within the class, further extending the class of problems our framework can accommodate. Finally, we numerically validate our theoretical results with several proof-of-concept examples with synthetic and real-world data, comparing against state-of-the-art algorithms.

大多数进化算法具有多个参数，它们的值大大影响性能。由于参数的常复相互作用，将这些值设置为特定问题（参数调整）是一个具有挑战性的任务。当最佳参数值在算法运行期间最佳参数值发生显着变化时，此任务变得更加复杂。然后是必要的动态参数选择（参数控制）。在这项工作中，我们提出了一个懒惰但有效的解决方案，即从一个适当缩放的幂律分布中随机地选择所有参数值（在那里这是有意义的）。为了展示这种方法的有效性，我们使用以这种方式选择的所有三个参数执行$（1 +（\ lambda，\ lambda））$遗传算法的运行时分析。我们展示该算法一方面可以模仿像$（1 + 1）$ EA这样的简单山羊，给出了onemax，领导者或最小生成树等问题的相同渐近运行时。另一方面，该算法对跳跃功能也非常有效，其中最佳静态参数与优化简单问题所需的静态参数非常不同。我们证明了具有可比性的性能保证，有时比静态参数所知的最佳性能更好。我们通过严格的实证研究来补充我们的理论结果，证实了渐近运行时期结果的建议。

在进化计算中使用非豁免主义时的一个希望是放弃当前最佳解决方案的能力，艾滋病们离开本地最佳效果。为了提高我们对这种机制的理解，我们对基本的非精英进化算法（EA），$（\ mu，\ lambda）$ ea进行严格的运行时分析，在最基本的基准函数上，具有本地最佳的基本基准函数跳跃功能。我们证明，对于参数和问题的所有合理值，$（\ mu，\ lambda）$ ~ea的预期运行时间除了下订单条款之外，至少与其Elitist对应的预期运行时间，$（\ mu + \ lambda）$〜ea（我们对跳转功能进行第一个运行时分析以允许此比较）。因此，$（\ mu，\ lambda）$ ~ea将本地最优方式留给劣质解决方案的能力不会导致运行时优势。我们补充了这个下限的下限，即对于参数的广泛范围，与我们的下限不同，与下顺序不同。这是一个在多模态问题上的非精英算法的第一个运行时结果，除了下订单术语。

在机器学习中最大化的是一项基本任务，在本文中，我们研究了经典的Matroid约束下的删除功能强大版本。在这里，目标是提取数据集的小尺寸摘要，即使在对手删除了一些元素之后，该数据集包含高价值独立集。我们提出了恒定因素近似算法，其空间复杂性取决于矩阵的等级$ k $和已删除元素的数字$ d $。在集中式设置中，我们提出$（4.597+o（\ varepsilon））$ - 近似算法，带有摘要大小$ o（\ frac {k+d} {\ varepsilon^2} \ log \ log \ frac \ frac {k} }）$将$（3.582 + o（\ varepsilon））$（k + \ frac {d} {\ varepsilon^2} \ log \ frac {k} {k} {\ varepsilon}） $摘要大小是单调的。在流设置中，我们提供$（9.435 + o（\ varepsilon））$ - 带有摘要大小和内存$ o的近似算法$（k + \ frac {d} {\ varepsilon^2} \ log \ log \ frac {k} {k} {k} {k} {k} {k} { \ varepsilon}）$;然后，将近似因子提高到单调盒中的$（5.582+o（\ varepsilon））$。

跳跃功能是随机搜索启发式理论中的{最多研究的非单峰基准，特别是进化算法（EA）。他们对我们的理解显着改善了EASE逃离当地最优的理解。然而，他们的特殊结构 - 离开本地最佳的结构只能直接跳到全球最优 - 引发代表性这种结果的问题。出于这个原因，我们提出了一个扩展的$ \ textsc {jump} _ {k，\ delta} $ jump函数，其中包含宽度$ \ delta $的低适合度vally以距离$ k $从全局最佳v $开始。我们证明了几个以前的结果延伸到这一更普遍的类：对于所有{$ k \ le \ frac {n ^ {1/3}} {\ ln {n}} $}和$ \ delta <k $，最佳$（1 + 1）$〜EA的突变率是$ \ FRAC {\ delta} $，并且快速$（1 + 1）$〜EA运行比经典$（1 + 1）$更快〜ea在$ \ delta $中的一个超级指数。但是，我们还观察到一些已知结果不概括：随机本地搜索算法具有停滞检测，其比$ \ textsc的$ k $ k $ k $ k $ k $ k $ k $ x $ \ textsc {跳} _K $，在某些$ \ textsc {jump} _ {k，\ delta} $实例上以$ n $的因子多项式慢。计算地，新类允许使用更宽的健身谷的实验，特别是当它们远离全球最佳时。

本文展示了如何适应$ k $ -MEANS问题的几种简单和经典的基于采样的算法，以使用离群值设置。最近，Bhaskara等人。 （Neurips 2019）展示了如何将古典$ K $ -MEANS ++算法适应与异常值的设置。但是，他们的算法需要输出$ o（\ log（k）\ cdot z）$ outiers，其中$ z $是true Outliers的数量，以匹配$ o（\ log k）$ - 近似值的$ k的近似保证$ -Means ++。在本文中，我们以他们的想法为基础，并展示了如何适应几个顺序和分布式的$ k $ - 均值算法，但使用离群值来设置，但具有更强的理论保证：我们的算法输出$（1+ \ VAREPSILON）z $ OUTLIERS Z $ OUTLIERS在实现$ o（1 / \ varepsilon）$ - 近似目标函数的同时。在顺序世界中，我们通过改编Lattanzi和Sohler的最新算法来实现这一目标（ICML 2019）。在分布式设置中，我们适应了Guha等人的简单算法。 （IEEE Trans。知道和数据工程2003）以及Bahmani等人的流行$ K $ -Means $ \ | $。 （PVLDB 2012）。我们技术的理论应用是一种具有运行时间$ \ tilde {o}（nk^2/z）$的算法，假设$ k \ ll z \ ll n $。这与Omacle模型中此问题的$ \ Omega（NK^2/z）$的匹配下限相互补。

在机器学习，游戏理论和控制理论中解决各种应用，极限优化已经是中心。因此，目前的文献主要集中于研究连续结构域中的这些问题，例如，凸凹minalax优化现在在很大程度上被理解。然而，最小的问题远远超出连续域以混合连续离散域或甚至完全离散域。在本文中，我们研究了混合连续离散的最小问题，其中最小化在属于欧几里德空间的连续变量上，最大化是在给定地面集的子集上。我们介绍了凸子蒙皮最小新的类问题，其中物镜相对于连续变量和子模块相对于离散变量凸出。尽管这些问题在机器学习应用中经常出现，但对于如何从算法和理论观点来解决它们的知之甚少。对于此类问题，我们首先表明获得鞍点难以达到任何近似，因此引入了（近）最优性的新概念。然后，我们提供了若干算法程序，用于解决凸且单调 - 子模块硬币问题，并根据我们最佳的概念来表征其收敛率，计算复杂性和最终解决方案的质量。我们所提出的算法迭代并组合离散和连续优化的工具。最后，我们提供了数字实验，以展示我们所用方法的有效性。

The NSGA-II is one of the most prominent algorithms to solve multi-objective optimization problems. Despite numerous successful applications, several studies have shown that the NSGA-II is less effective for larger numbers of objectives. In this work, we use mathematical runtime analyses to rigorously demonstrate and quantify this phenomenon. We show that even on the simple OneMinMax benchmark, where every solution is Pareto optimal, the NSGA-II also with large population sizes cannot compute the full Pareto front (objective vectors of all Pareto optima) in sub-exponential time when the number of objectives is at least three. Our proofs suggest that the reason for this unexpected behavior lies in the fact that in the computation of the crowding distance, the different objectives are regarded independently. This is not a problem for two objectives, where any sorting of a pair-wise incomparable set of solutions according to one objective is also such a sorting according to the other objective (in the inverse order).

我们考虑优化从高斯过程（GP）采样的矢量值的目标函数$ \ boldsymbol {f} $ sampled的问题，其索引集是良好的，紧凑的度量空间$（{\ cal x}，d）$设计。我们假设$ \ boldsymbol {f} $之前未知，并且在Design $ x $的$ \ \ boldsymbol {f} $ x $导致$ \ boldsymbol {f}（x）$。由于当$ {\ cal x} $很大的基数时，识别通过详尽搜索的帕累托最优设计是不可行的，因此我们提出了一种称为Adaptive $ \ Boldsymbol {\ epsilon} $ - PAL的算法，从而利用GP的平滑度-Ampled函数和$（{\ cal x}，d）$的结构快速学习。从本质上讲，Adaptive $ \ Boldsymbol {\ epsilon} $ - PAL采用基于树的自适应离散化技术，以识别$ \ Boldsymbol {\ epsilon} $ - 尽可能少的评估中的准确帕累托一组设计。我们在$ \ boldsymbol {\ epsilon} $ - 准确的Pareto Set识别上提供信息类型和度量尺寸类型界限。我们还在实验表明我们的算法在多个基准数据集上优于其他Pareto Set识别方法。