It has been observed in practice that applying pruning-at-initialization methods to neural networks and training the sparsified networks can not only retain the testing performance of the original dense models, but also sometimes even slightly boost the generalization performance. Theoretical understanding for such experimental observations are yet to be developed. This work makes the first attempt to study how different pruning fractions affect the model's gradient descent dynamics and generalization. Specifically, this work considers a classification task for overparameterized two-layer neural networks, where the network is randomly pruned according to different rates at the initialization. It is shown that as long as the pruning fraction is below a certain threshold, gradient descent can drive the training loss toward zero and the network exhibits good generalization performance. More surprisingly, the generalization bound gets better as the pruning fraction gets larger. To complement this positive result, this work further shows a negative result: there exists a large pruning fraction such that while gradient descent is still able to drive the training loss toward zero (by memorizing noise), the generalization performance is no better than random guessing. This further suggests that pruning can change the feature learning process, which leads to the performance drop of the pruned neural network. Up to our knowledge, this is the \textbf{first} generalization result for pruned neural networks, suggesting that pruning can improve the neural network's generalization.

Graph Neural Networks (GNNs) have been a prevailing technique for tackling various analysis tasks on graph data. A key premise for the remarkable performance of GNNs relies on complete and trustworthy initial graph descriptions (i.e., node features and graph structure), which is often not satisfied since real-world graphs are often incomplete due to various unavoidable factors. In particular, GNNs face greater challenges when both node features and graph structure are incomplete at the same time. The existing methods either focus on feature completion or structure completion. They usually rely on the matching relationship between features and structure, or employ joint learning of node representation and feature (or structure) completion in the hope of achieving mutual benefit. However, recent studies confirm that the mutual interference between features and structure leads to the degradation of GNN performance. When both features and structure are incomplete, the mismatch between features and structure caused by the missing randomness exacerbates the interference between the two, which may trigger incorrect completions that negatively affect node representation. To this end, in this paper we propose a general GNN framework based on teacher-student distillation to improve the performance of GNNs on incomplete graphs, namely T2-GNN. To avoid the interference between features and structure, we separately design feature-level and structure-level teacher models to provide targeted guidance for student model (base GNNs, such as GCN) through distillation. Then we design two personalized methods to obtain well-trained feature and structure teachers. To ensure that the knowledge of the teacher model is comprehensively and effectively distilled to the student model, we further propose a dual distillation mode to enable the student to acquire as much expert knowledge as possible.

The essential task of urban planning is to generate the optimal land-use configuration of a target area. However, traditional urban planning is time-consuming and labor-intensive. Deep generative learning gives us hope that we can automate this planning process and come up with the ideal urban plans. While remarkable achievements have been obtained, they have exhibited limitations in lacking awareness of: 1) the hierarchical dependencies between functional zones and spatial grids; 2) the peer dependencies among functional zones; and 3) human regulations to ensure the usability of generated configurations. To address these limitations, we develop a novel human-instructed deep hierarchical generative model. We rethink the urban planning generative task from a unique functionality perspective, where we summarize planning requirements into different functionality projections for better urban plan generation. To this end, we develop a three-stage generation process from a target area to zones to grids. The first stage is to label the grids of a target area with latent functionalities to discover functional zones. The second stage is to perceive the planning requirements to form urban functionality projections. We propose a novel module: functionalizer to project the embedding of human instructions and geospatial contexts to the zone-level plan to obtain such projections. Each projection includes the information of land-use portfolios and the structural dependencies across spatial grids in terms of a specific urban function. The third stage is to leverage multi-attentions to model the zone-zone peer dependencies of the functionality projections to generate grid-level land-use configurations. Finally, we present extensive experiments to demonstrate the effectiveness of our framework.

产品图像对于在电子商务平台中提供理想的用户体验至关重要。对于拥有数十亿种产品的平台，手动挑选和组织合格的图像非常耗时且耗尽劳动力。此外，要生成/选择的产品图像需要遵守众多且复杂的图像规则。为了解决这些挑战，在本文中，我们提出了一个新的学习框架，以便在电子商务中自动生成产品图像序列（AGPI）。为此，我们提出了一个多模式统一的图像序列分类器（MUISC），该分类器能够通过学习同时检测所有规则违规的类别。 MUISC利用文本审查反馈作为额外的培训目标，并利用产品文本描述提供额外的语义信息。根据离线评估，我们表明拟议的MUISC显着优于各种基线。除MUISC外，我们还将其他一些重要的模块集成在提出的框架中，例如主图像选择，不合格的内容检测和图像重复数据删除。借助所有这些模块，我们的框架在JD.com推荐平台中有效，有效地工作。到2021年12月，我们的AGPIS框架为约150万种产品生成了高标准图像，并获得了13.6％的拒绝率。

直到最近，研究人员才试图提供可证明的群体公平保证的分类算法。这些算法中的大多数都受到训练和部署数据遵循相同分布的要求造成的骚扰。本文提出了一种输入 - 不合时宜的团体公平算法，即Fairsmooth，用于改善分类模型的公平性，同时保持显着的预测准确性。开发了一种高斯参数平滑方法，以将基本分类器转换为平滑版本。通过仅使用有关该组的数据来学习一个最佳的单个平滑分类器，并且通过平均所有单个平滑的参数来生成所有组的总体平滑分类器。通过利用非线性功能分析的理论，将平滑的分类器重新构成NemyTSKII操作员的输出函数。进行理论分析是为了得出Nemytskii操作员的平滑状态并诱导特征差异的平滑歧管。从理论上讲，我们证明了平滑歧管具有一个全局LIPSCHITZ常数，该常数独立于输入数据的域，该域衍生了输入 - 不合时式认证的组公平性。

在过去的十年中，电子商务的自动产品描述生成已经取得了重大进步。产品文案旨在通过通过文本描述突出产品特征来吸引用户的兴趣并改善用户体验。随着电子商务平台提供的服务变得多样化，有必要动态地调整自动生成描述的模式。在本文中，我们将基于电子商务前缀的可控文案生成（EPCCG）系统部署到JD.com电子商务产品推荐平台中的经验。系统的开发包含两个主要组成部分：1）文案写作方面提取； 2）弱监督的方面标签； 3）具有基于前缀的语言模型的文本生成； 4）文案写作质量控制。我们进行实验以验证拟议的EPCCG的有效性。此外，我们将与EPCCG合作的已部署架构介绍到实时JD.com电子商务推荐平台以及部署以来的巨大回报。

图形神经网络（GNN）在解决图形结构数据（即网络）方面的各种分析任务方面已广受欢迎。典型的gnns及其变体遵循一种消息的方式，该方式通过网络拓扑沿网络拓扑的特征传播过程获得网络表示，然而，它们忽略了许多现实世界网络中存在的丰富文本语义（例如，局部单词序列）。现有的文本丰富网络方法通过主要利用内部信息（例如主题或短语/单词）来整合文本语义，这些信息通常无法全面地挖掘文本语义，从而限制了网络结构和文本语义之间的相互指导。为了解决这些问题，我们提出了一个具有外部知识（TEKO）的新型文本富裕的图形神经网络，以充分利用文本丰富的网络中的结构和文本信息。具体而言，我们首先提出一个灵活的异质语义网络，该网络结合了文档和实体之间的高质量实体和互动。然后，我们介绍两种类型的外部知识，即结构化的三胞胎和非结构化实体描述，以更深入地了解文本语义。我们进一步为构建的异质语义网络设计了互惠卷积机制，使网络结构和文本语义能够相互协作并学习高级网络表示。在四个公共文本丰富的网络以及一个大规模的电子商务搜索数据集上进行了广泛的实验结果，这说明了Teko优于最先进的基线。

在过去的几十年中，知识感知的方法增强了一系列自然语言处理应用。随着收集的动力，最近在文档摘要中引起了知识，这是自然语言处理应用之一。先前的作品报告说，知识包裹的文档摘要在产生卓越的消化方面表现出色，尤其是在信息性，连贯性和事实一致性方面。本文追求对将知识嵌入文档摘要的最先进方法论进行的首次系统调查。特别是，我们提出了新的分类法，以概括文档摘要观点下的知识和知识嵌入。我们进一步探讨了如何在嵌入文档摘要模型的学习体系结构时，尤其是深度学习模型的学习架构。最后，我们讨论了这个主题和未来方向的挑战。

恶意软件是跨越多个操作系统和各种文件格式的计算机的最损害威胁之一。为了防止不断增长的恶意软件的威胁，已经提出了巨大的努力来提出各种恶意软件检测方法，试图有效和有效地检测恶意软件。最近的研究表明，一方面，现有的ML和DL能够卓越地检测新出现和以前看不见的恶意软件。然而，另一方面，ML和DL模型本质上易于侵犯对抗性示例形式的对抗性攻击，这通过略微仔细地扰乱了合法输入来混淆目标模型来恶意地产生。基本上，在计算机视觉领域最初广泛地研究了对抗性攻击，并且一些快速扩展到其他域，包括NLP，语音识别甚至恶意软件检测。在本文中，我们专注于Windows操作系统系列中的便携式可执行文件（PE）文件格式的恶意软件，即Windows PE恶意软件，作为在这种对抗设置中研究对抗性攻击方法的代表性案例。具体而言，我们首先首先概述基于ML / DL的Windows PE恶意软件检测的一般学习框架，随后突出了在PE恶意软件的上下文中执行对抗性攻击的三个独特挑战。然后，我们进行全面和系统的审查，以对PE恶意软件检测以及增加PE恶意软件检测的稳健性的相应防御，对近最新的对手攻击进行分类。我们首先向Windows PE恶意软件检测的其他相关攻击结束除了对抗对抗攻击之外，然后对未来的研究方向和机遇脱落。

考虑到RDF三元组的集合，RDF到文本生成任务旨在生成文本描述。最先前的方法使用序列到序列模型或使用基于图形的模型来求解此任务以编码RDF三维并生成文本序列。然而，这些方法未能明确模拟RDF三元组之间的本地和全球结构信息。此外，以前的方法也面临了生成文本的低信任问题的不可忽略的问题，这严重影响了这些模型的整体性能。为了解决这些问题，我们提出了一种组合两个新的图形增强结构神经编码器的模型，共同学习输入的RDF三元组中的本地和全局结构信息。为了进一步改进文本忠诚，我们创新地根据信息提取（即）引进了强化学习（RL）奖励。我们首先使用佩带的IE模型从所生成的文本中提取三元组，并将提取的三级的正确数量视为额外的RL奖励。两个基准数据集上的实验结果表明，我们所提出的模型优于最先进的基线，额外的加强学习奖励确实有助于改善所生成的文本的忠诚度。