The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

Bilevel optimization plays an essential role in many machine learning tasks, ranging from hyperparameter optimization to meta-learning. Existing studies on bilevel optimization, however, focus on either centralized or synchronous distributed setting. The centralized bilevel optimization approaches require collecting massive amount of data to a single server, which inevitably incur significant communication expenses and may give rise to data privacy risks. Synchronous distributed bilevel optimization algorithms, on the other hand, often face the straggler problem and will immediately stop working if a few workers fail to respond. As a remedy, we propose Asynchronous Distributed Bilevel Optimization (ADBO) algorithm. The proposed ADBO can tackle bilevel optimization problems with both nonconvex upper-level and lower-level objective functions, and its convergence is theoretically guaranteed. Furthermore, it is revealed through theoretic analysis that the iteration complexity of ADBO to obtain the $\epsilon$-stationary point is upper bounded by $\mathcal{O}(\frac{1}{{{\epsilon ^2}}})$. Thorough empirical studies on public datasets have been conducted to elucidate the effectiveness and efficiency of the proposed ADBO.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

The Position Embedding (PE) is critical for Vision Transformers (VTs) due to the permutation-invariance of self-attention operation. By analyzing the input and output of each encoder layer in VTs using reparameterization and visualization, we find that the default PE joining method (simply adding the PE and patch embedding together) operates the same affine transformation to token embedding and PE, which limits the expressiveness of PE and hence constrains the performance of VTs. To overcome this limitation, we propose a simple, effective, and robust method. Specifically, we provide two independent layer normalizations for token embeddings and PE for each layer, and add them together as the input of each layer's Muti-Head Self-Attention module. Since the method allows the model to adaptively adjust the information of PE for different layers, we name it as Layer-adaptive Position Embedding, abbreviated as LaPE. Extensive experiments demonstrate that LaPE can improve various VTs with different types of PE and make VTs robust to PE types. For example, LaPE improves 0.94% accuracy for ViT-Lite on Cifar10, 0.98% for CCT on Cifar100, and 1.72% for DeiT on ImageNet-1K, which is remarkable considering the negligible extra parameters, memory and computational cost brought by LaPE. The code is publicly available at https://github.com/Ingrid725/LaPE.

Most existing distillation methods ignore the flexible role of the temperature in the loss function and fix it as a hyper-parameter that can be decided by an inefficient grid search. In general, the temperature controls the discrepancy between two distributions and can faithfully determine the difficulty level of the distillation task. Keeping a constant temperature, i.e., a fixed level of task difficulty, is usually sub-optimal for a growing student during its progressive learning stages. In this paper, we propose a simple curriculum-based technique, termed Curriculum Temperature for Knowledge Distillation (CTKD), which controls the task difficulty level during the student's learning career through a dynamic and learnable temperature. Specifically, following an easy-to-hard curriculum, we gradually increase the distillation loss w.r.t. the temperature, leading to increased distillation difficulty in an adversarial manner. As an easy-to-use plug-in technique, CTKD can be seamlessly integrated into existing knowledge distillation frameworks and brings general improvements at a negligible additional computation cost. Extensive experiments on CIFAR-100, ImageNet-2012, and MS-COCO demonstrate the effectiveness of our method. Our code is available at https://github.com/zhengli97/CTKD.

Deep learning methods have contributed substantially to the rapid advancement of medical image segmentation, the quality of which relies on the suitable design of loss functions. Popular loss functions, including the cross-entropy and dice losses, often fall short of boundary detection, thereby limiting high-resolution downstream applications such as automated diagnoses and procedures. We developed a novel loss function that is tailored to reflect the boundary information to enhance the boundary detection. As the contrast between segmentation and background regions along the classification boundary naturally induces heterogeneity over the pixels, we propose the piece-wise two-sample t-test augmented (PTA) loss that is infused with the statistical test for such heterogeneity. We demonstrate the improved boundary detection power of the PTA loss compared to benchmark losses without a t-test component.

基于深度学习的方法，例如物理知识的神经网络（PINN）和DeepOnets已显示出解决PDE受约束优化（PDECO）问题的希望。但是，现有方法不足以处理对优化目标具有复杂或非线性依赖性的PDE约束。在本文中，我们提出了一个新颖的双层优化框架，以通过将目标和约束的优化解耦来解决挑战。对于内部循环优化，我们采用PINN仅解决PDE约束。对于外循环，我们通过基于隐式函数定理（IFT）使用Broyden的方法来设计一种新颖的方法，该方法对于近似高度级别而言是有效且准确的。我们进一步介绍了高度级计算的理论解释和误差分析。在多个大规模和非线性PDE约束优化问题上进行了广泛的实验表明，与强基础相比，我们的方法可实现最新的结果。

在本文中，我们提出了一个新型的非线性观察者，称为神经观察者，以通过将神经网络（NN）引入观察者的设计，以实现线性时间传播（LTI）系统的观察任务和不确定的非线性系统。通过探索NN代表向NN映射矢量的方法，我们从LTI和不确定的非线性系统中得出了稳定性分析（例如，指数收敛速率），这些系统仅使用线性矩阵不平等（LMIS）为解决观察问题铺平了道路。值得注意的是，为不确定系统设计的神经观察者基于主动扰动拒绝控制（ADRC）的意识形态，该思想可以实时测量不确定性。 LMI结果也很重要，因为我们揭示了LMI溶液存在系统矩阵的可观察性和可控性。最后，我们在三个模拟案例上验证神经观察者的可用性，包括X-29A飞机模型，非线性摆和四轮转向车辆。

许多基于点的3D检测器采用点功能采样策略来提出一些分数以提高推断。这些策略通常基于固定和手工制作的规则，因此难以处理复杂的场景。与它们不同的是，我们提出了一个动态球查询（DBQ）网络，以根据输入特征自适应地选择输入点的子集，并为每个选定的点分配特征转换，并具有合适的接受场。它可以嵌入到一些最新的3D检测器中，并以端到端的方式进行训练，从而大大降低计算成本。广泛的实验表明，我们的方法可以在Kitti和Waymo数据集中将延迟降低30％-60％。具体而言，我们的检测器的推理速度分别可以在Kitti和Waymo数据集上具有可忽略的性能降解，可以达到162 fps和30 fps。

对于黑盒攻击，替代模型和受害者模型之间的差距通常很大，这表现为弱攻击性能。通过观察到，可以通过同时攻击多样的模型来提高对抗性示例的可传递性，并提出模型增强方法，这些模型通过使用转换图像模拟不同的模型。但是，空间域的现有转换不会转化为显着多样化的增强模型。为了解决这个问题，我们提出了一种新型的频谱模拟攻击，以针对正常训练和防御模型制作更容易转移的对抗性例子。具体而言，我们将频谱转换应用于输入，从而在频域中执行模型增强。从理论上讲，我们证明了从频域中得出的转换导致不同的频谱显着图，这是我们提出的指标，以反映替代模型的多样性。值得注意的是，我们的方法通常可以与现有攻击结合使用。 Imagenet数据集的广泛实验证明了我们方法的有效性，\ textit {e.g。}，攻击了九个最先进的防御模型，其平均成功率为\ textbf {95.4 \％}。我们的代码可在\ url {https://github.com/yuyang-long/ssa}中获得。