A normalizing flow (NF) is a mapping that transforms a chosen probability distribution to a normal distribution. Such flows are a common technique used for data generation and density estimation in machine learning and data science. The density estimate obtained with a NF requires a change of variables formula that involves the computation of the Jacobian determinant of the NF transformation. In order to tractably compute this determinant, continuous normalizing flows (CNF) estimate the mapping and its Jacobian determinant using a neural ODE. Optimal transport (OT) theory has been successfully used to assist in finding CNFs by formulating them as OT problems with a soft penalty for enforcing the standard normal distribution as a target measure. A drawback of OT-based CNFs is the addition of a hyperparameter, $\alpha$, that controls the strength of the soft penalty and requires significant tuning. We present JKO-Flow, an algorithm to solve OT-based CNF without the need of tuning $\alpha$. This is achieved by integrating the OT CNF framework into a Wasserstein gradient flow framework, also known as the JKO scheme. Instead of tuning $\alpha$, we repeatedly solve the optimization problem for a fixed $\alpha$ effectively performing a JKO update with a time-step $\alpha$. Hence we obtain a "divide and conquer" algorithm by repeatedly solving simpler problems instead of solving a potentially harder problem with large $\alpha$.

变形金刚在序列建模及以后取得了显着的成功，但相对于输入序列的长度，二次计算和记忆复杂性遭受了损失。利用技术包括稀疏和线性的注意力和哈希技巧；已经提出了有效的变压器来降低变压器的二次复杂性，但会显着降低准确性。作为响应，我们首先将计算注意图的线性注意力和残差连接解释为梯度下降步骤。然后，我们将动量引入这些组件，并提出\ emph {动量变压器}，该动量利用动量来提高线性变压器的精度，同时保持线性内存和计算复杂性。此外，我们制定了一种自适应策略，以根据二次优化的最佳动量计算模型的动量值。这种自适应动量消除了寻找最佳动量值的需求，并进一步增强了动量变压器的性能。包括图像生成和机器翻译在内的自回归和非自动回归任务的一系列实验表明，动量变压器在训练效率和准确性方面优于流行的线性变压器。

我们引入了一种称为吉祥物（具有最佳传输的多代理形状控制）的方法，以计算具有形状/形成/密度约束的剂的最佳控制溶液。例如，我们可能希望在代理商上应用形状约束 - 也许我们希望代理人沿着路径保持特定的形状，或者我们希望代理商分散以最大程度地减少碰撞。我们可能还希望一定比例的代理移动到一个目的地，而其他代理人则移至另一个目的地，并以最佳方式进行此操作，即源点性作业应该是最佳的。为了实现这一目标，我们利用地球移动器从最佳运输的距离将代理分配到适当的位置，以便可以满足某些形状。该成本都以终端成本以及最佳控制问题的运行成本引入。

多头注意力是最先进的变压器背后的推动力，它在各种自然语言处理（NLP）和计算机视觉任务中实现了出色的性能。已经观察到，对于许多应用，这些注意力头会学习冗余嵌入，并且大多数可以在不降低模型性能的情况下去除。受到这一观察的启发，我们提出了变压器的混合物（变压器-MGK）的混合物，这是一种新型的变压器架构，用每个头部的钥匙混合了变压器中的冗余头部。这些键的混合物遵循高斯混合模型，并使每个注意力头有效地集中在输入序列的不同部分上。与传统的变压器对应物相比，变压器-MGK会加速训练和推理，具有较少的参数，并且需要更少的拖船来计算，同时实现跨任务的可比性或更高的准确性。 Transformer-MGK也可以轻松扩展到线性注意力。我们从经验上证明了在一系列实用应用中变形金属MGK的优势，包括语言建模和涉及非常长序列的任务。在Wikitext-103和远程竞技场基准中，具有4个头部的变压器MGK具有与基线变压器具有8个头的可比性或更好的性能。

深度学习的一个有前景的趋势取代了具有隐式网络的传统馈送网络。与传统网络不同，隐式网络解决了一个固定点方程来计算推断。解决固定点的复杂性变化，具体取决于提供的数据和误差容差。重要的是，可以通过与前馈网络的STARK对比度训练隐式网络，其内存需求与深度线性缩放。但是，没有免费的午餐 - 通过隐式网络锻造BackPropagation通常需要解决从隐式功能定理引起的昂贵的Jacobian等方程。我们提出了无雅各比的BackPropagation（JFB），一种固定内存方法，这些方法旨在解决基于雅略族裔的基于雅代族人的方程。 JFB使隐式网络更快地培训，并明显更容易实现，而不会牺牲测试精度。我们的实验表明，使用JFB培训的隐式网络与给出相同数量的参数的前馈网络和现有的隐式网络具有竞争力。

There has been much interest recently in developing fair clustering algorithms that seek to do justice to the representation of groups defined along sensitive attributes such as race and gender. We observe that clustering algorithms could generate clusters such that different groups are disadvantaged within different clusters. We develop a clustering algorithm, building upon the centroid clustering paradigm pioneered by classical algorithms such as $k$-means, where we focus on mitigating the unfairness experienced by the most-disadvantaged group within each cluster. Our method uses an iterative optimisation paradigm whereby an initial cluster assignment is modified by reassigning objects to clusters such that the worst-off sensitive group within each cluster is benefitted. We demonstrate the effectiveness of our method through extensive empirical evaluations over a novel evaluation metric on real-world datasets. Specifically, we show that our method is effective in enhancing cluster-level group representativity fairness significantly at low impact on cluster coherence.

Previous work has shown the potential of deep learning to predict renal obstruction using kidney ultrasound images. However, these image-based classifiers have been trained with the goal of single-visit inference in mind. We compare methods from video action recognition (i.e. convolutional pooling, LSTM, TSM) to adapt single-visit convolutional models to handle multiple visit inference. We demonstrate that incorporating images from a patient's past hospital visits provides only a small benefit for the prediction of obstructive hydronephrosis. Therefore, inclusion of prior ultrasounds is beneficial, but prediction based on the latest ultrasound is sufficient for patient risk stratification.

Recently, e-scooter-involved crashes have increased significantly but little information is available about the behaviors of on-road e-scooter riders. Most existing e-scooter crash research was based on retrospectively descriptive media reports, emergency room patient records, and crash reports. This paper presents a naturalistic driving study with a focus on e-scooter and vehicle encounters. The goal is to quantitatively measure the behaviors of e-scooter riders in different encounters to help facilitate crash scenario modeling, baseline behavior modeling, and the potential future development of in-vehicle mitigation algorithms. The data was collected using an instrumented vehicle and an e-scooter rider wearable system, respectively. A three-step data analysis process is developed. First, semi-automatic data labeling extracts e-scooter rider images and non-rider human images in similar environments to train an e-scooter-rider classifier. Then, a multi-step scene reconstruction pipeline generates vehicle and e-scooter trajectories in all encounters. The final step is to model e-scooter rider behaviors and e-scooter-vehicle encounter scenarios. A total of 500 vehicle to e-scooter interactions are analyzed. The variables pertaining to the same are also discussed in this paper.

As one of the most popular micro-mobility options, e-scooters are spreading in hundreds of big cities and college towns in the US and worldwide. In the meantime, e-scooters are also posing new challenges to traffic safety. In general, e-scooters are suggested to be ridden in bike lanes/sidewalks or share the road with cars at the maximum speed of about 15-20 mph, which is more flexible and much faster than the pedestrains and bicyclists. These features make e-scooters challenging for human drivers, pedestrians, vehicle active safety modules, and self-driving modules to see and interact. To study this new mobility option and address e-scooter riders' and other road users' safety concerns, this paper proposes a wearable data collection system for investigating the micro-level e-Scooter motion behavior in a Naturalistic road environment. An e-Scooter-based data acquisition system has been developed by integrating LiDAR, cameras, and GPS using the robot operating system (ROS). Software frameworks are developed to support hardware interfaces, sensor operation, sensor synchronization, and data saving. The integrated system can collect data continuously for hours, meeting all the requirements including calibration accuracy and capability of collecting the vehicle and e-Scooter encountering data.

In this paper, we propose SceNDD: a scenario-based naturalistic driving dataset that is built upon data collected from an instrumented vehicle in downtown Indianapolis. The data collection was completed in 68 driving sessions with different drivers, where each session lasted about 20--40 minutes. The main goal of creating this dataset is to provide the research community with real driving scenarios that have diverse trajectories and driving behaviors. The dataset contains ego-vehicle's waypoints, velocity, yaw angle, as well as non-ego actor's waypoints, velocity, yaw angle, entry-time, and exit-time. Certain flexibility is provided to users so that actors, sensors, lanes, roads, and obstacles can be added to the existing scenarios. We used a Joint Probabilistic Data Association (JPDA) tracker to detect non-ego vehicles on the road. We present some preliminary results of the proposed dataset and a few applications associated with it. The complete dataset is expected to be released by early 2023.