Off-policy evaluation (OPE) is a method for estimating the return of a target policy using some pre-collected observational data generated by a potentially different behavior policy. In some cases, there may be unmeasured variables that can confound the action-reward or action-next-state relationships, rendering many existing OPE approaches ineffective. This paper develops an instrumental variable (IV)-based method for consistent OPE in confounded Markov decision processes (MDPs). Similar to single-stage decision making, we show that IV enables us to correctly identify the target policy's value in infinite horizon settings as well. Furthermore, we propose an efficient and robust value estimator and illustrate its effectiveness through extensive simulations and analysis of real data from a world-leading short-video platform.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Transformer-based models have gained large popularity and demonstrated promising results in long-term time-series forecasting in recent years. In addition to learning attention in time domain, recent works also explore learning attention in frequency domains (e.g., Fourier domain, wavelet domain), given that seasonal patterns can be better captured in these domains. In this work, we seek to understand the relationships between attention models in different time and frequency domains. Theoretically, we show that attention models in different domains are equivalent under linear conditions (i.e., linear kernel to attention scores). Empirically, we analyze how attention models of different domains show different behaviors through various synthetic experiments with seasonality, trend and noise, with emphasis on the role of softmax operation therein. Both these theoretical and empirical analyses motivate us to propose a new method: TDformer (Trend Decomposition Transformer), that first applies seasonal-trend decomposition, and then additively combines an MLP which predicts the trend component with Fourier attention which predicts the seasonal component to obtain the final prediction. Extensive experiments on benchmark time-series forecasting datasets demonstrate that TDformer achieves state-of-the-art performance against existing attention-based models.

We propose GazeNeRF, a 3D-aware method for the task of gaze redirection. Existing gaze redirection methods operate on 2D images and struggle to generate 3D consistent results. Instead, we build on the intuition that the face region and eyeballs are separate 3D structures that move in a coordinated yet independent fashion. Our method leverages recent advancements in conditional image-based neural radiance fields and proposes a two-stream architecture that predicts volumetric features for the face and eye regions separately. Rigidly transforming the eye features via a 3D rotation matrix provides fine-grained control over the desired gaze angle. The final, redirected image is then attained via differentiable volume compositing. Our experiments show that this architecture outperforms naively conditioned NeRF baselines as well as previous state-of-the-art 2D gaze redirection methods in terms of redirection accuracy and identity preservation.

The explosive growth of dynamic and heterogeneous data traffic brings great challenges for 5G and beyond mobile networks. To enhance the network capacity and reliability, we propose a learning-based dynamic time-frequency division duplexing (D-TFDD) scheme that adaptively allocates the uplink and downlink time-frequency resources of base stations (BSs) to meet the asymmetric and heterogeneous traffic demands while alleviating the inter-cell interference. We formulate the problem as a decentralized partially observable Markov decision process (Dec-POMDP) that maximizes the long-term expected sum rate under the users' packet dropping ratio constraints. In order to jointly optimize the global resources in a decentralized manner, we propose a federated reinforcement learning (RL) algorithm named federated Wolpertinger deep deterministic policy gradient (FWDDPG) algorithm. The BSs decide their local time-frequency configurations through RL algorithms and achieve global training via exchanging local RL models with their neighbors under a decentralized federated learning framework. Specifically, to deal with the large-scale discrete action space of each BS, we adopt a DDPG-based algorithm to generate actions in a continuous space, and then utilize Wolpertinger policy to reduce the mapping errors from continuous action space back to discrete action space. Simulation results demonstrate the superiority of our proposed algorithm to benchmark algorithms with respect to system sum rate.

节点分类是图神经网络中的重要任务，但是大多数现有研究都认为来自不同类别的样本是平衡的。但是，类不平衡问题是普遍的，可能会严重影响模型的性能。减少数据集对模型培训的不利影响对于改善模型的性能至关重要。因此，基于传统算法级别的方法来重建新的损失函数FD损失。首先，我们提出样品不种种量的距离，以根据分布过滤边缘样品和简单样品。然后，根据不抗测量距离定义了权重系数，并在损耗函数加权项中使用，以便损耗函数仅集中在有价值的样本上。与节点分类任务中的现有方法相比，几个基准的实验表明，我们的损耗函数可以有效地解决样品节点不平衡问题并将分类精度提高4％。

从医用试剂染色图像中分割牙齿斑块为诊断和确定随访治疗计划提供了宝贵的信息。但是，准确的牙菌斑分割是一项具有挑战性的任务，需要识别牙齿和牙齿斑块受到语义腔区域的影响（即，在牙齿和牙齿斑块之间的边界区域中存在困惑的边界）以及实例形状的复杂变化，这些变化均未完全解决。现有方法。因此，我们提出了一个语义分解网络（SDNET），该网络介绍了两个单任务分支，以分别解决牙齿和牙齿斑块的分割，并设计了其他约束，以学习每个分支的特定类别特征，从而促进语义分解并改善该类别的特征牙齿分割的性能。具体而言，SDNET以分裂方式学习了两个单独的分割分支和牙齿的牙齿，以解除它们之间的纠缠关系。指定类别的每个分支都倾向于产生准确的分割。为了帮助这两个分支更好地关注特定类别的特征，进一步提出了两个约束模块：1）通过最大化不同类别表示之间的距离来学习判别特征表示，以了解判别特征表示形式，以减少减少负面影响关于特征提取的语义腔区域； 2）结构约束模块（SCM）通过监督边界感知的几何约束提供完整的结构信息，以提供各种形状的牙菌斑。此外，我们构建了一个大规模的开源染色牙菌斑分割数据集（SDPSEG），该数据集为牙齿和牙齿提供高质量的注释。 SDPSEG数据集的实验结果显示SDNET达到了最新的性能。

由于复杂且巨大的模型结构，大多数现有的显着对象检测（SOD）模型很难应用。尽管提出了一些轻巧的模型，但准确性几乎不令人满意。在本文中，我们设计了一种新颖的语义引导的上下文融合网络（SCFNET），该网络重点介绍了多层次特征的交互式融合，以进行准确有效的显着对象检测。此外，我们将知识蒸馏应用于SOD任务，并提供相当大的数据集KD-SOD80K。详细说明，我们通过未标记的图像将丰富的知识从经验丰富的老师转移到未经训练的SCFNET，使SCFNET能够学习强大的概括能力，以更准确地检测显着对象。基于知识蒸馏的SCFNET（KDSCFNET）具有与最先进的重量级方法相当的精度，该方法少于1M参数和174 fps实时检测速度。广泛的实验证明了所提出的蒸馏方法和SOD框架的鲁棒性和有效性。代码和数据：https：//github.com/zhangjincv/kd-scfnet。

第三代合作伙伴项目已开始研究2021年的第18版。人工智能（AI）空气界面是第18版的关键特征之一，其中选择了用于渠道状态信息的AI（CSI）反馈增强作为代表性。用例。本文提供了5G助长和6G中CSI反馈增强的AI的全面概述。首先介绍和讨论了5G效率的CSI反馈增强AI的范围，包括高架降低，准确性提高和渠道预测。然后，介绍并比较了三个代表性CSI反馈的代表性框架，包括单方面隐式反馈，基于双面自动编码器的隐式反馈和双面显式反馈。最后，已经确定和讨论了CSI反馈增强的AI标准化考虑因素，尤其是重点是评估，复杂性，协作，概括，信息共享，具有渠道预测的联合设计和互惠性。本文为基于AI的CSI反馈增强的标准化研究提供了指南。

通过大量多输入和多重输出实现的许多性能增长取决于发射机（基站）下链路通道状态信息（CSI）的准确性，这通常是通过在接收器（用户终端）估算并馈入的。到发射器。 CSI反馈的开销占据了大量的上行链路带宽资源，尤其是当传输天线数量较大时。基于深度学习（DL）的CSI反馈是指基于DL的自动编码器的CSI压缩和重建，并且可以大大减少反馈开销。在本文中，提供了有关该主题的最新研究的全面概述，首先是在CSI反馈中广泛使用的基本DL概念，然后对一些现有的基于DL的反馈作品进行分类和描述。重点是新型的神经网络体系结构和沟通专家知识的利用来提高CSI反馈准确性。还介绍了有关CSI反馈和CSI反馈与其他通信模块的联合设计的作品，并讨论了一些实际问题，包括培训数据集收集，在线培训，复杂性，概括和标准化效果。在本文的最后，确定了与未来无线通信系统中基于DL的CSI反馈相关的一些挑战和潜在的研究方向。