The neuron reconstruction from raw Optical Microscopy (OM) image stacks is the basis of neuroscience. Manual annotation and semi-automatic neuron tracing algorithms are time-consuming and inefficient. Existing deep learning neuron reconstruction methods, although demonstrating exemplary performance, greatly demand complex rule-based components. Therefore, a crucial challenge is designing an end-to-end neuron reconstruction method that makes the overall framework simpler and model training easier. We propose a Neuron Reconstruction Transformer (NRTR) that, discarding the complex rule-based components, views neuron reconstruction as a direct set-prediction problem. To the best of our knowledge, NRTR is the first image-to-set deep learning model for end-to-end neuron reconstruction. In experiments using the BigNeuron and VISoR-40 datasets, NRTR achieves excellent neuron reconstruction results for comprehensive benchmarks and outperforms competitive baselines. Results of extensive experiments indicate that NRTR is effective at showing that neuron reconstruction is viewed as a set-prediction problem, which makes end-to-end model training available.
translated by 谷歌翻译
We introduce a new method for diverse foreground generation with explicit control over various factors. Existing image inpainting based foreground generation methods often struggle to generate diverse results and rarely allow users to explicitly control specific factors of variation (e.g., varying the facial identity or expression for face inpainting results). We leverage contrastive learning with latent codes to generate diverse foreground results for the same masked input. Specifically, we define two sets of latent codes, where one controls a pre-defined factor (``known''), and the other controls the remaining factors (``unknown''). The sampled latent codes from the two sets jointly bi-modulate the convolution kernels to guide the generator to synthesize diverse results. Experiments demonstrate the superiority of our method over state-of-the-arts in result diversity and generation controllability.
translated by 谷歌翻译
由于生成对抗网络(GAN)的突破,3D可控制的肖像合成已大大提高。但是,用精确的3D控制操纵现有的面部图像仍然具有挑战性。虽然连接gan倒置和3D感知,但噪声到图像是一种直接的解决方案,但它效率低下,可能导致编辑质量明显下降。为了填补这一空白,我们提出了3D-FM GAN,这是一个专门为3D可控制的面部操作设计的新型有条件GAN框架,并且在端到端学习阶段后不需要任何调整。通过小心地编码输入面图像和3D编辑的基于物理的渲染,我们的图像生成器提供了高质量,具有身份的3D控制面部操纵。为了有效地学习这种新颖的框架,我们制定了两种基本的训练策略和一种新颖的乘法共同调制体系结构,可在天真的方案上显着改善。通过广泛的评估,我们表明我们的方法在各种任务上的表现优于先前的艺术,具有更好的编辑性,更强的身份保存和更高的照片真实性。此外,我们在大型姿势编辑和室外图像上展示了设计更好的概括性。
translated by 谷歌翻译
域自适应文本分类对于大规模预处理的语言模型来说是一个具有挑战性的问题,因为它们通常需要昂贵的额外标记数据来适应新域。现有作品通常无法利用跨域单词之间的隐式关系。在本文中,我们提出了一种新的方法,称为结构化知识(DASK)的域适应性,以通过利用单词级别的语义关系来增强域的适应性。 Dask首先构建知识图,以捕获目标域中的枢轴项(独立域单词)和非居式项之间的关系。然后在训练期间,DASK注入与源域文本的枢轴相关知识图信息。对于下游任务,这些注入知识的文本被馈入能够处理知识注入文本数据的BERT变体。多亏了知识注入,我们的模型根据与枢轴的关系学习了非客者的域不变特征。 DASK通过在使用伪标签训练期间通过候选枢轴的极性得分动态推断出具有域不变行为的枢轴。我们在各种跨域情绪分类任务上验证了DASK,并观察到20种不同领域对的基准的绝对性能提高了2.9%。代码将在https://github.com/hikaru-nara/dask上提供。
translated by 谷歌翻译
现有的GAN倒置和编辑方法适用于具有干净背景的对齐物体,例如肖像和动物面孔,但通常会为更加困难的类别而苦苦挣扎,具有复杂的场景布局和物体遮挡,例如汽车,动物和室外图像。我们提出了一种新方法,以在gan的潜在空间(例如stylegan2)中倒转和编辑复杂的图像。我们的关键想法是用一系列层的集合探索反演,从而将反转过程适应图像的难度。我们学会预测不同图像段的“可逆性”,并将每个段投影到潜在层。更容易的区域可以倒入发电机潜在空间中的较早层,而更具挑战性的区域可以倒入更晚的特征空间。实验表明,与最新的复杂类别的方法相比,我们的方法获得了更好的反转结果,同时保持下游的编辑性。请参阅我们的项目页面,网址为https://www.cs.cmu.edu/~saminversion。
translated by 谷歌翻译
在本文中,通过以自我监督的方式将基于几何的方法纳入深度学习架构来实现强大的视觉测量(VO)的基本问题。通常,基于纯几何的算法与特征点提取和匹配中的深度学习不那么稳健,但由于其成熟的几何理论,在自我运动估计中表现良好。在这项工作中,首先提出了一种新颖的光学流量网络(PANET)内置于位置感知机构。然后,提出了一种在没有典型网络的情况下共同估计深度,光学流动和自我运动来学习自我运动的新系统。所提出的系统的关键组件是一种改进的束调节模块,其包含多个采样,初始化的自我运动,动态阻尼因子调整和Jacobi矩阵加权。另外,新颖的相对光度损耗函数先进以提高深度估计精度。该实验表明,所提出的系统在基于基于基于基于基于基于基于基于学习的基于学习的方法之间的深度,流量和VO估计方面不仅优于其他最先进的方法,而且与几何形状相比,也显着提高了鲁棒性 - 基于,基于学习和混合VO系统。进一步的实验表明,我们的模型在挑战室内(TMU-RGBD)和室外(KAIST)场景中实现了出色的泛化能力和性能。
translated by 谷歌翻译
In this paper, we present a pure-Python open-source library, called PyPop7, for black-box optimization (BBO). It provides a unified and modular interface for more than 60 versions and variants of different black-box optimization algorithms, particularly population-based optimizers, which can be classified into 12 popular families: Evolution Strategies (ES), Natural Evolution Strategies (NES), Estimation of Distribution Algorithms (EDA), Cross-Entropy Method (CEM), Differential Evolution (DE), Particle Swarm Optimizer (PSO), Cooperative Coevolution (CC), Simulated Annealing (SA), Genetic Algorithms (GA), Evolutionary Programming (EP), Pattern Search (PS), and Random Search (RS). It also provides many examples, interesting tutorials, and full-fledged API documentations. Through this new library, we expect to provide a well-designed platform for benchmarking of optimizers and promote their real-world applications, especially for large-scale BBO. Its source code and documentations are available at https://github.com/Evolutionary-Intelligence/pypop and https://pypop.readthedocs.io/en/latest, respectively.
translated by 谷歌翻译
Link prediction is a crucial problem in graph-structured data. Due to the recent success of graph neural networks (GNNs), a variety of GNN-based models were proposed to tackle the link prediction task. Specifically, GNNs leverage the message passing paradigm to obtain node representation, which relies on link connectivity. However, in a link prediction task, links in the training set are always present while ones in the testing set are not yet formed, resulting in a discrepancy of the connectivity pattern and bias of the learned representation. It leads to a problem of dataset shift which degrades the model performance. In this paper, we first identify the dataset shift problem in the link prediction task and provide theoretical analyses on how existing link prediction methods are vulnerable to it. We then propose FakeEdge, a model-agnostic technique, to address the problem by mitigating the graph topological gap between training and testing sets. Extensive experiments demonstrate the applicability and superiority of FakeEdge on multiple datasets across various domains.
translated by 谷歌翻译
现有的远处监督的关系提取器通常依靠嘈杂的数据进行模型培训和评估,这可能导致垃圾堆放系统。为了减轻问题,我们研究了小型清洁数据集是否可以帮助提高远距离监督模型的质量。我们表明,除了对模型进行更具说服力的评估外,一个小的清洁数据集还可以帮助我们构建更强大的Denoising模型。具体而言,我们提出了一个基于影响函数的清洁实例选择的新标准。它收集了样本级别的证据,以识别良好实例(这比损失级别的证据更具信息性)。我们还提出了一种教师实习机制,以控制自举套件时中间结果的纯度。整个方法是模型不合时宜的,并且在denoising Real(NYT)和合成噪声数据集上都表现出强烈的性能。
translated by 谷歌翻译
深度神经网络(DNNS)在各个领域都取得了出色的性能。但是,DNNS对对抗性示例(AE)的脆弱性阻碍了他们的部署到关键的安全应用程序。本文提出了一个新颖的AE检测框架,以值得信赖的预测为止。除了通过区分AE的异常关系与其增强版本(即邻居)与两个前景:表示相似性和标签一致性来区分检测。与监督的学习模型相比,使用现成的自我监督学习(SSL)模型用于提取表示形式,并预测其高度信息代表能力的标签。对于干净的样本,它们的表示和预测与邻居密切一致,而AE的邻居差异很大。此外,我们解释了这一观察结果,并表明,通过利用这种差异可以有效地检测到AE。我们为超越的有效性建立了严格的理由。此外,作为一种插件模型,超越的范围可以轻松与受过对抗训练的分类器(ATC)合作,从而实现最先进的(SOTA)鲁棒性精度。实验结果表明,超越表现的基线较大,尤其是在自适应攻击下。在SSL上建立的强大关系网络的授权下,我们发现超出了检测能力和速度方面优于基准。我们的代码将公开可用。
translated by 谷歌翻译