Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones. Taking the former text as positive and the latter as negative samples, the PLM can be trained effectively for contextualized representation. However, the training of such a type of PLMs highly relies on the quality of the automatically constructed samples. Existing PLMs simply treat all corrupted texts as equal negative without any examination, which actually lets the resulting model inevitably suffer from the false negative issue where training is carried out on pseudo-negative data and leads to less efficiency and less robustness in the resulting PLMs. In this work, on the basis of defining the false negative issue in discriminative PLMs that has been ignored for a long time, we design enhanced pre-training methods to counteract false negative predictions and encourage pre-training language models on true negatives by correcting the harmful gradient updates subject to false negative predictions. Experimental results on GLUE and SQuAD benchmarks show that our counter-false-negative pre-training methods indeed bring about better performance together with stronger robustness.

超新星（SNE）是宇宙中最亮的物体之一，是标志着恒星一生末尾的强大爆炸。超新星（SN）类型是由光谱发射线定义的，但是获得光谱法在逻辑上通常是不可行的。因此，仅使用时间序列图像数据鉴定SNE的能力至关重要，尤其是鉴于即将到来的望远镜的广度和深度的增加。我们提出了一种用于快速超新星时间序列分类的卷积神经网络方法，观察到的亮度数据在波长和时间方向上都通过高斯过程回归平滑。我们将此方法应用于完整的持续时间和截断的SN时间序列，以模拟回顾性和实时分类性能。回顾性分类用于区分宇宙学上有用的IA SNE与其他SN类型的类型，并且此方法在此任务上的准确性> 99％。我们还能够在只有两个晚上的数据和98％的准确度回顾性的情况下以60％精度区分6种SN类型。