We propose a hypothesis only baseline for diagnosing Natural LanguageInference (NLI). Especially when an NLI dataset assumes inference is occurringbased purely on the relationship between a context and a hypothesis, it followsthat assessing entailment relations while ignoring the provided context is adegenerate solution. Yet, through experiments on ten distinct NLI datasets, wefind that this approach, which we refer to as a hypothesis-only model, is ableto significantly outperform a majority class baseline across a number of NLIdatasets. Our analysis suggests that statistical irregularities may allow amodel to perform NLI in some datasets beyond what should be achievable withoutaccess to the context.
translated by 谷歌翻译