Reading comprehension (RC)---in contrast to information retrieval---requiresintegrating information and reasoning about events, entities, and theirrelations across a full document. Question answering is conventionally used toassess RC ability, in both artificial agents and children learning to read.However, existing RC datasets and tasks are dominated by questions that can besolved by selecting answers using superficial information (e.g., local contextsimilarity or global term frequency); they thus fail to test for the essentialintegrative aspect of RC. To encourage progress on deeper comprehension oflanguage, we present a new dataset and set of tasks in which the reader mustanswer questions about stories by reading entire books or movie scripts. Thesetasks are designed so that successfully answering their questions requiresunderstanding the underlying narrative rather than relying on shallow patternmatching or salience. We show that although humans solve the tasks easily,standard RC models struggle on the tasks presented here. We provide an analysisof the dataset and the challenges it presents.
translated by 谷歌翻译