The damage personal attacks cause to online discourse motivates manyplatforms to try to curb the phenomenon. However, understanding the prevalenceand impact of personal attacks in online platforms at scale remainssurprisingly difficult. The contribution of this paper is to develop andillustrate a method that combines crowdsourcing and machine learning to analyzepersonal attacks at scale. We show an evaluation method for a classifier interms of the aggregated number of crowd-workers it can approximate. We applyour methodology to English Wikipedia, generating a corpus of over 100k highquality human-labeled comments and 63M machine-labeled ones from a classifierthat is as good as the aggregate of 3 crowd-workers, as measured by the areaunder the ROC curve and Spearman correlation. Using this corpus ofmachine-labeled scores, our methodology allows us to explore some of the openquestions about the nature of online personal attacks. This reveals that themajority of personal attacks on Wikipedia are not the result of a few malicioususers, nor primarily the consequence of allowing anonymous contributions fromunregistered users.
translated by 谷歌翻译