sub:assertion {
sub:paragraph c4o:hasContent "To measure the performance of the studied crowdsourcing approaches (contest and microtasks), we report on: i) inter-rater agreement computed with the Fleiss’ kappa metric in order to measure the consensus degree among raters (experts or MTurk workers); ii) precision to measure the quality of the outcome of P each crowd, computed as T P T +F P ." ;
a doco:Paragraph .
}