【摘要】 针对常见的众包图像注解的质量评估方法只能处理客观型注解信息的问题,提出一种针对主观型众包图像注解的质量评估方法。将主观型众包图像注解任务分为两个阶段:注解阶段和评分阶段。由注解者对图像进行主观型注解,再由评分者对注解进行评价打分。根据评分者的打分情况对注解质量及评分者偏好等进行估计,得到注解质量评估结果。实验结果显示,该方法相对于多数投票法和标记聚合法可以获得更高的精度和用户满意度。表明该方法可以更高效地运用于各种主观型众包图像注解任务。更多还原

【Abstract】 Common quality estimation methods for crowdsourcing image annotations are only applicable for objective image annotations. In light of this,we propose a novel quality estimation method for subjective crowdsourcing image annotations. An annotation task is divided into two stages: the annotation stage and the marking stage. Annotators make subjective annotation on images and then reviewers mark these annotations. Estimations on the annotation quality and the preference of reviewers are made according to their marking situation to obtain the estimation results of annotation quality. Experimental results show that our method is able to achieve higher precision and customer satisfaction relative to the majority voting method and the label aggregation method. This means that the method can be more effectively applied in various subjective crowdsourcing image annotation tasks.更多还原