【摘要】 计算机自动评分优势诸多,发展势头迅疾,将这一技术引入到实评之中,须对其效度进行论证。本研究依据Williamson等在2012年提出的计算机自动评分效度论证框架,对2016年11月我国新疆地区中国少数民族汉语水平等级考试(MHK)口语考试的实测数据进行了计算机自动评分的效度验证,结果显示:依据该框架对口语开放性试题计算机自动评分进行效度验证是可行的,MHK口语开放性试题的计算机自动评分与人工评分的依据相似,与专家评分的相关系数在0.8以上,评分的一致性和稳定性较高,多项指标明显优于人工评分。更多还原

【Abstract】 Automated scoring has various advantages in multiple areas, and it has undergone rapid development.Before applying automatic scoring technologies in real-time settings, researchers need to evaluate its validity. In this study, researchers conducted a validation test based on the framework of validity evaluation for automated scoring proposed by Williamson, et al. Data was collected at a Minzu Hanyu Shuiping Kaoshi(MHK)oral test which was administered in Nov. 2016 in Xinjiang, China. Results shows that: 1)it is feasible and valid to apply Williamson’s framework to assess automated scoring of open-ended questions in oral tests; 2)the automated scoring in MHK used similar criteria as manual scoring, and its correlation coefficient with the expert scoring was above 0.80. Automated scoring in MHK also shows high consistency and stability with multiple indicators significantly better than those of the manual scoring’s.更多还原