【摘要】 本研究采用Rasch模型针对基于课程标准开发的测试任务开展效度研究,以验证测试任务设计的有效性和评分方案制定的可靠性。测试成绩的单维度Rasch模型分析结果显示,测试任务拟合度符合预期,测试任务难度总体覆盖考生能力水平,残差数据符合单维性检验。由此可见,测试成绩反映拟测构念要素,测试任务信度较高,且具有较好的代表性,考生作答情况符合模型预期,测试任务整体设计有效。构答任务评分数据的多层面Rasch模型分析结果显示,评分员严厉度、考生能力和测试任务难度显示出较为理想的拟合;评分员一致性多面Rasch模型分析结果显示,评分员间严厉度差异显著低于考生能力跨度,4位评分员的加权均方拟合值和均方拟合值均在合理范围之间。由此可见,评分员使用评分方案的一致性较高,评分方案整体制定可靠。本研究旨在为基于课程标准研发高质量测试任务提供参考,并为Rasch模型在测试任务效度研究中的应用提供操作范式。更多还原

【Abstract】 The study uses the Rasch model to validate assessment tasks developed based on the curriculum standards to verify the effective development of the tasks and the reliable formulation of scoring rubrics.The Unidimensional Rasch model analysis of test scores reveals that the fit statistics accord with model expectations:test task difficulty is generally lined with examinee ability;the residual statistics suggest unidimensionality of the test.Therefore,the tasks can elicit the intended construct with high reliability and good representativeness,and examinees’ performance reasonably fits the Rasch model,thus establishing the overall validity of the tasks.The Many-facets Rasch analysis of rating data of extended constructive assessment tasks reveals that raters’ severity,examinees’ ability and task difficulty show an ideal fit.The Many-facets Rasch analysis of raters’ rating performance shows that raters’ severity differences are significantly lower than that of examinee ability.Rater severity as measured by infit MNSQ and outfit MNSQ are within an acceptable range.Therefore,rating is consistent and the overall rating scale design is appropriate.The study aims to develop high-quality assessment tasks for the curriculum standards and to provide an operational mechanism for the application of Rasch models in the field of assessment validity.更多还原