【作者】 陈冠宇； 陈平；

【Author】 Chen Guanyu;Chen Ping;Collaborative Innovation Center of Assessment Toward Basic Education Quality, Beijing Normal University;

【机构】 北京师范大学中国基础教育质量监测协同创新中心；

【摘要】 题组(testlet)是指具有共同刺激(stimulus)或相同特征的一组题目(Wang&Wilson, 2005a)。在心理与教育测验情境中,题组的设计与使用越来越常见。虽然题组设计可以减少被试的负担、提高测验的效率,但是却违反了项目反应理论(Item Response Theory, IRT)的局部独立性假设(Lee, Brennan,&Frisbie,2005)。因此,如果不正确处理题组内题目的局部依赖性问题,而直接对题组内的题目进行题目功能差异(Differential Item Functioning, DIF)分析则会产生误差。针对这一问题,已有研究者提出多种在题组设计情境下侦测DIF的方法,比如题组反应模型(Wang&Wilson, 2005b)、两因素多维IRT模型(Fukuhara&Kamata, 2011)、两水平题组反应模型(Beretvas&Walker,2012)以及随机权重的线性对数测验模型(Paek&Fukuhara, 2015)。但是,以往研究都基于模拟数据进行分析,也没有进行多种方法之间的比较。因此,本研究基于国际学生评估项目(Program for International Student Assessment, PISA) 2012年的实测数据进行DIF分析。研究采用两水平的题组反应模型(Two-Level Testlet Response Model, MMMT-2)拟合数据,该方法能够同时侦测DIF和题组功能性差异(Differential Testlet Functioning, DTLF)。对MMMT-2的分析基于广义线性混合模型(Generalized Linear Mixed Model, GLMM)的框架,并通过R软件编程实现相关分析。本研究将MMMT-2与题组反应模型和SIBTEST方法 (Douglas et al., 1996)进行全面比较,结果表明MMMT-2具有更强的统计性能和准确性。虽然得到较有意义的研究结果,但还有一些未来研究方向值得讨论:比如与两因素多维IRT模型进行比较,对MMMT-2进行进一步修正,或者采用模拟数据系统比较不同方法之间的差异。更多还原

【Abstract】 Testlet is a bundle of items that share a common stimulus(Wang & Wilson, 2005 a), and more and more educational and psychological assessments have started to adopt testlet design. Although testlet can reduce the burden of examinee and improve the efficiency of test, it violates the local independent assumption that is basic for item response theory(IRT)(Lee, Brennan, & Frisbie, 2005). Thus, applying the IRT-based differential item functioning(DIF) analysis to the items in testlets may produce bias. There are some researchers have focused on this problem, and already proposed several methods to detect the DIF under the context of testlet design, such as Rasch testlet model(Wang & Wilson, 2005 b), bifactor multidimensional IRT model(Fukuhara & Kamata, 2011), two-level testlet response model(Beretvas & Walker, 2012) and the random-weights linear logistic test model(LLTM)(Paek & Fukuhara, 2015). But all these relative methods had not been used for the real data, and there’s not comparison between these methods. This research performed such an analysis, using the empirical data from PISA 2012. This research used the two-level testlet response model to fit the empirical data. This method can detect the DIF and differential testlet functioning(DTLF) simultaneously. The analysis of the two-level testlet response model was under the framework of Generalized Linear Mixed Model, and R was used. Comparing this result to Rasch testlet model and SIBTEST method(Douglas et al., 1996), this study showed the power and accuracy of two-level testlet response model. Though the results are very useful, several future directions for research, such as using the bifactor multidimensional IRT model to estimate the DIF and DTLF, modifying the two-level testlet response model, and employing simulation data to fully compare these methods are worthy of further discussion.更多还原