【作者】 韩雨婷； 刘红云；

【Author】 Han Yuting;Liu Hongyun;School of Psychology, Beijing Normal University;

【机构】 北京师范大学心理学部；

【摘要】 基于计算机的交互式问题解决测验可以在仿真任务情境下实时记录学生的反应过程,并保存为过程数据。过程数据蕴含了关于学生认知与思维活动的丰富信息,对这些信息进行挖掘与分析有利于对学生潜在能力进行更加准确的估计。然而,过程数据形式新颖、结构复杂,如何充分合理地利用过程数据评估学生的潜在能力是一个兼具理论和实践意义的新议题。本研究围绕这一议题,提出了一种可以全面利用学生在问题解决测验中的反应过程信息,估计其潜在能力和题目状态转移参数的序列反应模型。本研究首先提出了将学生的整个反应过程表征为问题状态序列的信息提取方法,然后结合随机过程思想,提出了可以对整个问题状态序列进行分析的序列反应模型(SRM)及其参数估计方法。然后使用一个Monte Carol模拟研究探讨了样本量、序列长度和先验信息对于SRM参数估计返真性的影响,并使用该模型分析了国际学生评估项目(PISA)2012年计算化问题解决测验Tickets任务的过程数据。模拟研究结果表明,新方法具有良好的心理测量学性能,先验信息的有无对于SRM参数估计返真性的影响较小;样本量越大,状态转移参数估计的返真性越好;序列长度越长,潜在能力估计的返真性越好。实证研究结果表明,用SRM分析实际数据获得的能力估计值与专家评分标准相一致,可以正向预测学生在问题解决测验上的整体能力;并且由于SRM利用了学生完整的反应序列,因此对于能力水平的区分相较于结果变量更加合理与细致,还可以获得任务中每种行为特征(状态转移)的相对难易程度。本研究提出的序列反应模型在充分利用过程性信息的同时兼具可解释性,为更合理、准确地估计学生能力提供了方法支持。同时,SRM的分析结果可为基于计算机的交互式问题解决测验的编制,以及如何提高测验的信度提供有用的参考,为实证研究提供了理论基础和方法学指导。总之,本研究推动了过程数据建模方法的发展,为基于动态过程的心理与教育测评提供了更深入和细化的分析方法。更多还原

【Abstract】 The computer-based interactive problem-solving test can record students’ response processes in real time under virtual reality scenario, and save them as process data. Process data contains rich information about students’ cognitive and thinking activities. Making full use of these information can lead to more accurate estimation of students’ latent abilities. However, the form of process data is novel and its structure is complex. How to rationally and fully utilize process data to evaluate students’ latent abilities more accurately and comprehensively is a new issue with both theoretical and practical significance. Aiming at this problem, this study proposed a sequential response model(SRM), which can make full use of students’ complete response sequences in problem-solving test to estimate their latent abilities and state transition parameters. In this study, we first proposed an information extraction method which characterized the whole reprocess process as a sequence of problem states. To analyze the whole problem state sequence, combined with the idea of stochastic process, SRM and its parameter estimation method were proposed.Then, a Monte Carlo simulation study was used to explore the influence of sample size, sequence length and prior information on the accuracy of parameter recovery in SRM. And the proposed model was further used to analyze the process data of the Ticket task in the Programme for International Student Assessment(PISA) 2012 problem solving test. Simulation results showed that the new methods had good psychometric performance. The setting of prior information has little influence on the accuracy of parameter estimation; and the increase of sample size and response sequence can improve the parameter estimation accuracy. The empirical data analysis results showed that the ability estimates obtained from SRM were consistent with the scoring rules made by experts, and can significantly predicted students’ overall performance in the problem-solving test. In addition, compared with the result variable, SRM was more reasonable and detailed in distinguishing students’ capabilities, and can also obtain the relative difficulty of each state transition in the task, for it utilizing the complete response sequences. The proposed SRM can make full use of the process information and be interpretable at the same time, which provides method support for estimating students’ abilities in a more rational and accurate way. In the meantime, the fruitful results of SRM can provide useful reference for the development of computer-based interactive problem-solving test and the improvement of test reliability, which provides methodological guidance for empirical research. In summary, this study promotes the development of process data modeling method, and provides a more in-depth and detailed analysis method for psychological and educational assessment based on dynamic process.更多还原