Options
Controlling response shift bias: The use of the retrospective pre-test design in the evaluation of a master's programme
Author(s)
Date Issued
2008-11-15
Date Available
2013-03-14T17:20:46Z
Abstract
Student self-report measures of change are widely used in evaluation research to
measure the impact and outcomes of an educational programme or intervention.
Traditionally the measures used to evaluate the impact of an educational programme
on student outcomes and the extent to which students change is a comparison of the
student’s pre-test scores with their post-test scores. However, this method of evaluating
change may be problematic due to the confounding factor of response shift bias.
Response shift bias occurs when the student’s internal frame of reference of the
construct being measured, for example research ability or critical thinking, changes
between the pre-test and the post-test due to the influence of the educational programme. To control for response shift bias the retrospective pre-test method was
used to evaluate the outcomes achieved from students completing a research module
at master’s level. The retrospective pre-test method differs from the traditional pre-test-post-test design in that both post-test and pre-test perceptions of respondents are
collected at the same time. The findings indicated that response shift bias was evident
in student self-reports of change, especially in subjects the student had been
previously exposed to at undergraduate level. The retrospective pre-test design found
that the programme had significantly greater impact on outcomes that that identified
using the traditional pre‐test–post‐test design leading to the conclusion that students
may overestimate their ability at the commencement of an educational programme.
The retrospective pre-test design is not a replacement for the traditional pre‐test–post‐test measures but may be a useful adjunct in the evaluation of the impact of educational programmes on student outcomes.
Type of Material
Journal Article
Publisher
Taylor & Francis (Routledge)
Journal
Assessment and Evaluation in Higher Education
Volume
33
Issue
6
Start Page
699
End Page
709
Copyright (Published Version)
2008 Taylor & Francis
Language
English
Status of Item
Peer reviewed
This item is made available under a Creative Commons License
File(s)
Owning collection
Scopus© citations
117
Acquisition Date
Mar 28, 2024
Mar 28, 2024
Views
1471
Last Month
2
2
Acquisition Date
Mar 28, 2024
Mar 28, 2024
Downloads
726
Last Week
1
1
Last Month
7
7
Acquisition Date
Mar 28, 2024
Mar 28, 2024