2.50
Hdl Handle:
http://hdl.handle.net/10755/156877
Type:
Presentation
Title:
Assessing Response-Shift in At-Risk Groups Using the Retrospective Pre-Test
Abstract:
Assessing Response-Shift in At-Risk Groups Using the Retrospective Pre-Test
Conference Sponsor:Sigma Theta Tau International
Conference Year:2002
Conference Date:July, 2002
Author:Parshall, Mark
P.I. Institution Name:University of New Mexico Health Sciences Center
Title:Assistant Professor
Background: Assessing change due to interventions aimed at members of at-risk groups involves practical and methodological difficulties. Objective evaluation of behavior in the field settings may be expensive, impractical, or alter the behavior(s) of interest. Intervention endpoints often depend on pre- and post-test score differences using self-reports as proxies for at-risk behaviors. This is especially common in nursing research, hence awareness of the potential limitations of self-report difference scores for measuring change is critical. In this theoretical / methodological presentation, we discuss a conceptual framework for measuring change (Golembiewski, Billingsley, & Yeager, 1976) and a specific example of a measurement approach based in that framework. Concept: Validity of pre- to post-test score differences as a criterion of change depends on underlying assumptions: (1) the same construct or domain is being measured before and after the intervention; and (2) participants apply the same subjective scaling judgments to pre- and post-test measures. Accordingly, three types of change have been identified (Golembiewski et al.,1976): alpha change corresponds to a conventional pretest-posttest difference, provided that underlying assumptions of constancy in the conceptual domain and subjective scale calibration are satisfied; beta change implies that there is evidence that participants' subjective calibration judgments with respect to study measures have changed in virtue of intervention participation; gamma change implies that intervention participants have reconceptualized the constructs or domains being measured over the course of the study. Originally, the term "response-shift" was applied primarily to internal scale recalibration (beta change; Terborg et al., 1980). More recently, Schwartz and Sprangers (2000) applied the concept of response-shift not only to changes in subjective scale calibration (beta change) but also changes in values (e.g., relative importance) and changes in construct conceptualization (gamma change). . Beta change can be examined by including a pretest (PRE), a traditional posttest (POST), and a retrospective pretest (RPT). The RPT is administered concurrently with the POST, but asks participants how they now rate their status prior to the intervention. Because the RPT and POST are concurrent, participants apply the same subjective calibration to the parallel item sets. A significant difference between RPT and true PRE scores implies that beta change has occurred (i.e., participants are using a different subjective scale calibration after the intervention than before it) and that the appropriate criterion of change should be the RPT-POST rather than the PRE-POST difference (Terborg et al., 1980). If there is no significant PRE-RPT difference, then the conventional PRE-POST difference is appropriate. Testing for gamma change typically involves confirmatory factor analysis or related techniques. Objectives and Approach: The use of the RPT technique will be illustrated with examples from a completed study* involving an injury prevention intervention for workers in small construction companies (no more than 10 employees). This is a high-risk group in terms of fatality and injury severity rates. In the first year of the intervention, we screened for possible beta change with a limited number of RPT items related to presumed dimensions of a safety climate measure under development. Results were consistent with a beta-change response shift. Accordingly, in the second year of intervention, we conducted a full RPT concurrently with the delayed post-test, using the finalized safety climate questionnaire (20 Likert-scaled items). Results did not support the occurrence of beta change. Implications: Uncritical acceptance of a pre- to post-test score difference, or lack thereof, without testing whether a response-shift occurred, may be a hidden source of systematic error, even when sample size affords nominally adequate statistical power. Use of a limited number of items to screen for beta change may be attractive from a standpoint of minimizing response burden, but, if feasible, a full retrospective pretest is preferable.

Repository Posting Date:
26-Oct-2011
Date of Publication:
Jul-2002
Sponsors:
Sigma Theta Tau International

Full metadata record

DC FieldValue Language
dc.typePresentationen_GB
dc.titleAssessing Response-Shift in At-Risk Groups Using the Retrospective Pre-Testen_GB
dc.identifier.urihttp://hdl.handle.net/10755/156877-
dc.description.abstract<table><tr><td colspan="2" class="item-title">Assessing Response-Shift in At-Risk Groups Using the Retrospective Pre-Test</td></tr><tr class="item-sponsor"><td class="label">Conference Sponsor:</td><td class="value">Sigma Theta Tau International</td></tr><tr class="item-year"><td class="label">Conference Year:</td><td class="value">2002</td></tr><tr class="item-conference-date"><td class="label">Conference Date:</td><td class="value">July, 2002</td></tr><tr class="item-author"><td class="label">Author:</td><td class="value">Parshall, Mark</td></tr><tr class="item-institute"><td class="label">P.I. Institution Name:</td><td class="value">University of New Mexico Health Sciences Center</td></tr><tr class="item-author-title"><td class="label">Title:</td><td class="value">Assistant Professor</td></tr><tr class="item-email"><td class="label">Email:</td><td class="value">mparshall@salud.unm.edu</td></tr><tr><td colspan="2" class="item-abstract">Background: Assessing change due to interventions aimed at members of at-risk groups involves practical and methodological difficulties. Objective evaluation of behavior in the field settings may be expensive, impractical, or alter the behavior(s) of interest. Intervention endpoints often depend on pre- and post-test score differences using self-reports as proxies for at-risk behaviors. This is especially common in nursing research, hence awareness of the potential limitations of self-report difference scores for measuring change is critical. In this theoretical / methodological presentation, we discuss a conceptual framework for measuring change (Golembiewski, Billingsley, &amp; Yeager, 1976) and a specific example of a measurement approach based in that framework. Concept: Validity of pre- to post-test score differences as a criterion of change depends on underlying assumptions: (1) the same construct or domain is being measured before and after the intervention; and (2) participants apply the same subjective scaling judgments to pre- and post-test measures. Accordingly, three types of change have been identified (Golembiewski et al.,1976): alpha change corresponds to a conventional pretest-posttest difference, provided that underlying assumptions of constancy in the conceptual domain and subjective scale calibration are satisfied; beta change implies that there is evidence that participants' subjective calibration judgments with respect to study measures have changed in virtue of intervention participation; gamma change implies that intervention participants have reconceptualized the constructs or domains being measured over the course of the study. Originally, the term &quot;response-shift&quot; was applied primarily to internal scale recalibration (beta change; Terborg et al., 1980). More recently, Schwartz and Sprangers (2000) applied the concept of response-shift not only to changes in subjective scale calibration (beta change) but also changes in values (e.g., relative importance) and changes in construct conceptualization (gamma change). . Beta change can be examined by including a pretest (PRE), a traditional posttest (POST), and a retrospective pretest (RPT). The RPT is administered concurrently with the POST, but asks participants how they now rate their status prior to the intervention. Because the RPT and POST are concurrent, participants apply the same subjective calibration to the parallel item sets. A significant difference between RPT and true PRE scores implies that beta change has occurred (i.e., participants are using a different subjective scale calibration after the intervention than before it) and that the appropriate criterion of change should be the RPT-POST rather than the PRE-POST difference (Terborg et al., 1980). If there is no significant PRE-RPT difference, then the conventional PRE-POST difference is appropriate. Testing for gamma change typically involves confirmatory factor analysis or related techniques. Objectives and Approach: The use of the RPT technique will be illustrated with examples from a completed study* involving an injury prevention intervention for workers in small construction companies (no more than 10 employees). This is a high-risk group in terms of fatality and injury severity rates. In the first year of the intervention, we screened for possible beta change with a limited number of RPT items related to presumed dimensions of a safety climate measure under development. Results were consistent with a beta-change response shift. Accordingly, in the second year of intervention, we conducted a full RPT concurrently with the delayed post-test, using the finalized safety climate questionnaire (20 Likert-scaled items). Results did not support the occurrence of beta change. Implications: Uncritical acceptance of a pre- to post-test score difference, or lack thereof, without testing whether a response-shift occurred, may be a hidden source of systematic error, even when sample size affords nominally adequate statistical power. Use of a limited number of items to screen for beta change may be attractive from a standpoint of minimizing response burden, but, if feasible, a full retrospective pretest is preferable.<br/><br/></td></tr></table>en_GB
dc.date.available2011-10-26T15:13:41Z-
dc.date.issued2002-07en_GB
dc.date.accessioned2011-10-26T15:13:41Z-
dc.description.sponsorshipSigma Theta Tau Internationalen_GB
All Items in this repository are protected by copyright, with all rights reserved, unless otherwise indicated.