The Use of Standardized Test Score Reports to Inform Instruction Using the Deming Model

2.50
Hdl Handle:
http://hdl.handle.net/10755/602921
Category:
Full-text
Format:
Text-based Document
Type:
Presentation
Title:
The Use of Standardized Test Score Reports to Inform Instruction Using the Deming Model
Other Titles:
Leadership in Nursing Education [Session]
Author(s):
Schultz, Mary Anne; Chesebrough, Geri; Chesebrough, Geri
Lead Author STTI Affiliation:
Gamma Tau
Author Details:
Mary Anne Schultz, RN, mschultz@csusb.edu; Geri Chesebrough, RN, CNE
Abstract:
Session presented on Tuesday, November 10, 2015: When considering evidence-driven approaches to quality improvement in schools or departments  of nursing, seldom is a true quality improvement circle observed. Too often, faculty “discuss” results of standardized test scores in ways that could be considered anecdotal, episodic, time-limited and perhaps, idiosyncratic. That is to say that unfortunately, a one-time only end-of-term discussion of the output of one set of standardized test scores are put forth, instead of results in a series of exams, longitudinally. Discussion of results for selected content areas for a student cohort, then, is usually eclipsed by the limitations of time (and know-how) obviating the opportunity for true quality improvement strategizing. If faculty are given regular formal opportunities to discuss group results of high-stakes, proctored, content-mastery exams (such as those reported by Mountain Measurement, NCSBN and/or ATI), the chance to collaborate both with one course and across a set of courses might emerge if the Deming Plan-Do-Check-Act (PDCA) model is used. Structuring the groups with members representing like content domains (e.g. all medical-surgical nursing instructors) may provide a good beginning for planning a sole course for subsequent offerings but it falls short of true quality improvement if not restructured, in subsequent formal opportunities, to capture creative strategizing from a group of dissimilar content masters. That is, when teachers of courses across a curriculum are assembled, the opportunity to identify weak versus strong content areas traversing a spectrum of learning opportunities might arise. In this fashion, faculty might learn to understand, with depth, what is involved in the measurement of student outcomes, especially as course and program objectives are leveled and considered. This presentation allows the attendee to witness how a plan for faculty development, in which regular formal opportunities for faculty to view and critically analyze reports of standardized test scores, was designed. With an emphasis on assigning similar, then dissimilar, faculty content masters to groups, the authors will walk the participants through a series of report interpretations identifying the uses, then misuses, of the evidence. Prototypes of data users will be presented, starting with the occasional faculty user of data-those who obtain, then record, a test score for partial course credit in one time-limited course. The presentation progresses,  then, to a description of the power-user of data who will compile a set of scores for multiple student cohorts, over time, indicative of alleged teaching prowess alongside possible mitigating trends, again limited to a course over time. Finally, a plan from the program perspective, to align faculty from diverse content areas for more discriminating analysis of report data, will be discussed, in which emphasis is placed on the commonalities of categories of student performance, e.g. patient safety, comfort care, use of the nursing process. It is in this last crucial component, involving similar, then dissimilar faculty areas of expertise, in which true performance improvement might be realized, especially if faculty can truly engage in the continuous, not static, nature of this process (across, not just within, courses). Since the introduction of Deming quality improvement principles to health care environments began in the early 1990s, significant resistance has occurred through protestations that principles of Japan’s automotive industry improvements (where the model began) could never apply to the human condition. In recent years, however, as hospitals and other health care environments have demonstrated fewer adverse outcomes with use of the PDCA model for provider and administrator decision-making, the use of the model is more frequent. Seldom is it seen used in higher education with few references to PDCA action circles for faculty identified in the literature. It remains to be seen if use of action circles, with changing faculty membership over time, can be associated to improved student learning outcomes such as improved test scores, particularly in nursing where high-stakes content-mastery testing is common and necessary.
Keywords:
Deming Model; PDCA action circles; standardized test score reports
Repository Posting Date:
21-Mar-2016
Date of Publication:
21-Mar-2016
Other Identifiers:
CONV15H10
Conference Date:
2015
Conference Name:
43rd Biennial Convention
Conference Host:
Sigma Theta Tau International, the Honor Society of Nursing
Conference Location:
Las Vegas, Nevada, USA
Description:
43rd Biennial Convention 2015 Theme: Serve Locally, Transform Regionally, Lead Globally.`

Full metadata record

DC FieldValue Language
dc.language.isoen_USen
dc.type.categoryFull-texten
dc.formatText-based Documenten
dc.typePresentationen
dc.titleThe Use of Standardized Test Score Reports to Inform Instruction Using the Deming Modelen
dc.title.alternativeLeadership in Nursing Education [Session]en
dc.contributor.authorSchultz, Mary Anneen
dc.contributor.authorChesebrough, Gerien
dc.contributor.authorChesebrough, Gerien
dc.contributor.departmentGamma Tauen
dc.author.detailsMary Anne Schultz, RN, mschultz@csusb.edu; Geri Chesebrough, RN, CNEen
dc.identifier.urihttp://hdl.handle.net/10755/602921en
dc.description.abstractSession presented on Tuesday, November 10, 2015: When considering evidence-driven approaches to quality improvement in schools or departments  of nursing, seldom is a true quality improvement circle observed. Too often, faculty “discuss” results of standardized test scores in ways that could be considered anecdotal, episodic, time-limited and perhaps, idiosyncratic. That is to say that unfortunately, a one-time only end-of-term discussion of the output of one set of standardized test scores are put forth, instead of results in a series of exams, longitudinally. Discussion of results for selected content areas for a student cohort, then, is usually eclipsed by the limitations of time (and know-how) obviating the opportunity for true quality improvement strategizing. If faculty are given regular formal opportunities to discuss group results of high-stakes, proctored, content-mastery exams (such as those reported by Mountain Measurement, NCSBN and/or ATI), the chance to collaborate both with one course and across a set of courses might emerge if the Deming Plan-Do-Check-Act (PDCA) model is used. Structuring the groups with members representing like content domains (e.g. all medical-surgical nursing instructors) may provide a good beginning for planning a sole course for subsequent offerings but it falls short of true quality improvement if not restructured, in subsequent formal opportunities, to capture creative strategizing from a group of dissimilar content masters. That is, when teachers of courses across a curriculum are assembled, the opportunity to identify weak versus strong content areas traversing a spectrum of learning opportunities might arise. In this fashion, faculty might learn to understand, with depth, what is involved in the measurement of student outcomes, especially as course and program objectives are leveled and considered. This presentation allows the attendee to witness how a plan for faculty development, in which regular formal opportunities for faculty to view and critically analyze reports of standardized test scores, was designed. With an emphasis on assigning similar, then dissimilar, faculty content masters to groups, the authors will walk the participants through a series of report interpretations identifying the uses, then misuses, of the evidence. Prototypes of data users will be presented, starting with the occasional faculty user of data-those who obtain, then record, a test score for partial course credit in one time-limited course. The presentation progresses,  then, to a description of the power-user of data who will compile a set of scores for multiple student cohorts, over time, indicative of alleged teaching prowess alongside possible mitigating trends, again limited to a course over time. Finally, a plan from the program perspective, to align faculty from diverse content areas for more discriminating analysis of report data, will be discussed, in which emphasis is placed on the commonalities of categories of student performance, e.g. patient safety, comfort care, use of the nursing process. It is in this last crucial component, involving similar, then dissimilar faculty areas of expertise, in which true performance improvement might be realized, especially if faculty can truly engage in the continuous, not static, nature of this process (across, not just within, courses). Since the introduction of Deming quality improvement principles to health care environments began in the early 1990s, significant resistance has occurred through protestations that principles of Japan’s automotive industry improvements (where the model began) could never apply to the human condition. In recent years, however, as hospitals and other health care environments have demonstrated fewer adverse outcomes with use of the PDCA model for provider and administrator decision-making, the use of the model is more frequent. Seldom is it seen used in higher education with few references to PDCA action circles for faculty identified in the literature. It remains to be seen if use of action circles, with changing faculty membership over time, can be associated to improved student learning outcomes such as improved test scores, particularly in nursing where high-stakes content-mastery testing is common and necessary.en
dc.subjectDeming Modelen
dc.subjectPDCA action circlesen
dc.subjectstandardized test score reportsen
dc.date.available2016-03-21T16:39:31Zen
dc.date.issued2016-03-21en
dc.date.accessioned2016-03-21T16:39:31Zen
dc.conference.date2015en
dc.conference.name43rd Biennial Conventionen
dc.conference.hostSigma Theta Tau International, the Honor Society of Nursingen
dc.conference.locationLas Vegas, Nevada, USAen
dc.description43rd Biennial Convention 2015 Theme: Serve Locally, Transform Regionally, Lead Globally.`en
All Items in this repository are protected by copyright, with all rights reserved, unless otherwise indicated.