Show simple item record

dc.contributor.advisorTekian, Ara
dc.creatorRambihar, Sherryn P
dc.date.accessioned2018-07-27T17:55:43Z
dc.date.available2018-07-27T17:55:43Z
dc.date.created2018-05
dc.date.issued2018-04-05
dc.date.submittedMay 2018
dc.identifier.urihttp://hdl.handle.net/10027/22685
dc.description.abstractBackground: New guidelines for echocardiography training recommend assessment of core competencies using simulation, direct observation and multisource evaluation. Purpose: We developed a simulation-based echocardiography competence assessment tool (ECAT) of basic echocardiography skills. We collected validity evidence to evaluate our claims that the ECAT can be used for formative assessment. Methods: We conducted a mixed methods pilot of cardiology trainees (n=5 C1; n=5 C2; n=4 C3), who were oriented to the simulator and assessed using the ECAT. We video-recorded performance and collected participants’ simulated echocardiogram videos, which were assessed by four raters in real-time or off-line. After providing participants with video footage and rater feedback, we interviewed both participants and raters. For implications evidence, we used a simple content analysis to analyze interview data. For scoring evidence, we computed internal consistency and inter-rater reliability. For extrapolation evidence, we correlated ECAT scores with participants’ expertise, diagnostic scan quality, and a summative exam score. Results: Our analysis of the interview data revealed three themes: i) feedback stimulated change, ii) how feedback was delivered impacted participants’ perceived learning, and iii) assessment credibility influenced participants’ reception of feedback. The inter-rater reliability across all items was 0.837 (95% CI 0.81-0.87) and for ECAT score, ICC was 0.913 (95% CI 0.81-0.97). An exploratory factor analysis demonstrated a two-factor model, for which coefficient were 0.96 and 0.87 respectively. ECAT scores correlated with summative exam scores (r=0.66, p=0.02), were significantly higher with diagnostic quality scans (p=0.0007), and were associated with number of echocardiograms seen (p=0.014), performed (0.012) and interpreted (0.024). ECAT score increased according to level of training (p=0.01) with significant differences seen between first and second year trainees, and first and third year trainees, but no difference between second and third year trainees. Conclusions: Trainees and raters provided specific data for refining the ECAT process, feedback delivery, and technology for improved formative assessment. We suggest the collective evidence is favorable, supporting use of the ECAT in a simulation-based formative assessment. Our study provides practical insights for echocardiography educators and those seeking to develop instruments for formative assessment.
dc.format.mimetypeapplication/pdf
dc.subjectcompetence-based medical education
dc.subjectformative assessment
dc.subjectechocardiography
dc.subjectvalidation framework
dc.titleValidation of a Simulation-Based Tool For Formative Assessment of Echocardiography Skill Competence
dc.typeThesis
thesis.degree.departmentMedical Education
thesis.degree.grantorUniversity of Illinois at Chicago
thesis.degree.levelMasters
thesis.degree.nameMHPE, Master of Health Professions Education
dc.contributor.committeeMemberBrydges, Ryan
dc.contributor.committeeMemberPark, Yoon Soo
dc.contributor.committeeMemberLineberry, Matthew
dc.type.materialtext
dc.contributor.chairTekian, Ara


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record