Understanding the Nexus Between Item Analytics and CLO Attainment
Medical education increasingly relies on well-constructed MCQs to measure not only factual recall but higher-order thinking. In an integrated, outcome-based dental curriculum, aligning assessment items with Course Learning Outcomes (CLOs) is essential. This study synthesizes how psychometric indices—Discrimination Index, Difficulty Index, and reliability measures KR-20/KR-21—relate to the percentage of CLO attainment across ten CLOs in three courses. The goal is to inform curriculum developers and assessment committees about optimizing item banks to enhance learning outcomes.
Key Psychometric Indices and What They Mean for CLOs
The Difficulty Index (p-value) indicates how challenging a question is. In this study, items averaged a moderate difficulty, with CLOs concentrated around a 0.70 mean. Discrimination Index measures how well an item differentiates high- from low-performing students, showing a moderate ability in this cohort. KR-20 and KR-21 quantify internal consistency reliability for dichotomous items; KR-20 tends to be more precise by accounting for actual item difficulty, while KR-21 assumes equal item difficulty.
Main Findings on CLO Attainment and Indices
- <strongCLO attainment averaged 71.4%, with variation across CLOs (58%–81%). CLO 4 stood out with the highest attainment (81%), while CLO 10 lagged at 58%.
- <strongDifficulty Index showed a strong positive relationship with CLO attainment (r = 0.906, p < 0.01), suggesting that appropriately challenging items boost CLO achievement.
- <strongKR-21 reliability correlated positively with CLO attainment (r = 0.701, p < 0.05), indicating that test forms with higher KR-21 values tended to align better with CLO success; however, KR-20’s relationship with CLO attainment was weaker and non-significant (r = -0.238).
- <strongDiscrimination Index correlated strongly with KR-20 (r = 0.831, p < 0.01) but showed a negative, non-significant link to CLO attainment (r = -0.450, not significant). This implies that while discrimination reflects item performance, it may not directly predict CLO success when Difficulty is considered.
Regression Insights and Practical Implications
Backward regression identified the Difficulty Index as the sole significant predictor of CLO attainment, with an impressive R² of 0.82. In practical terms, about 82% of the variance in CLO attainment could be explained by item difficulty alone when considering the included indices. This underscores the importance of calibrating item difficulty to balance challenge and learning objectives. Discrimination, KR-20, and KR-21 contributed less to the predictive model once difficulty was accounted for, though they remain valuable for general test quality and reliability.
Implications for Curriculum Design and Assessment Practice
The study highlights several actionable takeaways for dental education programs operating under outcome-based frameworks. First, item-writing quality matters: poorly worded stems, ambiguous distractors, or miskeyed items can undermine discriminative power and CLO alignment. Second, a deliberate emphasis on maintaining a moderate to challenging Difficulty Index can promote deeper learning and better CLO attainment, in line with Kane’s reliability theory. Third, while KR-20 and KR-21 offer important reliability checks, they should be interpreted alongside the Difficulty Index to gauge how well test items reflect the intended CLOs.
Limitations and Opportunities for Future Research
Limitations include a restricted sample from three courses and ten CLOs within a single institution. The study invites future work to explore longitudinal trends, cross-institutional replication, and the role of qualitative data (rubrics, performance-based assessments, and student feedback) to contextualize item performance and CLO attainment.
Conclusion
In integrated outcome-based dental curricula, item Difficulty emerges as the strongest predictor of CLO attainment. While Discrimination and reliability indices remain essential for overall test quality, aligning item difficulty with CLO targets appears pivotal for fostering meaningful learning outcomes. Routine item analysis, informed by these indices, can guide targeted revisions to the question bank, enhancing both assessment validity and student competence.
