EDUR 8331
Applied Measurement

Topic: Assessing Rater Agreement

1. Start Here: Summary Presentation

• Document: Assessing Coder/Rater Agreement for Nominal Data (Video: Assessing Rater Agreement for Nominal Data) (Notes updated Fall 2021, video not updated)

• Document :Assessing Coder/Rater Agreement for Ordinal and Interval/Ratio Data (Video: Assessing Rater Agreement for Ordinal and Interval/Ratio Data)
  • JASP video: Coder/Rater Agreement in JASP with Cohen kappa, Feliss kappa, Krippendorff alpah, and ICC

2. Readings

• Supplemental Inter-coder Agreement
  •  Viera & Garrett (2005) Understanding Interobserver Agreement: The Kappa Statistic.
  •  Joyce (2013) Blog Entry: Picking the Best Intercoder Reliability Statistic for Your Digital Activism Content Analysis (PDF version of page)