EDUR 8331
Applied Measurement

Course Index, Syllabus, and Calendar
Fall 2021

Instructor: Bryan W. Griffin
Personal web page: http://www.bwgriffin.com

Announcements:  

• Updated Nov 2 - Activity 3 is now available
• New - Inputting Data into Excel from a Printed Questionnaire
• Updated - Test 2 due Oct 25 31 rather than Oct 18
• Coder Agreement with Nominal Data has been updated - first presentation in the section entitled Assessing Coder/Rater Agreement (nominal and ordinal/interval/ratio)
• Activity 1 Part 2 instructions now added
• Activity 1 Part 1 due Sept 12 rather than Sept 7

Add activity to review validity and reliability of a school pscy instrument - (like the WISC, KABC, RIAS) - maybe a Forum discussion post - use validity reliability info from manual, but also find independent review articles

RIAS - review of RIAS author results CJSP324458.qxd (myschoolpsychology.com)

Author PP presentation with interesting tables to review Slide 1 (myschoolpsychology.com) (e.g. table 5.7 used attenuation correction for reliability - inflated estimates, would paired t-test show mean differences?)

Non-author review of RIAS Nelson & Canivez (2012) RIAS Validity.pdf (eiu.edu)

Convergent Validity of the Reynolds Intellectual Assessment Scales (RIAS) Using the Woodcock—Johnson Tests of Cognitive Ability, Third Edition (WJ-III) With University Students (sagepub.com)

Factorial validity of the Reynolds Intellectual Assessment Scales for referred students - Beaujean - 2009 - Psychology in the Schools - Wiley Online Library

Or maybe find empirical assessment of three IQ tests and how well they correlate to achievement

Frontiers | The Predictive Validity of Four Intelligence Tests for School Grades: A Small Sample Longitudinal Study | Psychology (frontiersin.org) - pdf version available - see correlation tables

Course Index

A. Course Administration

1. Syllabus 
2. Course Calendar and Reading Assignments (see below).
3. Creating PDFs Files (used for submission of tests and activities)

B. Graded Activities

1. Test 1: Posted in Folio two weeks before the due date of  Sept 20 - revised due Sept 28; topics include the following:
   • Introductory Review (Measurement, Variables and Scales of Measurement, Descriptive Statistics, Graphical Displays, Correlation, Two-group t-test)
   • Reliability and Validity
   • Questionnaire Selection
2. Test 2: Posted in Folio two weeks before the due date of  Oct 18 - revised due Oct 31; topics include the following:.
   • Material covered on Test 1
   • Reading Factor Analysis Results
   • Composite Scores and Reverse Scoring
   • Questionnaire Development
   • Coding Textual Data and Assessing Rater Agreement
   • Electronic Questionnaires
3. Test 3: Posted in Folio two weeks before the due date of Dec 2; topics include all content covered in the course.
   • Material covered on Tests 1 and 2
   • Classroom Test Construction (Test and Standardized Scores, Developing Objectives, Content Validity, Item Development, Item and Test Analysis)
   • Item Response Theory and Rasch Models
4. Activity 1, Part 1: Scale Development
   • Activity 1 Part 1: Description and Instructions
   • Activity 1: Latent Variable Assignments and Sample Scales
   • Activity 1 Due Dates
     • Sept  7 - revised Sept 12 Post draft of scale items to the Activity Part 1 dropbox in Folio.
     • Oct 1 Post revised items in Activity 1 Part 1 dropbox in Folio (identify the best four items and the summary item).
     • tbd Enter revised items into electronic questionnaire; link will be provided in Folio.
5. Activity 1, Part 2: Scale Validation      
   • Activity 1 Part 2: Description and Instructions
   • Activity 2 Due Dates
     • due tbd  Post reliability estimates to the Activity 1 Part 2 dropbox in Folio
     • due tbd Post content validity and relationship validity evidence in Activity 1 Part 2 dropbox in Folio
6. Activity 2: Validity
   • Activity 2 Description and Instructions
   • Activity 2 due Sept 18
7. Activity 3: Test Item Analysis
   • Instructions
   • Test Data 
   • Item Analysis Excel Spreadsheet (let me know if you need a spreadsheet other than Excel)
   • Supplemental - Brief Item Analysis Review
   • Due Nov 17
8. Activity 4: Test Item Development (may be added)
   • Instructions to be added

C. Course Calendar and Content

Note: Calendar topics may be altered or relocated to reflect realized course pace during semester. 

Week 1 (Aug 11)

• Syllabus and Course Content (review during optional live chat - to be recorded)
• General Introduction via optional live chat, with recording in Folio if you cannot attend the optional chat
• Get to know Folio and the course

Weeks 2 and 3 (Aug 16 to Aug 23)

• Introduction to Measurement
• Statistics Refresher:
  • Descriptive Statistics
  • Graphical and Tabular Displays
  • Pearson Correlation
  • Two-group t-test
• Readings for the above material

Weeks 4 to 6 (Aug 30 to Sept 13)

• Reliability (Readings for Reliability)
• Validity (Readings for Validity)
• Questionnaire/Scale Selection (Readings for Questionnaire Selection)

Weeks 7 to 10 (Sept 20 to Oct 11)

• Reading Factor Scores (Readings for Factor Scores)
• Questionnaire/Scale Development (Readings for Questionnaire Development)
• Electronic Questionnaire with Google Docs (and Qualtrics) (Readings for Electronic Questionnaire)
• New Data Entry (into a Spreadsheet) from a Printed Questionnaires/Scales (Videos for Data Entry)
• Composite Scores and Reverse Scoring (Readings for Composite and Reverse Scoring) (note: move to immediately after reliability)
• Qualitative Data Analysis of Open-ended Items (Readings for Qualitative Data Analysis)
• Assessing Coder/Rater Agreement (nominal and ordinal/interval/ratio) (Readings for Assessing Coder Agreement)

Weeks 11 to 15 (Oct 18 to Nov 29)

• Classroom Test Development
  • Content Validity: Specifying Content (Readings for Content Validity)
  • Test Item Development (Readings for Test Item Development)
  • Test Item Analysis and Other Topics (Readings for Test Item Analysis)
• Item Response Theory and Rasch Models (may be added)