EDUR 8331
Applied Measurement
Course
Index, Syllabus, and Calendar
Fall 2019
Instructor: Bryan W.
Griffin
Personal web page:
http://www.bwgriffin.com
Announcements:
None at this time.
A. Course Administration
B. Graded Activities
C. Course Calendar and Content
Note: Calendar topics may be altered or relocated to reflect realized course pace during semester.
Week | Date | Topic |
1 |
1/11 |
Note - chat dates differ from week-start dates located on the left. Optional chats will occur Tuesdays each week.
1. Syllabus 2. Course Content Overview 3. Topics and readings for weeks 1 and 2 a. Measurement: Van Blerkom, Chapter 1 Introduction to Measurement b. Variables, Graphical Displays, and Descriptive Statistics: * Van Blerkom, - Chapter 18 The Language and Logic of
Statistics * Additional material from EDUR 8131 Introduction to Educational Statistics: Basics Review c. Pearson Correlation: * Van Blerkom, Chapter 22 Correlation * Additional material from EDUR 8131 Introduction to Educational Statistics: Correlation d. Two-group t-test: (not sure t-tests are needed this semester, so ignore for now) * Van Blerkom, t-test not covered by text * Material from EDUR 8131 Introduction to Educational Statistics: t-test e. SPSS introduction: *See presentation video below for SPSS basics with descriptive statistics, graphical displays, correlation, and t-test. * Material from EDUR 8131 Introduction to Educational Statistics: SPSS Tutorials 4. Presentations a. Notes: * Course Overview, Measurement, and Variables * Basic Statistical Concepts and SPSS * Pearson Correlation and SPSS b. Video: * Overview, Measurement, and Variables (48 minutes) * Basic Statstics Concepts and SPSS (39 minutes) * Pearson Correlation and SPSS (21 minutes) * Two-group t-test (to be added)
5. Non-graded Activity - complete if you need practice with correlation and t-test. Perform analyses of data found below to ensure your findings match those provided in the answers.
|
2 |
1/18 |
1. Topics and readings: same as week 1 above 2. Presentation: same as week 1 above
|
3 | 1/25 |
1. Topic and readings for week 3: Reliability
* Van Blerkom, Chapter 4 * Conceptual Introduction to Reliability (from EDUR 7130 Introduction to Educational Research) * Tavakol and Dennick (2011). Making Sense of Cronbach's Alpha, International Journal of Medical Education. * Gliem and Gliem (2003). Calculating, Interpreting, and Repoting Cronbach's Alpha Reliability Coefficient for Likert-Type Scales . * Koo and Li (2016). A Guideline for Selecting and Reporting Intraclass Correlation Coefficients for Reliabiloty Research. Journal of Chiropractic Medicine, 15, 155-163 (This publication is statistical in nature; read it for conceptual understanding and study Figure 1 to identify different types of ICC; the ICC will be explained in this week's presentation and video). * Cronbach's Alpha with SPSS * Add examples from published studies and standardized tests. 2. Data for illustrations Data: Employment Thoughts Merged Data Leisure Activities
Questionnaire (please complete):
http://goo.gl/forms/JrMtZmVHF5 3. Presentation a. Notes: *
Review of Reliability and Validity * Internal Consistency and Item Analysis b. Video: * Review of Reliability and Validity * Internal Consistency and Item Analysis (71 minutes)
4. Non-graded Activity: Internal Consistency
|
4 |
2/1 |
1. Topic and readings for week 4: Validity
a. Van Blerkom, Chapter 5 b. Current Views on Validity * Brown, T. (2010) Construct Validity: A Unitary Concept for Occupational Therapy Assessment and Measurement. Hong Kong Journal of Occupational Therapy, 20, 30-42. Very good overview of traditional and current views of validity. Study Table 1 carefully. * Cook, DA, & Beckman, TJ (2006). Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Applicaiton. The American Journal of Medicine, 119, 166.e7 - 166.e16. Anoher good review of traditional and current views of validity. c. Validity Evidence Based Upon Content (i.e., Content Validity) * Haynes, S.N., Richard, D.C.S., & Kubany, E.S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238-247. (See especially Content Validation Guidelines starting page 244.) * Review Table 1, p. 35-36 in the Brown (2010) Construct Validity article linked above, with focus on "1. Test Content Evidence." * Add Lawsche article on quantitative content validity d. Validity Evidence Based Upon Internal Structure and Relation to Other Variables * Review Table 1, p. 35-36 in the Brown (2010) Construct Validity article linked above; study the two subjects in Table 1 entitled "4. Internal Structure Evidence" and "5. Relations to Other Variables Evidence." 2. Example publications to showing various validity evidence as discussed by readings above. a. Ragheb, M. G., & Beard, J. G. (1982). Measuring
leisure attitude. Journal of Leisure Research, 14, 155-167. 3. Data for illustrations Data: Employment Thoughts Merged Data Leisure Activities
Questionnaire (please complete):
http://goo.gl/forms/JrMtZmVHF5 4. Presentation a. Notes: Validity Evidence b. Video: to be added
|
5 | 2/15 |
1. Topic and readings for week 5: Composite Scores, Reverse Scoring, and Questionnaires/Scales Selection a. Composite Scores and Reverse Scoring
- see presentation notes below 2. Presentation a. Notes: *
Composite Scores and Reverse Scoring b. Video: * Composite Scores and Reverse Scoring (50 minutes) * Searching for Developed Questionnaire Scales (41 minutes)
3. Non-graded Activity: Composite Scores
|
6 | 2/22 |
1. Topics and readings for week 6: Reading Factor Scores for Item Selection and Questionnaire/Scale Construction a. Reading Factor Analysis Results for Item Selection * For a simple introduction see Presentation Notes and Video below on Reading Factor Analysis Scores * Kootsa: Factor Analysis Explained (this is briefly reviewed in Factor Analysis video below; focus on interpretation of results) b. Questionnaire/Scale Development Construction * Frary (1996) Hints for Designing Effective Questionnaries * Fanning (2005) Formatting a Paper-based Survey Questionnaire: Best Practices * de Vaus (2002) Chapter 7: Questionnaire Construction * Folz (1996) Chapter 4: Survey Design and Implementation * Oskamp & Schultz (2005) Explicit Measures of Attitudes * Siniscalco and Auriat (2005) Module 8 Questionnaire Design (supplemental material; multi-national perspective) c. Questionnaire Development Checklist * Holmbeck, G. N., & Devine, K. A. (2009). Editorial: An author's checklist for measure development and validation manuscripts. Journal of Pediatric Psychology, 1-6, 2009. * This file contains only Holmbeck and Devine's checklist d. Likert-scaled Response Options * List of Step Descriptions for Five-point Likert Responses * Dr. Siegle offers these: Likert Response Options * Dr. Vagias provides these: Likert Response Options 2. Presentation a. Notes: *
Reading Factor Scores
b. Video: *
Reading Factor Scores
(58 minutes)
3. Non-graded Activity: Instrument Construction Steps |
7 | 3/1 |
1. Continue readings on Questionnaire Development above 2. Instructor note - add readings on Single-item measures, validity and reliability 3. Electronic Questionnaire Development
4. Presentation a. Notes: *
Creating Electronic Questionnaire with Google Docs * Using vlookup with Google Spreadsheet to change text to numbers. Youtube: https://www.youtube.com/watch?v=YMASsjPkngg b. Video: * No videos |
8 | 3/8 |
1. Topics and readings for week 8: Qualitative Data Analysis, Codebooks and Codesheets,and Assessing Coder Agreement a. Qualitative Data Analysis * Taylor-Powell and Renner (2003) Analyzing Qualitative Data; An easy to read, brief (10 pages) summary of Qualitative Data Analysis * LeCompte (2000) Analyzing Qualitative Data. Theory into Practice, 39, 146-154; A more theoretical introduciton to Qualitative Data Analysis * Kent Lofgren (2013) Qualitative analysis of interview data: A step-by-step guide. His video provides basic introduction to coding textual data. His focus is on interview data, but his presentation applies equally to written responses to open-ended items. Youtube: https://www.youtube.com/watch?v=DRL4PF2u9XA * Hilary Nixon (2014). Analyzing Qualitative Data. Her video provides a general introduction for the first 6 mintues, then starting at minute 7 she begins providing multiple examples of coding real text. Feel free to skip to minute 7. Youtube: https://www.youtube.com/watch?v=BnDUARfEu5I b. Codebooks c. Inter-coder Agreement * Viera & Garrett (2005) Understanding Interobserver Agreement: The Kappa Statistic. * Joyce (2013) Blog Entry: Picking the Best Intercoder Reliability Statistic for Your Digital Activism Content Analysis (PDF version of page)
2. Presentation a. Notes: *
Qualitative Data Analysis of Open-ended Item Responses, Qualitative Code
Sheets and Codebooks * Codebooks for Quantitative Data: Creating a codebook using a Questionnaire (to be added after data collection) *
Assessing Coder/Rater
Agreement for Nominal Data b. Video: * Qualitative Data Analysis - see youtube videos
linked above *
Assessing Rater Agreement for Nominal Data
3. Non-graded Activity: Non-graded Activity: Coding Open-ended Responses
|
3/15 |
Spring Break |
|
9 | 10/14 |
1. Continue readings from Qualitative Data Analysis, Code Books and Sheets
above * Instructor note: review and possibly add Siniscalco and Saito (2005) Module 10 Data Preparation and Management |
10 | 10/21 |
1. Topic and readings for week 10: Classroom Measurement
Van Blerkom Text and John Hills notes on Standard Scores * Chapter 2 Frames of Reference: Interpreting Test Scores * Chapter 3 Developing Objectives * Chapter 5 Validity - please re-read this chapter * Chapter 10 Producing and Administering Tests * Chapter 16 Standardized Tests * Chapter 17 Alternative Ways to Report Test Scores Hills 1983 Grade Equivalent Scores Hills 1983 Percentile Scores Hills 1984 Normal Curve Equivalent (NCE) Scores Hills 1983 Stanines Supplemental Reading on Standard Scores Hills 1984 IQ Scores Hills 1984 SAT and ACT Scores (may be dated material) 2. Presentation a. Notes: * Chapter 2 Frames of Reference - Normed vs Criterion Referenced * Chapter 3 Performance Objectives * Chapter 16 Standadized Tests * Chapter 17 Alternate Test Score Formats b. Video: * No Videos Planned (may add videos as brief review)
|
11 | 10/28 |
1. Continue readings from week 10 above |
12 | 11/4 |
1. Topic and readings for week 12: Writing Test Items
Van Blerkom * Chapter 6 Completion and Short-Answer Items * Chapter 7 Essay Items * Chapter 8 Multiple-choice Items * Chapter 9 True-false Items (and Variations) 2. Presentation a. Notes: * Chapter 6 Completion and Short-Answer Items * Chapter 8 Multiple-choice Items * Chapter 9 True-false Items (and Variations) b. Video: * No Videos Planned (may add videos as brief review) |
13 | 11/11 |
1. Topic and readings for week 13: Item Analysis
Van Blerkom * Chapter 11 Analyzing Tests * Grades: Reporting Student Performance (see presentation below) * Additional readings to be added
2. Presentation a. Notes: * Chapter 11 Analyzing Tests and Item Analysis * Grades: Reporting Student Performance b. Video: Short, Focused Summary * Item Analysis - quick summary: Introduction to Item Analysis and Distractor Analysis
More detail - watch in order presented e. Non-graded activity: Item Analysis of Test Responses (to be added) |
14 | 11/18 |
1. Topic and readings for week 14: Item Response Theory (IRT) and and Rasch
Models Wu, M. & Adams, R. (2007). Applying the Rasch model to psycho-social measurement: A practical approach. Baker, F. (2001) The Basics of Item Response Theory Chapter 1 The Item Characteristic Curve (see Cook's videos 2, 3, and 4 below as companions to Baker's chapters 1 and 2) Chapter 2 Item Characteristics Curve Models (see above) Chapter 3 Estimating Item Parameters (see Cook's video 4 below) Chapter 4 Test Characteristic Curve Chapter 5 Estimating Ability (i.e., the latent variable) Chapter 6 Information Function (see Cook's video 5 below) Chapter 7 Test Calibration (see Cook's video 6 below) Chapter 8 Test Characteristics (see Cook's video 6 below)
2. Presentation a. Notes: * to be added * Annotated examples, to be added b. Videos:
Karon Cook's introduction to IRT 1. Introduction to IRT: https://www.youtube.com/watch?v=SrdbllMYq8M 2. IRT Probability Model: https://www.youtube.com/watch?v=b6FKYV1-mEU 3. IRT Item Plots: https://www.youtube.com/watch?v=9wHvZX-YQeo 4. IRT Difficulty, Discrimination, and Characteristic Curves: https://www.youtube.com/watch?v=yoKSAMUpvvE More detail on Item Characteristic Curves/Functions: https://www.youtube.com/watch?v=FkPtz9i0SBA 5. IRT Item and Scale Information: https://www.youtube.com/watch?v=BRYipKwPmW4 6. IRT Applications: https://www.youtube.com/watch?v=K6qkwuWSZMw
Nick Shryane's introduction to IRT * https://www.youtube.com/watch?time_continue=183&v=QHIKJlcniHA * His presentation as a PDF file: Shryane 2011
|
11/25 |
Thanksgiving break |
|
15 | 12/2 |
1. Continue readings from week 14 above |
16 | 12/9 |
Finals week |