Identifying Reliability Information

With this exercise you’ll review and identify basic reliability information as presented in published reports.

Study 1: Hoffman and Nadelson

Study can be found here:

http://www.bwgriffin.com/gsu/courses/edur7130/readings/Hoffman&Nadelson_2010.pdf

Go to page 250 and read about the 18-item Goals Inventory.

1. What type of reliability evidence was presented?

2. Do the reliability coefficients presented show acceptable levels of consistency? How do we know?

3. In addition to administering questionnaires to collect data, the authors also interviewed participants and collected their responses. The authors coded those responses. Now go to page 252 and read the section entitled “Coding methodology”. Read to the top of page 254; what type of reliability was presented for the coding of interview responses?

4. Do the authors explain how the measure of reliability for coding of interview responses was calculated or determined?

Study 2: Boeckner et. al.

http://www.bwgriffin.com/gsu/courses/edur7130/readings/Boeckner_et_al_2002.pdf

5. What is the purpose of this study?

6. To assess equivalent forms reliability (they call it parallel forms), what must they do? I am not asking what they did, but from our study of reliability you should know the basic steps for establishing equivalent forms reliability. What are the steps the authors should take to establish equivalent forms reliability?

7. Did the authors follow the steps outlined for establishing equivalent forms reliability?

8. The issue for these authors is whether both versions produced similar responses from participants. Where can we find evidence about the level of agreement among scores from both versions?

9. Do you see any problematic correlations in this table? Any correlations lower than the lowest value typically accepted for reliability estimates?

10. Now look at Table 2; how does this table help us assess equivalent forms reliability?