Activity 4: Sample Size Exercise

1. Assume you wish to randomly sample from a population of 33,050 teachers across the southeast USA. With a 95% confidence level and an error rate of no more than +/- 4%, what size sample would be needed?

2. A researcher plans to interview students to learn whether they have ever tried smoking. About 1100 students attend the school the researcher wishes to sample. What size sample would you recommend to this researcher? Provide details about various limits you placed when deciding upon this sample size.

3. You wish to compare mathematics scores on the CRCT for students exposed to three different teaching methods across the state of Georgia. What size sample do you anticipate is needed?

4. Is there evidence of sex discrimination in salary among teachers in Georgia? In a study conducted in the 1999, the mean salary for male teachers was $32,324 and the mean salary for female teachers was $34,992 with a combined standard deviation of $5,336. Using information from the 1999 study, what size sample would be needed (with alpha = .05) for a new study of sex discrimination among teachers to have a 95% chance of detecting a mean difference in salaries if one exists in the population of GA teachers?

5. Student ratings of college instructors is common and widespread across the USA. Often such ratings consist of several instructional dimensions such as course organization, clarity of presentation, utility of course material, etc. Often the overall mean of responses to these items are used as a summary measure of teaching effectiveness for tenure, promotion, and merit pay evaluations. Of interest to many faculty is whether factors other than instructional activities predict (or possibly affect) the overall mean instructional rating. Assume a college instructor wishes to know whether (a) perceived autonomy among students, (b) student efficacy for the course material, and (c) student rating of the utility of the course content to their future studies or career predict mean instructional ratings. The instructor is unsure how these three predictors may relate to mean ratings, so the instructor wishes to select a sample that would give a 90% chance of detecting a moderate relationship (say d = .5) between mean ratings and any of the three predictors with alpha set at .05. What size sample is needed?

6. A doctoral student is planning a study of possible association between school average mathematics performance on the CRCT and the percentage of students in the given school who qualify for free or reduced lunch. Three previous studies report correlations between school SES and student performance of .17, .20, and .29 respectively. With this information, what sample size should the student seek to have an 80% chance of rejecting a false null?

7. When interviewing women teachers in Georgia, I plan to ask this following question: "Have you experienced sexual discrimination as a teacher in Georgia?" Research in other states suggest that about 75% of women teachers claim to have experienced some form of sexual discrimination in the workplace. For this study, set your confidence level of 95%. Also, there are about 15,000 women teachers in Georgia. (Suppose we did not know that 75% will likely claim discrimination, what size sample would then be needed?)

8. In a pilot study of only 45 students a researcher found a Multiple R² value of .16 (this is the squared correlation, r², between the DV and the predicted DV) when analyzing the pilot study data with multiple regression. The study data included a dependent variable (student mathematics test scores) and two independent variables ([a] number of homework problems successfully completed the week before the test and [b] the number of minutes per day spent in mathematics instructional activities). Now ready to move beyond the pilot testing phase, using information learned from the pilot study, what size sample should the teacher seek to detect an effect of this size (Multiple R² = .16) with two independent variables?

Answers provided below. Do not scroll down to review answers until each question above is covered in class.

Answers

1. Using the Excel spreadsheet for finding samples sizes of proportions, sample size would be about 590.

2. Since smoking is a yes/no type response, samples for proportions would work. Sample sizes for various margins of error and for alpha = .05 and .01 appear below:

3. Use the Maxwell and Delaney table to determine sample size here. Since no other information is given about what to expect in terms of group differences in mathematics, assume the smallest effect size provided in the table (d = .25) as a conservative estimate. There are three groups, alpha = .05  (since that is the only table value for alpha provided), and assume beta = .20 (so power is 1-.2 = .80). Sample size needed is 310 *  2 = 620 (or about 620/3 = 206.6 per group). CORRECTION: Since there are 3 groups, the sample size of 310 is per group, so the total sample size would be 310 * 3 = 930.

4. Again, use the Maxwell and Delaney table to determine sample size here. Since there is previous evidence about the size of the difference that can be expected, we can estimate the effect size d as follows:

d = (Mean1 - Mean2) / SD

d = ($32,324 - $34,992) /  $5,336

d = $2,668 / $5,336

d = -.5

There are two groups, d = -.5 (take the absolute value here [.5] for use of their table), alpha = .05, and power = .95 (i.e., 95% chance of detecting a mean difference), so the tabled sample size is 105 for the two groups or a total sample of 210 teachers.

5. To determine the needed sample size for the multiple regression study, use this regression sample size calculator:

http://www.danielsoper.com/statcalc/calc01.aspx

or the Excel file here:

http://www.bwgriffin.com/samplesize

To use this, we must know four parameters: (a) alpha, (b) number of predictors, (c) effect size f-squared, and (d) power. Each of these are:

(a) alpha = .05 (as set by the researcher above)

(b) number of predictors is 3 (perceived autonomy, student efficacy, and utility rating)

(d) power = .90 (i.e., "90% chance of detecting a moderate relationship")

(c) effect size f-square can be determined from d = .50. To find f-square value for a d = .50, d must be converted to f-square. Use this spreadsheet to find f-squared:

http://www.bwgriffin.com/gsu/courses/edur9131/content/sample_size_effect_size_conversion.xls

In the yellow column find "d" and enter the value of .5. The resulting f-square is reported as .0625. Enter .0625 as the Anticipated Effect Size (f²).

The resulting sample size to detect an moderate effect of .5 is about 177 evaluations.

6. Since the mean correlation is .22 ( [.17 + .20 + .29] / 3 = .22) and power is .80, the sample size needed to detect this correlation is 160 (or 157 for the exact sample size) with alpha = .05 or 214 CORRECTION -> 237 (or 233 with the exact sample size) with alpha = .01.

7. Responses can take form of yes/no, so sample size for proportions can be used. We know from previous research to expect about 75% to experience sexual discrimination, and alpha = .05 with a population of about 15000 women teachers. Below are sample sizes needed for this study. (Note sample sizes needed in parentheses if the we did not know that 75% are expected to claim discrimination.)

8. As with #5 above, four parameters must be known:

(a) alpha = was not specified in the scenario, so I will use both .01 and .05 in this example response;

(b) number of predictors = 2 ( number of homework problems completed successfully and time in minutes in instruction on mathematics)

(d) power = was not specified, so I will use .80 and .90

(c) The effect size: the value R² = .16 must be converted to f². In regression, R² is just the squared Pearson correlation, r², between the dependent variable and the predicted scores on the DV. The value f² can be converted from  r² using the conversion file below:  

http://www.bwgriffin.com/gsu/courses/edur9131/content/sample_size_effect_size_conversion.xls

In the yellow column find "r²" and enter the value of .16. The resulting f² is reported as .19. In the regression sample size page

http://www.danielsoper.com/statcalc/calc01.aspx

enter .19 as the Anticipated Effect Size (f²). The following sample sizes were obtained:

If alpha = .05 and power = .80, the n = 54

If alpha = .05 and power = .80, the n = 78

If alpha = .01 and power = .90, the n = 69

If alpha = .01 and power = .90, the n = 96