concepts they were to learn. This pre-lecture quiz contained questions that the class should not have known before the class as well as some they might have been able to answer. The quizzes were then marked and the class’ overall mean, median, and standard deviation were calculated.

The students were then taught the concept of Java interfaces as defined in the course curriculum. Dr. Kinley conducted a role-playing exercise with the experimental class. The next time the class met, they were given a nearly identical quiz with slightly reorganized questions but with the same information as the pre-lecture quiz. Again, the quizzes were marked and the class’ overall mean, median, and standard deviation were calculated.

 

The Role-Play

One student (call him Jeff) was told to leave the room, and another (call her Jill) was told to come to the front of the room. Jill was given a list of activities that she would “know” how to do if asked. Once she felt comfortable with the list of activities, Jeff was invited back. The instructor told Jeff to give Jill different instructions. After a few moments of Jeff asking Jill to do activities that she could not do, the instructor explained that this is how life is without Java-style interfaces. Once again, Jeff was told to leave the room. This time Jill was given a set of activities she had to be able to do and what kind of parameters went with each instruction, and they were then written on the board. Jeff was invited back in and found himself greeted by a list of things that Jill could do, as well as what she needed to know to do them. The instructor asked if it would be easier to tell Jill what to do this time. As expected, Jeff confirmed it was easier and found that Jill would react to his instructions in the way he expected. The instructor explained that the writing on the board was a kind of Java-style interface.

 

Initial Instructor Reaction

Immediately after the classes, Kinley explained that the students were very receptive to the role-play exercise. He said their enthusiasm toward class participation increased when they got to stand up and do something rather than just sit and listen during the class. The mood in the classroom was almost cheerful and he felt that he had the attention of the entire class through the exercise. Kinley said that he believed the students in the experimental section learned the concept of Java-style interfaces much more quickly than the control class. He also believed that they were more able to apply their knowledge of interfaces than the class that had not participated in the role-play. He expressed his confidence in the effectiveness of the role-play in getting the students to understand and apply interfaces.

 

Statistical Results

However, the statistical results are not as straightforward as the professor’s anecdotal confirmation of the hypothesis. Percent change (Table 3) was calculated for each student. Data in this chart reflects the mean, median, and standard deviation of the changes, and not the change in mean, median, and standard deviations. In short, the control section seemed to show more improvement than the experiment section. On average, the students in the control section improved their scores 11% while the experimental section showed a much smaller mean improvement of 3%. The experiment section started with half of the scores above 67%, but finished with a median of 60%. Seven percent of the students in that class moved below the average, indicating that either the tail end increased (fewer very low scores) or there were

more very high scores than before the lesson. Combining this 7% decrease in median with the fact that half of the students’ scores decreased significantly (-10% median in change) leads to the conclusion that many students did worse than average, but fewer students had extremely low scores. This spread in the spectrum is verified by the large deviation in the change of each of the student’s score.

 

Control Section Mean Accuracy Median Score Standard Deviation

Experiment Section Mean Accuracy Median Score Standard Deviation

Pre-Quiz
50%
60%
33%

Pre-Quiz
56%
67%
39%

Post-Quiz
61%
60%
15%

Post-Quiz
59%
60%
23%

Change
11%
5%
30%

Change
3%
-10%
45%

Table 3: Results of the experiment.

 

The median change for the control section is positive, which suggests that more than half of the students in the class had increased scores. The mean increased quite a bit (11%) as well, suggesting that there are more high scores, but fewer perfect scores.

C ontrol 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Experimental

p r e

po st

pr e

po st

Figure 1: Interfaces Test Results.

 

The standard deviation of each section indicates that the scores clumped together much more closely around the mean after the lesson. This signifies that more of the students in both cases are ending up with a closer to average mastery of the class; each individual’s skill in Java-style interfaces is becoming more “normal.” In summary, both sections’ scores coalesced into closer-knit groups around their averages (Figure 1), but many more students in the experimental section experienced a decrease in scores. In both cases, there is a variance of scores below the average that shrinks after the lesson. It seems that the people who were having the most trouble in the experimental section were helped by the role-play, but some people in that section suffered as well.

 

Experiment 2: Threads

The second experiment was conducted on the understanding of threads. The control class was taught this concept using the standard lecture and group work method. The experiment class performed a simple role-playing exercise to supplement the standard classroom routine. During this experiment, Dr. Laxer taught the control section and Dr. Kinley taught the experimental group. This time, with the use of two different instructors, a possible difference in teaching styles had to be