Research+Paper

Research Proposal in Word format:

TCH 501 Research Proposal

Instant Response Systems: Developing Higher Achievement through Instant Feedback Department of Teacher Education Shippensburg University Michael Ginter May 2, 2012 In Partial Fulfillment of TCH 501 Effective Teaching: Theory and Practice Dr. Han Liu

Over the last few decades, technology has significantly altered the manner in which teachers are able to present material to their students. Chalkboards gave way to overhead projectors which in turn gave way to interactive whiteboards. Unfortunately, student response to these technologies has all too often remained the same – writing down the information with pen and paper. This creates a delay between student response and teacher feedback since the teacher must collect, check and return student work. When available, appropriate technology must be incorporated on the other side of the teacher’s desk in order to promote instant feedback and successful learning. The underutilization of Instant Response (IR) systems is a glaring gap in the integration of student-based learning technology into today’s classrooms.  The main question to be examined through the course of the research is, “Do Instant Response systems, when implemented in a systematic manner, increase student achievement and motivation?” When examining this question, it is also important to address the following sub-questions: “Are IR systems effective learning tools?” and “Are students more willing to fix mistakes when presented with immediate feedback? Since the 1960’s there have been different interactive response systems actively utilized in higher education and secondary schools. Roger Lowery (2005), in his paper //Teaching and Learning with Interactive Student Response Systems//, traces the history of such systems beginning with wired response systems and progressing through the present day wireless and web-based systems. No matter what generation of these IR systems Lowery was referring to, the implementation goal always remained the same. Teachers were interested in both improving student learning and improving teacher effectiveness.  Tuz-Chien Liu (2003), in his paper //The Features and Potential of Interactive Response System//, lays a solid foundational knowledge of what the IR system is and its potential for improving student learning and teacher effectiveness. The traditional set up of the IR system, as seen in Liu’s figure to the right, provides for a highly efficient classroom in which “the [IR system] can immediately collect and process all students’ responses, helping teachers understand all student’s learning statuses, and helping them make decisions instantly during class” (p. 318). Liu concludes though that the existence of the technology does not ensure greater learning, instead “the essential issue is how to apply the [IR system] to various instructional and learning activates” (p. 319)  Much has been said since on the effective implementation of the IR systems, as documented by Fies and Marshall (2006) in //Classroom Response Systems: A Review of the Literature//. The authors highlight 24 different publications that address the pedagogical theory behind, or the implementation of, IR systems. Their general conclusion, drawn from an aggregate of sources, is that IR system “environments lead to greater gains than traditional learning environments,” (p.106) especially when used as an individual learning aid. Keng Siau (2006), author of //Use of a Classroom Response System to Enhance Classroom Interactivity,// attributes this individual learning gain to the ability of IR systems to increase interactivity which in turn influences attitude and achievement. Through the course of the article, Siau goes on to empirically and scientifically prove this claim and concludes that students were “more engaged, more attentive, and more involved in class” (p.400). While it is logical that more engaged and attentive students will achieve higher, the link between these two ideas must be confirmed.  A greater level of interaction in the classroom means a greater level of immediate feedback from the teacher to the student. It is here that the link between interaction and achievement becomes clear. Michael Epstein et al. (2002) examine this topic in the article //Immediate Feedback Assessment Technique Promotes Learning and Corrects Inaccurate First Responses//, by conducting three different studies in which students were presented with multiple choice test items and then interacted with the teacher (through prearranged paper or electronic means) in order to garner immediate feedback. All three studies returned the same result that such a process “promotes acquisition and the retention of test materials” (p.199).  Though there is the potential for IR systems to increase interactivity, attentiveness, and retention, they must not be viewed as a “magic bullet” for education (Beatty, 2006). The effectiveness of an IR system is limited to the effectiveness of the pedagogical implementation and the quality of questions that are asked. While there exists the potential for every possible type of question (multiple-choice, true/false, open-ended), multiple-choice and true/false are the most common. Epstein (2002) doubts whether multiple-choice questions can truly promote new learning and Ian Beatty (2006), in //Designing Effective Questions for Classroom Response System Teaching//, echoes that sentiment when he states that the very characteristics of effective IR system question can differ from the characteristics of effective exam question or homework question. Good IR system questions accomplish the triad of content goal, process goal, and metacognitive goal (Beatty, 2006). In other words, the student must demonstrate not only the cognitive skills to follow the correct process and demonstrate mastery of the content but also the cognitive skills to consider the context the problem is placed in. The implementation of such questions only increases the effectiveness of the IR system as an educational tool when matched with sound pedagogy.  This proposed study brings together the ideas of using IR systems as an interactive means of increasing motivation through well-crafted questions and the importance of immediate feedback for increasing retention. Fies and Marshall (2006) conclude that a “tightly controlled comparison in which the only difference is the use of, or lack of use, of an IR system” (p.106) is one topic that is lacking from current IR system research. This proposed study attempts to address this topic by narrowing down the only difference between two classes to the use of an IR system that provides immediate feedback and the possibility for immediate correction of answers. <span style="font-family: "Times New Roman","serif"; font-size: 16px;">Based upon the research question above, the research design will include both quantitative and qualitative components. The examination of an increase in student achievement will be the quantitative piece which will be evaluated through a multi-subject experiment. The examination of a change in student attitude will be the qualitative piece which will be evaluated through survey data. <span style="font-family: "Times New Roman","serif"; font-size: 16px;">The participants will be my two sections of Algebra 2 which are approximately the same size. There are 22 students in the one class (Class A) who will complete warm-up questions each day using a conventional warm-up journal and paper/pen and 24 in the other section (Class B) who will transition from pen/paper journals to an the IR system to conduct daily warm-ups. <span style="font-family: "Times New Roman","serif"; font-size: 16px;">The control group, Class A, will conduct daily review in the form of 5 warm-up exercises that will be placed into a warm-up journal and graded by the teacher. These questions will take the form of open-ended questions, not multiple choice or true/false, and are to be completed independently each day with only the help of individual notes on the given topic and the teacher. The teacher will provide feedback in the warm-up journal that students will see the next day. Students have the opportunity the next day to fix any mistakes for credit. Therefore, the study will track both student achievement on the questions and how many students fixed answers from the previous day’s material. <span style="font-family: "Times New Roman","serif"; font-size: 16px;"> For the first half of the year, Class B will use the procedure listed above. For the second half of the year, each day, Class B will enter the class room and get their own IR clicker that will be used to complete the warm-up exercises. The students will be presented with the same 5 open-ended review questions that were presented to Class A and students will still be expected to complete the questions independently with only the help of their notes and the teacher. The answers will still be kept in a warm-up journal. The difference is that when students enter their answers, the instant response system will tell them immediately which answers are correct and which answers are not. This provides the students with the immediate chance to change answers as necessary instead of waiting until the next day. The IR system can easily track how many responses were recorded for each question in order to collect data on how many students fixed answers from that day’s material. <span style="font-family: "Times New Roman","serif"; font-size: 16px;"> The quantitative data will be collected by tracking the daily achievement of students from both of the classes (i.e. what percent of the questions does student X get correct on any given day). When students are using warm-up journals, this can be done with a simple tally sheet inside each journal and when students use the IR system, the program will collect this data automatically. In the same manner, data can be collected as to how many students elected to fix incorrect answers. The qualitative data will be gathered through a common survey that will be conducted with both classes at the end of the semester or individual student interviews. The survey and student interviews will measure the student’s general attitude toward the daily review questions (were they helpful, how much effort did you put into them, how often did you feel like correcting your answers …) and the student’s attitude toward the feedback provided to them on incorrect answers. <span style="font-family: "Times New Roman","serif"; font-size: 16px;"> The qualitative data collected (what percent of the questions does student X get correct on any given day) can easily be aggregated into an overall average for each student and overall average for each class. Since both classes will be following the same warm-up procedures for the first half of the year, the comparison of the overall class averages will set the baseline of comparison for the second half of the year. By looking at such a large set of data (at least 80 days of 5 questions each day – approximately 400 questions for each student), it will give a statistically significant idea of the average difference in achievement on warm-ups between the two classes. Furthermore, the test data from the first half of the year should be aggregated in a similar way in order to set up a baseline of achievement difference between the two classes. Note that any outliers (i.e. students who withdraw, move, refused to participate in classroom activities) will be eliminated from the data analysis. <span style="font-family: "Times New Roman","serif"; font-size: 16px;"> Once the baselines are determined from the first half of the year, the data collected from the second half of the year will be compared in the same manner. This will give a second set of comparison achievement rates, but this time Class B will have undergone the IR system intervention. Comparing the two rates of difference in achievement (on both warm-up questions and on test scores) will provide an indication as to whether or not the intervention was a success in terms of increasing achievement. <span style="font-family: "Times New Roman","serif"; font-size: 16px;"> Analyzing the data in this manner presents the following benefits. First, Class B can be compared to itself in terms of achievement on the warm-up questions before and after the intervention, but given the mathematical topics covered in the first and second half of the year, this type of self-comparison can be misleading. Class B can be compared to Class A in terms of achievement, but without the baseline that is established during the first half of the year, there would be no way to know if the differences between the two classes is simply caused by the different students or the implementation of the IR system. By examining the change in the difference of achievement rates, this problem is avoided. <span style="font-family: "Times New Roman","serif"; font-size: 16px;">The survey and interview data will need to be aggregated and analyzed for trends in affective student behavior. For example, did the student’s attitude change in a positive way when they were presented with immediate feedback? Did the students even read the feedback when it was presented in the warm-up journals? How often did the students take advantage of fixing their answers? <span style="font-family: "Times New Roman","serif"; font-size: 16px;">It is impossible to predict the results, but based upon research the students who receive immediate feedback about their answers will be more likely to correct those answers. The increase in students correcting answers should show an increase in test scores because the students will be more actively engaged in the learning process through self-monitoring. The results of the comparison of the achievement difference between the classes should show a trend toward higher achievement with the IR system since students will progressively be more engaged with the material by correcting their mistakes and misinterpretations.
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">1. Problem Statement **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">2. Research Questions **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">3. Literature Review **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">4. Research Design **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">5. Participants **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">6. Intervention and Procedure **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">7. Data Collection **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">8. Data Analysis **
 * <span style="font-family: "Times New Roman","serif"; font-size: 16px;">9. Results Prediction **

<span style="font-family: "Times New Roman","serif"; font-size: 16px;">References <span style="font-family: "Times New Roman","serif"; font-size: 16px;">Beatty, I. D., Gerace, W. J., Leonard, W. J., & Dufresne, R. J. (2006). Designing Effective Questions for Classroom Response System Teaching. //American Journal of Physics//, //74//(1), 31-39. doi: 10.1119/1.2121753 <span style="font-family: "Times New Roman","serif"; font-size: 16px;">Epstein, M. L., Lazarus, A. D., Calvano, T. B., Matthews, K. A., Hendel, R. A., Epstein, B. B., & Brosvic, G. M. (2002). Immediate Feedback Assessment Technique Promotes Learning and Corrects Inaccurate First Responses. //The Psychological Record//, (52), 187-201. <span style="font-family: "Times New Roman","serif"; font-size: 16px;">Fies, C., & Marshall, J. (2006). Classroom Response Systems: A Review of the Literature. //Journal of Science Education and Technology//, //15//(1), 101-109. doi: 10.1007/s10956-006-0360-1 <span style="font-family: "Times New Roman","serif"; font-size: 16px;">Liu, T. (n.d.). The Features and Potential of Interactive Response Systems. //International Conference on Computers in Education 2003 Proceedings Book// (pp. 315-322). <span style="font-family: "Times New Roman","serif"; font-size: 16px;">Lowery, R. C. (n.d.). Teaching and Learning with Interactive Student Response Systems: A Comparison of Commercial Products in the Higher-Education Market. //Southwestern Social Science Association Meeting, New Orleans 2005//. Retrieved from people.uncw.edu/lowery/swssa%20ms.pdf <span style="font-family: "Times New Roman","serif"; font-size: 16px;">Siau, K., Sheng, H., & Nah, F. (2006). Use of a Classroom Response System to Enhance Classroom Interactivity. //IEEE Transactions on Education//, //49//(3), 398-403. doi: 10.1109/TE.2006.879802