According to a study by Ertner et al. (2007), peer feedback is important in maintaining the quality of discussion postings over the course of a semester. However, instructor feedback was still viewed as more important than peer feedback. Participants indicated that peer feedback was important on two fronts.  First, when peer feedback was received that was not explicit or clear, or was not helpful, the receiver tended to be motivated to consider how to write their own responses in a way that would be more helpful to the receiver. In addition, in the process of writing feedback, students were able to realize the shortcomings of their own posts, and, according to interview and survey responses, were able to use this knowledge to construct higher quality responses to posted discussion questions. However, this was not reflected in study results which showed no significant improvement in the quality of discussion postings over time. The study also indicated an increase in the importance of timeliness in the feedback process. Ultimately, while participants viewed instructor feedback as more important, peer feedback was valued for several reasons, including its use as a guide for constructing responses and for its less complex (and perhaps less rigorous) way of presenting information. Limitations of the study included low rigor of the discussion questions, narrow range of scores which created a ceiling effect, lack of timeliness of responses due to the mediation of the instructor, impact on critical feedback due to concerns about grades, and the low number of postings that were actually geared toward answering the discussion questions.

The study focused on the interesting idea of using peer feedback as a replacement for instructor feedback during online learning. However, the hypotheses presented ideas that were not addressed in the literature review, as did the tools used in the survey process. The literature review, while it indicated some of the benefits of online discussion, did not directly address why these benefits were valuable.  There was no comparison in the literature review of the nature of peer feedback as compared to instructor feedback, other than to say that giving and receiving peer feedback could sometimes be seen as invalid due to lack of peer expertise (especially in terms of negative feedback). There was no indication about why peer feedback would be preferable.  There was a significant omission in the literature review concerning the use of Bloom’s Taxonomy. Because this was the focal point of evaluating student responses, it would be reasonable to expect that significant time would be spent in analyzing its usefulness not only in terms of responses, but in terms of the discussion questions used to elicit those responses.  If one is trying to “evaluate” a response for its higher order thinking, then the questions asked should be written accordingly.  No discussion of either of these points was included in the literature review, nor in the study design, but was only considered as an afterthought in the discussion.  While one of the concluding statements discussed the possibility of using peer feedback to supplement or replace instructor feedback, and while this was addressed extensively in the literature review, this was strictly inferential and not based on direct evidence from the study. Another apparent flaw in the study design occurred in the quantity of discussion questions and required responses as compared to the responses given. Although only seventeen discussion questions were asked, which required one answer and one response for each student, the researchers analyzed 778 total responses, acknowledging in the discussion that only 160 of these responses were actually related to the peer feedback being studied.  The researchers cited that they didn’t eliminate the extraneous responses because then they wouldn’t have had enough data to draw valid conclusions; however, I propose that this invalidates the entire study.  If the responses analyzed weren’t tied to the hypotheses, then they should have been eliminated.  If there weren’t enough responses, then perhaps the study should have been replicated to ensure that there were.  Just acknowledging an error in study design does not improve the quality of the data. The researchers did a successful job of calibrating the analysis of responses to ensure inter-rater reliability; however, as mentioned, the narrow range of possible scores probably resulted in some false positivity here. There was much less margin for error on the part of the reviewers. Had the researchers used a more detailed rubric, perhaps scoring on a scale of 0-6 to coincide with each level of Bloom’s Taxonomy, calibration may have been much more difficult, or perhaps impossible; however, as indicated by the researchers, there would have been a much better indication of whether or not the quality of the responses improved over time. Finally, there was no control group because the number of participants was so small. Every participant operated under the same conditions. The small number of participants calls into question the generalizability of this study. The authors called this an exploratory study. In my opinion, this is simply a way to acknowledge that the study didn’t really have a lot to tell its readers.

In terms of my research focus, I think there are a lot of ways that creativity could have contributed to this study.  Was it necessary to use only questions to evaluate the effectiveness of peer feedback? What if the study had focused on creation of products, outlines, or graphic organizers? Would this feedback prove more valuable?  What about creating a control situation where students answer some questions (at an appropriately high level) without a rubric, and then answer the next questions using the rubric, with peer feedback given in both situations?  Perhaps students could write responses, then in an online chat situation, compare their responses together and use the rubric to co-construct a BETTER response due to the collaborative process. Virtual discussions are the next best thing to being there–and I would wager that the resultant learning would improve.  What about using a matrix to evaluate the strength of a response rather than a numerical scale? This would, I think, have addressed the problem of low quality discussion questions; the matrix would only be useful if the questions were of sufficient complexity. What if the STUDENTS wrote the questions, using Bloom’s Taxonomy as a guide, responded to each other’s questions, and then gave peer responses to the responses for their own questions? To me, this study was a prime example of Einstein’s quote about insanity: “Insanity: doing the same thing over and over again and expecting different results.” If nothing was changed about the way the discussion questions were posed, it is no wonder that the results were inconclusive.

Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., Lei, K., & Mong, C. (2007). Using Peer Feedback to Enhance the Quality of Student Online Postings: An Exploratory Study. Journal of Computer-Mediated Communication, 12(2), 412–433. https://doi.org/10.1111/j.1083-6101.2007.00331.x