Why Use Simulated Peer Review
A primary value of adapted journal article teaching is the
emphasis on student-generated analysis,
discussion and debate. For a professor to tell students what they
should think ahead of time defeats the main point of the exercise.
This kind of learning activity provides strong and convenient
incentive for professors to extend their traditional
"stand-and-deliver" teaching style. And, we believe, teaching in
this way will stimulate students to become more intellectually
engaged and active in their learning.
While this view is often supported by the facts, professors
should ask "How will my students ever learn to think for
themselves, if I don't require it?" Of course, many students are
not knowledgeable enough, and they will find such lessons
difficult. But how else can they learn to do difficult things if
they are not challenged? Thinking is a skill that can be learned.
The main point of this kind of teaching is to help students learn
to think about academic matters, not just memorize answers for
exams. In short, knowing the WHAT of a given discipline is not
enough. Students also need to know the WHY, HOW, AND SO WHAT.
A major goal of this kind of instruction is to show students how
to develop their creative and critical thinking capabilities.
Simulating peer review is a logical way to spur students to be more
insightful, as well as show how to scrutinize scholarly work before
publication. To guide student thinking, we provide in the lesson
plan some 24 questions that the students should address in their
simulated peer review. In terms of pedagogy, this exercise embraces
every level of Bloom's Taxonomy.
- Students (preferably grouped as learning teams) are assigned an
ARR and conduct a simulated peer review.
- Some form of deliverable should be required (written analysis,
poster, class presentation, etc.).
- Student reviews should graded primarily for their critical and
- Professor should follow up to show how a professional would
have evaluated the ARR.
Procedure & Scaffolding
Students should write or give an oral presentation that applies
creative and critical thinking to the paper. Poster reviews are
another option for a deliverable. Students should conduct the
analysis of the paper's primary sections sequentially. The inquiry
is probably most effective if scaffolding questions are
Note: simply stating an opinion is not
sufficient. Where opinions are presented, they must be
defended with facts and logic.
- Was there an explicit hypothesis? If not, what was the implicit
- How reasonable does the rationale seem? Why does it seem
reasonable or not reasonable?
- What are some alternative ideas that were not
- Does this research seem scientifically important? Does it have
practical importance? Why or why not?
- Is the design adequate? Why or Why not?
- How well do the control groups serve as checks on variables
that could influence results other than what is being tested? Why
or why not?
- Describe the negative control group and its function? Are there
important variables that the control group does not account
- Is there a positive control group? Is one needed? Why or why
- Is double-blind testing needed and used? Why or why not?
- Do the data-collecting approaches or devices seem Appropriate?
Are they sensitive enough for what is being tested?
- What other approaches or devices might have been used?**
- Do the results support the hypothesis or not? How convincing is
- Do you notice anything of potential importance in the data that
was not commented on by the authors?**
- Is the variance in data large enough to suggest that some
variables are not being controlled? What might these be?
- Apart from the statistical effect, what is the magnitude of the
'treatment' effect? Is it large enough to be of much practical
- Summarize how the authors discussed the results in terms of
their original hypothesis.
- Did they point out implications that go beyond the
- What implications did the authors perceive that go beyond the
- Do you perceive any other
- What ideas for future research did the authors generate?
- What ideas for future research do you generate?**
- Note any important information or ideas that were not commented
on by the authors and explain the implications.**
- Does the author state a 'take-home lesson?
- How would you state the take-home lesson?**
**Creative ideas are expected
This kind of exercise is best performed by small student teams
(4-6/group), because the intellectual challenge could exceed the
capabilities of any one undergraduate. This kind of teaching should
have special appeal to the Colleges of Business, Engineering,
Medicine, and Veterinary Medicine, where team learning is already
an accepted practice.
The formalisms of proper collaborative learning will be
explained to professors to assure that the process is optimized. To
summarize the basic principles: teams should be heterogeneous,
every member of the team must have a defined role (our lesson plan
explains the roles for APS), an academic deliverable is required
(in this case, simulated peer review), and a significant portion of
the grade must be a group grade (grading rubric options are
Working as a group confers other special advantages, which we
hope the professor will explain to the students (see "Instructions
to Teams" below). Students may have had bad experiences with group
work in the past (such as non-contributing group members or poor
social dynamics). The professor should explain how this group-work
approach will help students, both in terms of academic ability but
also in terms of social attitudes and skills. Tell students that
they will benefit from this exercise because it:
- Encourages them to think for themselves, yet in
the context of a team effort.
- Authentically reflects how scholarship is usually done these
days - by a team of researchers who bring different knowledge and
skills to the problem. The old days of a lone scientist locked up
in an "Ivory Tower" are mostly gone. Science is now so advanced
that no one scientist knows enough or is smart enough to do it all
alone. Scientists must learn how to work in teams, often with
people who were former strangers, often located in another city or
even another country.
- Rewards them for thinking "outside the box" and generating good
- Requires them to assert their ideas and thinking, but at the
same time to be able to defend them with peers.
- Teaches them to disagree without being disagreeable.
- Teaches them to accept and benefit from legitimate criticism of
- Requires them to contribute to the welfare of the group, rather
than to be selfish
- Allows them the experience of being on a winning team, one that
has learned how to work together, sharing the joy of accomplishing
something none of them could have done alone. They can be proud to
have been an important part of the group.
How to Promote Team Cohesion
- Keep group small (4-6 students/group)
- Assign specific team roles (leader, librarian, editor,
presenter, whip, etc.)
- Group grade
- Require public display of the deliverable (class presentation,
poster, web blog, etc.)
How to Eliminate Malingering
- Promote individual role functions, team identity, bonding,
- Have each individual show the professor a draft of answers to
all scaffolding questions BEFORE the team deliberation begins.
- Include some form of individual grading along with the group
Combining in-class work and out-of-class work on a simulated
peer review is one form of "blended learning." Performance of
out-of-class work by a learning team is most conveniently done in
Internet environments that let team members share the same digital
A very useful part of the simulated peer review is for a team to
learn about various Web 2.0 environments and decide which one they
would like to do their work in. Perhaps their very first task is to
assign Web 2.0 options and have team members report back to the
team how the software works and the pros and cons of its use in
simulated peer review. The advantages of making such a decision
- Team members have an "ice-breaking" activity that helps them
get to know each other and form working relationships
- They learn about available technologies, some of which they
will probably wish to use later for other purposes.
- The team has to think about what information and ideas it needs
to collect, as related to the scaffolding questions, and which
environment would be the most "user-friendly" and Appropriate for
Most of these Web 2.0 systems operate in "the cloud," that is
the software and data remain on a Web server, accessible at any
time from any computer. We will provide guidance about the useful
software collaborative environments through e-mail, Web documents,
Grading of Simulated Peer Review
To be fair, grading team performance requires combining team and
individual grades. One approach is to give a grade for each team,
with all members getting the same grade, and averaging that in some
weighted fashion with an individual grade derived from scores from
team members, content test scores or other related work.
A suggested scheme for integrating team and individual grades is
derived from that used by others (
The process uses anonymous peer evaluations on a form teams fill
out at the end of the semester. Students give each team member a
score based on their contributions to group projects throughout the
course. To stop a student from being too generous to friends, a
limit has to be placed. Multiple options exist. A professor can
insist that every score has to differ by at least 5% (or 10% or
whatever). That way, top performers will always get recognized, as
will bottom performers. Then, individuals can be given a
proportional amount of the group grade as their individual
It may be advisable to give a practice peer evaluation about
one-third or one-half of the way through the semester, so that each
team member can made formative adjustments in their work and
interaction with fellow team members. Low-scoring students are
advised to fix things, perhaps by talking to the group and asking
how to compensate for their perceived weakness. The professor can
always intervene to helping solve group-dynamic problems.
The rubric below can be used for presentations that are written,
oral, or poster. Items and scoring are easily modified. We chose to
put emphasis on items that require critical and creative
What is the problem the research is trying to address?
What is the main question being asked about the problem?
Are there other relevant questions that could be asked?
How did the authors frame the question to make it a testable
Summarize the methods used to test the hypothesis.
What other methods could be appropriate?
Identify the dependent and independent variables.
What was the control? Was it adequate? Why or why not?
Summarize the results obtained.
In what way did the results confirm (or refute) the
What ideas did the results generate for the authors?
What ideas did the results generate for you (not stated by
*Awarded only for exceptional insights.
- Alberts, Bruce. (2009). Redefining science education. Science,
- Raphael, Ellen. (2009). Teaching peer review. The Scientist,
May. p. 24