See all articles from this issue 

The peer-review process is fundamental to the rigor and credibility of social science research. Yet the current model of preparing early scholars (e.g., graduate students, new professionals) to participate in the peer-review process primarily relies on informal training, or in some cases, no training at all. Early scholars can feel insufficiently prepared to engage in the peer-review process, which can create anxiety and uncertainty around the peer-review process and, sometimes, poorer-quality reviews. At the 2018 NCFR Annual Conference, we developed a workshop to serve as an intentional space for early scholars to acquire new skills, facilitate dialogue with professional peers, and cultivate best practices for future reviews.

In advance of the workshop, we reviewed available literature on issues related to, guidelines for, and academic conversations around constructing a peer review. We also requested input from established scholars, journal editors, and professional colleagues across a variety of social science disciplines to gain insights and feedback on the peer-review process. Specifically, established scholars and journal editors contributed their insights about paths to becoming a peer reviewer and their advice for early scholars on what makes a good review. Finally, we compiled examples of real reviews from editors as well as from our own prior publishing experiences in our workshop materials. We used these real-world reviews to provide workshop attendees with annotated examples of “helpful” and “unhelpful” reviews (which were de-identified) and hands-on exercises for small group discussion during the workshop.

The 75-minute workshop was well attended and included a lively discussion during and even following the workshop. Attendees had a variety of publishing and reviewing experience, and half the session was dedicated to roundtable hands-on exercises and small-group discussion. One of the goals of the NCFR Students and New Professionals (SNP) group is to ensure that early scholars have a chance to engage, question, and reflect on a professional topic—this workshop provided a much-needed arena for attendees to ask questions and talk with their peers about the peer-review process. Attendees left the session with new skills, strategies, and resources on conducting academic reviews. The enthusiasm and active engagement with the workshop materials and discussion demonstrated to us the importance and relevance of this topic for graduate students and new professionals. Below, we highlight key ideas and discussion points from the workshop.

Advice for SNP

• Orientation. Often the first questions are, What is a peer review? And how and when do I get my foot in the door? Resources, peer, and faculty mentors can help orient you to the process before preparing to engage in peer reviews.
• Mentorship! Early scholars should first work with a more seasoned scholar; an adviser or trusted graduate mentor may be willing to do a co-review.
• Publish. Once you start to publish, offers to review will happen automatically. Plus, receiving reviews is a good way to learn how to be a better reviewer.

What Makes a Good Reviewer?

• Communicate. Articulate clearly major and minor changes needed to improve the manuscript.
• Keep Big Picture in Mind. You are not a copy editor—stay focused on big ideas. Can you identify a meaningful narrative within the manuscript? Do the methods and results relate to the research questions? Does the discussion connect to the literature?
• Adhere to Rigor. Are the hypotheses or research questions clear? Do methods serve to answer research questions? Are results and methods transparent? Are limitations acknowledged?

Tips of the Trade:

• Submit your review on time! Submitting before the deadline is always preferred. If you are going to be late with a review, contact the editor ahead of time.
• Logistics: Create separate comments for the editor and the author(s). Proofread your review. Read the journal’s guidelines and adhere to the template (if provided).
• Maintain Professionalism. Always be respectful when engaging with author(s) and editor(s). We are all people putting our best foot forward in our various roles.

We encourage SNP to seek out mentorship for training and guidance as you engage in the peer-review process—mentorship that was invaluable to us. For established scholars, we encourage the compilation of resources and the creation of applied exercises for your students to help them enter the world of peer review. Our workshop experience demonstrates that more, not fewer, conversations should be had surrounding the seemingly “black box” of peer review. Below we provide resources that may be helpful. We also suggest the NCFR journals’ reviewer guidelines available through each journal information page (https://www.ncfr.org/journals). 

Resources

Duncan, M. (2018, June 18). The 3 types of peer reviewers. Chronicle Vitae. Retrieved from https://chroniclevitae.com/news/2069-the-3-types-of-peer-reviewers?cid=VTEVPMSED1

Jones, L. (2017). Reviewer guidelines and best practices. Retrieved from https://editorresources.taylorandfrancisgroup.com/reviewers-guidelines-and-best-practice/

Milardo, R. M. (2015). Crafting effective peer reviews. In Crafting scholarship in the behavioral and social sciences: Writing, reviewing, and editing (pp. 147–170). New York, NY: Taylor & Francis.

Voice of Young Scientists. (2016). Peer review: Nuts and bolts. Retrieved from http://senseaboutscience.org/wp-content/uploads/2016/09/peer-review-the-nuts-and-bolts.pdf

Wilson, J. (2017). Episode 5: Making Sense of Science for the Public. [Podcast episode.] Retrieved from https://soundcloud.com/wileysocietyupdates/episode-5-making-sense-of-science-for-the-public