Challenges of Designing and Implementing Simulation Models of Peer Review

Files in This Item:
Access to this item has been restricted by the copyright holder until:2019-05-05
File Description SizeFormat 
Abstract for ABM04 workshop in Bolzano (2019).pdf93.34 kBAdobe PDFDownload
Title: Challenges of Designing and Implementing Simulation Models of Peer Review
Authors: Feliciani, Thomas
Lucas, Pablo
Luo, Junwen
Shankar, Kalpana
Permanent link:
Date: 4-May-2019
Online since: 2019-04-25T06:50:36Z
Abstract: Science relies on peer review. Through this mechanism, manuscripts are selected for publication and grant proposals for funding. However, the processes of peer review do not operate in a vacuum; they reflect the priorities, norms, and practices of the institutions in which they are embedded, such as scientific communities, funding agencies, publishers, and scholarly societies, each with their own perspectives and logics (Bollen et al. 2014; Benner & Sandstrom 2000). Peer review is a multi-level system. At the macro level a funding agency sets its priorities and goals for funding based on national priorities and legal mandates. At the meso level, funding agencies use peer review to select which proposals to fund, but also integrate their own strategic objectives (gender balance, geographical diversity, disciplinary needs for example) into the selection process. At the micro level, individual reviewers and panels bring their own perspectives to bear on the review processes. In particular, the dynamics of meso- and micro-level complexity provides an area of exploration that could benefit from simulation studies for two reasons. Simulation studies help us understand what features of the peer review process emerge from different norms, relationships, attitudes and behaviors of the actors and organizations involved. These methods also allow us to develop and test policy recommendations for the improvement of peer review in these same organizations. In our own project we started by mapping existing simulation models of peer review and identified knowledge gaps in the literature, then started developing a simulation model to address these gaps. We found that numerous researchers had studied peer review systems by means of formal and computational modeling, such as agent-based models (ABM) (Squazzoni & Takács 2011). We counted 44 papers on simulation models of peer review published since 1969: some were used to compare the efficiencies of alternative peer review systems (e.g. Kovanis et al. 2017); some compared different behavioral strategies of authors, editors or reviewers (e.g. Thurner & Hanel 2011; Squazzoni & Gandelli 2013); some sought the origin of the issues of peer review, such as biases, high costs and inefficiencies (e.g. Righi & Takács 2017).
Type of material: Conference Publication
Keywords: Peer reviewMulti-level systemPerspectivesLogicsSimulation modelsAgent-based models
Other versions:
Language: en
Status of Item: Peer reviewed
Conference Details: Fourth Agent-Based Models of Organizational Behavior (ABMO4) Workshop: Modeling Organizational and Institutional Complexity, Bolzano, Italy, May 3-4, 2019
Appears in Collections:Information and Communication Studies Research Collection

Show full item record

Google ScholarTM


This item is available under the Attribution-NonCommercial-NoDerivs 3.0 Ireland. No item may be reproduced for commercial purposes. For other possible restrictions on use please refer to the publisher's URL where this is made available, or to notes contained in the item itself. Other terms may apply.