Options
Challenges of Designing and Implementing Simulation Models of Peer Review
Date Issued
2019-05-04
Date Available
2019-04-25T06:50:36Z
Abstract
Science relies on peer review. Through this mechanism, manuscripts are selected for publication and grant proposals for funding. However, the processes of peer review do not operate in a vacuum; they reflect the priorities, norms, and practices of the institutions in which they are embedded, such as scientific communities, funding agencies, publishers, and scholarly societies, each with their own perspectives and logics (Bollen et al. 2014; Benner & Sandstrom 2000). Peer review is a multi-level system. At the macro level a funding agency sets its priorities and goals for funding based on national priorities and legal mandates. At the meso level, funding agencies use peer review to select which proposals to fund, but also integrate their own strategic objectives (gender balance, geographical diversity, disciplinary needs for example) into the selection process. At the micro level, individual reviewers and panels bring their own perspectives to bear on the review processes. In particular, the dynamics of meso- and micro-level complexity provides an area of exploration that could benefit from simulation studies for two reasons. Simulation studies help us understand what features of the peer review process emerge from different norms, relationships, attitudes and behaviors of the actors and organizations involved. These methods also allow us to develop and test policy recommendations for the improvement of peer review in these same organizations.
In our own project we started by mapping existing simulation models of peer review and identified knowledge gaps in the literature, then started developing a simulation model to address these gaps. We found that numerous researchers had studied peer review systems by means of formal and computational modeling, such as agent-based models (ABM) (Squazzoni & Takács 2011). We counted 44 papers on simulation models of peer review published since 1969: some were used to compare the efficiencies of alternative peer review systems (e.g. Kovanis et al. 2017); some compared different behavioral strategies of authors, editors or reviewers (e.g. Thurner & Hanel 2011; Squazzoni & Gandelli 2013); some sought the origin of the issues of peer review, such as biases, high costs and inefficiencies (e.g. Righi & Takács 2017).
Other Sponsorship
Science Foundation Ireland (SFI)
Type of Material
Conference Publication
Web versions
Language
English
Status of Item
Peer reviewed
Conference Details
Fourth Agent-Based Models of Organizational Behavior (ABMO4) Workshop: Modeling Organizational and Institutional Complexity, Bolzano, Italy, May 3-4, 2019
This item is made available under a Creative Commons License
File(s)
No Thumbnail Available
Name
Abstract for ABM04 workshop in Bolzano (2019).pdf
Size
93.34 KB
Format
Owning collection
Views
666
Acquisition Date
Mar 19, 2024
Mar 19, 2024
Downloads
149
Last Week
1
1
Last Month
3
3
Acquisition Date
Mar 19, 2024
Mar 19, 2024