- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Faculty Research and Publications /
- Reimagining peer review as an expert elicitation process
Open Collections
UBC Faculty Research and Publications
Reimagining peer review as an expert elicitation process Marcoci, Alexandru; Vercammen, Ans; Bush, Martin; Hamilton, Daniel G.; Hanea, Anca; Hemming, Victoria; Wintle, Bonnie C.; Burgman, Mark; Fidler, Fiona
Abstract
Journal peer review regulates the flow of ideas through an academic discipline and thus has the power to shape what a research community knows, actively investigates, and recommends to policymakers and the wider public. We might assume that editors can identify the ‘best’ experts and rely on them for peer review. But decades of research on both expert decision-making and peer review suggests they cannot. In the absence of a clear criterion for demarcating reliable, insightful, and accurate expert assessors of research quality, the best safeguard against unwanted biases and uneven power distributions is to introduce greater transparency and structure into the process. This paper argues that peer review would therefore benefit from applying a series of evidence-based recommendations from the empirical literature on structured expert elicitation. We highlight individual and group characteristics that contribute to higher quality judgements, and elements of elicitation protocols that reduce bias, promote constructive discussion, and enable opinions to be objectively and transparently aggregated.
Item Metadata
Title |
Reimagining peer review as an expert elicitation process
|
Creator | |
Contributor | |
Publisher |
BioMed Central
|
Date Issued |
2022-04-05
|
Description |
Journal peer review regulates the flow of ideas through an academic discipline and thus has the power to shape what a research community knows, actively investigates, and recommends to policymakers and the wider public. We might assume that editors can identify the ‘best’ experts and rely on them for peer review. But decades of research on both expert decision-making and peer review suggests they cannot. In the absence of a clear criterion for demarcating reliable, insightful, and accurate expert assessors of research quality, the best safeguard against unwanted biases and uneven power distributions is to introduce greater transparency and structure into the process. This paper argues that peer review would therefore benefit from applying a series of evidence-based recommendations from the empirical literature on structured expert elicitation. We highlight individual and group characteristics that contribute to higher quality judgements, and elements of elicitation protocols that reduce bias, promote constructive discussion, and enable opinions to be objectively and transparently aggregated.
|
Subject | |
Genre | |
Type | |
Language |
eng
|
Date Available |
2022-05-27
|
Provider |
Vancouver : University of British Columbia Library
|
Rights |
Attribution 4.0 International (CC BY 4.0)
|
DOI |
10.14288/1.0413707
|
URI | |
Affiliation | |
Citation |
BMC Research Notes. 2022 Apr 05;15(1):127
|
Publisher DOI |
10.1186/s13104-022-06016-0
|
Peer Review Status |
Reviewed
|
Scholarly Level |
Faculty; Researcher
|
Copyright Holder |
The Author(s)
|
Rights URI | |
Aggregated Source Repository |
DSpace
|
Item Media
Item Citations and Data
Rights
Attribution 4.0 International (CC BY 4.0)