-
Canadian Medical Association Journal Feb 1981
Topics: Peer Review; Research; Research Support as Topic
PubMed: 7459784
DOI: No ID Found -
PloS One 2020This paper analyzes the concordance between bibliometrics and peer review. It draws evidence from the data of two experiments of the Italian governmental agency for...
This paper analyzes the concordance between bibliometrics and peer review. It draws evidence from the data of two experiments of the Italian governmental agency for research evaluation. The experiments were performed by the agency for validating the adoption in the Italian research assessment exercises of a dual system of evaluation, where some outputs were evaluated by bibliometrics and others by peer review. The two experiments were based on stratified random samples of journal articles. Each article was scored by bibliometrics and by peer review. The degree of concordance between the two evaluations is then computed. The correct setting of the experiments is defined by developing the design-based estimation of the Cohen's kappa coefficient and some testing procedures for assessing the homogeneity of missing proportions between strata. The results of both experiments show that for each research areas of science, technology, engineering and mathematics the degree of agreement between bibliometrics and peer review is-at most-weak at an individual article level. Thus, the outcome of the experiments does not validate the use of the dual system of evaluation in the Italian research assessments. More in general, the very weak concordance indicates that metrics should not replace peer review at the level of individual article. Hence, the use of the dual system in a research assessment might worsen the quality of information compared to the adoption of peer review only or bibliometrics only.
Topics: Bibliometrics; Humans; Italy; Peer Review; Peer Review, Research; Publishing
PubMed: 33206715
DOI: 10.1371/journal.pone.0242520 -
Acta Medica Portuguesa 2012
Topics: Peer Review; Periodicals as Topic; Portugal
PubMed: 23211193
DOI: No ID Found -
Journal of General Internal Medicine Dec 2011
Topics: Editorial Policies; Forecasting; Humans; Peer Review
PubMed: 21971601
DOI: 10.1007/s11606-011-1901-8 -
Andrology Nov 2016
Topics: Peer Review
PubMed: 28079312
DOI: 10.1111/andr.12301 -
Journal of Comparative Physiology. A,... Jan 2024A frequent complaint of editors of scientific journals is that it has become increasingly difficult to find reviewers for evaluating submitted manuscripts. Such claims...
A frequent complaint of editors of scientific journals is that it has become increasingly difficult to find reviewers for evaluating submitted manuscripts. Such claims are, most commonly, based on anecdotal evidence. To gain more insight grounded on empirical evidence, editorial data of manuscripts submitted for publication to the Journal of Comparative Physiology A between 2014 and 2021 were analyzed. No evidence was found that more invitations were necessary over time to get manuscripts reviewed; that the reviewer's response time after invitation increased; that the number of reviewers who completed their reports, relative to the number of reviewers who had agreed to review a manuscript, decreased; and that the recommendation behavior of reviewers changed. The only significant trend observed was among reviewers who completed their reports later than agreed. The average number of days that these reviewers submitted their evaluations roughly doubled over the period analyzed. By contrast, neither the proportion of late vs. early reviews, nor the time for completing the reviews among the punctual reviewers, changed. Comparison with editorial data from other journals suggests that journals that serve a smaller community of readers and authors, and whose editors themselves contact potential reviewers, perform better in terms of reviewer recruitment and performance than journals that receive large numbers of submissions and use editorial assistants for sending invitations to potential reviewers.
Topics: Animals; Publishing; Peer Review
PubMed: 37318565
DOI: 10.1007/s00359-023-01642-w -
Journal of Digital Imaging Feb 2018One method for addressing existing peer review limitations is the assignment of peer review cases on a completely blinded basis, in which the peer reviewer would create... (Review)
Review
One method for addressing existing peer review limitations is the assignment of peer review cases on a completely blinded basis, in which the peer reviewer would create an independent report which can then be cross-referenced with the primary reader report of record. By leveraging existing computerized data mining techniques, one could in theory automate and objectify the process of report data extraction, classification, and analysis, while reducing time and resource requirements intrinsic to manual peer review report analysis. Once inter-report analysis has been performed, resulting inter-report discrepancies can be presented to the radiologist of record for review, along with the option to directly communicate with the peer reviewer through an electronic data reconciliation tool aimed at collaboratively resolving inter-report discrepancies and improving report accuracy. All associated report and reconciled data could in turn be recorded in a referenceable peer review database, which provides opportunity for context and user-specific education and decision support.
Topics: Automation; Data Mining; Databases, Factual; Humans; Peer Review; Research Report
PubMed: 28744581
DOI: 10.1007/s10278-017-0006-2 -
BMC Medicine Jun 2019Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may... (Review)
Review
BACKGROUND
Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals.
METHODS
Comprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes.
RESULTS
After screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items). We also extracted 2026 statements related to peer reviewers' tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items).
CONCLUSIONS
Peer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors.
Topics: Humans; Peer Review; Periodicals as Topic
PubMed: 31217033
DOI: 10.1186/s12916-019-1347-0 -
Translational Vision Science &... Jul 2020
Topics: Peer Review, Research
PubMed: 32864195
DOI: 10.1167/tvst.9.8.r -
Journal of Insect Science (Online) Jan 2021
Topics: Entomology; Female; Humans; Male; Peer Review; Periodicals as Topic; Sexism
PubMed: 33394046
DOI: 10.1093/jisesa/ieaa141