Should reviews also be evaluated and by whom? Should the influence of a review on the overall assessment depend on the reviewer’s reputation?

Home Forums An open forum to discuss the future of peer review Should reviews also be evaluated and by whom? Should the influence of a review on the overall assessment depend on the reviewer’s reputation?

This topic contains 2 replies, has 3 voices, and was last updated by  Anonymous 3 years, 2 months ago.

  • Author
    Posts
  • #2374 Score: 0

    admin
    Keymaster

    Disclosing the reviewer’s comments enables an evaluation of the review itself. Is this something useful? Who should be able to evaluate a review? The entire research community or only other reviewers of the same research object? Should the reputation of a reviewer matter on the influence of his/her opinion on the overall assessment of a research object?

  • #2468 Score: 0

    ChrisSampson87
    Participant

    An ‘evaluation’ of the reviews certainly already takes place, but in most cases at present this is only by the editor(s). However – as peer-review currently proceeds – the purpose of this evaluation is probably only to ascertain whether or not it is necessary to seek comments from another reviewer. Clearly this is part of the pre-publication review process. I do not think any additional pre-publication review-of-reviews is necessary.

    If review comments were open and could be read following publication, editors’ accountability in carrying out this type of evaluation would increase. This could improve the review process.

    I believe reviews should be open to evaluation following publication. Scholarly output gains a lot of credibility from the label of ‘peer-reviewed’, and it should be possible to evaluate whether such credibility is justified. Such evaluation should be carried out by the scholarly community as a whole; not just those with vested interest.

    It should go without saying that reputation (of author, reviewer or editor) is irrelevant.

  • #2469 Score: 0

    Anonymous
    1 pt

    I agree  with Chris that any evaluation is the job of editors before they accept an article
    for publication. Since all the review comments would be in public domain it
    is their concern to evaluate them or not. There is therefore no need for
    evaluation of review comments at the stage of open peer review process. However,
    others should be allowed to comment on review comments so that no reviewer gets
    away with invalid reviews. The process of evaluation, if introduced as a
    mandatory process, might create vested interests.

You must be logged in to reply to this topic.