publishing as systematic?

Thanks to Amy Buckland for sending “Effects of Editorial Peer Review: A Systematic Review” my way. It’s from 2002 and was cited in a blog by Heather Morrison.

True, it’s an old study for me to blog about, but I was happy that Amy thought of me and I was amused by the study. Its authors looked at previous studies and concluded that more studies were necessary before a conclusion could be reached about the positive impact of blind review by referees.

I’m not surprised that there was nothing concrete showing the benefit of peer review. (Anyone know if there is now?) Is there anything systematic about publishing? How do you even go about showing the benefits? Compare different versions of manuscripts?

Peer review is about people, as is acquisitions, as is copyediting, as is design. Even though biomedical journals contain real data and high-level theory, we’re still dealing with people, people with fixed ideas about how information should be presented, biases, jealousies, and preferences that have more to do with themselves than with the work they’re reviewing.