During conference season, when you’re rushing from session to session, peer review is something you often hear about in snatches of conversation as you’re running by. “[Professor X] must have reviewed that paper, otherwise it would have been accepted”. Or “I knew getting in at [Journal Y] was a problem because they don’t like [Theory Z]“.

Peer review can have a really big effect on someone’s academic career, because it determines whether your research ever gets published or presented. In theory, the peer review process in the academic world works like this.  An author sends a article about their research to an academic conference or to an academic journal. The conference organizer or the journal editor removes the identifying information (such as the author’s name and institutional affiliation) to reduce any possibility of reviewers being biased in their assessment by knowing who wrote the article. Then the article is sent to the reviewers, who read it and say whether they think the article should be accepted or rejected, and the conference organizer or journal editor decides whether to accept the article to be published or presented.

In most peer review processes, at least in my field, the author doesn’t know who the reviewers are. The journal editor or the conference organizer does, but he or she is expected to assign the article to reviewers with some expertise or familiarity with the article’s subject. The reviewers’ expertise helps to identify and weed out articles with major problems, like poorly thought out research questions, bad methodology, or weak reasoning. Using multiple reviewers is also supposed to balance out any bias or limitations that might result from relying on the opinion of a single reviewer.

Well, that’s the way it’s supposed to work.

A few weeks ago, researcher and consultant CV Harquail sent out a Tweet about this research article about the peer review process, Continue reading →