This past week, I had the opportunity to read Jonah Lehrer’s Imagine –  the book that’s part of the controversy about plagiarism and fabrication in Lehrer’s writing. Imagine has been pulled by its publisher as a result of that controversy, so pointing out additional problems with it may now be somewhat redundant. (The copy I read came from my public library.) Nevertheless, after reading it, I want to outline the problems I found with the referencing in the book – an area which other commentators have also raised concerns about.

I have a professional interest in how scientific research findings get communicated to general audiences, but as a reader I’m in some disbelief that a book with this many obvious problems ever got published as it is.  The fact that it did goes beyond the ethical debates around Lehrer’s own conduct, and should make publishers question their editorial processes and where they choose to invest resources in a project.

Referencing in non-fiction writing is important. It’s important because:

• it gives factual support to the writer’s claims;
• it demonstrates that the writer is familiar enough with existing knowledge about the subject to be able to make an informed contribution of new knowledge;
• it tells readers who want to learn more about the subject where they can find other sources of information; and,
  

  An example of a reference list

• it gives credit where credit is due for work by other writers or researchers.

Accurate referencing is particularly important for a book like Imagine, which positions itself as an investigation of the “science of creativity”. Since no single author or researcher could possibly cover this huge subject on their own, they’re going to have to draw on other sources for at least some of their material – and those sources need to be acknowledged.  

Imagine is written for a popular (non-scientist) audience, so obviously it shouldn’t be held to the same rigorous standard of referencing as, say, a scientific research article in a peer-reviewed academic journal. However, the book does include a reference list (p. 255-263), so to some extent it sets its own standard that referencing is expected. If a book is structured to include a reference list, it’s reasonable for the reader to expect that everything in the book that should be referenced is going to be referenced, and is going to be referenced accurately.

Before I outline what I found, I’ll present a few disclaimers/boundaries. I am not trained as a neuroscientist, so I can’t tell whether the neuroscience research described in Imagine is accurately reported or referenced, and I can’t address the allegations of selective “cherry-picking” from that body of research.  Instead, I checked the reference list while I was reading the book, and I marked points in the text where I thought there should have been a reference – and there wasn’t – or where I felt that a referenced topic was close enough to my own expertise that I could reasonably assess whether the referenced material was accurately reported.

The first problematic reference in Imagine is within its first few pages, on page xvi of the Introduction:  “A recent survey of psychology papers published between 1950 and 2000 revealed that less than 1 percent of them investigated aspects of the creative process”. Clearly, this statement should have a reference, since it reports factual material from a secondary source. But there is no reference included in the book’s reference list.  So I ran a search using five EbscoHost reference databases (PsycInfo, PsycArticles, Academic Search Premier, Business Source Complete, and ERIC) that include psychology journals, as well as searching Google Scholar – and I could not find a published article that fit all of the identifiers of the study as described in the sentence. The sentence contributes to setting up the premise of the book:  i.e., “how creativity works” hasn’t been extensively researched, so the book’s contribution is to examine this important yet overlooked topic. Given this premise, the lack of a reference for this study is, in my opinion, a serious omission.

Other problems that I found relating to referencing published academic research:

• Page 140 includes a discussion of research on trends in scientific production. The referenced article is this one. However, from the description of the article’s contents and findings, it appears that the article being discussed is actually this one.
• Page 141 contains a discussion of an “epic study of nearly every musical produced on Broadway between 1877 and 1990, analyzing the teams behind 2,258 different productions”. The study in the referenced article looked at musicals produced from 1945 to 1989, and examined 474 musicals with 2,092 participants.
• A discussion starting on page 162 describes a research study on the effects of dissent on group decision making, and explains how subjects were placed in groups and asked to answer questions, with a “plant” in some groups shouting a wrong answer. The referenced study investigates the impact of stable or changing group membership on a group’s creativity. It does not report on an experiment structured like the one described.

Three other academic studies that were mentioned did not have references.

There are also problems with incomplete or inaccurate references to other types of sources. For example:

• Keith Johnstone, a theatrical improvisation teacher, is quoted directly on page 103. The source of the quote is not referenced. A Google search indicates that Malcolm Gladwell uses the same quote in his book Blink. I do not have a copy of that book, so I could not determine if a source was identified there.
• Page 108 contains a discussion of how Rolling Stones guitarist Keith Richards came up with the guitar riff for the song Satisfaction. There is no reference for the direct quotes, although there’s contextual information that makes the source relatively easy to find. However, there is a discrepancy between the content of the direct quotes in the source transcript and the content of the direct quotes in Imagine:

Imagine: “There, in some sort of ghostly version, is the opening of a song.”
NPR transcript: “There, in some sort of ghostly version, is (singing) da, da, da, da, da – I can’t get no satisfaction.”

• Footnote 3 on page 101 describes the “cinematic improv” of actor Mike Myers in a scene in an Austin Powers film (the Austin Powers film containing the scene is not identified, but it’s fairly easy to discover through Google that it’s the first one). However, there’s no reference for the assertion that “Myers made it up on the spot”. According to this interview with the film’s director, the basis of the scene was spontaneous, but what appeared in the film was not constructed quite in the way that Imagine suggests it was.

And finally, I found four factual statements (e.g. on page 180: “the United States, for example, is 81 percent urbanized”) that were not referenced, along with two instances of discrepancy between the information in the text and the information in the reference. There are also extended discussions of three topics – the development of Silicon Valley, the creative process at Pixar, and Shakespeare’s creative inspirations –  which only have general references to secondary sources and/or to author interviews, making it difficult to determine which material is from the secondary sources and which material is the author’s own work.

All of these problems – in addition to the other problems in Imagine that have already been identified – seriously make me question how this book ever got published as it is.

In Imagine’s acknowledgements (p. 265-267), Lehrer thanks an editor, a number of readers who reviewed drafts of various parts of the book, and a copy editor. It’s entirely possible that some of those readers looked at conceptual or structural issues, and thus missed any referencing problems.  I don’t know the specific details of the editorial or production process that Imagine went through – but I find it disturbing that apparently there was no point in the process when the references in the reference list were cross-verified with the text, or when unreferenced material was identified and flagged for correction.

I am not sure what lesson the publishing industry will take away from the saga of Imagine, other than being very careful with any subsequent work by its author. But as this commentator indicates, the publishing industry has put itself at risk for these kinds of catastrophes by cutting back or eliminating fact-checking resources. I suppose I fall into the particular sub-category of reader that Willingham describes as having “an eye for detail” – but, in fairness, for many of the referencing problems I identified in Imagine, I was able to find the correct information in less than a few seconds through Google. And using a research database to search journal content is not that much different from running a Google search. Neither kind of search is particularly time-consuming, highly skilled, or prohibitively expensive.

Hopefully, another lesson of the Imagine story for publishers will be that investing in fact-checking is worth the cost – not only to avoid the financial and reputational consequences of uncorrected errors, but also to be sure that authors are fulfilling their part of the responsibility for the accuracy of their work.

UPDATE: A writer has alleged that a quote in Imagine was borrowed from a piece he wrote, which he discovered because Imagine did not reference the source of the quote. The complete article is available here.