This past week, I had the opportunity to read Jonah Lehrer’s Imagine – the book that’s part of the controversy about plagiarism and fabrication in Lehrer’s writing. Imagine has been pulled by its publisher as a result of that controversy, so pointing out additional problems with it may now be somewhat redundant. (The copy I read came from my public library.) Nevertheless, after reading it, I want to outline the problems I found with the referencing in the book – an area which other commentators have also raised concerns about.
I have a professional interest in how scientific research findings get communicated to general audiences, but as a reader I’m in some disbelief that a book with this many obvious problems ever got published as it is. The fact that it did goes beyond the ethical debates around Lehrer’s own conduct, and should make publishers question their editorial processes and where they choose to invest resources in a project.
Referencing in non-fiction writing is important. It’s important because:
- it gives factual support to the writer’s claims;
- it demonstrates that the writer is familiar enough with existing knowledge about the subject to be able to make an informed contribution of new knowledge;
- it tells readers who want to learn more about the subject where they can find other sources of information; and,
- it gives credit where credit is due for work by other writers or researchers.
Accurate referencing is particularly important for a book like Imagine, which positions itself as an investigation of the “science of creativity”. Since no single author or researcher could possibly cover this huge subject on their own, they’re going to have to draw on other sources for at least some of their material – and those sources need to be acknowledged.
Imagine is written for a popular (non-scientist) audience, so obviously it shouldn’t be held to the same rigorous standard of referencing as, say, a scientific research article in a peer-reviewed academic journal. However, the book does include a reference list (p. 255-263), so to some extent it sets its own standard that referencing is expected. If a book is structured to include a reference list, it’s reasonable for the reader to expect that everything in the book that should be referenced is going to be referenced, and is going to be referenced accurately.
Before I outline what I found, I’ll present a few disclaimers/boundaries. I am not trained as a neuroscientist, so I can’t tell whether the neuroscience research described in Imagine is accurately reported or referenced, and I can’t address the allegations of selective “cherry-picking” from that body of research. Instead, I checked the reference list while I was reading the book, and I marked points in the text where I thought there should have been a reference – and there wasn’t – or where I felt that a referenced topic was close enough to my own expertise that I could reasonably assess whether the referenced material was accurately reported.
The first problematic reference in Imagine is within its first few pages, on page xvi of the Introduction: “A recent survey of psychology papers published between 1950 and 2000 revealed that less than 1 percent of them investigated aspects of the creative process”. Clearly, this statement should have a reference, since it reports factual material from a secondary source. But there is no reference included in the book’s reference list. So I ran a search using five EbscoHost reference databases (PsycInfo, PsycArticles, Academic Search Premier, Business Source Complete, and ERIC) that include psychology journals, as well as searching Google Scholar – and I could not find a published article that fit all of the identifiers of the study as described in the sentence. The sentence contributes to setting up the premise of the book: i.e., “how creativity works” hasn’t been extensively researched, so the book’s contribution is to examine this important yet overlooked topic. Given this premise, the lack of a reference for this study is, in my opinion, a serious omission.
Other problems that I found relating to referencing published academic research:
- Page 140 includes a discussion of research on trends in scientific production. The referenced article is this one. However, from the description of the article’s contents and findings, it appears that the article being discussed is actually this one.
- Page 141 contains a discussion of an “epic study of nearly every musical produced on Broadway between 1877 and 1990, analyzing the teams behind 2,258 different productions”. The study in the referenced article looked at musicals produced from 1945 to 1989, and examined 474 musicals with 2,092 participants.
- A discussion starting on page 162 describes a research study on the effects of dissent on group decision making, and explains how subjects were placed in groups and asked to answer questions, with a “plant” in some groups shouting a wrong answer. The referenced study investigates the impact of stable or changing group membership on a group’s creativity. It does not report on an experiment structured like the one described.
Three other academic studies that were mentioned did not have references.
There are also problems with incomplete or inaccurate references to other types of sources. For example:
- Keith Johnstone, a theatrical improvisation teacher, is quoted directly on page 103. The source of the quote is not referenced. A Google search indicates that Malcolm Gladwell uses the same quote in his book Blink. I do not have a copy of that book, so I could not determine if a source was identified there.
- Page 108 contains a discussion of how Rolling Stones guitarist Keith Richards came up with the guitar riff for the song Satisfaction. There is no reference for the direct quotes, although there’s contextual information that makes the source relatively easy to find. However, there is a discrepancy between the content of the direct quotes in the source transcript and the content of the direct quotes in Imagine:
Imagine: “There, in some sort of ghostly version, is the opening of a song.”
NPR transcript: “There, in some sort of ghostly version, is (singing) da, da, da, da, da – I can’t get no satisfaction.”
- Footnote 3 on page 101 describes the “cinematic improv” of actor Mike Myers in a scene in an Austin Powers film (the Austin Powers film containing the scene is not identified, but it’s fairly easy to discover through Google that it’s the first one). However, there’s no reference for the assertion that “Myers made it up on the spot”. According to this interview with the film’s director, the basis of the scene was spontaneous, but what appeared in the film was not constructed quite in the way that Imagine suggests it was.
And finally, I found four factual statements (e.g. on page 180: “the United States, for example, is 81 percent urbanized”) that were not referenced, along with two instances of discrepancy between the information in the text and the information in the reference. There are also extended discussions of three topics – the development of Silicon Valley, the creative process at Pixar, and Shakespeare’s creative inspirations – which only have general references to secondary sources and/or to author interviews, making it difficult to determine which material is from the secondary sources and which material is the author’s own work.
All of these problems – in addition to the other problems in Imagine that have already been identified – seriously make me question how this book ever got published as it is.
In Imagine’s acknowledgements (p. 265-267), Lehrer thanks an editor, a number of readers who reviewed drafts of various parts of the book, and a copy editor. It’s entirely possible that some of those readers looked at conceptual or structural issues, and thus missed any referencing problems. I don’t know the specific details of the editorial or production process that Imagine went through – but I find it disturbing that apparently there was no point in the process when the references in the reference list were cross-verified with the text, or when unreferenced material was identified and flagged for correction.
I am not sure what lesson the publishing industry will take away from the saga of Imagine, other than being very careful with any subsequent work by its author. But as this commentator indicates, the publishing industry has put itself at risk for these kinds of catastrophes by cutting back or eliminating fact-checking resources. I suppose I fall into the particular sub-category of reader that Willingham describes as having “an eye for detail” – but, in fairness, for many of the referencing problems I identified in Imagine, I was able to find the correct information in less than a few seconds through Google. And using a research database to search journal content is not that much different from running a Google search. Neither kind of search is particularly time-consuming, highly skilled, or prohibitively expensive.
Hopefully, another lesson of the Imagine story for publishers will be that investing in fact-checking is worth the cost – not only to avoid the financial and reputational consequences of uncorrected errors, but also to be sure that authors are fulfilling their part of the responsibility for the accuracy of their work.
UPDATE: A writer has alleged that a quote in Imagine was borrowed from a piece he wrote, which he discovered because Imagine did not reference the source of the quote. The complete article is available here.
Great analysis. I absolutely appreciate your “lessons learned” for the publishing industry — now if only the editors and publishers choose to review books with as keen an eye for detail as you’ve done here, we should be in good shape!
Fascinating article! It is quite shocking that this happened, and eye-opening indeed.
“Hopefully, another lesson of the Imagine story for publishers will be that investing in fact-checking is worth the cost – not only to avoid the financial and reputational consequences of uncorrected errors, but also to be sure that authors are fulfilling their part of the responsibility for the accuracy of their work.”
Not going to happen. You make many fair points, but the commercial/trade publishing business does not play by any of the rules of academia. Authors who do not have (or didn’t once have) a huge reputation like Lehrer’s get extremely low advances — some as low as $5,000 or $10,000 — in return for which they are expected to research, write and revise up to 100,000 words. Oh and fact-check them.
I’ve commercially published two non-fiction books with a lot of original work and some use of secondary sources. The first has endnotes, index and bibliography, the second none of these. Like all news journalists trained in legacy media, I do fully attribute all material used that I did not personally gather. But my editor and copy editor would not typically fact-check nor would I expect them to. Some authors hire and pay their own fact-checkers, yet another cost they must carve out of their (low) advance which some of us actually live on with few other sources of income.
The model is tilted in favor of cutting corners to save costs. The “reputational consequences” seem to be nil. The lying/cheating author will, we all know, resurface. His or her editor will be fine. His or her agent will be fine. There ARE no consequences…unlike a newspaper or magazine story which, if you press the editor, will print a correction in a later issue and which can, and sometimes will, fire a staffer and never again use that freelancer.
Thanks for your comments. To clarify, when I spoke of the “reputational consequences”, I was referring primarily to the consequences for the publishing company of releasing a flawed product, not necessarily to consequences for the author. But I can’t agree with you that “there ARE no consequences”, at least in this particular case; as indicated by some of the stories linked in the post, Lehrer has lost several long-term contracts as a result of the problems with his work.
I’d also point out that “the rules of academia” and the practices of trade/commercial publishers are not as different as you suggest. Academic publishers (journals and books) also have differing degrees of fact-checking rigour; some review everything very closely, some don’t check at all. And in either situation, whether the primary responsibility for fact-checking lies with the author or with the publisher, readers have a right to expect accurate and complete material.
I wasn’t really aware that it was the pub’s responsibility to fact check (seriously, I’m not being flip.) It would seem to me the onus for responsible fact usage and truthfulness should be on the author. I agree that readers should be able to expect accurate material, but as a responsible reader, I also take what I see and read with a grain of salt most times. Now it seems like you can’t trust anything you don’t see for yourself. Suppose everything had been properly sited (the fact that references were sited at all might lend a false sense of legitimacy to the work,) how do we know the work cited was accurate or on the level? Maybe I am just a cynical person…
Wow. This is a really powerful piece. It is amazing that “fact checking” is something that seems to be an area that can be cut back. Ridiculous.
As a writer I pride my self on the research that I have done. I enjoy the watchful eye that follows me. Without this, where is the integrity of the written piece or the publication venue that it appears in?
I left publishing years ago, partly because publishers were doing everything possible to relegate tasks to computers so they could get rid of expensive people, and the work was becoming increasingly clerical and boring. Whenever I saw something in a manuscript that I knew was wrong, or at least wondered about, I would double check (and this was before the Internet, so it consumed time), but systematic fact checking was expressly discouraged. It always seemed wrong-headed to me — isn’t it important to make sure the author got things right? — but that’s how it went.
Thanks for your diligence in following up on this, even though the book was pulled. This stuff really IS important.
This has definetly changed my life. I am inspired. Bravo
Wow, you indeed did a very thorough research on this one. From your critical analysis, i did learnt something from you. Now whenever i read something important or interesting from a book, i will do some research and not just read and believe whatever it says. Thank you!
It’s so interesting to me that these things keep happening. I write for UC Berkeley’s newspaper, the Daily Californian, and just the other day, an entire post was taken down for self-plagiarism: http://www.dailycal.org/2012/09/12/but-you-looked-different-on-facebook/
I wonder why no one’s checking anymore.
Intelligence, good writing and integrity do not always go together. We would be fools to think that Lehrer is the only one to have been negligent with the rigours of providing a complete and accurate bibliography. Translators of non fiction and proofreaders of academic papers and theses need to ensure vigilance in this regard too. I have no time for lies, however pretty or well.written they are! Thank you for your systematic, methodical post.
The reality of commercial publishing (as Caitlin Kelly points out above) is that publishers do not have the resources and returns these days to justify close-focus third-party fact checking. As a non-fiction writer myself, i know that the onus is on the writer to ensure their material is verified to the best of their ability; and nobody, probably, is going to follow that up in any detail – even though the book still goes through the usual publisher processes involving copy-editing, line-editing and multiple proofing. Sometimes (and I have had this) a well-meaning editor at the publisher end will actually introduce referencing errors – I once had an editor strip out about half the content of my footnotes, because they were ‘too long’. I had them all put back again.
We have to draw distinction between inadvertent or even egregious referencing errors, and any deliberate deception by the author.
Thanks for sharing this with me! Fascinating. I can’t believe I didn’t know about all this controversy surrounding it when I read it. That does explain why sometimes I found his “evidence” kind of hard to swallow. 🙂