An unexpected benefit of mask-wearing is being able to say things without being heard. That came in handy a few weeks ago at my local public library, when I uttered “what in the actual f***?” at the completely unexpected sight of a new Jonah Lehrer book.

As you might remember, Lehrer’s writing career derailed nearly 10 years ago when numerous instances of plagiarism and inaccuracy were uncovered in his books and magazine articles. Two of his books were withdrawn from sale by their publishers, and most of the publications he wrote for dropped him as a contributor and removed his discredited articles from their websites. Lehrer gave a controversial speech of apology and co-operated with author Jon Ronson for a chapter in Ronson’s book on public shaming, but since then his writing has mostly been seen on his own blog and in two books that slipped by without much notice.

Mystery, the new book, is indeed mysterious in that it seems to have sprung out of nowhere. I couldn’t find any online news about a new Lehrer book on the way, and there doesn’t seem to have been much advance promotion for the book. So, being intrigued by that additional layer of mystery, of course I read it.

I posted about Lehrer’s previous problems several times on this blog, including an analysis of his now-withdrawn book Imagine which became a featured post on WordPress and attracted thousands of readers. I don’t say this to be boastful, but to be forthright about the reality that I’m familiar with the controversial history of Lehrer’s writing – and that inevitably colours how I look at his current work. I didn’t deliberately go through Mystery looking for things to pick on, but I have to be truthful and say that there were several places where information in the book just didn’t read or feel right. When I encountered those places, I did additional research on the information that Lehrer presented.

The book follows much the same format as Lehrer’s previous work: posing a question and then bringing in scientific ideas and research to provide an answer. Lehrer’s strength as a writer is his descriptive skills: providing evocative details and quotes without bogging down the narrative. The fundamental problem with Mystery, though, is that it doesn’t hang together as a whole. I guess the question that drives the book is “why people want to understand things that are different or unusual”, but the answer to that question seems pretty self-evident: they want to know what’s going on. Under that broad umbrella question there’s a whole range of stories, most of which are good reads on their own, but I didn’t come away from the book feeling that it posed a clear question or that it answered whatever question it posed.

However, as someone who does social science research, I was irritated that Lehrer continues to quote single studies as proof of something definitive or counter-intuitive. Study results that go against generally accepted knowledge can be incredibly important, because they can encourage people to think differently about a topic. But a single study generally isn’t considered to prove anything unless subsequent studies come up with similar results.

Lehrer also seems to retain his habit of selectively choosing studies that make his point. In one chapter, provocatively titled “The Power of Comic Sans”, he discusses a study whose results indicated that when students read classroom materials presented in unfamiliar fonts, they remembered the material better.  This effect is called “disfluency”. This research topic is not my area of expertise, so I certainly don’t claim to know more about it than Lehrer does.

But when I Googled “disfluency and retention” I came across this 2018 study. It claims that “evidence for this disfluency effect, however, has been mixed, suggesting possible moderating factors” – in other words, there may be individual characteristics or circumstances that affect how or whether the material is retained – and this meta-analysis (an analysis using data from other studies) indicating that “there was no effect of perceptual disfluency on recall”. (There’s also a newer article claiming that the methodology of the meta-analysis is problematic.) Clearly, this is an area of research without definitive answers and with multiple perspectives, so to summarize its findings as (my paraphrase) “Comic Sans makes kids pay attention” is misleading. Lehrer also habitually describes almost every study he cites as “recent”, when many of them were published more than a decade ago. It’s entirely possible that the results of the cited study have been refined or disproven since then.

Similarly, elsewhere in the “Comic Sans” chapter, Lehrer discusses the famous 1960 Volkswagen “Lemon” car ad. He explains that the ad was successful because it upended conventions of what car advertising should look like, in addition to promoting a small foreign car when bigger domestic cars were the norm in North America. However, what he completely omits to mention, and I don’t understand why, is that another ad earlier in the same ad campaign – the “Think Small” ad – has been more widely hailed as the truly groundbreaking ad.

Another example: Lehrer discusses a study conducted by two psychologists and a magician – the study is described on page 9 of this article by the magician –  in which people were shown videos of magic tricks, and then asked if they wanted to see more tricks or find out how the trick they saw was done. Lehrer describes a single study in which “subjects [watched] a series of videos documenting impressive magical feats, such as making a girl levitate” (p. 76), and says that the participants in the study were more interested in seeing more tricks. But from the description in the article I’ve linked, it appears there were two separate studies, not one study. In the first, the subjects watched a single video, of “an appearing helicopter illusion“, and then were given the choice of seeing more tricks or learning the trick’s secret. Sixty percent wanted to see another “amazing magic trick”.

In the second study, the subjects watched one of three different videos – including the levitation illusion that Lehrer describes – with each illusion filmed from a different perspective. The subjects were then given the choice of rewatching the video  or guessing how the illusion was done, which is very different from being told how it was done. In that study, the subjects’ choice to rewatch or guess seemed to depend on which of the three illusions they saw. Lehrer confuses the outcome from the “helicopter” study with the “levitation” study, but still claims that people were more interested in the magic: a claim that is only supported by the results of one of the two studies (it could also be debated whether 60% is accurately described as “most”).

As in his previous books, Lehrer occasionally rewords a direct quote to fit his purposes. A footnote on p. 90 reads: “As [Gertrude] Stein put it, ‘If a reader doesn’t know that a question is a question when he reads it, then a question mark can’t tell him.’” Anyone familiar with Stein’s work will recognize that this quote doesn’t sound like the abstract way she usually wrote, and, indeed, the original quote, from her book Lectures in America, is “A question is a question, anybody can know that a question is a question and so why add to it the question mark when it is already there when the question is already there in the writing.” Admittedly that wording is dense to get through, so a rewording or paraphrasing could be helpful to readers. But Lehrer presents the rewording as a direct quote from Stein, which it clearly isn’t.

None of these problems are fatal enough to sink the entire credibility of the book. And to his credit, Lehrer does provide a full reference list of his sources, so he appears to be mindful of the reality that his work is going to be scrutinized – although, if he is truly set on redeeming his history, it seems odd to include How We Decide in his author biography on Mystery’s dust jacket (that is one of his books that was withdrawn from publication for having questionable content). But as Daniel Engber said in pointing out multiple problems in Lehrer’s last book, “Science journalism can go wrong in many ways, very few of which require significant acts of fraud.” From my own exploration of just some of the information presented in Mystery, Lehrer’s work still has problems with accuracy, and still needs to be viewed with caution and with a healthy sense of skepticism.