Why brain-based lie detection is not ready for "prime time"

We are in a new and interesting legal world. Although to date, no US court cases have used brain-based lie detection techniques as evidence, several cases have sought such evidence and settled out of court. fMRI is the most frequent type of brain-based lie detection technology, with two companies, Cephos and No Lie MRI providing this service in the legal domain. There have also been attempts made to use EEG for deception detection. Notably, such a technique was used in part to prosecute a young woman for murder in India in 2008.

I am far from the first to point out that this technology is highly exploratory and not accurate enough to be used in the court of law. My goal here is to outline a good number of the reasons this is the case.

9. We do not know how accurate these techniques are. Although the two aforementioned companies boast lie detection accuracy rates of 90%+, these cannot be independently verified by an independent lab as the methods used by these companies are trade secrets. For example, there are few peer-reviewed studies of the putative EEG-based marker of deception, the P300, and most come from the lab that is commercially involved with a company trying to sell the technique as a product. Interestingly, an independent lab studying the effect of countermeasures on the technique found an 82% hit rate in controls (not the 99% accuracy claimed by the company), and this was reduced to 18% when countermeasures were used!

8. In the academic literature, where we do have access to methodology, we are limited to testing typical research participants: undergraduate psychology majors (although see this). For a lie detection method to be valid, it would need to be shown as accurate in a wide variety of populations, varying in age, education, drug use, etc. This population is not likely to be skilled in deception as a career criminal might, and it has been shown that the more often one lies, the easier it is to lie. Most fMRI-based lie detection techniques are based on the assumption that lying is hard to do, and thus requires the brain to use more energy. If frequent lying makes lying easy, then it could be the case that practiced liars don't have this pattern of brain activity.
     Although a fair amount has been made lately about WEIRD subjects, participants in these studies are actually beyond WEIRD: they are almost exclusively right handed, and predominantly male.

7. Along this same line, the "lies" that are told in these studies rarely have an impact on the lives of the student participants. Occasionally, an extra reward is given if the participant is able to "trick" the system, but in the real world, with reputations and civil liberties at stake, one might imagine that one might do a better job at tricking the scanner. However, being instructed to lie about a low-stakes laboratory situation is not the same as the high-stress situations where this technology would be used in real-life. Occasionally, a study will try to ameliorate this situation by using a mock crime (such as a theft) as the deceptive stimuli. However, these are also of limited use as participants know that the situation is contrived.

6. Like traditional polygraph tests, it is possible to fool brain-based lie detection systems with countermeasures. Indeed, in an article in press at NeuroImage, Ganis and colleagues found that deliberate countermeasures on the part of their participants dropped deception detection from 100% to 30%. Most studies of fMRI lie detection have found more brain activation for lies than truth, suggesting that it is more difficult for participants to lie. However, is this still the case with well-rehearsed lies? What about subjects performing mental arithmetic during truth to fool the scanner?
    
5. A general lack of consistency in the findings in the academic literature. To date, there are ~25 published, peer-reviewed studies of deception and fMRI. Of these studies there are at least as many brain areas implicated in deception, including the anterior prefrontal area, ventromedial prefrontal area, dorsolateral prefrontal area, parahippocampal areas, anterior cingulate, left posterior cingulate, temporal and subcortical caudate, right precuneous, left cerebellum, insula, putamen, caudate, thalamus, and various regions of temporal cortex! Of course, we know better than to believe that there is some dedicated "lying region" of the brain, and given the diversity of deception tasks (everything from "lie about this playing card" to "lie about things you typically do during the day"), the diversity of regions is not surprising. However, the lack of replication is a cause for concern, particularly when we are applying science to issues of civil liberties.

4. An additional issue surrounds the fact that many of these studies are not properly balanced. In other words, participants are instructed to lie more or less often than they are instructed to tell the truth.

3. There is a large difference between group averages and finding deception within an individual. Knowing that on average, brain region X is significantly more active in a group of subjects during deception than during truth does not tell you than for subject 2 on trial 9 than deception was likely to occur due to the differences in activation. Of course, some studies are trying to study this level of analysis, but right now they are the majority.

2. Some things that we think that are not true are not necessarily lies. Most of us believe we are above-average drivers, and smarter and more attractive than most even when these beliefs are not true. Memories, even so-called "flash-bulb" memories are not fool proof.

1. Are all lies equivalent to the brain?  Are lies about knowledge of a crime the same in the brain as white lies such as "no, honey those pants don't make you look fat" or lies of omission or self-deceiving lies?

My picks for the top studies of 2010

Presented in no particular order, here are the ten studies of 2010 that I found the most interesting. Enjoy!

1. A Wandering Mind is an Unhappy Mind
I wrote about this study here. Authors used an iPhone app to obtain "what are you doing?", "what are you thinking about?" and "how happy are you right now?" data. It turns out that we are thinking about something other than what we are doing about half of the time, and these are the times we are least happy.

2. Electrical Enhancement of Mathematical Ability.
I wrote about it here. The authors used transcranial direct-current stimulation as participants were learning an novel digit vocabulary. The stimulation (if done in the proper direction) facilitated this type of learning.

3. Ego Depletion only Happens if you Believe it will Happen.
I wrote about it here. Ego depletion is the idea that self-control is like a muscle, and if you use too much of it, you need to rest it like a muscle. The authors show here that people who believed in ego depletion had depleted continuous self-control, whereas people who believed that self-control was not a limited resource did not. Critically, giving people one of the two beliefs also changed behavior in the predicted direction.

4. Habit Formation in the Real World.
Have you heard that it takes 28 days to form a new habit? This is a very common notion, but is it true? The authors found that it can take up to 60 days for people to report a new behavior feeling automatic.

5. Vollenweider's Review of Psychadelic Drugs
Everything you ever wanted to know about psychedelics, but were afraid to ask. Good review of neurobiology of these drugs and their potential therapeutic uses in depression, OCD, anxiety, etc.

6. Eye Position Predicts what Number you have in Mind
When asked to generate "random" numbers, people have a hard time being truly random. In this study, people were asked to generate random numbers while their eye movements were monitored. It was found that the number a person generated could be predicted from the position of the eyes: in particular, that eyes in the lower left part of the visual field predicted small numbers and eyes in the upper right predicted large numbers.

7. Willful Modulation of Brain Activity in Disorders of Consciousness
Until this paper, there had been limited reports of patients in minimally conscious states able to perform mental imagery tasks as measured with fMRI. However, we also know that diagnosis of these states is prone to errors. How common is it for minimally conscious patients to be able to follow mental directions? 54 patients were tested, and it was found that only 5 could perform the tasks.

8. Putting Brain Training to the Test
Making software to "train your brain" into being smarter, more attentive, and less forgetful is a lucrative industry, but there was little evidence for or against this kind of training leading to cognitive enhancement. This massive study used over 11,000 participants and found that although participants got better at the tasks in the games, no improvement was found in non-trained real-world tasks. Save your money, folks.

9. Outcome Reporting Among Drug Trials
All clinical trials need to be registered through the NIH at clinicaltrials.gov. This policy was designed to prevent Big Pharma from running near-infinite numbers of trials on a drug, then only reporting ones with positive results. This study examines differences between studies funded by industry, by government and by private organizations. Industry-funded trials had 85% positive results, while government-funded trials had 50% positive results. Humph.
 
10. Prediction of Individual Brain Maturity
 Human brains mature rather late and at different rates. The issue of brain maturity has been key to policy decisions about how to punish adolescent criminals and how to insure adolescent drivers. This study uses fMRI to show how mature an individual brain is. 

Killingsworth MA, & Gilbert DT (2010). A wandering mind is an unhappy mind. Science (New York, N.Y.), 330 (6006) PMID: 21071660

Cohen Kadosh R, Soskic S, Iuculano T, Kanai R, & Walsh V (2010). Modulating neuronal activity produces specific and long-lasting changes in numerical competence. Current biology : CB, 20 (22), 2016-20 PMID: 21055945

Job V, Dweck CS, & Walton GM (2010). Ego depletion--is it all in your head?: implicit theories about willpower affect self-regulation. Psychological science : a journal of the American Psychological Society / APS, 21 (11), 1686-93 PMID: 20876879

Vollenweider FX, & Kometer M (2010). The neurobiology of psychedelic drugs: implications for the treatment of mood disorders. Nature reviews. Neuroscience, 11 (9), 642-51 PMID: 20717121

Loetscher, T., Bockisch, C., Nicholls, M., & Brugger, P. (2010). Eye position predicts what number you have in mind Current Biology, 20 (6) DOI: 10.1016/j.cub.2010.01.015

Monti, M., Vanhaudenhuyse, A., Coleman, M., Boly, M., Pickard, J., Tshibanda, L., Owen, A., & Laureys, S. (2010). Willful Modulation of Brain Activity in Disorders of Consciousness New England Journal of Medicine, 362 (7), 579-589 DOI: 10.1056/NEJMoa0905370

Owen, A., Hampshire, A., Grahn, J., Stenton, R., Dajani, S., Burns, A., Howard, R., & Ballard, C. (2010). Putting brain training to the test Nature, 465 (7299), 775-778 DOI: 10.1038/nature09042

Bourgeois FT, Murthy S, & Mandl KD (2010). Outcome reporting among drug trials registered in ClinicalTrials.gov. Annals of internal medicine, 153 (3), 158-66 PMID: 20679560

Dosenbach NU, Nardos B, Cohen AL, Fair DA, Power JD, Church JA, Nelson SM, Wig GS, Vogel AC, Lessov-Schlaggar CN, Barnes KA, Dubis JW, Feczko E, Coalson RS, Pruett JR Jr, Barch DM, Petersen SE, & Schlaggar BL (2010). Prediction of individual brain maturity using fMRI. Science (New York, N.Y.), 329 (5997), 1358-61 PMID: 20829489

Sunday soundbites: Happy Boxing Day!

YGERC6QPEMCG

Here is an article I wish I'd written: links between the disease semantic dementia and the insomniac memory loss in Gabriel Garcia Marquez' One Hundred Years of Solitude.

PLoS ONE publishes a study flying in the face of placebo theory showing that patients aware of placebo treatment still show a placebo effect. Study is here, and skeptical analysis found here and here.

Andrew Gelman has a great analysis of why summary plots don't show the whole story.

Nature shows the value of networking in science.

Counting calories? You can blame Lavoisier for first understanding the body as a combustion engine, according to Nature's history of nutrition information.

The journal Biology Letters published this paper by a group of 8th graders. It will be freely available online until the New Year.

How much are those Nature papers really worth? A new study examines citation rates of Nature and Psychological Review.

Bad Science has a great "Year in Nonsense" article for the cynical.

Have some fun doing recreational probability with Understanding Uncertainty.

The science of a good first impression, over at Pop Economics.

Why no one bats .299 in late September

This paper shows that people strive for round-number goals, showing evidence from Major League baseball players, high school students taking the SAT, and from laboratory subjects answering hypothetical surveys of behavior.

As can be seen in the figure, baseball players are 4 times more likely to end the season with a 0.300 batting average than a 0.299 average! How does this happen? Players that are at 0.298 or 0.299 are more likely to have at-bats (rather than having a pinch hitter), they are slightly more likely to have hits at those at bats, and once a batter hits the magic 0.300 point, batters often take walks and sit out for pinch hitters.

The SAT takers were 10-20 percentage points more likely to re-take the test if they had an exam ending in -90 (e.g. 1190) than one ending in -00 (e.g. 1200).

Last, the authors gathered laboratory participants and asked them how they would react given certain situations. To give an example situation, imagine running laps around a track and you are getting tired. You have run either 28, 29, 30 or 31 laps (depending on what condition you are in). Do you want to run one more lap? They found that participants in the just under a round number condition (29 laps) were more likely to run one more, and participants in the just over the round number (31) were less likely to do one more.

Are these round number goals rational? In other words, is a baseball player more likely to get a lucrative contract with a 0.300 batting average than a 0.299? Do highly selective colleges have round number cut-offs for admissions? The authors examined data from university admissions that showed no discontinuities in the probability of admission as a function of SAT score, suggesting that such round number goals are not, in fact, rational.

Pope D, & Simonsohn U (2010). Round Numbers as Goals: Evidence From Baseball, SAT Takers, and the Lab. Psychological science : a journal of the American Psychological Society / APS PMID: 21148460

Sunday soundbites: Rocky Mountain High edition

Not neuro-related, but terrifying: 5% of Boston high school students report carrying guns to school, and think that more people carry guns than actually do.

Nature recognizes the role (they don't seem sure what kind, though) of blogs in scientific discourse.

The New York Times has a nice review of Antonio Damasio's new book The Self Comes to Mind.

Psychology Today has a top ten psychology studies of the year article.

In the spirit of top tens, ScienceNow also has a top ten stories of the year.

Nate Silver of the enviably-fantastic Five Thirty Eight, and himself an orthodox Bayesian, gives a Bayesian take on WikiLeaks' Jullian Assange.

Taking advantage of the open scanner hours now that people are leaving for winter break? Neuroskeptic has the prayer for you.

105 Major League Baseball players have theraputic exemptions for ADHD drugs.

Accelerometers on smart phones used to train prosthetic limbs.

Barking up the Wrong Tree presents an article showing that the Flynn effect (the phenomenon of population IQs rising over time) makes it difficult to have an objective standard for retardation when it comes to capital punishment cases.

Roundup of epic visualizations

A map of the world revealed by networks of Facebook friendships.

Play with the New York Times' tool for visualizing census data.

Spatial distributions of tourists and locals in various cities.

How did your schools compare with others within district, state, country?

Asteroid discoveries from the last 30 years.

Sunday soundbites

Weekly fun dataset for hobby-statisticians.

More statistical goodness on diabetes rates and state lines.

Neurophilosophy has an interview with Suzanne Corkin about HM.

Visualize yourself eating to lose weight? A good review of a recent study at NPR.

The Chronicle of Higher Education has a short piece reviewing a paper showing that we find bearded salesmen more trustworthy. (I dunno... facial hair freaks me out....)

Barking up the Wrong Tree reviews a study showing that narcissism scores are on the rise.

Maybe it's because parents are giving their children more unique names.

The Economist has a good interview with Oliver Sacks. (Though shame on the Economist for mixing up "affect" and "effect").

The Neurocritic has a great article on how blogging and peer review interact (or not).

Cool interview with author of recent paper on visual hallucinations.

A good analysis of the "slut gene".

Book review: Addiction: A Disorder of Choice by Gene Heyman

I was apprehensive about reading this book. I was worried that it was going to be a hyper-conservative and moralizing tome. What I found instead was a provocative and well-researched book that I highly recommend to anyone interested in addiction, public policy or psychiatry.

As is clear from the book’s title, Heyman asserts that the dominant paradigm of drug addiction (that it is a “chronic and relapsing brain disease”) is not correct, and that by viewing addiction as the result of a series of willful actions, we have a better understanding of the course of drug addiction and its treatment.

Why is this view controversial? It strikes at the heart of the paradoxical way that we view drugs and drug addiction in the modern US. Heyman explains: “The reasoning behind this view is that if addiction is a disease, then science will soon find an effective treatment for it, as had been the case for many other diseases, but if that addiction is a matter of choice, then the appropriate response is punishment… The core assumption of this viewpoint is that there are but two possible responses to addiction: treatment or punishment”. As a good liberal, I was uncomfortable starting the book because I was worried that Heyman would be blaming the victims of addiction, and potentially creating policy that would take away treatment options, leaving them worse off.

Heyman remains unfortunately agnostic on what ought to be done about drug abuse, but focuses instead on the science and history of drug abuse and drug policy. Here are his central arguments for addiction being a choice:

The same drugs have different effects on people that depend on cultural context

Across time, culture and place, the reactions to the same drug are markedly different. If drug abuse were a biological reflex, this would not be the case. Of course, some of the “cultural contexts” listed by Heyman (smoking versus injecting heroin, for example) can dramatically influence the metabolism of the drug, which is a biological context that can influence receptor binding, etc.

The more interesting piece of this argument was that studying only those in treatment for drug addiction biases many studies of drug addicts. These people have higher co-morbidity with other mental illnesses and are more likely to relapse. For most people, a drug habit runs a natural course, beginning in a person’s late teens to early twenties and ending by age 30. Heyman profiles the majority population of invisible “successful addicts” who start and end drug-taking behavior as free choices.

The right motivations get just about anyone to quit

When people are given enough motivation, nearly all can quit taking drugs. 85% of drug addicted doctors and airline pilots faced with the possibility of passing all future random drug tests or losing their jobs will quit taking drugs. “Change your incentives, change your behavior, change your brain”.

The genetic basis for addiction is not compelling.

Heyman uses religiosity and twin studies to prove this point. Religion is a learned and voluntary behavior, and twins tend to have similar degrees of religiosity. However, the religiosity data only show a 0.3-0.4 correlation between even identical twins, so I agree with Heyman that the argument is a little weak.

The book misses a great opportunity to go into the neuroscience of addiction, as strengthening neural circuits associated with habit, and down-regulating neurotransmitter systems affected by the drugs could be very potent reasons why drug taking is so difficult to stop.

Near the end of the book is an argument of choice behavior, largely based on the very cool research of Drazen Prelec and colleagues. While it takes Heyman a while to set up the background, it is worth following as it has a lot of explanatory power for habitual behavior of all kinds, not just drug addictions. In brief, as the negative consequences of taking drugs accumulate slowly while the positive feeling of taking drugs is immediate (albeit diminishing over time), it takes a long-term perspective to see that drug-taking is a losing prospect in the long-term. The same goes with any bad habit: while just one cupcake will not cause you to gain 20 pounds, the habit of eating cupcakes over time will accumulate those 20 pounds. It takes a strong future orientation to see that the immediate deliciousness of the cupcake is not worth the long-term metabolic cost of the cupcake. Heyman refines his argument here by pointing out that while it is silly to say that someone would choose to be an addict, one becomes an addict from the accumulation of choices. “The point is that one day of heroin does not mean addiction, just as eating desert once does not make one fat. Of course as the days accumulate, the characteristics of addiction emerge, and as the deserts accumulate, fat cells get bigger”

One disappointment is the frequent straw-man argument along the lines of “It is not likely that anyone ever referred to recovery from obsessive compulsive disorder or schizophrenia as ‘going cold turkey’”.  These arguments seem to appeal to the emotions behind addiction rather than the science. Also, cognitive behavioral therapy, which involves the willful changing of one’s beliefs and behaviors, is an effective strategy for combatting other mental illnesses such as OCD, only showing how nuanced the idea of will in mental illness really is.

Sunday soundbites

Does kicking back and watching TV make you happier? Absolutely not. In fact, scheduling your free time might make you happier.

A very pretty statistical visualization.

Interesting Google talk on industry scientists hired by corporations to sow seeds of doubt in the public's mind about science.

Speaking of evil corporate interests, The New York Times has an article about the increasing boldness of ghostwriting in the pharmaceutical industry.

And more Big Pharma shenanigans here.

Nature has a short-but-sweet quip on the value of basic research.

I think that this guy should read that last one.

Freakonomics blog has a piece on how the same study can lead to opposite-sounding headlines in different papers.

This would surely be nominated for an IgNoble if not for the null result: do dogs get contagious yawns from people? Unclear.

More IgNoble fodder: this might actually be the most inspiring paper I've ever read!

Science reviews two feminist critiques of neuroscience.

PLoS has this article about D4 variants associated with sexual promiscuity. I can't wait to see what kind of headlines we see about this one.

What is the real value of effective writing?

This shocking article, written by a man who makes a living writing college papers for other people, has had a lot of mileage around the web lately.

I had two immediate reactions to the essay: “I would love to invite this guy to a dinner party, he sounds really interesting” and “this is just another example of the profoundly broken economics of the American higher educational system”

“In the midst of this great recession, business is booming. At busy times, during midterms and finals, my company's staff of roughly 50 writers is not large enough to satisfy the demands of students who will pay for our work and claim it as their own.”  Stated the pseudonymous Mr. Dante.

Alex Reid wrote about some economic observations from the article. “… it's a little sad that people who are clearly accomplished writers (to be able to produce quickly good academic material across the disciplines) are willing to work for such little pay.”  This is in stark opposition to my own reaction, which was along the lines of “wow, I could increase my post-doc salary by a substantial margin by doing this!”  And recall that despite my whining, my salary is quite reasonable when compared to my adjunct peers in the humanities.  I whole-heartedly agree with Mr. Reid’s assertion that we should really start questioning our paradigms about college education.

To review: our students can’t afford not to get a college degree, and end up paying smart individuals who might otherwise be teaching them if it wasn’t more worth their while economically to pass them through the system. These students are in college because it’s just “what you do” to get a job that doesn’t involve flipping burgers.  What kind of education do people need to have for a typical job? How do we best scale this to the largest number of people?

Of course, this ghostwriting problem is far from limited to the college population. This week’s Nature had this article about “editorial services” that help with everything from experimental logic to type setting that help researchers get work published.  Such a service operates in a massive ethical gray area of authorship ethics – if a service organizes your ideas and suggests a critical control experiment, is that not a unique intellectual contribution?

Although both cases are very different, it is evident that the inability to clearly communicate one’s ideas is a primary barrier to academic and life success, and although we do not compensate teachers for this skill, its value is shown on the black market.