(postscript added July-Aug 2011, small edit made Feb 2012)
A quote from pages 162-163 of the Penguin paperback edition of the 2003 pop psychology book The Essential Difference by Prof. Simon Baron-Cohen:
"Sally and I decided it would be good to try to get some quantitative measures of his social understanding and degree of autistic traits, so we asked him to take the "Reading the Mind in the Eyes" Test. Richard scored 25 out of 36. People typically score on average 30 out of 36, so Richard's score was significantly lower than one would have expected."
"Richard" in this quote is the Fields Medal-winning mathematician Richard Borcherds, whose diagnostic consultation with Professor Baron-Cohen to see if he had Asperger syndrome was written about in detail in the book. In the book it is not clear whether or not a clinical diagnosis was made; Baron-Cohen referring to Borcherds as "an example of someone whose AS has not been an obstacle to achievement in his adult life...", but later stating that "he is not currently severe enough in his symptoms to warrant a diagnosis in adulthood...." The Reading the Mind in the Eyes Test is a test that was developed by the Autism Research Centre of the University of Cambridge, which has Baron-Cohen as a Director.
Here is a quote from page 18 of the Icon Books paperback edition of the 2010 feminist pop psychology book Delusions of Gender by Dr Cordelia Fine:
"In the Austrian study, women scored higher than men on the Reading the Mind from the Eyes test. However, the difference was small: women, on average, correctly guessed 23 of the 36 items; men, 22."
Dr Fine was not referring to sick or abnormal men and women here, she was discussing men and women in general, and the study that she cited as the source of these figures was a large study (n=423) of "the general population" of Austria, that was published in 2006. So, the findings of the Austrian study, as reported by Fine, appear to show that a normal score on the Reading the Mind in the Eyes Test (RMET) is much lower than that claimed by Prof. Baron-Cohen back in 2003. The study was a large one, so the findings should be fairly reliable.
Things don't add up when we compare the quotes from the two different books. Baron-Cohen was most certainly wrong when he wrote that a typical, normal score on that test is 30. So far I have had a close look at eight different studies that have involved giving a normal control group the same 36-question version of the Reading the Mind in the Eyes Test (RMET), and in all of those studies the normal control group's mean score was under 30. Baron-Cohen even contradicted his own research findings when he claimed in the book published in 2003 that 30 was a normal score on the 36-question RMET. A study co-authored by Prof. Baron-Cohen that was published in 2001 in an autism journal indicated that a normal score on the RMET for “general population controls” is around 26 out of 36. I have not yet been able to get a hold of the full-text of the Austrian study, but if Fine's account of the scores is correct a normal score on the RMET is nowhere near 30.
Should it be this difficult to find out what a normal score on a test is? Of course it shouldn't! A person should be able to trust a professor when he says that you have scored below normal on a test. With a score of 25 out of 36 in the RMET, and a normal score in the RMET being well under 30 and possibly as low as 22 out of 36 for men, it looks like Richard Borcherd's ability to read facial expressions in the eyes is completely normal, maybe even good, and possibly even better than the average woman's score. But Borcherds is supposed to be autistic, and autistic people are emotion-blind, aren't we? And women have super powers of emotion-reading by virtue of our hormones, don't we? [Why do I feel so confused?] So any woman should be far superior to some autistic male mathematician in a test of identifying facial expressions, surely? Well, maybe not.
Why does this matter? It matters because the idea that autistic people cannot read emotions is an article of faith and also an industry in the world of academic autism research and autism charities. The Reading the Mind in the Eyes Test has been cited as a test of empathizing, by virtue of it being regarded as a test of reading facial expressions, and this test has been used very widely in researching possible social deficits in people who have a variety of different medical or psychiatric conditions unrelated to autism. The popularity of this test (which presumably researchers have to pay a licensing fee to the ARC to use) I guess might be the result of the belief that it has been a useful and valid instrument in identifying autistic deficits. The idea that an inability to read emotions is a core deficit of autism has been widely believed and widely sold. Teaching autistic kids how to read facial expressions has been marketed as a therapy for autism, and I’m sure many parents of kids who have been given an autism diagnosis have bought products that are promoted as useful in this area believing that they are making an important investment in their child’s future wellbeing. Baron-Cohen’s Autism Research Centre has been marketing a DVD aimed at parents of autistic kids: “This BAFTA nominated children's animation features vehicles with emotions and has been shown to help children with autism improve in emotion recognition.” As Baron-Cohen has explained himself in an interview, the development of this DVD was funded by British taxpayers to the tune of half a million pounds, but the licence is now owned by the ARC. Free copies of this DVD have been distributed to UK families with autistic kids, but it is also for sale, and some of the profits are given to the autism charity Autism Speaks, which is much despised by many autistic adults and family members of autistic people. The ARC has also been marketing for a number of years a very expensive CD-ROM that features actors acting out (acted, fake, fake-looking) emotions which is supposed to be educational for autistic people. The author of this work is listed as Simon Baron-Cohen. Prof. Simon Baron-Cohen has also written a book titled Teaching Children with Autism to Mind-read which covers facial expressions.
The idea of an autistic deficit in reading facial expressions is the basis of an industry that turns over a lot of money, and when this amount of cash is sloshing around, the numbers should add up. But they don’t add up, and I’m not done with investigating this matter. So I’ll soon be jumping onto my broom and flying on down to my favourite university library to check out the full text of the Austrian study of the RMET first-hand, and I’ll let you know what I find.
P.S. July-August 2011
I've completed my meta-analysis of studies that have used the 36 item RMET on a normal control group or a normal placebo group. I have not yet found the time to write a lengthy discussion of interesting stuff that I've found, but you can see the figures that I've calculated from my findings at the article below. Yes, there is something strange about the German version of the RMET, but it is also true that a normal score on the RMET is lower than 30.
I have found that a normal score on the 36 item Reading the Mind in the Eyes Test, as created and revised by Baron-Cohen's team, is 26.03061. This would make the score in this test of the mathematician Richard Borcherds of 25 out of 36 COMPLETELY NORMAL. Does this mean that he can't really be autistic? No. I have yet to complete a similar review of studies to find out what the average score of autistic study subjects is on this test, but one figure that has been given is 21.9, which is only about four points below the mean score for non-autistic study subjects, and is certainly not evidence of a serious deficit in facial expression reading as a general feature of autism. The Reading the Mind in the Eyes Test cannot be used to diagnose autism, and it is too unreliable to use to draw any conclusions about individuals.
So, really, what is a normal score on the Reading the Mind in the Eyes Test?http://incorrectpleasures.blogspot.com/2011/06/so-really-what-is-normal-score-on.html
References
Fine, Cordelia (2010) Delusions of Gender: The Real Science Behind Sex Differences. Icon Books, 2010.
Baron-Cohen, Simon (2003) The essential difference. Penguin Books, 2003.
Martin Voracek and Stefan G. Dressler (2006) Lack of correlation between digit ratio (2D:4D) and Baron-Cohen’s “Reading the Mind in the Eyes” test, empathy, systemising, and autism-spectrum quotients in a general population sample. Personality and Individual Differences. Volume 41, Issue 8, December 2006. P. 1481-1491.
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V9F-4KMYFM7-1&_user=10&_coverDate=12%2F31%2F2006&_rdoc=1&_fmt=high&_orig=browse&_origin=browse&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=3924796307d73f3a34f02f8b20f47e19
Baron-Cohen S, Wheelwright S, Hill J, Raste Y, Plumb I.(2001) The "Reading the Mind in the Eyes" Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Child Psychiatry, and Allied Disciplines. 2001 Feb;42(2):241-51.
http://www.ncbi.nlm.nih.gov/pubmed/11280420
Reading the mind in the eyes (test)
http://glennrowe.net/BaronCohen/Faces/EyesTest.aspx
Autism Research Centre
http://www.autismresearchcentre.com/arc/default.asp
Good catches ... looking forward to more of your research . How nice to see that someone has decided to look behind the curtain in The land of OZ. (Love the unintended pun, wish I could draw cartoon of SMC as the Wizard .)
ReplyDeleteNice metaphor, Krex.
ReplyDeleteFunny, my husband and I just got done taking that test here http://glennrowe.net/BaronCohen/Faces/EyesTest.aspx
ReplyDeleteWe both scored 22--making us on the border of normal, according to the scores of this particular "test". We tended to disagree with the test on 1 face. at least.
One more thing..."the transporters" dvd is a shameless take-off of Thomas the Tank Engine, and friends.
ReplyDeleteObviously, Thomas IS a very useful engine. He's also been the favorite of auties for years.
It all seems so opportunistic...but maybe it's just me..........
Well, R. B. based on the evidence that I have at hand presently, it appears that your test scores are normal for Austria, a nation full of autistic people. I guess this interpretation makes sense when one considers that Wittgenstein, Mozart, the writer Thomas Bernhard, the poet Franz Grillparzer, and Hans Asperger are all Austrians. It must be one interesting place.
ReplyDeleteCorrection - they were all Austrians.
ReplyDeleteFrance has it's share of auties today. Only in Toulouse is it a cash cow. I figure it's because they say, "There's nothing wrong with that kid. He's just like us!"
ReplyDeleteThe French are so rude, you know...(I'm French, by the way...)cold and have no theory that anyone else is better than them.
...... if you haven't seen it already , just wanted to share....http://biologyfiles.fieldofscience.com/2011/05/autism-lupron-geiers-and-what-can.html
ReplyDeleteScientist would attest, that the best psychopaths fake empathy to manipulate people ....so why do some "scientist" seem to imply that autistics are psychopaths ...we suck at lying, faking emotion or manipulating any body .
Oh, krex...you must be one of the non-scary psychopaths. You know, the ones that don't lie or manupulate people. You're an abberation.But you're still a psychopath, because if you aren't metally ill, then we've got to come up with a new label for children. Autism MUST be a mental illness.
ReplyDeleteBecause,we say so.
Psych = soul/essence....path=where something leads....does sound scary .
ReplyDeleteWhat "they"=(people who call themselves experts),should be scared of is autistics who are not so blinded by emotion, denial and unearned respect for "offical titles" that they believe everything they are told without any faith in their own ability to use logical reasoning to reach their own conclusions.
I'm sure there is a new title for "those we dare to look behind the curtain"....just needs to have a nice Latin term to make it sound all sciency and official .
"Qui audemus respicere velum syndrome"
ReplyDeleteIt's a bit of a mouthful!
The colloquial term for this syndrome is "troublemaker".
I downloaded the Voracek study and had a look at the results. According to the Voracek study the mean RMET score for males is 22.35 with a standard deviation of 4.52. The standard deviation, for those not statistically inclined, is the average difference between the individual scores and the sample or population mean. Any individual score within the mean plus or minus one standard deviation is considered "normal" and clinical distinctions are usually reserved for people who score outside the mean plus or minus two standard deviations (roughly 2.5% of the population if normally distributed).
ReplyDeleteSo, we can say this:
1) Any male with a score above 17.83 is perfectly typical.
2) The clinical threshold for males, as measured by the RMET, is any score below 13.31.
I should point out that each question on the RMET is a four-option multiple choice. In order to achieve a score of 13.31 all one has to do is be able to rule-out one of the four options and then guess randomly among the remaining three answers. With this information and having looked at the RMET I do not conclude that the RMET is useful in diagnosing a deficit in facial reading.
That's pretty amazing. The only problem with your argument is that that study used the/a German version of the RMET, and it seems to give lower scores than the original English language version. There are lots of different versions of this test, and they aren't comparable, another factor making research using the RMET quite meaningless.
ReplyDeleteI'm not particularly impressed with Baron-Cohen's work either (I mean Simon, I love Borat). I'm not autistic, and I consider myself pretty good at reading people. I scored 30 out of 36. Some of the ones I got wrong were pretty dubious IMO, but this sort of thing is also going to be a little subjective, obviously. There is one where I still insist I'm right, so secretly I believe I got 31.
ReplyDeleteI didn't have a problem with that test, but his survey analysis seems to be laden with confirmation bias. Perhaps his comment about Richard Borcherds is a case in point - the truth is Richard scored normal (slightly better, in fact) on the test.
He ran a different survery on "empathising" and "systematising". The latter word I consider rather clumsy. Anywho, I study mathematics (this is how I stumbled on this) and it turns out he tested mathematicians from Cambridge (1st year undergrads, probably a fairly nerdy bunch) who scored 21.4 vs 16.4 for the general population (32+ is supposed to indicate a possibility of being autistic).
But looking at the survery, two of the questions directly concerned having an interest in mathematics. One was "are you fascinated by numbers and patterns?" and the other was something similar. But hang on, that eliminates about 2 points (most people hate maths) from the gap of 5 already!
Don't believe anything you have heard in some study in psychology until you have read the survey. In this case, my regard for it's conclusions fell very quickly.
Another irritating thing, whilst I see the sense of the empathising quotient, something that I forgot to mention, is Baron-Cohen's "systemising quotient".
ReplyDeleteWhen referring to high scores (SQs), he repeatedly uses the phrase "good at", as in "good at systemising". But most of the questions on the survey are things like "do you collect stamps?", "do you keep your fridge neatly organized?". However one answers those questions, it doesn't mean you are good at *anything!*
Secondly, the word "systemising" is hopelessly vague, which handily means it can come to mean almost anything (or at least, anything that isn't empathising). I am sure both mathematics and stamp-collecting come under "systemising" (most things do). Personally I am interested in mathematics and don't give a shit about stamp collecting. All that connects these two things ("systemising" is frankly too vague to count) is that nerdy people tend to be attracted to both.
Dan, I think your points are very valid criticisms of Baron-Cohen's work. Regarding the bias towards characterizing mathematicians as low E and high S people, I'm reminded of Cordelia Fine's great book. I recall she devoted a fair chunk of her book to the many forces reinforcinig the stereotype that females can't do maths. This is a link to the book's website:
ReplyDeletehttp://www.cordeliafine.com/delusions_of_gender.html
My analysis of the RMET found that the weighted average normal score was 26.03061, so the famous mathematician's score of 25 can be regarded as normal. The difference of one cannot be counted as meaningful, as it has been noted that the test-retest reliability of one foreign version of this test is poor (Hallerback et al 2009) and this most likely appies to the original version too. "For these reasons, the test must never be used for confiming or refuting a diagnosis of an autism spectrum disorder."
ReplyDeleteTo be fair to Baron-Cohen, at the scoring key for the RMET in the appendix of his 2003 book The Essential Difference SB-C gives a range of 22-30 as the range of a "typical score", but with such a big range, it raises the question of the usefulness of this test. It's strange that he specified 30 as a typical score in the RMET in the main body of the book, and he shouldn't have been using this test at all in the context of an individual diagnosis.
ReplyDeleteI inadvertantly ran into a comment that made me think again about the faces test.
ReplyDeleteThe pictures are paid actors. Perhaps there is something in the actors face that shows insincerity or an undisclosed private pain (just examples) that a highly detail oriented mind would pick up on that a typical NT mind would overlook.
I've found NT's to be kind of clueless when it comes to certain aspects of another's character. I mean, they catch the fluff stuff that really doesn't matter, except to grease the social wheels.
One face in particular I found highly confusing, and so did my husband. Why? Was it the most apparent dis-harmony of spirit versus facade?
I completely agree. There is a real problem with using images of acted expressions and actors for this test. Actors act like actors. They go to special actor school to learn this stuff.
ReplyDeleteThis study shows how much of an issue it is the difference between facial expression of acted emotion and genuine emotion:
http://affect.media.mit.edu/pdfs/11.Hoque-Picard-FrustratedSmile-FG.pdf
One of the authors of this paper was featured in this recent interesting New Scientist article:
http://www.newscientist.com/article/mg21128191.600-specs-that-see-right-through-you.html?full=true
Interesting stuff. I got 4 out of 4 on the "frustration vs delight" test.
ReplyDeleteAlthough the software identifies correctly only about 64% of the expressions correctly. I think I could do a little better than that.
Lili, it is curious that you should mention Cordelia Fine, (I had a look at that) women in maths and such. I know it sounds hard to believe, but most of the mathematics undergraduates at universities in Europe are women.
ReplyDeleteIn Britain the percentage of male maths uni students is 63%. France is 60%. USA is 57-58%. Scandinavia (Norway, Denmark, etc.) is about 60%. However Germany, Italy, Spain, Poland, Portgual and other countries have at least 50-60% female. The English language seems to be one denominator. Germany has a lot of female maths students in particular. Poland is probably the next biggest.
To say this is due to genetic factors in the various countries would be daft. Sacha should stick that in his pipe and give it a good smoke.
Where's this test Dan? I want to have a crack at it!
ReplyDeleteDan wrote:
ReplyDelete"I know it sounds hard to believe, but most of the mathematics undergraduates at universities in Europe are women."
If this is true, someone should tell S B-C, and someone should also tell Cordelia Fine. Maybe people would argue that these stats only reflect attitudes and not aptitudes. But a person can't develop skill and expertise in an area that they have never had a go at. This is apparently a main criticism of the trendy theory of "multiple intelligences", that it confuses motivation with innate talent. Researchers still seem to be arguing about the origins of savantism - motivation or innate talent, is synaesthesia or autism an important element, or all of the above?
Um, the test was in one of the links you provided! It's not really advertised as a "test", but it has the pictures of people and then the answers in the acknowledgements.
ReplyDeleteOh, yes there are test images in the journal paper that I linked to in these comments.
ReplyDeleteI did lousy - got 1 out of 4 right!