So, really, what is a normal score on the Reading the Mind in the Eyes Test?

A while ago I noticed some interesting and potentially important discrepancies regarding scores reported by researchers for the Reading the Mind in the Eyes Test ...... (the rest of introduction to be written sometime) What I have done here is to do my own meta-analysis of all of the eligible studies that I can find that include a normal control group or a group of healthy subjects in a normal, non-treated condition doing the Reading the Mind in the Eyes Test (RMET) (revised 2001) with 36 items. I have included some studies that used foreign language versions of this test with 36 items, but I have noted them and analysed their data separately. The point of all this is to establish with some authority what is a normal score on the RMET, and to investigate possible biasing effects of different versions and possible confounding influences like intelligence, socio-economic background or education levels. What I found was that at least one foreign version was probably easier than the original English version, that control groups consisting of university students tend to do better on the tests than control groups that are more representative of the general public, and that a normal score on this test isn't much higher than scores reported elsewhere as typical of autistic study subjects. Many questions must be asked about the RMET and the theories about empathy and autism that have informed the creation of this test, which has been very widely used by medical, psychiatric and psychological researchers.

Search method

Searched PubMed, the private ScienceDirect journal paper database, the search form for journal publications at the Autism Research Centre and the online archives of the journals Psychological Medicine and Personality and Individual Differences for terms such as “RMET” combined with “reading”, and “reading the mind in the eyes test” and “reading the mind in the eyes” or simply “reading”. I really have no idea how much of the relevant literature in psychology journals that isn’t covered by PubMed I’ve missed. If any readers know of any published studies that meet the criteria listed below, but which are not included here, I’d appreciate if you’d let me know the details.

Eligibility criteria and characteristics

Journal papers with a publication date before the publication date of the revised version of the test, 2001, were excluded.

The journal paper must include reporting of one or more studies that involved testing a control group of normal healthy adults or adolescents or a placebo group of normal healthy adult or adolescent study subjects with the revised Reading the Mind in the Eyes Test as described in the 2001 study PMID: 11280420 with 36 test items.

Reports of studies of children or reports of studies that used the child version of the RMET were excluded.

Reports of studies that used foreign language versions of the RMET test are included, but the version noted.

Reports of studies of timed and also untimed RMET testing are included, but timing is noted.

Reports of studies that used versions of the RMET test that had more or less than 36 items, or did not specify item number were excluded.

A mean score correct or incorrect or a mean percentage correct or incorrect for the RMET must be given for the control group or the normal healthy placebo group.

Any control group that has evidently been used in more than one study can only be included once.

Potentially eligible studies but not available or readable

PMID: 12101592 Senju et al 2002 in Japanese

PMID: 20677440 Tylec et al 2010 paper in Polish

PMID: 20712172 Nietlisbach et al 2010 HC=16 RMET negative finding re RMET

Eligible studies

PMID: 11280420 Baron-Cohen et al 2001 divided into 3 groups community N=122 (M=55 F=67) M=26.0 F=26.4 total mean=26.2 uni students n=103 (M=53 F=50) mean=28.0 IQ116 n=14 mean=30.9 graph 3 groups separately

PMID: 12559146 Richell et al 2003 Controls=18 non-psychopathic male prisoners screened RMET revised 36 (4 items excluded) mean=26.3

PMID: 15145468 Craig et al 2004 HC=16 (M=11 F=5) mean IQ 110.25 RMET 2000 36 mean=27.63

PMID: 16403703 Havet-Thomassin et al 2006 HC=17 (M=15 F=2) screened matched Uneducated RMET 36 French version Mean=27.6

PMID: 16916074 Meyer & Shean 2006 N=146 (M=51 F=95) BUT reduced to N=142 (outliers disc) mean=28.85

PMID: 16979876 Irani et al 2006 HC=10 screened matched uni=based RMET 2001 score est 80% =28.8

PMID: 17137561 Domes et al 2007 HC=30 all male screened RMET 2001 36 German version? Placebo mean=69.4 correct = 24.984

PMID: 18058213 Kaland et al 2008 HC=20 IQ122.7 RMET child and also adult 36 item tests used. adult test mean=28.10

PMID: 18774263 Hirao et al 2008 HC=20 (m=10 F=10) screened matched R-H educated IQ111.2 RMET 2001 36 not timed mean=67.6% =24.3

PMID: 19173104 Pardini, Nichelli 2009 HC=120 (sex ratio roughly =) screened uni educated sample RMET 2001 36 validated Italian version 4 mean scores for diff =no age groups 26.79 25.3 23.5 21.6 mean=24.3

PMID: 19470361 Smeets et al 2009 subjects=64 (M=32 F=32) (German?) uni students screened for physical illness RMET 2001 36 German version? male controls N=16 mean=0.79 low cort N=8 mean=0.83 high cort N=8 mean=0.80 female control N=16 mean=0.79 low cort N=8 mean= 0.84 high cort N=8 mean=0.71 weighted mean for all groups= 0.792 = 28.5

PMID: 19934046 Rodrigues et al 2009 GG phenotype n=47 uni students RMET 36 20.55% incorrect mean=28.6 correct

PMID: 19460187 Fertuck et al 2009 HC=25 part uni-based screened matched RMET 2001 36 mean=25.00

PMID: 19545579 Euteneuer et al 2009 HC=23 matched screened average edn, prob community sample RMET 2001 36 German version mean=19.91

PMID: 20026084 de Aachaval et al 2010 HC=20 RMET 2001 36 mean=27.3

PMID: 20452061 Schimansky et al 2010 HC=39 (roughly equal sex ratio) middle-class RMET 2001 36 mean=25.5

PMID: 20471104 Gooding et al 2010 HC=110 uni students, screened RMET 2001 36 timed mean=27.76

PMID: 20503068 Kirchner et al 2010 HC=21 (M=15 F=6) edn 13.8 years IQ 110.1 mean=25.1

PMID: 20538499 Bodden et al 2010 N=21 (M=15 F=6) screened educated RMET 63.6% =22.9

PMID: 20589713 Valla et al 2010 all subjects normal N=144 (m=65 f=79) uni students, screened RMET 2001 36 M mean=27.08 F mean=27.27 total mean=27.18

PMID: 20591417 Harrison, Tchanturia, Treasure 2010 HC=90 (female=90) screened est. IQ=113.27 years of edn.=15.94 RMET 36 untimed positive study 80.33% correct = score of 29

PMID: 20811770 Ahmed & Stephen Miller 2011 HC=123 (M=34% F=56%) uni-based pool RMET 2001 36 mean=27.28

PMID: 20919762 Girardi, Macpherson, Abrahams 2011 HC=20 (F=5 M=15) screened RMET 36 est. HC mean 25 correct

PMID: 21147235 Brune et al 2011 N=23 prob community average edn RMET 36 German version mean=19.91

PMID: 21205056 Kornreich et al 2011 HC=25 (M=17 F=8) RMET revised 36 French version mean=23

PMID: 21420177 Wolkenstein et al 2011 HC=20 screened RMET 2001 36 assumed to be German version %correct=65.56 =mean score=23.60

PMID: 21596443 Gooding & Pflum 2011 normal uni-based group n=68 (M=49 F=19) est high IQ RMET 2001 36 timed mean=27.2

Uzefovsky et al 2011 placebo group n=39 all male RMET 36 placebo group 9.74 errors =26.26 correct [This study report was part of a paper that included a number of reports of studies]

doi: 10.1016/j.paid.2010.06.010 Cook & Saucier 2010 subjects=88 (F=42 R-H=81) screened uni psych students RMET 2001 36 mean M=26.14 meanF=27.26 mean=26.7

doi:10.1016/j.paid.2006.06.009 Voracek & Dressler 2006 all subjects normal N=423 (F=217 M=206) looks like a genuine community sample (psych students excluded!) RMET 2001 36 German translation means male 22.35 female 23.31 mean=22.8

Ineligible studies

PMID: 11912109 Gregory et al 2002 RMET 1997
PMID: 14982254 Dorris et al 2004 RMET children’s version 28 item
PMID: 15919556 Dziobek et al 2005 RMET shortened 24 item version
PMID: 16477515 Golan et al 2006 no scores found for RMET
PMID: 17765214 Bull et al 2008 RMET 25 item
PMID: 18155447 Kettle et al 2008 no total means found
PMID: 18633782 Chapman et al 2006 RMET child version
PMID: 19048440 Shur et al 2008 RMET 1999 42 item
PMID: 19370436 Hallerback et al 2009 used 28 item and 24 item tweaked Swedish versions of the child version
PMID: 19451755 Szily, Keri 2009 no total score or % for RMET found
PMID: 19828142 Muller et al 2010 only 20 RMET photos used
PMID: 20102669 Harrison, Sullivan, Tchanturia, Treasure 2010 suspect that same control group as used for PMID: 20591417, don’t use (re-use) positive study
PMID: 20452047 Adler et al 2010 no clear report of RMET scores found
PMID: 20457166 Castelli et al 2010 not all 36 RMET used
PMID: 20545452 Geraci et al 2010 test Italian version 37 items
PMID: 20570111 Gibson et al 2010 no score for RMET found
PMID: 20889365 Tsuruya et al 2011 20 item Japanese version RMET
PMID: 21151817 Preisler et al 2010 reported RMET 2001 as having 40 questions
PMID: 21423444 Pincus et al 2010 Subjects did test in a scanner – not conditions comparable to normal testing
PMID: 21515640 Gunther Morr et al 2011 used adapted version of child version of RMET
PMID: 21641964 Luminet et al 2011 HC placebo group divided into LA and HA groups that scored differently in tests, but could not find total numbers for these groups.
doi: 10.1016/j.paid.2010.03.027 Ali & Chamorro-Premuzic 2010 no clear report of RMET scores found

Guidelines for judging elite or community control groups

If the control group are described as university students or undergraduates then that control group is in the elite group.

If a university is mentioned in conjunction with recruitment of controls then that control group is an elite control group.

If the control group are described as employees or are recruited through a place of work then that control group is in the elite group.

If the control group are described as having mean IQ or estimated mean IQ of 105 or above then that control group is in the elite group.

Mean years of education reported for control group can be used to categorize control group.

Clues as to social class can be used to categorize control group.

Prisoners are not an elite control group.

If there is specific reference to the control group being representative of the general population, or a community-based control group or excluding university students, then that control group is a community control group.

Separate control groups with differing characteristics used in the same study can be divided into elite and community categories if separate mean scores are given.

Weighted Means of:

all eligible studies 26.03061 total study control subjects 2046

French version of test only 24.86190 total study control subjects 42

definitely German version of test 22.69170 total study control subjects 466

Definitely non-German version 26.99245 total study control subjects 1486

non-elite control groups 23.61746 total study control subjects 603

non-elite control groups non-German studies 26.36306 total study control subjects 157

elite/uni student control groups 27.00085 total study control subjects 110


Many thanks to Handymath.com for the use of their Weighted Average Calculator
http://www.handymath.com/mission.html


Copyright Lili Marlene 2011.