Monday, July 30, 2012

Lili is happy to help by pointing out so many things that you've done wrong

(This post added to Wednesday August 1st 2012and also Thursday August 2nd 2012)

Rothen, Nicholas, Meier, Beat and Ward, Jamie (2012) Enhanced memory ability: Insights from synaesthesia.  Neuroscience & Biobehavioral Reviews. Received 19 February 2012. Revised 7 May 2012. Accepted 15 May 2012. Available online 23 May 2012.
["In Press, Uncorrected Proof"]

This paper was presented to the 13th International Multisensory Research Forum at the University of Oxford from June 19-22 2012.

Two of the authors of this paper have apparently written a chapter titled “Synaesthesia and memory” for the upcoming academic book Oxford Handbook of Synaesthesia.

In a post that I wrote a while ago I asserted that there were many problems with this in press journal paper. It has taken a while but I’ve finally found the time to explain my claim.

There are two major aspects of this journal paper which I find the most objectionable, but there are many things to be annoyed about in this inexplicably sloppy paper. One is the disappointing fact that Rothen, Meier and Ward have joined the crowd of gullible researchers who have discussed Daniel Tammet at length uncritically as a synaesthete memory savant, in the face of abundant and easily-accessible evidence that practically every aspect of the story of Daniel Tammet (formerly Daniel Corney) has been called into question. Daniel Tammet omitted his past as a World Memory Championship (WMC) competitor in 1999 and 2000, and his participation in 2002 as a subject in a study of WMC participants from his autobiography and self-descriptions in media interviews. These omissions are highly questionable and important.  Rothen, Meier and Ward’s ostrich-like ignorance of the evidence for scepticism about Daniel Tammet is highlighted in the way that they have discussed in their paper a study of which it is known that Tammet was one of the subjects, but they show no awareness that he was included in the study, and they starkly fail to note that the findings of that group study directly contradict important aspects of the official account of Daniel Tammet’s cognitive exceptionalities. The authors have made the same mistake that researchers have been making for a long time, in spite of the efforts of myself and others to highlight the fact that Daniel Tammet was one of the subjects studied in the often-cited “Routes to Remembering” study by Maguire, Valentine, Wilding and Kapur published in 2002-3. Rothen, Meier and Ward discuss this study of World Memory Championship participants on page 4 of their paper, and also Anders Ericsson’s commentary on that study, and point out that most of the superior memorizers used the method of loci memory method. The authors mentioned faces as one of the types of memory tasks that were studied in the “Routes to Remembering” study, so I must presume that they are aware that nothing unusual was found regarding face memory in the study. Unfortunately there is nothing in the paper to indicate that Rothen, Meier or Ward had any awareness that Daniel Tammet was one of the people studied in the “Routes to Remembering” study, even though there is ample evidence that he was.  The “Routes to Remembering” study was discussed in a separate section from Rothen, Meier and Ward’s lengthy discussion of Daniel Tammet.

In addition to failing to connect Tammet with the “Routes to Remembering” study, the authors of the review also managed to misrepresent one of the findings from a study of Tammet in a way that obscures one of the starkest contradictions between the official account of Tammet and findings of the “Routes to Remembering” study. The reviewers asserted that in a test of Tammet in a 2007 study “A recognition memory test for faces yielded unremarkable performance”. Not true. When Baron-Cohen and his team studied Tammet in the study published in 2007 in the Journal of Consciousness Studies, Tammet’s memory for faces was tested and the conclusion was that it “appears impaired”. Tammet’s face recognition rate was barely above chance level, while his accuracy in identifying faces as unfamiliar was somewhat better. Tammet’s subnormal performance in face memory in this study is inconsistent with the finding in the “Routes to Remembering” study that the superior memorizers as a group performed better than the normal control group in face memory, and I think their score in the face recognition test was significantly superior to the controls. I could find nothing in the “Routes to Remembering” paper to indicate that any individual in the study showed a deficit in face memory, or any type of cognitive deficit. Rothen, Meier and Ward failed to report this important inconsistency.

In their discussion of a study of Tammet, researchers Rothen, Meier and Ward asserted that “Whereas controls show a benefit of chunking of verbal material in memory tasks, associated with increased activity in lateral prefrontal cortex (Bor et al., 2003; Bor and Owen, 2007), Tammet failed to show this chunking-effect either behaviourally or in terms of neural correlates (Bor et al., 2007). This is consistent with the idea that he is able to impose his own internal organisation on ‘unchunked’ sequences, thereby benefiting less from an externally imposed strategy.” I think this is a confusing and misleading way to present the facts of what was found in the 2007 study of Tammet by Bor, Billington and Baron-Cohen, in which Tammet is generally known to have been the subject but was given the name DT. The results of the digit span task are not presented in a table, and the description of results by Bor, Billington and Baron-Cohen is as clear as mud, but they do assert that “DT showed hyperactivity in bilateral LPFC compared to normal controls” when presented with numerical stimuli, LPFC being the lateral prefrontal cortex. They then go on to explain that this activation pattern can be associated with chunking processes and “it is possible that the over-activity in LPFC in DT reflects chunking processes.” There is no reason to dismiss the possibility that DT was using a conscious memory strategy such as chunking, in fact the evidence indicates that he was, but in a way that was different than the controls, who presumably were not former World Memory Championship competitors, so that should surprise no one, not even Rothen, Meier and Ward.

The authors’ ignorance of Tammet’s past colours every aspect of their review of research about him. On page 13 they cite Tammet’s supposed autistic interest in numbers as an explanation of why he showed some superiority in the digit-span task in contrast with performance by other grapheme-colour synaesthetes. The authors failed to consider the use of mnemonic strategy as a probable explanation for Tammet’s advantage.

It is worth noting that one of the authors, UK synaesthesia researcher Dr Jamie Ward, is a serial offender in terms of writing accounts of Tammet that ignore important information about his past. In Ward’s 2008 popular science book The Frog Who Croaked Blue Dr Ward wrote about Tammet and there is some indication that he had spoken to Tammet. Amongst his discussion of Tammet Dr Ward explained that extraordinary memory feats can be performed with the use of the method of loci, and it is clear from the details in this paragraph that Ward has read the “Routes to Remembering” study by Maguire, Valentine, Wilding and Kapur but disappointingly Dr Ward showed no awareness that Tammet was one of the subjects of that study (Ward 2008 p.113), and Ward showed no scepticism about Tammet’s self-account. Dr Ward also failed to caution the book’s readers that the face memory findings of the “Routes to Remembering” study directly contradict Tammet’s self-account of his face memory ability: “As with other people with autism, he is little interested in faces and he believes that his memory for faces is poor.”

Perhaps it might seem like I'm making too big a deal of Daniel Tammet, but I can't get over the fact that Rothen, Meier and Ward wrote more about Tammet as a single case study of memory superiority associated with synaesthesia than any other case except Shereshevskii, while it is known that both Shereshevskii and Tammet have performed as mnemonists and most likely both used established memory techniques that don't require nor necessarily involve synaesthesia, while there are many who doubt that Tammet is a synaesthete at all, and while completely ignoring at least one well-documented and researched case study of superior memory who apparently is a genuine synaesthete. In addition to failing to note that Tammet/Corney was a study subject in the “Routes to Remembering” study, Rothen, Meier and Ward also failed to note that another of the superior memorizer participants in that study, given the anonymous name of TM, has reported in another document as reporting coloured and flavoured numbers, most likely an experience of synaesthesia. TM (whose real name appears to be Tom Morton) was written about in a 1994 journal paper and in the 1997 book Superior Memory by Wilding and Valentine, and in that book a brief description of TM’s synaesthesia can be found, if you have a sharp eye. Perhaps Rothen, Meier and Ward could be excused for completely overlooking TM as an interesting case study of a synaesthete elite memory performer, even though they have clearly read the paper of one of the studies in which he was a subject. The authors of the “Routes to Remembering” study couldn’t have done less to indicate that TM was included in their study, and to my knowledge Wilding, Valentine and colleagues never explicitly identified TM as a synaesthete. But on the other hand, if an Australian housewife can eventually put the pieces of evidence together to conclude that an interesting case study of memory superiority is also a synaesthete and that he was studied in the “Routes to Remembering” study, then why shouldn’t a team of university academics writing a review paper about memory superiority and synaesthesia also be aware of this very pertinent fact? At the very least, if they had been regular readers of my blog, they would have discovered this interesting fact. 

Rothen, Meier and Ward’s careless and incomplete but state-of-the-art accounts of Daniel Tammet and the “Routes to Remembering” study are my biggest gripes about this paper, but these flaws are certainly not isolated issues. There’s so much more in this paper that is questionable. I’ve only identified issues that have jumped out at me - I have not made an exhaustive check of the facts as presented in Rothen, Meier and Ward’s paper.

On page four of the review paper the authors had already strayed well into questionable territory before they blundered upon Tammet: “However, it is to be noted that rather than expertise, memory deficits are more common in autism which, in contrast to synaesthesia, is a developmental disorder (cf. Happé, 1999).” Only one old journal paper is cited to back up this assertion, but I think it is a claim that requires much greater explanation, support and qualification. It appears that not all autistic individuals display memory deficits and not all studies find memory deficit as a characteristic of autism, and to assert that memory deficits are more common in autism than memory expertise, one would need to have at hand good evidence about the rate of savantism or memory expertise in autism, and that is itself an area of controversy.

On the next page the authors asserted that “More recently, it has been suggested that Shereshevskii may also have had autism which together with his synaesthesia may have been the basis for his exceptional memory (Baron-Cohen et al., 2007; Bor et al., 2007).” Those papers do include discussion of the idea of a combination of autism and synaesthesia as a basis for exceptional memory or savantism, but there’s no speculation about Shereshevskii and autism or Asperger syndrome (AS) in those papers. In the Neurocase paper it is only stated that “(It is unknown if Shereshevsky had AS, as his case predates the recognition of AS.)”

Just about every journalist or researcher who has written about Jill Price, known as AJ in the neuropsychology literature, has made a hash of it. It is no surprise that this review paper conformed to the general pattern of a lack of care. From page five of the review: “AJ reported having synaesthetic mental calendars that she was able to use for virtually perfect autobiographical memory and a perfect memory for world events (Parker et al., 2006; Simner et al., 2009b). However, as with Shereshevskii and Tammet she appeared to train herself to use this system, in this case when she was traumatised by a move from the East to the West coast of the USA at the age of 8 years.” The reviewers make it sound like AJ/Price is very much responsible for and knowledgeable about her own superior autobiographical memory (also known as hyperthymestic syndrome). I’ve read the 2006 journal paper about Price and her autobiography and many other documents about her, and I’ve found nothing to support this idea. Here is a quote from her book The Woman Who Can't Forget: "Perhaps one reason that I remember days so well is that my brain seems to love to organize time. One of the unusual ways it does so, which intrigued the scientists because again it was so unprecedented, is with visual that I just "see" in my mind." Of course, there was nothing unprecedented about mental  number-forms for months and years - Sir Francis Galton published illustrations of this type of synaesthesia in 1880. One matter needs to be set straight; Price/AJ never reported herself as being a synaesthete nor claimed to have synaesthesia. Even though descriptions of her time-line synaesthesia can be found in both her book and the 2006 Neurocase paper about her, neither Price nor the researchers identified it as synaesthesia. It was presented as a mystery in both pieces of writing. One could presume that this incomprehension was not sincere, but maybe one shouldn’t. It seems odd that here the reviewers assert that Tammet is self-trained, an idea that they hadn’t already explored in their review. To get another fact straight about Jill Price; even though the researchers who first wrote about her described her mental representations of time as "mental calendars" they are not calendars at all. Rothen, Meier and Ward have thoughtlessly copied the sloppy description of other researchers, who should have known better, because they had all the facts at hand to see that Price was experiencing a type of "number forms" or number lines, but applied to the representation of time, and they also should have been aware that this is regarded as a type of synaesthesia. Synaesthesia researchers Rothen, Meier and Ward should certainly have been knowledgeable enough about this subject to know that time-space synaesthesia is not the same as a calendar, it is a line or a collection of different lines for different units of time. Price included hand-drawn illustrations of her time-space synaesthesia on pages 30 and 31 of her autobiography. They are completely typical of time-space synaesthesia; linear representations with idiosyncratic forms. 

The reviewers continue, on the subject of Jill Price/AJ: “Moreover, the superior recall was limited to events of interest to her which might suggest some autistic traits. Nevertheless, her memory was superior in general (on the WMS – Wechsler Memory Scale), although her IQ was normal (on the WAIS – Wechsler Adult Intelligence Scale) and other cognitive abilities (executive functioning, language and face recognition) were impaired.” The reviewers failed to mention Price/AJ’s perfect score for face perception in the Benton Face Test, which is apparently a group of tests of facial emotion recognition. This piece of information doesn’t sit well against the picture of autistic tendencies that the reviewers have painted.

I suspect that I could sit here for a week or more writing a full account of everything that is wrong with Rothen, Meier and Ward’s paper, but I think I have made my point. It’s a pity, because I thought there were a lot of interesting ideas presented in their examination of competing theories as explanations of memory advantage in synaesthesia. But elegant theories aren’t worth a cup of spit if they conflict with the evidence, and evidence isn’t much use to science if researchers don’t know about it, or won’t acknowledge it.

Other References

Wilding, John M. and Valentine, Elizabeth R. (1997) Superior memory. Psychology Press, 1997.

Maguire, Eleanor A., Valentine, Elizabeth R., Wilding, John M. & Kapur, Narinder (2002-3) Routes to remembering: the brains behind superior memory. Nature Neuroscience. Volume 6 Number 1 January 2003 p.90-95. Published online: 16 December 2002 doi:10.1038/nn988

Parker, Elizabeth S., Cahill, Larry, & McGaugh, James L. (2006) A case of unusual autobiographical remembering. Neurocase. Volume 12 Issue 1 February 2006. p. 35 – 49.

Baron-Cohen S, Bor D, Billington J, Asher JE, Wheelwright S and Ashwin C. (2007) Savant Memory in a Man with Colour Form-Number Synaesthesia and Asperger Syndrome. Journal of Consciousness Studies. volume 14, number 9-10, September-October 2007, p. 237-251.

Bor, D, Billington, J, Baron-Cohen, S. (2007) Savant memory for digits in a case of synaesthesia and Asperger syndrome is related to hyperactivity in the lateral prefrontal cortex. Neurocase. 2007 Oct;13(5):311-9.


Anonymous said...

Great post.

Is there really noone else on the entire internet who has noticed all the problems with this paper?

Aussie housewives: 1
Scientists: 0


Lili Marlene said...

Thanks Tomas. Mediocrity has done nothing to hold back the progress of this paper and its authors, but we can only hope that criticism might force some changes. The authors have written a chapter on chapter titled “Synaesthesia and memory” for the upcoming academic book Oxford Handbook of Synaesthesia, and the paper was presented to a big international forum at Oxford this June.