Object familiarity modulates the relationship between visual object imagery and haptic shape perception

“The function of the PMv zone is not clear, although it is the locus of a number of interesting multisensory interactions (Graziano et al., 1997; Ehrsson et al., 2004; Rizzolatti and Craighero, 2004).”

“These activations were corrected for multiple comparisons by the false discovery rate (FDR) method (Genovese et al., 2002).”

“…(corrected p<0.05) was achieved by imposing a threshold for the volume of clusters comprising contiguous voxels that passed a voxel-wise threshold of p<0.05, using a 3D extension (implemented in BrainVoyager QX) of the 2D Monte Carlo simulation procedure described by Forman et al. (1995).”

“Procedures Imagery session Stimuli and tasks: In the VI task (Figure 1a), participants heard pairs of high-imagery nouns, rated 5 or higher on a seven-point imageability scale (Paivio et al., 1968; Gilhooly and Logie, 1980).”

“Subjects used EPs relevant for shape information, initial enclosure and subsequent contour-following (Lederman and Klatzky, 1987), during both HS tasks.”

“For example, the lateral occipital complex (LOC), an object-selective area in the ventral visual pathway (Malach et al., 1995), is also shape-selective during haptic perception (Amedi et al.”

“For example, the OFC may be involved in rapidly collecting gist input to evaluate hypotheses about object identity by generating analogies with existing representations (Bar et al., 2006; Bar, 2007).”

“The function of the PMv zone is not clear, although it is the locus of a number of interesting multisensory interactions (Graziano et al., 1997; Ehrsson et al., 2004; Rizzolatti and Craighero, 2004).”

“For example, the OFC may be involved in rapidly collecting gist input to evaluate hypotheses about object identity by generating analogies with existing representations (Bar et al., 2006; Bar, 2007).”

“Procedures Imagery session Stimuli and tasks: In the VI task (Figure 1a), participants heard pairs of high-imagery nouns, rated 5 or higher on a seven-point imageability scale (Paivio et al., 1968; Gilhooly and Logie, 1980).”

“The IFG zone was probably involved in top-down generation and control of imagery processes (Kosslyn et al., 1993; Ishai et al., 2000; Mechelli et al., 2004) and the Pul/LP thalamic zone could reflect processes of attention-shifting within a visual buffer that maintains visual images (Kosslyn et al.”

“In support of this idea, the LOC is bilaterally active during VI (Amedi et al., 2001, 2005a), while the left LOC is active during imagery of either geometric or material object properties in response to visually presented word-cues (Newman et al., 2005) and also when generating mental images of…”

“All participants were right-handed, based on the validated subset of the Edinburgh handedness inventory (Raczkowski et al., 1974).”

“The function of the PMv zone is not clear, although it is the locus of a number of interesting multisensory interactions (Graziano et al., 1997; Ehrsson et al., 2004; Rizzolatti and Craighero, 2004).”

“…to that used in other studies of the LOC (Amedi et al., 2005a (4mm); James et al., 2002 (6mm); Stilla and Sathian, 2008 (4mm)); and is substantially smaller than that used in many earlier studies (Amedi et al., 2003; Stoesz et al., 2003; Zhang et al., 2004 (all 8mm); De Volder et al., 2001 (15mm)).”

“Also, there was no significant difference in HS accuracy between experiments, although the non-significantly greater accuracy for familiar HS is consistent with findings that haptic identification of familiar objects is both fast (Klatzky and Lederman, 1995) and accurate (Klatzky et al., 1985).”

“The IFG zone was probably involved in top-down generation and control of imagery processes (Kosslyn et al., 1993; Ishai et al., 2000; Mechelli et al., 2004) and the Pul/LP thalamic zone could reflect processes of attention-shifting within a visual buffer that maintains visual images (Kosslyn et al.”

“Recent work suggests two types of VI: object imagery, referring to pictorial representations of objects, and spatial imagery, involving schematic representations of the spatial relations between object parts (Kozhevnikov et al., 2005; Blajenkova et al., 2006).”

“Another argument that has been advanced against a role for VI in HS perception is that congenitally blind people show similar activations (e.g., Pietrini et al., 2004).”

“Although it is theoretically possible that participants might have generated a haptic rather than visual image, vision is well known to be the preferred modality for shape processing, and shape is less salient than material object properties (such as texture) for haptic perception (Klatzky et al., 1987).”

“The IFG zone was probably involved in top-down generation and control of imagery processes (Kosslyn et al., 1993; Ishai et al., 2000; Mechelli et al., 2004) and the Pul/LP thalamic zone could reflect processes of attention-shifting within a visual buffer that maintains visual images (Kosslyn et al.”

“…in the VI task (in both experiments) is ample for generating visual images, as evidenced by the high accuracy rate of our participants and RTs ranging from 1.1-2.0s in various image generation studies (Kosslyn et al., 1988, 1995; Doricchi et al., 1995; Dean et al., 2005; Slotnick et al., 2005).”

“The avIPS focus is close to a retinotopically identified region designated IPS1 (Swisher et al., 2007), while the pIPS focus is in the vicinity of a neighboring retinotopic area designated IPS2 (Swisher et al., 2007) and also of an area involved in processing visual surface orientation, and…”

“Amedi et al. (2005b) report that the Talairach coordinates for the bisensory region of the LOC, termed LOtv, are remarkably consistent, the average x, y, and z coordinates being 51, -56, -7 for the right LOC and -47, -61, -5 for the left.”

“The LOC is a strong candidate for the site of this representation, given its modality-independent engagement during shape perception (Amedi et al., 2001, 2002, 2007; James et al., 2002; Zhang et al., 2004; Stilla and Sathian, 2008).”

“, 1995), is also shape-selective during haptic perception (Amedi et al., 2001; James et al., 2002; Zhang et al., 2004; Stilla and Sathian, 2008) but the underlying reasons for this crossmodal recruitment remain uncertain.”

“Recent work suggests two types of VI: object imagery, referring to pictorial representations of objects, and spatial imagery, involving schematic representations of the spatial relations between object parts (Kozhevnikov et al., 2005; Blajenkova et al., 2006).”

“We did not find the deactivations in auditory and somatosensory cortex previously reported by Amedi et al. (2005a) during VI; instead, these areas were non-selectively active during both the VI and WnW conditions.”

“Activations were localized with respect to 3D cortical anatomy with the help of an MRI atlas (Duvernoy, 1999).”

“We did not find the deactivations in auditory and somatosensory cortex previously reported by Amedi et al. (2005a) during VI; instead, these areas were non-selectively active during both the VI and WnW conditions.”

“The smIPS focus is near a region involved in transforming visual coordinate information into motor programs for reaching (Grefkes et al., 2004): this has been proposed as the human homolog of the macaque medial intraparietal area, MIP (Grefkes and Fink, 2005).”

“The common activations included sites in frontoparieto-cingulate cortex and the LOC, similar to previous reports of HS-specific activity (Bodegard et al., 2001;Zhang et al., 2004;Stilla and Sathian, 2008).”

“We did not observe any systematic differences for familiar and unfamiliar objects, and there is no evidence that object familiarity alters the nature of EPs used for HS perception (Lederman and Klatzky, 1987, 1990), although we are not aware that a direct comparison has been made.”

“The IPS is involved in visuo-haptic processing of both object shape and location (Stilla & Sathian, 2008; Gibson et al., 2008), the main difference being concomitant LOC activity during shape processing but concomitant FEF activity during location processing.”

“parietal operculum in the field termed OP3 (Eickhoff et al., 2007).”

“…involved in top-down generation and control of imagery processes (Kosslyn et al., 1993; Ishai et al., 2000; Mechelli et al., 2004) and the Pul/LP thalamic zone could reflect processes of attention-shifting within a visual buffer that maintains visual images (Kosslyn et al., 1993; Kosslyn, 1994).”

“Also, there was no significant difference in HS accuracy between experiments, although the non-significantly greater accuracy for familiar HS is consistent with findings that haptic identification of familiar objects is both fast (Klatzky & Lederman, 1995) and accurate (Klatzky et al., 1985).”

“D objects, and imagery of object construction (Jancke et al., 2001).”

“…to that used in other studies of the LOC (Amedi et al., 2005a (4mm); James et al., 2002 (6mm); Stilla and Sathian, 2008 (4mm)); and is substantially smaller than that used in many earlier studies (Amedi et al., 2003; Stoesz et al., 2003; Zhang et al., 2004 (all 8mm); De Volder et al., 2001 (15mm)).”

“…in the VI task (in both experiments) is ample for generating visual images, as evidenced by the high accuracy rate of our participants and RTs ranging from 1.1-2.0s in various image generation studies (Kosslyn et al., 1988, 1995; Doricchi et al., 1995; Dean et al., 2005; Slotnick et al., 2005).”

“The 4 mm filter is within the 3- to 6 -mm range recommended to reduce the possibility of blurring together activations that are in fact anatomically and/or functionally distinct (White et al., 2001).”

“VI activated a widespread network of brain regions in occipitotemporal and frontoparietal cortex, including the LOC, consistent with previous studies (Ishai et al., 2000; Wise et al., 2000; De Volder et al., 2001; Amedi et al., 2001, 2005a; Newman et al., 2005).”

“…to that used in other studies of the LOC (Amedi et al., 2005a (4mm); James et al., 2002 (6mm); Stilla and Sathian, 2008 (4mm)); and is substantially smaller than that used in many earlier studies (Amedi et al., 2003; Stoesz et al., 2003; Zhang et al., 2004 (all 8mm); De Volder et al., 2001 (15mm)).”

“Participants heard stimulus pairs that could be either two real, lowimagery words (rated 4 or lower on the seven-point imageability scale), two non-words (pronounceable nonsense syllables that did not form real English words (Gibbs and Van Orden, 1998; Pexman et al., 2001) and which are non-imageable by definition), o…”

“, 2008), together with these various prior findings, we proposed that the role of the IPS in haptic perception of unfamiliar shape may be in the assembly of object shape representations from component parts, with the aid of spatial imagery (Lacey et al., 2009).”

“, 2004), and close to a region implicated in visuo-haptic shape matching (Saito et al., 2003).”

“…to that used in other studies of the LOC (Amedi et al., 2005a (4mm); James et al., 2002 (6mm); Stilla and Sathian, 2008 (4mm)); and is substantially smaller than that used in many earlier studies (Amedi et al., 2003; Stoesz et al., 2003; Zhang et al., 2004 (all 8mm); De Volder et al., 2001 (15mm)).”

“…2005a), while the left LOC is active during imagery of either geometric or material object properties in response to visually presented word-cues (Newman et al., 2005) and also when generating mental images of shape cued by familiar sounds in both blind and sighted individuals (De Volder et al.,…”

“We previously suggested that vision and touch may share a representation of shape, flexibly accessible via both top-down and bottom-up pathways (Lacey et al., 2007a).”

“…in the VI task (in both experiments) is ample for generating visual images, as evidenced by the high accuracy rate of our participants and RTs ranging from 1.1-2.0s in various image generation studies (Kosslyn et al., 1988, 1995; Doricchi et al., 1995; Dean et al., 2005; Slotnick et al., 2005).”

“The companion paper (Deshpande et al., 2009) provides converging evidence for a differential role of VI during HS perception of familiar and unfamiliar objects, based on task-specific analyses of functional and effective connectivity.”

“…HS would involve more finger movements than familiar HS, which could affect the signal-to-noise ratio and also result in greater activation in the superior parietal lobule (SPL) for unfamiliar HS, relating to increased kinesthetic attention and tactile working memory load (Stoeckel et al., 2004).”

“abstract and meaningless, unlike a previous study (Lacey and Campbell, 2006) that used items and components of caving and climbing equipment that showed obvious signs of meaningful, purposive design, but were unfamiliar to the participants.”

“We previously suggested that vision and touch may share a representation of shape, flexibly accessible via both top-down and bottom-up pathways (Lacey et al., 2007a).”

“The VI task was adapted from a neuropsychological study which showed that a visually agnosic patient had deficits in VI (Mehta et al., 1992).”

“…2007), while the pIPS focus is in the vicinity of a neighboring retinotopic area designated IPS2 (Swisher et al., 2007) and also of an area involved in processing visual surface orientation, and referred to as the human homolog of the macaque caudal intraparietal area, CIP (Shikata et al., 2008).”

“…in the VI task (in both experiments) is ample for generating visual images, as evidenced by the high accuracy rate of our participants and RTs ranging from 1.1-2.0s in various image generation studies (Kosslyn et al., 1988, 1995; Doricchi et al., 1995; Dean et al., 2005; Slotnick et al., 2005).”

“LOC activity was more extensive on the left, possibly because imagery was triggered by verbal cues: this accords with left lateralization of LOC activity during visual object recognition when object naming is required (Large et al., 2007).”

“Keywords Multisensory; cross-modal; visual cortex; lateral occipital complex; somatosensory cortex; fMRI Introduction Visual cortical areas are routinely active during touch in normally sighted individuals (Sathian and Lacey, 2007).”

“Because of the sequential nature of haptic processing, associated mental image generation may be easier using categorical rather than coordinate relations (Findlay et al., 1994).”

“We define a multisensory representation as one that can be both encoded and retrieved in more than one sensory modality (Sathian, 2004), as opposed to a visual image which can only be encoded in one modality although its retrieval might be triggered by cues in multiple modalities.”

“…in the VI task (in both experiments) is ample for generating visual images, as evidenced by the high accuracy rate of our participants and RTs ranging from 1.1-2.0s in various image generation studies (Kosslyn et al., 1988, 1995; Doricchi et al., 1995; Dean et al., 2005; Slotnick et al., 2005).”

“Several studies have also shown that hearing and touch can be used to stimulate visual imagery (de Volder et al. 2001; Lacey et al. 2010).”

“Touch can even modulate visual imagery (Lacey et al., 2010).”

“Indeed, functional imaging studies in humans showed that tactile object recognition activates a cortical network that includes VLPF areas, in addition to temporal, inferior, and opercular parietal, and ventral premotor areas (Reed et al., 2004; Lacey et al., 2010; Savini et al., 2010).”

“Analysis of activation and effective connectivity profiles in a set of studies (Deshpande et al. 2010; Lacey et al. 2010, 2014) led to the proposal of a conceptual model (Lacey et al. 2009b, 2014; Lacey and Sathian 2015) (Fig.”

“A number of previous brain imaging studies on haptic processing [49, 50] and auditory-based sensory substitution [51] in blindfolded sighted subjects have suggested that neural activity related to visual imagery may account for the activation in the occipital cortex.”

“Lacey et al. (2010) addressed the controversy of whether haptic shape perception depends on visual imagery by investigating LOC activity and connectivity modulations by the familiarity of objects (Deshpande et al., 2010).”

“The significant paths resulting from this are again tested for significant differences between religious and non-religious groups. makes it easier to interpret the relationship between activity and connectivity (Lacey et al., 2010a).”

“In fact, it was shown recently that object familiarity modulates the relationship between visual object imagery and haptic shape perception [91].”

“, [43]; but see [44]), but the actual function of these areas during tactile information processing remains poorly understood.”

“, 2001) or different sensory modalities (Berger & Ehrsson, 2013, 2014, 2017; Lacey et al., 2010) but can also effectively reshape future perception of real stimuli through neural plasticity mechanisms across sensory modalities.”

“Functional activation of the visual cortex by non-visual tactile stimulation has been reported in people with blindness [18,24–33], or with normal vision [17].”

“A few other studies have also implicated IFG in haptic object tasks (Binkofski et al. 1999; Stoeckel et al. 2003; Reed et al. 2004; Lacey et al. 2010).”

“Importantly, these findings were not a mere result of visual imagery as expected at the beginning of this line of research (Deshpande et al., 2010; Lacey et al., 2010).”

“In this respect, it is interesting to note that LOC activity is only directly correlated between tasks for objIMG and fHS, not for objIMG and uHS (Lacey et al., 2010) and not for spIMG and HS regardless of familiarity (present results).”

“…study of haptic perception with blind participants as relevant because studies with sighted participants might be resulting in an underestimation of haptic abilities (Heller and Ballesteros 2006), since there are reports of enhanced haptic sensitivity in the blind (e.g., Sathian and Prather 2006).”

“Conversely, the unfamiliar shape network is mainly characterized by bottom–up somatosensory inputs to LOC (Deshpande et al, 2010 ; Lacey, Flueckiger, Stilla, Lava, & Sathian, 2010 ; Deshpande et al, 2008 ).”

“…objects show greater overlap with visual Frontiers in Human Neuroscience www.frontiersin.org March 2012 | Volume 6 | Article 49 | 7 object recognition and have stronger top-down input than those involved in haptic recognition of unfamiliar objects (Deshpande et al., 2010; Lacey et al., 2010).”

“, 2012), recognition of familiar tactile objects (Lacey et al., 2010) and activation for tactile shape over texture (Reiner et al.”

“Accordingly, we chose this task as a supportive test for comparison, assuming that our categorization entailed both motor coordination (exploring the shape) as well as potential visual imagery (see discussion by [Lacey et al., 2010] on the use of visual imagery during haptic object processing).”

“The authors revealed that, during a tactile exploration of familiar objects, the LOC activation’s magnitude was correlated with the magnitude of activation evoked in this area by a visual object imagery condition (Lacey et al., 2010).”

“Top-down effects from imagery on haptic shape perception have been previously documented [45–49] but these mostly engage imagery of higher-level visual features as object size, shape, mental rotation (reviewed in Kosslyn et al.”

“, 2004) and activations evoked by visual imagery were found to be more correlated with those evoked by the haptic perception of familiar (compared to unfamiliar) shapes (Lacey et al., 2010).”

“, 2014) and review the evidence for this model from studies designed to explicitly test the visual imagery hypothesis discussed above (Lacey et al., 2010; Lacey et al., 2014).”

“In fact, visual imagery has been found to play a critical role in mediating tactile perception and recognition in normally sighted individuals (see Lacey et al., 2010, Scocchia et al., 2009 ), as also is suggested by the involvement of the occipital cortex in haptic discrimination (e.g., Merabet et…”

“However, in contrast to previous studies on tactile shape exploration, 4,9 activation of visual areas did not appear to be specific to the texture since the classifier was not able to discriminate the rough versus smooth conditions within the visual cortex.”

“For example, Lacey et al. (2010) conducted two experiments: the first involved a haptic shape task that relied on unfamiliar (or novel) objects, and the second involved familiar objects.”

“…studies have found haptic object-selectivity — often de fi ned by a contrast of haptic shape versus haptic texture — in the LOC (Amedi et al. 2001; Lacey et al., 2010; Stilla and Sathian, 2008; Zhang et al., 2004; Stoesz et al., 2003; Prather et al., 2004; Peltier et al., 2007; for a review, see…”

“…anterior intraparietal (AIP) and dorsolateral prefrontal cortex have found evidence to support this, finding these areas to be especially active when comparing shapes across modalities (Murata et al., 2000; Grefkes et al., 2002; Ricciardi et al., 2006; Lacey et al., 2010; Helbig et al., 2012).”

“Prior studies examining the encoding of visual or verbal stimuli have reported different patterns of cortical activity for each stimulus type (Deshpande, Hu, Lacey, Stilla, & Sathian, 2010; Gottlieb, Uncapher, & Rugg, 2010; Lacey, Flueckiger, Stilla, Lava, & Sathian, 2010; Harrison & Tong, 2009; Hocking & Price, 2009;…”

“Haptic shape perception is modulated by object familiarity and visual imagery preference (e.g., Craddock & Lawson, 2008; Lacey et al., 2010).”

“Furthermore, LOC preferential activity to familiar (iconic) objects was shown in a visual imagery with haptic perception task (Deshpande et al., 2010, Lacey et al., 2010).”

“In fact, the tactile perception of familiar shapes has been shown to be closely linked to visual object imagery 34 .”

“…of the shape selection process, we will also test for the impact of shape familiarity on task performance, since previous research have shown that form discrimination at least in part rely on prior knowledge of recognized forms (Augustine et al., 2011; Lacey et al., 2010; Lederman & Klatzky, 1990).”