The emulation theory of representation: Motor control, imagery, and perception

“More recently, forward models have been used to describe the predictive nature of perception and imager”

“It has been proposed that bodily internal models can be reenacted and used in simulation [199, 200], emulation [201], or other (Bayesian) generative schemes [202] to support higher cognition in individual and social domains.”

“Predictive coding models are now well-established in accounting for various features of perception (Rao and Ballard, 1999;Yuille and Kersten, 2006), cognition (Grush, 2004), and motor control (Wolpert and Ghahramani, 2000) (see Bubic et al., 2010 for a review).”

“Grush, R. (2004). The emulation theory of representation: Motor”

“For example, it is conceivable that a dynamical perceptual– motor process might become detached from its original informational and behavioral context to form an emergent representation, which could then be deployed in mental practice, imagery, off-line planning, or strategic reasoning (cf. Barsalou, 1999; Grush, 2004…”

“…inputs: for example, producing overt speech is thought to involve the sending of an “efference copy” of the speech motor plan to speech perception areas, forming the basis for a predictive model of what the utterance will sound like, and inhibiting the ensuing auditory response (Grush, 2004).”

“We add to the Wolpert and Kawato scheme two features addressed by Grush (2004) and Hurley (2004).”

“of the same perceptual-motor loops – and internally generated brain dynamics – might realize increasingly complex cognitive functions such as planning, imagery, and conscious thought (Buzsáki et al., 2014; Grush, 2004; Hesslow, 2002; Jeannerod, 2001; Pezzulo, 2011; Pezzulo and Castelfranchi, 2009).”

“…of prediction have much greater value (Friston and Stephan, 2007), as they may signal unsuccessful learning, a major change in the surroundings or noise and smaller changes in the body or the environment, corresponding to normal (plant or world) drifts which typically occur over time (Grush, 2004).”

“Grush (2004) takes a related standpoint, but goes one step further by stating that motor imagery results from the control centers of the brain driving an emulator of the body, whereby the normal efferent flow is disengaged from the muscle system.”

“An emulator is defined as " a device that mimics the input/output operation of a target system " (such as the motor system; Grush, 2004, p. 387).”

“Forward models are ‘neural simulators’ of the musculo-skeletal system and the environment (Clark and Grush 1999, Grush 2003, Wolpert et al. 2003), and thus allow prediction of the future state of the system given the present state and a certain input (a state specifies a particular body…”

“In particular, this renders perception similar to hallucination [1,10].”

“From the standpoint of the emulation theory of representation (Grush 2004), what is present in motor imagery, but not in the ISrelated processes, is probably willed generation of efference copy and analysis of ''mock'' sensory (mainly proprioceptive and partly visual) signals.”

“Cf. Wolpert (1997), Wolpert and Kawato (1998), Haruno et al. (2001), and Wolpert et al. (2003) on selection among forward models; Grush (2004) on continuous combination of real and simulated input; and Grush (1995) linking motor control and mindreading.”

“…dynamics have been at the heart of a large number of influential and very diverse theories of mind, brain and skilled behavior more generally (Helmholtz 1860/1962; Gregory 1980; Grush 2004; Turvey 2010; Noë 2004; Stepp and Turvey 2010; Port and Van Gelder 1995; Thompson 2007; Chemero 2009).”

“Critically, the forward model can also be used imaginatively, in mental practice, to evaluate outcomes without performing them (e.g., Grush, 2004).”

“that we can predict (cf., emulation theory; Grush, 2004); ensuring we keep to well-trodden paths in the space of all the physical and physiological variables that underwrite our existence.”

“…proposed by Jeannerod (1994, 2001, 2006), and more recently motor simulation, along with prediction as its most prominent cognitive function, has become a commonly accepted framework for a wide range of cognitive domains (Grush, 2004; Kilner et al., 2007; Bubic et al., 2010; Pezzulo et al., 2013).”

“A recent model (Grush, 2004) proposes that there should be domain-general type imagery, called supramodal or amodal imagery.”

“The representations they call on are indexical functional (Agre & Chapman, 1987), pushmi pullyu (Millikan, 1995), action-oriented (Clark, 1997), or emulator representations (Grush, 2004).”

“Decety, J., Jeannerod, M., and Prablanc, C. (1989).”

“In fact, to the extent that this exploration is coupled with an intention to approach (or avoid) a space, it would also be accompanied by activation in brain regions implicated in motor imagery and/or planning of voluntary motor movement (Crammond, 1997; Decety, 1996; Dieber et al., 1998; Grush, 2004; Hanakawa, Dimyan…”

“Pioneering work from Chris Eliasmith (2003) and Rick Grush (2004) did not appeal to Friston’s approach but independently provided influential philosophical frameworks congenial to PP.”

“joint angle positions) that interact according to the laws of dynamics and mechanics [61].”

“The second component consists of simulation mechanisms tapping into the observer's motor system (Blakemore & Decety 2001; Grush 2004; Wilson & Knoblich 2005).”

“, [111–115]).”

“These ideas resonate with the ‘emulation’ framework discussed in relation to action control, and generalized to motor imagery, perception, and prediction [112,113].”

“This prospective signal may be related to the “efference copy” proposed in previous models (von Holst and Mittelstaedt, 1950; Troyer and Doupe, 2000), including so-called emulator (or “forward”) models (Grush, 2004; Callan et al., 2007; Ghasia et al., 2008).”

“Simulation has been contrasted with motor imagery (e.g., O’Shea & Moran, 2017; Willems, Toni, Hagoort, & Casasanto, 2009) and with emulation (e.g., Grush, 2004), and there is debate about the roles of conscious representations and internal models in simulation (e.g., Hesslow, 2012; Pezzulo,…”

“There is evidence that nervous systems contain and manipulate analog models of their environment (Grush, 2004; Waskan, 2003).”

“This article considers recent evidence from invertebrates that demonstrates the predictive modulation of sensory processes by motor output, and discusses to what extent this supports the conclusion that insect nervous systems also implement forward models.”

“This resonance would be part of an emulation process (Grush, 2004) that plays a role in representing and understanding the behavior of conspecifics (Wilson & Knoblich, 2005).”

“…a combination of ideas with a long history, going back to associationist philosophers such as David Hume (1739) and Alexander Bain (1868) and similar ideas have been suggested by several modern authors (too numerous to be credited here but see for instance Grush (2004) or Decety and Grèzes (2006)).”

“…central aim is realizing complete robotic architectures that, on the one hand, include principles and constraints derived from animal and human cognition and, on the other hand, have realistic embodiment, sensors and effectors, and learn to act autonomously in complex, open-ended (and social).”

“This issue is an instance of the general distinction that has been drawn between processes leading to the generation of a motor plan (that include predictions of the sensory consequences of the action), and processes related to the evaluation of sensory feedback (Blakemore and Sirigu, 2003; Grush, 2004; Wolpert et al.…”

“Embodied views of cognition suggest that these internal representations are grounded in the systems of perception and action that are used to interact with the world (e.g., Barsalou, 1999; Glenberg, 1997; Grush, 2004; Pecher & Zwaan, 2005).”

“We use the term ‘emulation’ to denote this core function of the dFPN [11,12].”

“…motor/perceptual resonance and action simulation: While resonance is driven exogenously and automatically (i.e., preattentively) by the perception of external events, simulation may be generated endogenously in the absence of external stimuli (cf. Grush, 2004) and may be modulated by attention.”

“’’ Within this account, imagery could be an anticipatory phase of perception like a ‘‘disposition to see’’ (see also Freyd, 1987; Grush, 2004; Kosslyn & Sussman, 1994; Ryle, 1949) that takes place all the time; only when the perceptual pickup of information is either interrupted or delayed, imagery becomes subjectivel…”

“Like overt motor execution, motor imagery may entail the generation of an action plan (inverse model) as well as a prediction of the actionʼ sensory consequences (forward model) (e.g., Grush, 2004; Wolpert & Ghahramani, 2000).”

“…(de Lange et al., 2006; Vargas et al., 2004) studies have showed that there is a clear proprioceptive influence on motor imagery performance in healthy subjects, favouring the view that motor imagery relies on the generation of a complete motor plan that is prevented from operating on the body.”

“This can be operationalized via a Kalman filter (Grush, 2004), which informally speaking determines to what extent the existing model is “trusted” over the current prediction error or vice versa.”

“To address this issue, it is important to disentangle planningrelated activity from the effects of motor execution and sensory feedback (Blakemore and Sirigu, 2003; Davidson and Wolpert, 2003; Grush, 2004).”

“It has been suggested that during imagery, the brain internally simulates the predicted sensory consequences of a movement via a forward model (Grush, 2004), and the cerebellum seems well suited to support this function (Miall and King, 2008; Wolpert et al.”

“Importantly, imagery tasks are typically argued to be mediated by internal simulation and prediction processes (Tian & Poeppel, 2010, 2012; Grush, 2004; Miall & Wolpert, 1996; Sirigu et al., 1996; Jeannerod, 1994, 1995).”

““Simulative” processes (e.g., “emulation,” Grush 2004) are likely to be recruited to achieve better temporal and spatial predictions of a partner’s actions.”

“External signals are assessed by exteroceptive inference of the causes of these signals; that is, by comparing desired to anticipated and actual sensory states (Grush, 2004).”

“This notion is largely consistent with the tenets of the neural emulation theory, according to which inner models of the body and the environment can be run offline to produce “imagery, estimate outcomes of different actions, and evaluate and develop motor plans” (Churchland 2002; Grush 2004).”

“Mental imagery has been linked to the concept of internal forward models by the argument that the subjective feeling in mental imagery is the result of the internal estimation of the perceptual consequences following the internal simulation of an action (Grush, 2004).”

“Motor imagery has been at the centre of many theoretical discussions and has been conceptualised in terms of motor representations (Jeannerod, 1994, 1995, 1997), action prototypes coupled with memory processes (Annett, 1996) and forward models (Grush, 2004; Schwoebel, Boronat, & Coslett, 2002; Wilson, 2003; Wolpert &…”