The Wick in the Candle of Learning: Epistemic Curiosity Activates Reward Circuitry and Enhances Memory

“The two sessions in Experiment 2 were conducted and analyzed in Matlab (Mathworks, Inc., Natick, MA) using the Psychophysics Toolbox (Brainard, 1997) and the Eyelink Toolbox (Cornelissen, Peters, & Palmer, 2002).”

“A reward is an object or event that elicits approach and is worked for (Wise, 2004).”

“Philosophers and psychologists have described curiosity as an appetite for knowledge, a drive like hunger and thirst (Loewenstein, 1994), the hunger pang of an ‘‘info-vore’’ (Biederman & Vessel, 2006, p. 247), and ‘‘the wick in the candle of learning’’ (William Arthur Ward, cited by Wikiquote,…”

“Because pupil dilation is known to be linked to arousal, attention, interest, and cognitive effort (Beatty, 1982; Hess & Polt, 1960), and because anticipatory pupillary responses increase following a stimulus that predicts rewards (O’Doherty, Dayan, Friston, Critchley, & Dolan, 2003), the observed…”

“…Fissell, Noll, & Fiez, 2000), including social rewards such as benevolent reciprocity (Fehr & Camerer, 2007; King-Casas et al., 2005), social cooperation (Rilling et al., 2002), altruistic punishment (de Quervain et al., 2004), and winning an auction (Delgado, Schotter, Ozbay, & Phelps, 2008).”

“These regions are thought to be involved in successful verbal memory encoding (Brewer, Zhao, Desmond, Glover, & Gabrieli, 1998; Paller & Wagner, 2002; Wagner et al., 1998).”

“, 2005), social cooperation (Rilling et al., 2002), altruistic punishment (de Quervain et al.”

“2) and was part of Broca’s area, which is important for language comprehension (Bookheimer, 2002).”

“…be linked to curiosity because a growing body of evidence suggests that activity in the human striatum is correlated with the level of reward signals (Hare, O’Doherty, Camerer, Schultz, & Rangel, 2008; Knutson, Westdorp, Kaiser, & Hommer, 2000; McClure, York, & Montague, 2004; O’Doherty, 2004).”

“This interpretation is consistent with the recent finding that unexpected punishments and unexpected rewards produce very similar BOLD responses in the putamen at the time of outcomes (Knutson et al., 2000; Seymour, Daw, Dayan, Singer, & Dolan, 2007).”

“…& Fiez, 2003; Delgado, Nystrom, Fissell, Noll, & Fiez, 2000), including social rewards such as benevolent reciprocity (Fehr & Camerer, 2007; King-Casas et al., 2005), social cooperation (Rilling et al., 2002), altruistic punishment (de Quervain et al., 2004), and winning an auction…”

“Recent computational neural network models suggest another compatible interpretation involving memory (Frank, Loughry, & O’Reilly, 2001; O’Reilly & Frank, 2006).”

“The latter type of curiosity is called diversive perceptual curiosity and can be found in various animals (Berlyne, 1954).”

“When the 968 Volume 20—Number 8 answer appeared (0–1,000 ms after the onset of the answer), the average PDR differed significantly across the three trial types, p < .03 (one-tailed), prep .91, d 0.18.”

“This internal representation in the lateral PFC, particularly in the left IFG, is a crucial component of long-term memory consolidation and is critical in the learning of new information (Paller & Wagner, 2002).”

“…(Delgado, Locke, Stenger, & Fiez, 2003; Delgado, Nystrom, Fissell, Noll, & Fiez, 2000), including social rewards such as benevolent reciprocity (Fehr & Camerer, 2007; King-Casas et al., 2005), social cooperation (Rilling et al., 2002), altruistic punishment (de Quervain et al., 2004), and…”

“The dorsal striatum responds to magnitude of primary rewards and reward prediction (Leon & Shadlen, 1999; Rogers et al., 1999) and shows sustained phasic activations during reward expectation (Watanabe, 1996).”

“In reinforcement learning, a novelty bonus is used to motivate the choice of unexplored strategies (Kakade & Dayan, 2002).”

“The dorsal striatum responds to magnitude of primary rewards and reward prediction (Leon & Shadlen, 1999; Rogers et al., 1999) and shows sustained phasic activations during reward expectation (Watanabe, 1996).”

“…and psychologists have described curiosity as an appetite for knowledge, a drive like hunger and thirst (Loewenstein, 1994), the hunger pang of an ‘‘info-vore’’ (Biederman & Vessel, 2006, p. 247), and ‘‘the wick in the candle of learning’’ (William Arthur Ward, cited by Wikiquote, 2008).”

“Previous studies have found that the expectation of feedback is sufficient for activation of the caudate (Aron et al., 2004).”

“There are also studies that have reported striatal activations in response to negatively valenced stimuli (Knutson, Adams, Fong, & Hommer, 2001) or during nonreward activity such as working memory and motor preparation (Cairo, Liddle, Woodward, & Ngan, 2004; Simon et al., 2002).”

“Fifth, interest has a physiological/neurological basis; brain activations differ when a learner is and is not engaged with interest (e.g., Hidi, 2006; Kang et al., 2009; Palmer, 2009; Panksepp, 1998).”

“Participants recalled significantly more answers to high curiosity questions, compared to low curiosity questions (70.6% SE= ± 2.60 vs. 54.1% SE= ± 3.04; paired-sample t (17) =5.64, p <0.001; Figure 3A) replicating earlier findings (Kang et al., 2009).”

“The curiosity studies by Kang et al. (2009) and Gruber et al. (2014) found greater activity within the lateral PFC during curiosity-inducing questions.”

“This provides the foundation for alternative theories such as costly talk (as in Kartik, Ottaviani, and Squintani 2007; Ying Chen 2007; and Kartik 2009) or the level-k model (as in Crawford 2003; Cai and Wang 2006).”

“, and White, T. L. (1994). Behavioral inhibition, behavioural activation, and affective responses to impending reward and punishment: the”

“studies using different paradigms, such as those involving interesting trivia questions (Kang et al., 2009), inherently pleasurable music (Salimpoor et al.”

“…for the arousal of epistemic emotions is discrepant information and appraisals of cognitive incongruity that can trigger surprise and curiosity (Kang et al., 2009), confusion, frustration, and boredom when the incongruity cannot be resolved (D’Mello & Graesser, 2012; D’Mello et al., 2014),…”

“An fMRI study showed that learning the answers to trivia questions about which one is curious activates the brain’s reward system in much the same manner as receiving monetary, gustatory, or social rewards (Kang et al., 2009).”

“Dopamine potentiates exploration in behavior and cognition (Depue & Collins, 1999; Kang et al., 2009; Panksepp, 1998).”

“When people are curious about the answers to trivia questions, their pupils dilate more (Kang et al. 2009)—by a small (8% vs. 4”

“…models in adult cognitive science, looking at exploration/exploitation trade-offs and decision-making under uncertainty (e.g., Daw, Niv, & Dayan, 2005; Daw, O'Doherty, Dayan, Seymour, & Dolan, 2006; Kaelbling, 1993; Kang et al., 2009; McClure, Daw, & Read Montague, 2003; Sutton & Barto, 1998).”

“Although the focus on knowledge identified within this review tends to be on knowledge as something to be internalized and recalled (e.g., Kang et al. 2009; Litman et al. 2005), research moving forward should examine ways in which curiosity aligns with educational goals of developing understanding…”

“…current research on curiosity tends to focus on information-seeking behavior for knowledge that has only shortterm consequences (e.g., information about an outcome that would be revealed a few seconds later; Bennett et al. 2016) or little practical value (e.g., trivia questions; Kang et al. 2009).”

“Neuroimaging studies have also shown that the subjective feeling of curiosity in the face of knowledge gap (e.g., seeing a trivia question) activates the reward network in the same way that extrinsic rewards do (e.g., Gruber et al., 2014; Kang et al., 2009).”

“See Loewenstein ð1994Þ, Kang et al. ð2009Þ, or Golman and Loewenstein ð2012Þ.”

“Recent studies have begun to identify the structures associated with curiosity and motivated information seeking more generally (Bromberg-Martin and Hikosaka, 2009, 2011; Gruber et al., 2014; Kang et al., 2009; Phillips et al., 2012).”

“Individuals may also experience surprise, curiosity, and confusion when incoming information is inconsistent with prior knowledge (D’Mello & Grasser, 2012), epistemic beliefs (Muis, Pekrun, et al., 2015), or recently processed information (Kang et al., 2009).”

“…humans has further supported this idea, demonstrating that people are more willing to wait and pay for information about which they’re more curious (Kang et al., 2009) and that high-curiosity information is associated with activation in brain areas known to respond to rewards, including the…”

“, a manageable challenge) therefore enhances learning, piquing curiosity and attention in the process (Kang et al., 2009; Abuhamdeh and Csikszentmihalyi, 2012a,b; Gottlieb et al., 2013; Kidd et al., 2014; Baranes et al., 2015; Daddaoua et al., 2016; Oudeyer et al., 2016; Brydevall et al., 2018).”

“Two previous papers also found that human curiosity (rather than its relief) for nonvalenced trivia questions was associated with enhanced activation in reward regions (21, 44).”

“…2011) and has been found to promote the exploration of new knowledge (Berlyne, 1954, 1960; Litman et al., 2005) and to enhance memory for new information (Gruber, Gelman, & Ranganath, 2014; Kang et al., 2009; Marvin & Shohamy, 2016; Middlebrooks, McGillivray, Murayama, & Castel, 2016).”

“While previous studies found that test accuracy for trivia questions is linked to a learner's curiosity for the questions (e.g., Gruber et al., 2014; Kang et al., 2009), it was previously unclear whether the learning effects attributed to curiosity would be better explained by curiosity-related variables, such as prio…”

“Curiosity—the desire for information—is well recognized as a human blessing; it facilitates learning (Gruber, Gelman, & Ranganath, 2014; Kang et al., 2009), propels discoveries (Simon, 2001), and enriches life (Ruan, Hsee, & Lu, 2015; Kashdan & Silvia, 2009).”

“Curiosity during learning was associated with improved recall one to two weeks later, indicating its plasticity facilitating effect (Kang et al., 2009).”

“Experimental inductions of curiosity map onto this definition, the most common of which is presenting trivia questions that participants cannot answer but may desire to know the answer (Gruber, Gelman, & Ranganath, 2014; Kang et al., 2009).”

“Importantly, the motivation to explore cognitively likewise appears to involve the reward system: discovering the answer to a trivia question about which one is curious triggers the brain’s reward system in much the same way that monetary, social, or food rewards do (Kang et al., 2009).”

“Epistemic curiosity, as a trait, can explain individual differences in academic achievement [19], and experimental manipulations have demonstrated that degree of interest or motivation for receiving particular information determines whether that information is subsequently retained [12] [20].”

“M. J. Kang et al. (2009) found that curiosity to know the correct answer, following the making of an error, enhanced subsequent memory.”

“…empirical evidence shows that epistemic emotions are active (D’Mello& Graesser, 2012, 2014) and influential during learning, affecting intrinsic motivation (Kang et al., 2009), type of learning strategies used (Muis et al., 2015), and learning outcomes (D’Mello, Lehmann, Pekrun, & Graesser, 2014).”

“Curiosity has been found to enhance learning in studies using self-report measures of knowledge (Jones, 1979) as well as knowledge tests (Arnone et al., 1994; Kang et al., 2009).”

“001)), which replicates previous findings (Gruber et al. 2014; Kang et al. 2009).”

“, 2002), which has been previously linked to curiosity (Kang et al., 2009; Marvin and Shohamy, 2016).”

“…such as self-reported curiosity or stimulus novelty, correlate with BOLD signals in striatum and dopaminergic midbrain regions in humans 9-11 , 32-34 If information acquisition is indeed driven by dopamine reward system in general, we should expect two features from dopaminergic responses;…”

“This condition likely evoked curiosity (Inagaki & Hatano, 1977), which enhances motivation and is posited to boost learning (Kang et al., 2009).”

“The stimuli consisted of 60 trivia questions, adopted from Kang et al. (2009) and taken from various Internet trivia sites.”

“Recognizing the importance of integrating findings of psychological investigations and neuroscientific research on human motivation and rewards, neuroscientists have started to conduct such research (e.g., Gruber et al. 2014; Kang et al. 2009; Lee and Reeve 2013; Ma et al. 2014; Murayama et al. 2010).”

“Moreover, because users of the treatment version know that the uncertainty will resolve soon, curiosity may be associated with a positive feeling of anticipation (Kang et al. 2009; Lee andQiu 2009; Noordewier and Van Dijk 2015; Nowlis, Mandel, and McCabe 2004).”

“Previous investigations proposed that a key factor driving epistemic curiosity is an ''information gap " – a discrepancy between what one knows and what one would like to know – and therefore that curiosity should peak when one has a little bit of knowledge, but diminish if a subject knows too little or too much about…”

“…(Mussel, 2012), problem solving (Nair & Ramnarayan, 2000), intellectual task performance (Coutinho, Wiemer-Hastings, Skowronski, & Britt, 2005), cognitive motivation (Cacioppo, Petty, Feinstein, & Jarvis, 1996), and knowledge acquisition (Arnone, Grabowski, & Rynd, 1994; Kang et al., 2009).”

“Given that interesting materials are in and of themselves well remembered (Kang et al., 2009; see also Fig.”

“It can be seen as a a personality trait ([31]), a psycho-emotional state aroused by external situations([41], [30]),[5],...) or as a malleable skill that can be trained through teaching specific practices.”

“, 2008), curiosity and intrinsic motivation (Hess, Emery, & Neupert, 2012; Kang et al., 2009; Murayama & Kuhbandner, 2011), as well as individual difference variables such as anxiety (Murty et al.”

“Moreover, research consistently demonstrates that an interest in obtaining information supports memory of that information (Adcock et al., 2006; Gruber & Otten, 2010; Kang et al., 2009; Loewenstein, 1994).”

“This finding is seemingly at odds with studies and theories that suggest confidence is related to curiosity in an inverse u-form shape (Kang et al. 2009; for a review of these findings see Butlin 2010).”

“Consistent with our identification of differences in frontocentral EEG activity as an FRN, we found that reconstructed ACC activity significantly differed as a function of prediction error sign, both for positive reward prediction errors relative to negative reward prediction errors (t(14) = 2.”

“People attach inherent value to acquiring information (Gottlieb, Oudeye, Lopes, & Baranes, 2013) and are willing to give up time and resources to acquire it (Kang et al., 2009).”

“Because curiosity-driven investigation has previously been associated with reward anticipation and positive valence (17), we examined whether the increased arousal level coincides with a positive or negative valence.”

“This line of work is primarily motivated by recent findings in cognitive neuroscience that interest activates the so-called “reward network” in the brain, including the ventral striatum and the midbrain (Gruber et al., 2014; Kang et al., 2009).”

“1a), the suppressive effect of average surprise should be confirmed in other tasks probing the willingness-to-pay or the willingness-to-wait for answers (Kang et al., 2009; Marvin and Shohamy, 2016).”

“…curiosity (Renner, 2006), and epistemic curiosity (Berlyne, 1954; Litman, 2008), and examined in different lines of research such as psychology (e.g., Litman, 2005; Loewenstein, 1994), neurobiology (e.g., Kang et al., 2009), and education (e.g., Eren, 2011; Engel, 2011; Richards et al., 2013).”

“, physics, writing, baseball) develops and deepens, they are more likely to reengage that content to seek information; seeking behavior activates the reward circuitry of the brain (e.g., Gottlieb, Oudeyer, Lopes, & Baranes, 2013; Kang et al., 2009).”

“…with high curiosity are better remembered than answers to low-curiosity questions (Fastrich et al., 2018; Galli et al., 2018; Gruber et al., 2014; Kang et al., 2009; Marvin & Shohamy, 2016; McGillivray et al., 2015; Mullaney, Carpenter, Grotenhuis, & Burianek, 2014; Stare, Gruber, Nadel,…”

“In contrast, an expanding literature proposes that people prefer intermediate dif fi culty 16 in a range of conditions including curiosity about trivia questions 5,17 , choices among sensorimotor activities 18 , infant attention 19 and esthetic appreciation 20,21 .”

“Indeed, Kang et al. (2009) link curiosity to pupil dilation, which is a well-known, reliable measure of attention (Kahneman, 1973).”

“…neuroscience studies have examined the concept of intrinsic motivation in their investigations, but these investigations have only discovered the shared reward-related neural bases between intrinsic motivation and incentive motivation (Kang et al., 2009; Mizuno et al., 2008; Murayama et al., 2010).”

“Ultimately, we hope this line of work will be integrated into studies of the neural bases of information-seeking and curiosity — key drivers of search behavior in foraging [48,49 ,50], and to search more generally [7].”