Attention, learning, and the value of information

May combine two claims.

In that task, the informative cue had value, and lateral intraparietal area neurons continued selecting the cue, showing higher activity if the target E rather than a distractor was in their receptive field.

“Nevertheless the informative cue had value, and neurons in the lateral intraparietal area continued selecting the cue, showing much higher activity if the “E” rather than a distractor was in their receptive field (Balan and Gottlieb, 2009; Balan et al., 2008; Oristaglio et al., 2006) (Fig. 4B).”

• cites Balan and Gottlieb, 2009Not checked yet

  Not checked in the free run.

• cites Balan et al., 2008 sourceNot checked yet

  Not checked in the free run.

• cites Oristaglio et al., 2006 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In a task where monkeys reported orientation of a peripheral target while maintaining central fixation, overt saccades had no value and were actively suppressed.

“Monkeys had to perform the task while maintaining their gaze straight ahead (on the central fixation point), so that overt saccades had no value and would have been punished with a loss of reward - and indeed, monkeys actively suppressed the saccades.”

• cites Oristaglio et al., 2006 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The interference from the CS− was not due to lower motivation but was spatially specific for saccades whose target was at the same location as the CS−.

“Notably, this interference was not due to lower motivation but was a spatially specific for saccades whose target was at the same location as the CS−, showing that attention was inhibited specifically at the CS− location.”

May combine two claims.

Some parietal cells responded selectively if the 'E' was in their receptive field and instructed release of the left bar; other cells had the complementary preference.

“Some of the cells, like that shown in Fig. 4C, responded selectively if the “E” was in their receptive field and instructed release of the left bar; other cells had the complementary preference, responding best if the “E” was in their receptive field and instructed release of the left bar (not shown).”

May combine two claims.

The CS− evoked inhibition interfered with the monkeys' performance and lowered their rate of reward.

“The CS− evoked inhibition interfered with the monkeys’ performance and lowered their rate of reward.”

May combine two claims.

Saccades following a low-reward cue (CS−) were markedly impaired, showing both longer reaction times and lower accuracy relative to saccades to other locations.

“Saccades on these trials were markedly impaired, showing both longer reaction times and lower accuracy relative to saccades to other locations.”

May combine two claims.

Parietal neurons encoded the location of the relevant cue, and some cells had only a target selection response, responding more if the cue rather than a distractor was in the RF regardless of manual release.

“As I mentioned in the previous section, parietal neurons encoded the location of the relevant cue, and some of the cells had only a target selection response, responding more if the cue rather than a distractor was in the RF regardless of the manual release (Fig. 4B).”

• cites Oristaglio et al., 2006 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The effects grew rather than abating with training and, in both neural responses and behavior, were larger after familiar relative to novel CS.

“Nevertheless the effects grew rather than abating with training and, in both neural responses and behavior, were larger after familiar relative to novel CS (Fig. 5D, bottom vs. top).”

Probably not a substantial claim: may not pass S4 (based).

The neural response to target selection can be viewed as an internal decision that seeks to maximize a utility function, increasing benefit and minimizing cost.

“Consistent with the decision framework, I propose that the neural response to target selection can be viewed as an internal decision that seeks to maximize a utility function (i.e., increase a benefit and minimize a cost).”

Probably not a substantial claim: may not pass S4 (based).

Neural responses to target selection have no good explanation in sensory or motor terms.

“Gaining this appreciation is necessary for explaining a range of observations regarding the neural responses to target selection, which have no good explanation in sensory or motor terms.”

Classified as probably not a substantial claim. May combine two claims.

Neurons in the lateral intraparietal area and frontal eye field have spatial receptive fields and saccade-related responses, and respond selectively to stimuli likely to attract attention in a variety of tasks.

“Neurons in these areas have spatial receptive fields and saccade-related responses, and respond selectively to stimuli that are likely to attract attention in a variety of tasks.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A shift of attention can remain covert or be accompanied by saccades generated by a network including portions of the basal ganglia, superior colliculus, and frontal eye field.

“A shift of attention can remain covert – generating only an improvement in perceptual discrimination – or can be accompanied by saccades – rapid eye movements that are generated by a network of cortical and subcortical structures that includes portions of the basal ganglia, the superior colliculus and the frontal eye field (Schall et al., 2011; Stanford et al., 2010).”

• cites Schall et al., 2011 sourceNot checked yet

  Not checked in the free run.

• cites Stanford et al., 2010 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

These cells are not selective for visual features but encode an abstract quantity of target selection, discriminating between targets and distractors in a variety of tasks.

“These cells are not selective for visual features, but encode a more abstract quantity of target selection – i.e., discriminate between targets and distractors in a variety of tasks (Gottlieb and Balan, 2010; Thompson and Bichot, 2005).”

• cites Gottlieb and Balan, 2010Not checked yet

  Not checked in the free run.

• cites Thompson and Bichot, 2005Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Attention produces widespread effects throughout early and late visual areas, increasing the signal from the attended item and suppressing noise from unattended distractors.

“These studies have shown that attention produces widespread effects throughout early and late visual areas, which collectively increase the signal from the attended item and suppress noise from unattended distractors (Reynolds and Heeger, 2009).”

• cites Reynolds and Heeger, 2009 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Both the motor decision and stimulus selection frameworks fail to fully capture empirical observations and leave open significant questions about eye movement control.

“Moreover, as I will describe below, both views fail to fully capture the empirical observations and leave open significant questions about eye movement control.”

May combine two claims.

These cells encode the specific act of target selection and can provide feedback regarding this selection to earlier visual areas and downstream movement structures that generate shifts of gaze.

“Not specifically sensory or motor, these cells seem to encode the specific act of target selection, and can provide feedback regarding this selection both to earlier visual areas and to downstream movement structures that generate shifts of gaze.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

As a system controlling a sensory organ, the eye, oculomotor decisions must be optimized for sampling information.

“However, consistent with an attention interpretation I emphasize that, as a system controlling a sensory organ - the eye - this decision must be optimized for sampling information.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The source of attentional modulations lies, at least in part, in two sensorimotor areas associated with rapid eye movements (saccades).

“These studies also suggest that the source of attentional modulations lies, at least in part, in two sensorimotor areas associated with rapid eye movements (saccades).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The reinforcement learning framework has been particularly successful in explaining behavioral and neuronal results.

“The computational framework of reinforcement learning, originally developed in the machine learning field (Sutton and Barto, 1998), has been particularly successful in explaining behavioral and neuronal results.”

• cites Sutton and Barto, 1998Not checked yet

  Not checked in the free run.

May combine two claims.

In the tea-making task, gaze is selectively allocated to targets of arm actions and not leg actions, and this selectivity cannot be explained by action value alone but reflects higher uncertainty of arm movements and thus more to gain from new information.

“The subject’s gaze however, is very selectively allocated to the targets of the arm and not the leg actions. This selectivity cannot be explained in terms of action value alone, but reflects the fact that the arm movements have higher uncertainty and thus more to gain from new information.”

May combine two claims.

The learning rate β depends on global task properties such as volatility or uncertainty and is suggested to be modulated by noradrenergic inputs.

“This rate depends on global task properties such as the volatility or uncertainty of a given task, and has been suggested to be modulated by noradrenergic inputs (Cohen et al., 2007; Nassar et al., 2012).”

• cites Cohen et al., 2007 sourceNot checked yet

  Not checked in the free run.

• cites Nassar et al., 2012 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Eye movements and attention, as specific types of selection, satisfy a utility function, seeking to maximize benefit and minimize cost.

“Therefore it seems likely that as specific types of selection, eye movements and attention would also satisfy a utility function – i.e., seek to maximize a benefit and minimize a cost.”

• AssumptionThe authors state this as an assumption: The ultimate goal of any act of selection is to maximize an organism's biological fitness.

May combine two claims.

An integrated account of learning and decision formation holds that value representations are maintained in cortical and sensorimotor structures and are dynamically updated based on feedback from dopaminergic cells.

“They suggest an integrated account of learning and decision formation, whereby value representations are maintained in cortical and sensorimotor structures and are dynamically updated based on feedback from dopaminergic cells (Kable and Glimcher, 2009; Sugrue et al., 2005).”

• cites Kable and Glimcher, 2009 sourceNot checked yet

  Not checked in the free run.

• cites Sugrue et al., 2005 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The learning rate β takes values between 0 and 1 and determines how quickly the agent updates predictions.

“One quantity, β, is a learning rate that takes values between 0 and 1 and determines how quickly the agent updates itse predictions.”

May combine two claims.

In a dynamic payoff task, monkeys apportioned choices in proportion to recent reward history, and lateral intraparietal area neurons increased selective responses in proportion with the target's expected reward.

“Monkeys apportioned their choices in proportion to the recent history of rewards, and neurons in the lateral intraparietal area increased their selective responses in proportion with the target’s expected reward: firing for a saccade directed toward the receptive field increased monotonically (blue traces, dashed to solid), while firing for a saccade to a different location decreased monotonically”

• cites Sugrue et al., 2004 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Models describing links between attention and decision formation share the theme that attention influences accumulation of evidence toward the attended option, making the subject more likely to select that option.

“A common theme in these models is that attention influences the accumulation of evidence toward the attended option, making the subject more likely to select that option (Krajbich et al., 2010).”

• cites Krajbich et al., 2010 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The normalization model portrays the attention field as a box with an output but no inputs, and focuses on its sensory effects.

“However, the model portrays this field as a box with an output but no inputs, and focuses on its sensory effects.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The normalization model proposes that a spatially selective attention field is fed back to the visual system and multiplicatively scales visual inputs in spatially specific fashion, followed by divisive normalization based on local competition, resulting in a biased visual representation where the attended stimulus is more strongly represented.

“As illustrated in Fig. 1B, the model proposes that a spatially selective “attention field” is fed back to the visual system and multiplicatively scales visual inputs in spatially specific fashion. Followed by divisive normalization based on local competition with other visual inputs (“suppressive drive”), this attentional influence results in a biased visual representation where the attended”

• cites Reynolds and Heeger, 2009 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Target selection neurons encode the relative value of alternative actions and integrate multiple sources of evidence pertinent to this estimation.

“Taken together these studies suggest the powerful hypothesis that target selection neurons encode the relative value of alternative actions, and that they integrate multiple sources of evidence pertinent to this estimation.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The attention field conforms to properties of the target selection response, being sensitive to spatial location but not visual features.

“The “attention field” conforms to the properties of the target selection response – i.e., it is sensitive to spatial location but not visual features.”

May combine two claims.

Subjects estimate the reliability of a sensory stimulus based on prior experience with that stimulus and use this knowledge to modulate future learning based on that cue.

“A central finding emerging from these studies is that subjects estimate the reliability of a sensory stimulus based on their prior experience with that stimulus, and use this knowledge to modulate their future learning based on that cue.”

• cites Pearce and Mackintosh, 2010Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

In all current studies of decision formation, value is defined in terms of primary rewards: the value of a saccade target in a laboratory task is defined by the juice obtained by making the saccade.

“As I described in the preceding section, in all current studies of decision formation “value” is defined in terms of primary rewards: the value of a saccade target in a laboratory task is defined by the juice that the monkey obtains by making the saccade (Fig. 1C).”

May combine two claims.

Information value depends critically on the subject's uncertainty and is more closely related to the learning side of the Rescorla-Wagner equation.

“A salient property of this process is that information value depends critically on the subjects’ uncertainty and, in the Rescorla-Wagner equation is more closely related with the right side of the equation – the act of learning or modifying expectations.”

Classified as probably not a substantial claim. May combine two claims.

These models begin by assuming that attention exists but do not explain how it may come to be.

“These models begin by assuming that attention exists, but do not explain how it may come to be – e.g., why subjects may attend to a specific object in the first place.”

Classified as probably not a substantial claim. May combine two claims.

The prediction error term δ describes how surprised the agent is by an outcome, defined as the difference between the agent's estimate and the actual outcome at the previous step, and provides a trigger for learning to reduce prediction error.

“The second quantity is the prediction error term (δ), which describes how “surprised” the agent is by a particular outcome - i.e., how well or poorly it had predicted that outcome. This quantity, defined as the difference between the agent’s estimate and the actual outcome at the previous step (δ = r−Vt−1), provides a trigger for learning (or updating expectations) so as to reduce the error in”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Casting target selection as an internal value estimation would bridge the conceptual gap in attention research.

“Casting target selection as an internal value estimation would seem to bridge the conceptual gap in attention research.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A recent normalization model of attention has been particularly successful in explaining a large number of sensory effects using a simple biologically-plausible circuit.

“A recent “normalization” model of attention has been particularly successful in explaining a large number of sensory effects using a simple biologically-plausible circuit (Reynolds and Heeger, 2009).”

• cites Reynolds and Heeger, 2009 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Associability is a property of an individual cue and can differentially weight available cues, whereas learning rate β is applied globally to a context or task.

“While the learning rate β is applied globally to a context or task, associability is a property of an individual cue and can differentially weight the available cues.”

• cites Pearce and Mackintosh, 2010Not checked yet

  Not checked in the free run.

May combine two claims.

The associability modification entails a complex, hierarchical learning mechanism and an executive process that, having learnt the predictive validity of a sensory cue, guides moment-by-moment information selection.

“As I discuss in detail in the following sections, this apparently simple modification entails a complex, hierarchical learning mechanism. It entails an executive process which, having previously learnt the predictive validity of a sensory cue, guides the moment by moment information selection – i.e., has in effect learnt how to learn.”

May combine two claims.

In an experiment where monkeys were initially uncertain about reward size, dopamine cells had transient excitatory response to a cue signaling larger-than-expected reward, transient inhibition to a cue signaling lower-than-expected reward, and nearly no response to a cue providing no new information.

“Dopamine cells had a transient excitatory response to a stimulus that signaled a larger-than expected reward (“Info-big”) and a transient inhibition to a stimulus that signaled a lower-than expected reward (“Info-small”), but had nearly no response to a stimulus that provided no new information (“Rand”, blue traces).”

• cites Bromberg-Martin and Hikosaka, 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

To decide where to shift gaze or attend, the brain may keep track of the values of alternative options and make choices according to this value representation.

“A straightforward implication of this idea is that, to decide where to shift gaze or where to attend, the brain may simply keep track of the values of the alternative options and make choices according to this value representation.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The gated accumulation model does not attempt to explain target selection itself; it asks how visual selection, once generated, gives rise to an overt saccade.

“However, the model does not attempt to explain target selection itself; it simply asks how visual selection, once it has been generated, gives rise to an overt saccade.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent models of sensorimotor transformation for eye movement control propose a process of gated accumulation, whereby accumulation in saccade movement cells is insulated from visual selection until an eye movement becomes appropriate.

“Recent models synthesizing these findings have proposed a process of gated accumulation, whereby the accumulation of information in saccade movement cells is insulated from visual selection unless (or until) an eye movement becomes appropriate (Lo and Wang, 2006; Purcell et al., 2012; Schall et al., 2011).”

• cites Lo and Wang, 2006 sourceNot checked yet

  Not checked in the free run.

• cites Purcell et al., 2012 sourceNot checked yet

  Not checked in the free run.

• cites Schall et al., 2011 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In natural behavior, eye movements rarely harvest primary rewards; instead, they sample information.

“In natural behavior however, eye movements rarely harvest primary rewards. Instead, they sample information.”

May combine two claims.

Parietal neurons encoding target selection report an action value representation (the term V in Rescorla-Wagner) and update this representation dynamically.

“As described above, parietal neurons encoding target selection are thought to report an action value representation – the term V in the Rescorla-Wagner equation – and to update this representation in dynamic fashion (Sugrue et al., 2004).”

• cites Sugrue et al., 2004 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Attention for action assigns low weight to cues that predict an uncertain reward and high weight to cues that make consistent predictions.

“As shown in the left panel of Fig. 2B, the system of “attention for action” is thought to assign low weight (associability) to cues that predict an uncertain reward, but a high weight for cues that make consistent predictions.”

• cites Pearce and Mackintosh, 2010Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The brain has several types of attention mechanism (attention for action, attention for learning, and attention for liking) that have different neuronal substrates and serve different behavioral roles.

Classified as probably not a substantial claim. May combine two claims.

The associability hypothesis postulates that attention for action assigns weight based on the reliability of a cue's predictions, and attention for learning assigns weight based on the variance of a cue's predictions.

“The associability hypothesis postulates that the systems of “attention for action” and “attention for learning” assign weight based, respectively, on the reliability and variance of a cue’s predictions (Pearce and Mackintosh, 2010).”

• cites Pearce and Mackintosh, 2010Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The brain has several types of attention mechanism (attention for action, attention for learning, and attention for liking) that have different neuronal substrates and serve different behavioral roles.

Probably not a substantial claim: may not pass S3 (load-bearing).

It is unclear whether the brain encodes the more cognitive type of reliability postulated by the associative learning field, which requires learning of complex relationships between separate events.

“It is unclear however whether the brain encodes the more cognitive type of reliability that is postulated by the associative learning field, which is not embedded in the stimulus itself but requires learning of complex relationships between separate events.”

May combine two claims.

Humans infer hidden models of complex tasks, and lower animals also possess this capacity.

“Converging evidence shows that humans indeed infer hidden models of complex tasks (Acuna and Schrater, 2010; Braun et al., 2010; Daw et al., 2011; Gershman and Niv, 2010; Yakushijin and Jacobs, 2011), and indirect evidence from tasks involving schemas or contextual associations suggests that lower animals also possess this capacity (Balan and Gottlieb, 2006; Braun et al., 2010; Johnson et al., 201”

• cites Acuna and Schrater, 2010Not checked yet

  Not checked in the free run.

• cites Braun et al., 2010 sourceNot checked yet

  Not checked in the free run.

• cites Daw et al., 2011 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A recent study using an information choice task proposed that the cognitive type of reliability is encoded in midbrain dopaminergic cells, but the findings remain open to alternative interpretations.

“While a recent study using an “information choice task” proposed that this type of reliability is encoded in midbrain dopaminergic cells (Bromberg-Martin and Hikosaka, 2009), the findings remain open to alternative interpretations.”

• cites Bromberg-Martin and Hikosaka, 2009 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The early dopamine response seems to signal a superordinate property of informativeness (or reliability) independent of a specific message, and to correspond to the monkeys' behavioral preference for the informative cue.

“This early response seems to signal a superordinate property of “informativeness” (or reliability) that is independent of a specific message, and to correspond to the monkeys’ behavioral preference for the informative cue.”

Classified as probably not a substantial claim. May combine two claims.

Monkeys modulate their anticipatory licking based on stimulus-reward associations and will stop licking when observing a low-reward cue.

“It is well known that monkeys modulate their anticipatory licking based on stimulus-reward associations, and will stop licking when observing a low-reward cue (Fiorillo et al., 2003).”

• cites Fiorillo et al., 2003 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The brain explicitly learns and potentially represents the reliability of the predictions generated by a cue.

“According to the associability idea, to achieve this type of control the brain will explicitly learn (and potentially represent) the reliability of the predictions generated by a cue (Pearce and Mackintosh, 2010).”

• cites Pearce and Mackintosh, 2010Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Subjects direct attention based on stimulus-reward associations and may gaze longer at high-reward versus low-reward cues.

“In addition as I mentioned above, subjects direct attention based on stimulus-reward associations, and may have gazed for longer periods at the high-reward versus the low-reward cue (e.g., the green cross versus green wave in Fig. 3A) (Hogarth et al., 2010).”

• cites Hogarth et al., 2010Not checked yet

  Not checked in the free run.

May combine two claims.

In the information choice task, reward outcomes were on average equal and fixed in all conditions, so monkeys could not increase their physical rewards with a specific choice.

“Notably, the reward outcomes themselves were on average equal and fixed in all conditions, so that monkeys could not increase their physical rewards with a specific choice.”

• cites Bromberg-Martin and Hikosaka, 2009 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Information selection can be remarkably complex, with most complexity deriving from the indirect benefits of acquiring information, which can be computed only by forward planning across a sequence of steps.

“However, even as they demonstrate the feasibility of this computation, the studies show that information selection can be remarkably complex. Most of the complexity derives from the indirect benefits of acquiring information, which can be computed only by means of forward planning across a sequence of steps.”

May combine two claims.

Both attention for action and attention for learning are value-neutral: they give equal weight to stimuli predicting low or high rewards, provided these make reliable predictions.

“Importantly however, both systems are value-neutral in the sense that they do not depend on expected reward: they give equal weight to stimuli predicting low or high rewards, provided these make reliable predictions.”

• cites Pearce and Mackintosh, 2010Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The brain has several types of attention mechanism (attention for action, attention for learning, and attention for liking) that have different neuronal substrates and serve different behavioral roles.

May combine two claims.

Monkeys reliably selected the informative target, suggesting they had an intrinsic preference for information.

“Nevertheless, monkeys reliably selected the informative target suggesting that they had an intrinsic preference for information.”

• cites Bromberg-Martin and Hikosaka, 2009 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The capacity for covert attention has been well-established in psychophysical research, and its correlates are found also in the frontal eye field.

“These neural responses are in some respect not surprising because the capacity for covert attention has been well-established in psychophysical research, and its correlates are found also in the frontal eye field (Schall et al., 2011; Thompson et al., 2005).”

• cites Schall et al., 2011 sourceNot checked yet

  Not checked in the free run.

• cites Thompson et al., 2005Not checked yet

  Not checked in the free run.

May combine two claims.

A model-free prediction error mechanism can be prohibitively slow in complex tasks and may erroneously assign credit to irrelevant steps.

“Such a mechanism can be prohibitively slow in complex tasks, and may erroneously assign credit to irrelevant steps (e.g., value the earlier decision to study for a final exam less than the more recent decision to wear a white shirt) (Rothkopf et al., 2007).”

• cites Rothkopf et al., 2007 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The value of an eye movement lies in reducing uncertainty and increasing the expected reward (probability of success) of a future action.

“An alternative explanation however emerges from studies of eye movements in natural behaviors, which suggests that the value of an eye movement lies in reducing uncertainty and increasing the expected reward (probability of success) of a future action (Ballard and Hayhoe, 2009; Hayhoe et al., 2012; Rothkopf et al., 2007; Tatler et al., 2011).”

• cites Ballard and Hayhoe, 2009Not checked yet

  Not checked in the free run.

• cites Hayhoe et al., 2012 sourceNot checked yet

  Not checked in the free run.

• cites Rothkopf et al., 2007 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Dopamine neurons had two types of responses on the information choice task: a prediction error response at the time of the actual cue, and an earlier response at the time of selection that was slightly stronger for informative than uninformative cues.

“Dopamine neurons (Fig. 3B) had two types of responses on the task. At the time of the actual information (marked “Cue”) in Fig. 3B, the neurons emitted the customary prediction error response which, as described above, was excitatory for a “good news” (big reward) cue and inhibitory for a “bad news” (small reward) cue. Of particular interest however was a response that preceded the actual cue and”

• cites Bromberg-Martin and Hikosaka, 2009 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The process of building internal models entails an executive mechanism that learns how to learn, infers the relevant dimension, and constrains moment-by-moment sampling of information to focus on the relevant dimension.

“As mentioned above in relation with the associability equation (eq. 2), this process entails an executive mechanism that learns how to learn. This mechanism infers the relevant dimension – e.g., using direct instruction or learning on longer time scales - and constrains the moment by moment sampling of information to focus on the relevant dimension.”

May combine two claims.

Attentional influences from attention for liking are difficult to overcome and may underlie maladaptive reactions in psychiatric disorders, such as enhanced susceptibility of addicted patients to drug-related cues.

“These attentional influences are difficult to overcome and may underlie maladaptive reactions in psychiatric disorders, such as the enhanced susceptibility of addicted patients to drug-related cues (Flagel et al., 2011).”

• cites Flagel et al., 2011 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The distinction between model-free and model-based computations is fundamental and may explain key differences between attention for action and attention for liking.

“As I discuss in the final section, the distinction between model-free and model-based computations is fundamental and may explain key differences between an “attention for action” and “attention for liking” mechanism.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Task-related and exploratory attention rely on separate neural circuits involving the medial frontal cortex versus the substantia nigra, amygdala, and parietal lobe.

“Neuropsychological studies in rats suggest that task-related and exploratory attention rely on separate neural circuits that involve, respectively, the medial frontal cortex (Maddux and Holland, 2011) versus the substantia nigra, amygdala and the parietal lobe (Maddux et al., 2007).”

• cites Maddux and Holland, 2011Not checked yet

  Not checked in the free run.

• cites Maddux et al., 2007 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The brain generates intrinsic rewards when it senses learning progress, measured by tracking prediction errors over time.

“One solution, emerging from the field of developmental robotics, is that the brain generates intrinsic rewards when it senses learning progress, which it measures by tracking its prediction errors over time (Oudeyer et al., 2007).”

• cites Oudeyer et al., 2007 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The intrinsic reward mechanism may motivate learning in the absence of external reward and has been effective in producing curiosity-like behaviors in robots.

“This mechanism may motivate learning even in the absence of an external reward, and has been very effective in producing curiosity-like behaviors - whereby robots remain spontaneously interested in activities of intermediate complexity where they improve their predictions but disengage from random (unlearnable) or from overlearned and “boring” tasks.”

• cites Oudeyer et al., 2007 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Novelty acts as an exploration bonus in reward seeking tasks and is encoded in dopamine cells as an intrinsic bonus for exploration.

“Novelty has been proposed to act as an exploration bonus in reward seeking tasks (Wittmann et al., 2008), and to be encoded in dopamine cells as an intrinsic bonus for exploration (Redgrave and Gurney, 2006).”

• cites Wittmann et al., 2008 sourceNot checked yet

  Not checked in the free run.

• cites Redgrave and Gurney, 2006 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Neural responses with combinatorial (mixed) properties are hallmarks of goal-directed cognitive control, and in the context of information selection may embody the bank of knowledge necessary for selecting cues.

“However, neural responses with combinatorial (mixed) properties are hallmarks of goal-directed cognitive control (Rigotti et al., 2010), and in the context of information selection may embody the bank of knowledge that is necessary for selecting cues.”

• cites Rigotti et al., 2010 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Exploratory attention is mediated by a separate system of attention for learning which allocates resources to uncertain rather than reliable cues.

“As described in Fig. 2B, studies of associative learning propose that exploratory attention is mediated by a separate system of “attention for learning” which, in contrast with “attention for action”, allocates resources to uncertain rather than reliable cues (Fig. 2B, center panel).”

• cites Pearce and Mackintosh, 2010Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The brain has several types of attention mechanism (attention for action, attention for learning, and attention for liking) that have different neuronal substrates and serve different behavioral roles.

Probably not a substantial claim: may not pass S3 (load-bearing).

Goal-directed exploration is motivated by reward expectations generalized based on prior experience.

“An alternative account is that goal directed exploration is not motivated by learning progress but by reward expectations that are generalized based on prior experience (Dayan, 2012).”

• cites Dayan, 2012Not checked yet

  Not checked in the free run.

May combine two claims.

Dopamine recipient structures also encode model-based prediction errors useful for detecting relevant information, and dopamine responses depend on actions as well as stimuli.

“Recent evidence from functional imaging experiments in humans suggests that dopamine recipient structures also encode model-based prediction errors that may be useful for detecting relevant information (Daw et al., 2011), and single-neuron studies in rats and monkeys show that dopamine responses depend not only on stimuli but also on actions (Morris et al., 2006; Takahashi et al., 2012) suggesting”

• cites Daw et al., 2011 sourceNot checked yet

  Not checked in the free run.

• cites Morris et al., 2006 sourceNot checked yet

  Not checked in the free run.

• cites Takahashi et al., 2012Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Neural responses to uncertainty or surprise have been reported in multiple structures and linked with variables such as arousal, anxiety, risk preference, or global learning rates.

“Neural responses to uncertainty or surprise have been reported in multiple structures (den Ouden et al., 2010; Fiorillo et al., 2003; Kepecs et al., 2008; McCoy and Platt, 2005; O'Neill and Schultz, 2010; Preuschoff et al., 2006; Preuschoff et al., 2008; Schultz et al., 2008; So and Stuphorn, 2012; Tobler et al., 2009) and have been linked with variables such as arousal, anxiety, risk preference,”

• cites den Ouden et al., 2010 sourceNot checked yet

  Not checked in the free run.

• cites Fiorillo et al., 2003 sourceNot checked yet

  Not checked in the free run.

• cites Kepecs et al., 2008 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

After prolonged training, the effects involved plasticity of the visual response, becoming insensitive to context and automatically transferring to a different task in which the pre-trained CS no longer predicted rewards.

“Moreover after prolonged training the effects seemed to involve plasticity of the visual response, since they became insensitive to context and automatically transferred to a different task in which the pre-trained CS no longer predicted rewards (Fig. 5E).”

Classified as probably not a substantial claim.

Other forms of automatic attention produced by salience or surprise are vital heuristics for allocating resources in very uncertain conditions, when the brain has not yet learnt how to learn.

“This raises the possibility that other forms of automatic attention that are produced by salience or surprise (Boehnke et al., 2011; Karacan and Hayhoe, 2008; Wittmann et al., 2008), rather than being mere weaknesses of a control mechanism, are vital heuristics for allocating resources in very uncertain conditions, when the brain has not yet learnt how to learn.”

• cites Boehnke et al., 2011 sourceNot checked yet

  Not checked in the free run.

• cites Karacan and Hayhoe, 2008Not checked yet

  Not checked in the free run.

• cites Wittmann et al., 2008 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The manual modulations were not free-standing limb motor responses but modulatory effects on visual selection, consistent with the later finding that reversible inactivation produced visual but not skeletal motor defects.

“These manual modulations were not free-standing limb motor responses but modulatory effects on visual selection (i.e., the effects were not seen if a distractor appeared in the receptive field; Fig. 4C, right), a conclusion consistent with the later finding that reversible inactivation produced visual but not skeletal motor defects (Balan and Gottlieb, 2009).”

• cites Balan and Gottlieb, 2009Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

To generate adaptive exploration, the brain must distinguish between at least two types of uncertainty: reducible and irreducible.

“It has been previously noted that to generate adaptive exploration the brain must distinguish between at least two types of uncertainty (Oudeyer et al., 2007; Payzan-LeNestour and Bossaerts, 2011; Yu and Dayan, 2005).”

• cites Oudeyer et al., 2007 sourceNot checked yet

  Not checked in the free run.

• cites Payzan-LeNestour and Bossaerts, 2011Not checked yet

  Not checked in the free run.

• cites Yu and Dayan, 2005 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Model-based accounts suggest that the distinction between attention for action and attention for learning may not be clear cut, and that even when the brain orients toward uncertain cues, it is with the goal of learning or reducing uncertainty.

“Model-based accounts however, suggest that this distinction may not be quite as clear cut, and that, even when the brain orients toward uncertain cues, it is with the goal of learning, or reducing the uncertainty regarding that cue.”

May combine two claims.

These findings are difficult to explain in a purely visual framework that casts target selection as a disembodied bias term, and are puzzling in an action-based framework that asks whether parietal areas are involved in skeletal or ocular actions.

“These findings are difficult to explain in a purely visual framework that casts target selection as a disembodied bias term (Fig. 1B). They are also puzzling in an action based framework that asks whether parietal areas are involved in skeletal or ocular actions (Snyder et al., 2000).”

• cites Snyder et al., 2000 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Generalization may call upon the same executive mechanisms of learning to learn: to generalize effectively the brain must know how to generalize, i.e., compare relevant aspects of different task structures.

“Interestingly, this form of generalization may call upon the same executive mechanisms of “learning to learn” that we discussed the previous section: to generalize effectively the brain must know how to generalize, i.e., compare the relevant aspects of the different task structures (Bavelier et al., 2012).”

• cites Bavelier et al., 2012 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In a well-known motion discrimination paradigm, the direction of motion of a sensory cue is thought to be discriminated by cells in the middle temporal area, while lateral intraparietal cells select the appropriate action (e.g., a specific saccade).

“For example in a well-known motion discrimination paradigm, the direction of motion of a sensory cue is thought to be discriminated by cells in the middle temporal area, while lateral intraparietal cells select the appropriate action (e.g., a specific saccade) (Gold and Shadlen, 2007).”

• cites Gold and Shadlen, 2007 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

By recognizing their tight links with selective attention we can use the oculomotor system to gain insight into these mechanisms.

“By recognizing their tight links with selective attention we can use the oculomotor system to gain insight into these mechanisms.”

May combine two claims.

Computing information value optimally is a costly and time-consuming operation that requires inference and advance planning for multiple future steps, and can itself be suboptimal in complex tasks.

“As we have seen in the preceding sections, computing information value optimally is a costly and time-consuming operation that requires inference and advance planning for multiple future steps, and can itself be suboptimal in complex tasks (Wilson and Niv, 2011).”

• cites Wilson and Niv, 2011Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

While some of these computations have been addressed in research with human observers, their cellular mechanisms remain largely unknown.

“While some of these computations have been addressed in research with human observers, their cellular mechanisms remain largely unknown.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

A goal-directed mechanism may assign value to stimuli based on their relevance to future actions, while an agnostic system simply prioritizes stimuli that signal changes in reward expectation.

“A goal-directed mechanism may assign value to stimuli based on their relevance to future actions, while an agnostic system simply prioritizes stimuli that signal changes in reward expectation.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

These operations include building internal models of external events, guiding behavior based on curiosity and exploration, and generating (and controlling) emotional biases in information processing.

“These include building internal models of external events, guiding behavior based on curiosity and exploration, and generating (and controlling) emotional biases in information processing.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

By appreciating the cognitive dimension of eye movement control we can begin use the full power of this system as a window into cognitive function.

“Second, by appreciating the cognitive dimension of eye movement control we can begin use the full power of this system as a window into cognitive function.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

To understand this question - as well as complex properties of the target selection response - we will need to understand the visual learning mechanisms by which the brain assigns meaning to visual cues, and the cognitive systems that assign value to these cues.

“To understand this question – as well as complex properties of the target selection response - we will need to understand the visual learning mechanisms by which the brain assigns meaning to visual cues, and the cognitive systems that assign value to these cues.”

Probably not a substantial claim: may not pass S2 (checkable).

Attention is the core cognitive system that mediates our active search for information - whether information is sought for a foreseeable, well-practiced action or in a more open-ended, exploratory fashion.

“I have argued that attention is the core cognitive system that mediates our active search for information – whether information is sought for a foreseeable, well-practiced action or in a more open-ended, exploratory fashion.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Existing research has used the oculomotor system to study variables involved in decision formation.

“As mentioned in the opening sections, existing research has used the oculomotor system to study variables involved in decision formation.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The need for selective attention becomes clear when we consider that, in addition to analyzing visual information, the brain must solve another highly complex task - namely, determine the significance and value of that information.

“The need for selective attention however becomes clear when we consider that, in addition to analyzing visual information the brain must solve another highly complex task – namely, determine the significance and value of that information.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

This view is consistent with reinforcement learning research, but it is not well integrated with studies of oculomotor control.

“While this view is consistent with reinforcement learning research, it is not well integrated with studies of oculomotor control.”

Probably not a substantial claim: may not pass S2 (checkable).

This distinct form of selection can entail highly sophisticated operations that may engage much of our cognitive function.

“As I have discussed above, this distinct form of selection can entail highly sophisticated operations that may engage much of our cognitive function.”

Classified as probably not a substantial claim.

This framework purports to explain oculomotor decisions as a sensory-to-motor transfer without invoking the concept of selective attention.

“This framework purports to explain oculomotor decisions as a sensory-to-motor transfer without invoking the concept of selective attention.”

Probably not a substantial claim: may not pass S2 (checkable).

This integration has become necessary for understanding core open questions in attention control - i.e., how the brain decides when and to what to attend.

“First, as I described in the earlier sections, this integration has become necessary for understanding core open questions in attention control – i.e., how the brain decides when and to what to attend.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A unique hallmark of higher intelligence is a vast capacity for learning and prediction.

“While all living organisms take actions that bring biological rewards, a unique hallmark of higher intelligence is a vast capacity for learning and prediction (Friston, 2010).”

• cites Friston, 2010 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The findings are consistent with several - not mutually exclusive - mechanisms.

“The findings are consistent with several – not mutually exclusive – mechanisms.”