A State-Transition-Free Delayed-Feedback Task Elicits Heterogeneous Human Responses.

Humans and nonhuman animals learn to perform actions by associating actions with outcomes. In everyday life, outcomes sometimes occur only after a delay, and at an unexpected moment. The ability to connect actions and delayed outcomes has received less attention than performance in tasks where rewards follow the most recent action.

Hippocampal Sharp-Wave Ripples Decrease during Physical Actions Including Consummatory Behavior in Immobile Rodents.

Hippocampal sharp-wave ripples (SWRs) are intermittent, fast synchronous oscillations that play a pivotal role in memory formation. It has been well established that SWRs occur during "consummatory behaviors," e.g.

Goal-oriented inference of environment from redundant observations.

The agent learns to organize decision behavior to achieve a behavioral goal, such as reward maximization, and reinforcement learning is often used for this optimization. Learning an optimal behavioral strategy is difficult under the uncertainty that events necessary for learning are only partially observable, called as Partially Observable Markov Decision Process (POMDP). However, the real-world environment also gives many events irrelevant to reward delivery and an optimal behavioral strategy.

Reward expectation enhances action-related activity of nigral dopaminergic and two striatal output pathways.

Neurons comprising nigrostriatal system play important roles in action selection. However, it remains unclear how this system integrates recent outcome information with current action (movement) and outcome (reward or no reward) information to achieve appropriate subsequent action. We examined how neuronal activity of substantia nigra pars compacta (SNc) and dorsal striatum reflects the level of reward expectation from recent outcomes in rats performing a reward-based choice task.

State-transition-free reinforcement learning in chimpanzees (Pan troglodytes).

The outcome of an action often occurs after a delay. One solution for learning appropriate actions from delayed outcomes is to rely on a chain of state transitions. Another solution, which does not rest on state transitions, is to use an eligibility trace (ET) that directly bridges a current outcome and multiple past actions via transient memories.

Rat hippocampal CA1 region represents learning-related action and reward events with shorter latency than the lateral entorhinal cortex.

The hippocampus and entorhinal cortex are deeply involved in learning and memory. However, little is known how ongoing events are processed in the hippocampal-entorhinal circuit. By recording from head-fixed rats during action-reward learning, here we show that the action and reward events are represented differently in the hippocampal CA1 region and lateral entorhinal cortex (LEC).

Automated and parallelized spike collision tests to identify spike signal projections.

The spike collision test is a highly reliable technique to identify the axonal projection of a neuron recorded electrophysiologically for investigating functional spike information among brain areas. It is potentially applicable to more neuronal projections by combining multi-channel recording with optogenetic stimulation. Yet, it remains inefficient and laborious because an experimenter must visually select spikes in every channel and manually repeat spike collision tests for each neuron serially.

Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior.

We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors.

Computerized intertemporal choice task in chimpanzees (Pan troglodytes) with/without postreward delay.

In intertemporal choice (ITC) tasks, animals are presented with alternative choices between a smaller reward that becomes available sooner and a larger reward that becomes available later. To equate the duration of a trial across the 2 options, postreward delays (PRDs) are inserted after the delivery of the reward. Animals need to incorporate this to increase the long-term reward rate.

Differential Changes in the Lateralized Activity of Identified Projection Neurons of Motor Cortex in Hemiparkinsonian Rats.

In the parkinsonian state, the motor cortex and basal ganglia (BG) undergo dynamic remodeling of movement representation. One such change is the loss of the normal contralateral lateralized activity pattern. The increase in the number of movement-related neurons responding to ipsilateral or bilateral limb movements may cause motor problems, including impaired balance, reduced bimanual coordination, and abnormal mirror movements.

Monitoring and Updating of Action Selection for Goal-Directed Behavior through the Striatal Direct and Indirect Pathways.

The basal ganglia play key roles in adaptive behaviors guided by reward and punishment. However, despite accumulating knowledge, few studies have tested how heterogeneous signals in the basal ganglia are organized and coordinated for goal-directed behavior. In this study, we investigated neuronal signals of the direct and indirect pathways of the basal ganglia as rats performed a lever push/pull task for a probabilistic reward.

Area-specific Modulation of Functional Cortical Activity During Block-based and Trial-based Proactive Inhibition.

Animals can suppress their behavioral response in advance according to changes in environmental context (proactive inhibition: delaying the start of response), a process in which several cortical areas may participate. However, it remains unclear how this process is adaptively regulated according to contextual changes on different timescales. To address the issue, we used an improved stop-signal task paradigm to behaviorally and electrophysiologically characterize the temporal aspect of proactive inhibition in head-fixed rats.

Distinct Laterality in Forelimb-Movement Representations of Rat Primary and Secondary Motor Cortical Neurons with Intratelencephalic and Pyramidal Tract Projections.

Two distinct motor areas, the primary and secondary motor cortices (M1 and M2), play crucial roles in voluntary movement in rodents. The aim of this study was to characterize the laterality in motor cortical representations of right and left forelimb movements. To achieve this goal, we developed a novel behavioral task, the Right-Left Pedal task, in which a head-restrained male rat manipulates a right or left pedal with the corresponding forelimb.

In Vivo Spiking Dynamics of Intra- and Extratelencephalic Projection Neurons in Rat Motor Cortex.

In motor cortex, 2 types of deep layer pyramidal cells send their axons to other areas: intratelencephalic (IT)-type neurons specifically project bilaterally to the cerebral cortex and striatum, whereas neurons of the extratelencephalic (ET)-type, termed conventionally pyramidal tract-type, project ipsilaterally to the thalamus and other areas. Although they have totally different synaptic and membrane potential properties in vitro, little is known about the differences between them in ongoing spiking dynamics in vivo. We identified IT-type and ET-type neurons, as well as fast-spiking-type interneurons, using novel multineuronal analysis based on optogenetically evoked spike collision along their axons in behaving/resting rats expressing channelrhodopsin-2 (Multi-Linc method).

A balanced motor primitive framework can simultaneously explain motor learning in unimanual and bimanual movements.

Certain theoretical frameworks have successfully explained motor learning in either unimanual or bimanual movements. However, no single theoretical framework can comprehensively explain motor learning in both types of movement because the relationship between these two types of movement remains unclear. Although our recent model of a balanced motor primitive framework attempted to simultaneously explain motor learning in unimanual and bimanual movements, this model focused only on a limited subset of bimanual movements and therefore did not elucidate the relationships between unimanual movements and various bimanual movements.

Similarity in Neuronal Firing Regimes across Mammalian Species.

Unlabelled: The architectonic subdivisions of the brain are believed to be functional modules, each processing parts of global functions. Previously, we showed that neurons in different regions operate in different firing regimes in monkeys. It is possible that firing regimes reflect differences in underlying information processing, and consequently the firing regimes in homologous regions across animal species might be similar.

Balanced motor primitive can explain generalization of motor learning effects between unimanual and bimanual movements.

Motor learning in unimanual and bimanual planar reaching movements has been intensively investigated. Although distinct theoretical frameworks have been proposed for each of these reaching movements, the relationship between these movements remains unclear. In particular, the generalization of motor learning effects (transfer of learning effects) between unimanual and bimanual movements has yet to be successfully explained.

Anomalous neuronal responses to fluctuated inputs.

The irregular firing of a cortical neuron is thought to result from a highly fluctuating drive that is generated by the balance of excitatory and inhibitory synaptic inputs. A previous study reported anomalous responses of the Hodgkin-Huxley neuron to the fluctuated inputs where an irregularity of spike trains is inversely proportional to an input irregularity. In the current study, we investigated the origin of these anomalous responses with the Hindmarsh-Rose neuron model, map-based models, and a simple mixture of interspike interval distributions.

Spatial information enhanced by non-spatial information in hippocampal granule cells.

The hippocampus organizes sequential memory composed of non-spatial information (such as objects and odors) and spatial information (places). The dentate gyrus (DG) in the hippocampus receives two types of information from the lateral and medial entorhinal cortices. Non-spatial and spatial information is delivered respectively to distal and medial dendrites (MDs) of granule cells (GCs) within the molecular layer in the DG.

Different modulation of common motor information in rat primary and secondary motor cortices.

Rodents have primary and secondary motor cortices that are involved in the execution of voluntary movements via their direct and parallel projections to the spinal cord. However, it is unclear whether the rodent secondary motor cortex has any motor function distinct from the primary motor cortex to properly control voluntary movements. In the present study, we quantitatively examined neuronal activity in the caudal forelimb area (CFA) of the primary motor cortex and rostral forelimb area (RFA) of the secondary motor cortex in head-fixed rats performing forelimb movements (pushing, holding, and pulling a lever).

Immediate return preference emerged from a synaptic learning rule for return maximization.

Animals including human often prefer immediate returns to larger delayed returns. It holds true in the human communications. Standard interpretation of the immediate return preference is that an animal might subjectively discount the value of a delayed reward, and that might choose the larger valued one.

Reinforcement learning for discounted values often loses the goal in the application to animal learning.

The impulsive preference of an animal for an immediate reward implies that it might subjectively discount the value of potential future outcomes. A theoretical framework to maximize the discounted subjective value has been established in the reinforcement learning theory. The framework has been successfully applied in engineering.