Reinforcement learningReinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.
Q-learningQ-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.
Basal ganglia diseaseBasal ganglia disease is a group of physical problems that occur when the group of nuclei in the brain known as the basal ganglia fail to properly suppress unwanted movements or to properly prime upper motor neuron circuits to initiate motor function. Research indicates that increased output of the basal ganglia inhibits thalamocortical projection neurons. Proper activation or deactivation of these neurons is an integral component for proper movement.
StriatumThe striatum, or corpus striatum (also called the striate nucleus), is a nucleus (a cluster of neurons) in the subcortical basal ganglia of the forebrain. The striatum is a critical component of the motor and reward systems; receives glutamatergic and dopaminergic inputs from different sources; and serves as the primary input to the rest of the basal ganglia. Functionally, the striatum coordinates multiple aspects of cognition, including both motor and action planning, decision-making, motivation, reinforcement, and reward perception.
Basal gangliaThe basal ganglia (BG), or basal nuclei, are a group of subcortical nuclei, of varied origin, in the brains of vertebrates. In humans, and some primates, there are some differences, mainly in the division of the globus pallidus into an external and internal region, and in the division of the striatum. The basal ganglia are situated at the base of the forebrain and top of the midbrain. Basal ganglia are strongly interconnected with the cerebral cortex, thalamus, and brainstem, as well as several other brain areas.
Deep reinforcement learningDeep reinforcement learning (deep RL) is a subfield of machine learning that combines reinforcement learning (RL) and deep learning. RL considers the problem of a computational agent learning to make decisions by trial and error. Deep RL incorporates deep learning into the solution, allowing agents to make decisions from unstructured input data without manual engineering of the state space. Deep RL algorithms are able to take in very large inputs (e.g.
Medium spiny neuronMedium spiny neurons (MSNs), also known as spiny projection neurons (SPNs), are a special type of GABAergic inhibitory cell representing 95% of neurons within the human striatum, a basal ganglia structure. Medium spiny neurons have two primary phenotypes (characteristic types): D1-type MSNs of the direct pathway and D2-type MSNs of the indirect pathway. Most striatal MSNs contain only D1-type or D2-type dopamine receptors, but a subpopulation of MSNs exhibit both phenotypes.
MemoryMemory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered, it would be impossible for language, relationships, or personal identity to develop. Memory loss is usually described as forgetfulness or amnesia. Memory is often understood as an informational processing system with explicit and implicit functioning that is made up of a sensory processor, short-term (or working) memory, and long-term memory.
NeurotransmitterA neurotransmitter is a signaling molecule secreted by a neuron to affect another cell across a synapse. The cell receiving the signal, or target cell, may be another neuron, but could also be a gland or muscle cell. Neurotransmitters are released from synaptic vesicles into the synaptic cleft where they are able to interact with neurotransmitter receptors on the target cell. The neurotransmitter's effect on the target cell is determined by the receptor it binds to.
Working memoryWorking memory is a cognitive system with a limited capacity that can hold information temporarily. It is important for reasoning and the guidance of decision-making and behavior. Working memory is often used synonymously with short-term memory, but some theorists consider the two forms of memory distinct, assuming that working memory allows for the manipulation of stored information, whereas short-term memory only refers to the short-term storage of information.
Baddeley's model of working memoryBaddeley's model of working memory is a model of human memory proposed by Alan Baddeley and Graham Hitch in 1974, in an attempt to present a more accurate model of primary memory (often referred to as short-term memory). Working memory splits primary memory into multiple components, rather than considering it to be a single, unified construct. Baddeley & Hitch proposed their three-part working memory model as an alternative to the short-term store in Atkinson & Shiffrin's 'multi-store' memory model (1968).
LearningLearning is the process of acquiring new understanding, knowledge, behaviors, skills, values, attitudes, and preferences. The ability to learn is possessed by humans, animals, and some machines; there is also evidence for some kind of learning in certain plants. Some learning is immediate, induced by a single event (e.g. being burned by a hot stove), but much skill and knowledge accumulate from repeated experiences. The changes induced by learning often last a lifetime, and it is hard to distinguish learned material that seems to be "lost" from that which cannot be retrieved.
ForgettingForgetting or disremembering is the apparent loss or modification of information already encoded and stored in an individual's short or long-term memory. It is a spontaneous or gradual process in which old memories are unable to be recalled from memory storage. Problems with remembering, learning and retaining new information are a few of the most common complaints of older adults. Studies show that retention improves with increased rehearsal. This improvement occurs because rehearsal helps to transfer information into long-term memory.
Action selectionAction selection is a way of characterizing the most basic problem of intelligent systems: what to do next. In artificial intelligence and computational cognitive science, "the action selection problem" is typically associated with intelligent agents and animats—artificial systems that exhibit complex behaviour in an agent environment. The term is also sometimes used in ethology or animal behavior. One problem for understanding action selection is determining the level of abstraction used for specifying an "act".
Forgetting curveThe forgetting curve hypothesizes the decline of memory retention in time. This curve shows how information is lost over time when there is no attempt to retain it. A related concept is the strength of memory that refers to the durability that memory traces in the brain. The stronger the memory, the longer period of time that a person is able to recall it. A typical graph of the forgetting curve purports to show that humans tend to halve their memory of newly learned knowledge in a matter of days or weeks unless they consciously review the learned material.
Cortico-basal ganglia-thalamo-cortical loopThe cortico-basal ganglia-thalamo-cortical loop (CBGTC loop) is a system of neural circuits in the brain. The loop involves connections between the cortex, the basal ganglia, the thalamus, and back to the cortex. It is of particular relevance to hyperkinetic and hypokinetic movement disorders, such as Parkinson's disease and Huntington's disease, as well as to mental disorders of control, such as attention deficit hyperactivity disorder (ADHD), obsessive–compulsive disorder (OCD), and Tourette syndrome.
Action potentialAn action potential occurs when the membrane potential of a specific cell rapidly rises and falls. This depolarization then causes adjacent locations to similarly depolarize. Action potentials occur in several types of animal cells, called excitable cells, which include neurons, muscle cells, and in some plant cells. Certain endocrine cells such as pancreatic beta cells, and certain cells of the anterior pituitary gland are also excitable cells.
Long-term memoryLong-term memory (LTM) is the stage of the Atkinson–Shiffrin memory model in which informative knowledge is held indefinitely. It is defined in contrast to short-term and working memory, which persist for only about 18 to 30 seconds. LTM is commonly labelled as "explicit memory" (declarative), as well as "episodic memory," "semantic memory," "autobiographical memory," and "implicit memory" (procedural memory). The idea of separate memories for short- and long-term storage originated in the 19th century.
Learning rateIn machine learning and statistics, the learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function. Since it influences to what extent newly acquired information overrides old information, it metaphorically represents the speed at which a machine learning model "learns". In the adaptive control literature, the learning rate is commonly referred to as gain. In setting a learning rate, there is a trade-off between the rate of convergence and overshooting.
Interference theoryThe interference theory is a theory regarding human memory. Interference occurs in learning. The notion is that memories encoded in long-term memory (LTM) are forgotten and cannot be retrieved into short-term memory (STM) because either memory could interfere with the other. There is an immense number of encoded memories within the storage of LTM. The challenge for memory retrieval is recalling the specific memory and working in the temporary workspace provided in STM.
