Reinforcement learningReinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.
Sequential analysisIn statistics, sequential analysis or sequential hypothesis testing is statistical analysis where the sample size is not fixed in advance. Instead data are evaluated as they are collected, and further sampling is stopped in accordance with a pre-defined stopping rule as soon as significant results are observed. Thus a conclusion may sometimes be reached at a much earlier stage than would be possible with more classical hypothesis testing or estimation, at consequently lower financial and/or human cost.
Q-learningQ-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.
Temporal difference learningTemporal difference (TD) learning refers to a class of model-free reinforcement learning methods which learn by bootstrapping from the current estimate of the value function. These methods sample from the environment, like Monte Carlo methods, and perform updates based on current estimates, like dynamic programming methods. While Monte Carlo methods only adjust their estimates once the final outcome is known, TD methods adjust predictions to match later, more accurate, predictions about the future before the final outcome is known.
CUSUMIn statistical quality control, the CUsUM (or cumulative sum control chart) is a sequential analysis technique developed by E. S. Page of the University of Cambridge. It is typically used for monitoring change detection. CUSUM was announced in Biometrika, in 1954, a few years after the publication of Wald's sequential probability ratio test (SPRT). E. S. Page referred to a "quality number" , by which he meant a parameter of the probability distribution; for example, the mean.
Design of experimentsThe design of experiments (DOE or DOX), also known as experiment design or experimental design, is the design of any task that aims to describe and explain the variation of information under conditions that are hypothesized to reflect the variation. The term is generally associated with experiments in which the design introduces conditions that directly affect the variation, but may also refer to the design of quasi-experiments, in which natural conditions that influence the variation are selected for observation.
Martingale (probability theory)In probability theory, a martingale is a sequence of random variables (i.e., a stochastic process) for which, at a particular time, the conditional expectation of the next value in the sequence is equal to the present value, regardless of all prior values. Originally, martingale referred to a class of betting strategies that was popular in 18th-century France. The simplest of these strategies was designed for a game in which the gambler wins their stake if a coin comes up heads and loses it if the coin comes up tails.
Doob's martingale convergence theoremsIn mathematics specifically, in the theory of stochastic processes Doob's martingale convergence theorems are a collection of results on the limits of supermartingales, named after the American mathematician Joseph L. Doob. Informally, the martingale convergence theorem typically refers to the result that any supermartingale satisfying a certain boundedness condition must converge.
AI alignmentIn the field of artificial intelligence (AI), AI alignment research aims to steer AI systems towards humans' intended goals, preferences, or ethical principles. An AI system is considered aligned if it advances the intended objectives. A misaligned AI system pursues some objectives, but not the intended ones. It can be challenging for AI designers to align an AI system because it can be difficult for them to specify the full range of desired and undesired behaviors.
GristmillA gristmill (also: grist mill, corn mill, flour mill, feed mill or feedmill) grinds cereal grain into flour and middlings. The term can refer to either the grinding mechanism or the building that holds it. Grist is grain that has been separated from its chaff in preparation for grinding. Watermill List of ancient watermills and List of early medieval watermills The Greek geographer Strabo reports in his Geography a water-powered grain-mill to have existed near the palace of king Mithradates VI Eupator at Cabira, Asia Minor, before 71 BC.
WatermillA watermill or water mill is a mill that uses hydropower. It is a structure that uses a water wheel or water turbine to drive a mechanical process such as milling (grinding), rolling, or hammering. Such processes are needed in the production of many material goods, including flour, lumber, paper, textiles, and many metal products. These watermills may comprise gristmills, sawmills, paper mills, textile mills, hammermills, trip hammering mills, rolling mills, wire drawing mills.
WindmillA windmill is a structure that converts wind power into rotational energy using vanes called sails or blades, by tradition specifically to mill grain (gristmills), but in some parts of the English-speaking world the term has also been extended to encompass windpumps, wind turbines, and other applications. The term wind engine is also sometimes used to describe such devices. Windmills were used throughout the high medieval and early modern periods; the horizontal or panemone windmill first appeared in Persia during the 9th century, and the vertical windmill first appeared in northwestern Europe in the 12th century.
SawmillA sawmill (saw mill, saw-mill) or lumber mill is a facility where logs are cut into lumber. Modern sawmills use a motorized saw to cut logs lengthwise to make long pieces, and crosswise to length depending on standard or custom sizes (dimensional lumber). The "portable" sawmill is simple to operate. The log lies flat on a steel bed, and the motorized saw cuts the log horizontally along the length of the bed, by the operator manually pushing the saw.
Mill (grinding)A mill is a device, often a structure, machine or kitchen appliance, that breaks solid materials into smaller pieces by grinding, crushing, or cutting. Such comminution is an important unit operation in many processes. There are many different types of mills and many types of materials processed in them. Historically mills were powered by hand or by animals (e.g., via a hand crank), working animal (e.g., horse mill), wind (windmill) or water (watermill). In modern era, they are usually powered by electricity.
Hardy spaceIn complex analysis, the Hardy spaces (or Hardy classes) Hp are certain spaces of holomorphic functions on the unit disk or upper half plane. They were introduced by Frigyes Riesz , who named them after G. H. Hardy, because of the paper . In real analysis Hardy spaces are certain spaces of distributions on the real line, which are (in the sense of distributions) boundary values of the holomorphic functions of the complex Hardy spaces, and are related to the Lp spaces of functional analysis.
OpenAIOpenAI is an American artificial intelligence (AI) research laboratory consisting of the non-profit OpenAI, Inc. and its for-profit subsidiary corporation OpenAI, L.P.. OpenAI conducts research on artificial intelligence with the declared intention of developing "safe and beneficial" artificial general intelligence, which it defines as "highly autonomous systems that outperform humans at most economically valuable work".
Artificial intelligenceArtificial intelligence (AI) is the intelligence of machines or software, as opposed to the intelligence of human beings or animals. AI applications include advanced web search engines (e.g., Google Search), recommendation systems (used by YouTube, Amazon, and Netflix), understanding human speech (such as Siri and Alexa), self-driving cars (e.g., Waymo), generative or creative tools (ChatGPT and AI art), and competing at the highest level in strategic games (such as chess and Go).
PolicyPolicy is a deliberate system of guidelines to guide decisions and achieve rational outcomes. A policy is a statement of intent and is implemented as a procedure or protocol. Policies are generally adopted by a governance body within an organization. Policies can assist in both subjective and objective decision making. Policies used in subjective decision-making usually assist senior management with decisions that must be based on the relative merits of a number of factors, and as a result, are often hard to test objectively, e.
Policy analysisPolicy analysis or public policy analysis is a technique used in the public administration sub-field of political science to enable civil servants, nonprofit organizations, and others to examine and evaluate the available options to implement the goals of laws and elected officials. People who regularly use policy analysis skills and techniques on the job, particularly those who use it as a major part of their job duties are generally known by the title Policy Analyst.
Public policyPublic policy is an institutionalized proposal or a decided set of elements like laws, regulations, guidelines, and actions to solve or address relevant and real-world problems, guided by a conception and often implemented by programs. The implementation of public policy is known as public administration. Public policy can be considered to be the sum of a government's direct and indirect activities and has been conceptualized in a variety of ways. They are created and/or enacted on behalf of the public typically by a government.
