Reinforcement learningReinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.
Q-learningQ-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.
DiscountingIn finance, discounting is a mechanism in which a debtor obtains the right to delay payments to a creditor, for a defined period of time, in exchange for a charge or fee. Essentially, the party that owes money in the present purchases the right to delay the payment until some future date. This transaction is based on the fact that most people prefer current interest to delayed interest because of mortality effects, impatience effects, and salience effects.
Discounted cash flowThe discounted cash flow (DCF) analysis, in finance, is a method used to value a security, project, company, or asset, that incorporates the time value of money. Discounted cash flow analysis is widely used in investment finance, real estate development, corporate financial management, and patent valuation. Used in industry as early as the 1700s or 1800s, it was widely discussed in financial economics in the 1960s, and U.S. courts began employing the concept in the 1980s and 1990s.
Temporal difference learningTemporal difference (TD) learning refers to a class of model-free reinforcement learning methods which learn by bootstrapping from the current estimate of the value function. These methods sample from the environment, like Monte Carlo methods, and perform updates based on current estimates, like dynamic programming methods. While Monte Carlo methods only adjust their estimates once the final outcome is known, TD methods adjust predictions to match later, more accurate, predictions about the future before the final outcome is known.
Exponential discountingIn economics exponential discounting is a specific form of the discount function, used in the analysis of choice over time (with or without uncertainty). Formally, exponential discounting occurs when total utility is given by where ct is consumption at time t, is the exponential discount factor, and u is the instantaneous utility function. In continuous time, exponential discounting is given by Exponential discounting implies that the marginal rate of substitution between consumption at any pair of points in time depends only on how far apart those two points are.
Hyperbolic discountingIn economics, hyperbolic discounting is a time-inconsistent model of delay discounting. It is one of the cornerstones of behavioral economics and its brain-basis is actively being studied by neuroeconomics researchers. According to the discounted utility approach, intertemporal choices are no different from other choices, except that some consequences are delayed and hence must be anticipated and discounted (i.e., reweighted to take into account the delay). Given two similar rewards, humans show a preference for one that arrives sooner rather than later.
Valuation using discounted cash flowsValuation using discounted cash flows (DCF valuation) is a method of estimating the current value of a company based on projected future cash flows adjusted for the time value of money. The cash flows are made up of those within the “explicit” forecast period, together with a continuing or terminal value that represents the cash flow stream after the forecast period. In several contexts, DCF valuation is referred to as the "income approach".
Time preferenceIn economics, time preference (or time discounting, delay discounting, temporal discounting, long-term orientation) is the current relative valuation placed on receiving a good or some cash at an earlier date compared with receiving it at a later date. Time preferences are captured mathematically in the discount function. The higher the time preference, the higher the discount placed on returns receivable or costs payable in the future. One of the factors that may determine an individual's time preference is how long that individual has lived.
