Convolutional neural networkConvolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.
Residual neural networkA Residual Neural Network (a.k.a. Residual Network, ResNet) is a deep learning model in which the weight layers learn residual functions with reference to the layer inputs. A Residual Network is a network with skip connections that perform identity mappings, merged with the layer outputs by addition. It behaves like a Highway Network whose gates are opened through strongly positive bias weights. This enables deep learning models with tens or hundreds of layers to train easily and approach better accuracy when going deeper.
Deep learningDeep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.
Master's degreeA master's degree (from Latin magister) is a postgraduate academic degree awarded by universities or colleges upon completion of a course of study demonstrating mastery or a high-order overview of a specific field of study or area of professional practice. A master's degree normally requires previous study at the bachelor's level, either as a separate degree or as part of an integrated course.
Academic degreeAn academic degree is a qualification awarded to a student upon successful completion of a course of study in higher education, usually at a college or university. These institutions often offer degrees at various levels, usually divided into undergraduate and postgraduate degrees. The most common undergraduate degree is the bachelor's degree, although some educational systems offer lower level undergraduate degrees such as associate and foundation degrees. Common postgraduate degrees include master's degrees and doctorates.
Associate degreeAn associate degree or associate's degree is an undergraduate degree awarded after a course of post-secondary study lasting two to three years. It is a level of academic qualification above a high school diploma and below a bachelor's degree. The first associate degrees were awarded in the UK (where they are no longer awarded) in 1873 before spreading to the US in 1898. In the United States, the associate degree may allow transfer into the third year of a bachelor's degree.
Artificial neural networkArtificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.
Bachelor's degreeA bachelor's degree (from Middle Latin baccalaureus) or baccalaureate (from Modern Latin baccalaureatus) is an undergraduate academic degree awarded by colleges and universities upon completion of a course of study lasting three to six years (depending on institution and academic discipline). The two most common bachelor's degrees are the Bachelor of Arts (BA) and the Bachelor of Science (BS or BSc).
Honours degreeHonours degree has various meanings in the context of different degrees and education systems. Most commonly it refers to a variant of the undergraduate bachelor's degree containing a larger volume of material or a higher standard of study, or both, rather than an "ordinary", "general" or "pass" bachelor's degree. Honours degrees are sometimes indicated by "Hons" after the degree abbreviation, with various punctuation according to local custom, e.g. "BA (Hons)", "B.A., Hons", etc.
Law degreeA law degree is an academic degree conferred for studies in law. Such degrees are generally preparation for legal careers. But while their curricula may be reviewed by legal authority, they do not confer a license themselves. A legal license is granted by examination, and exercised locally. The law degree can have local, international, and world-wide aspects, such as in England and Wales, where the Legal Practice Course or passing Solicitors Qualifying Examination (SQE) is required to become a solicitor or the Bar Professional Training Course (BPTC) to become a barrister.
Engineer's degreeAn engineer's degree is an advanced academic degree in engineering which is conferred in Europe, some countries of Latin America, North Africa and a few institutions in the United States. The degree may require a thesis but always requires a non-abstract project. Through the Canadian Engineering Accreditation Board (CEAB), Engineers Canada accredits Canadian undergraduate engineering programs that meet the standards of the profession.
Computer data storageComputer data storage is a technology consisting of computer components and recording media that are used to retain digital data. It is a core function and fundamental component of computers. The central processing unit (CPU) of a computer is what manipulates data by performing computations. In practice, almost all computers use a storage hierarchy, which puts fast but expensive and small storage options close to the CPU and slower but less expensive and larger options further away.
Transformer (machine learning model)A transformer is a deep learning architecture that relies on the parallel multi-head attention mechanism. The modern transformer was proposed in the 2017 paper titled 'Attention Is All You Need' by Ashish Vaswani et al., Google Brain team. It is notable for requiring less training time than previous recurrent neural architectures, such as long short-term memory (LSTM), and its later variation has been prevalently adopted for training large language models on large (language) datasets, such as the Wikipedia corpus and Common Crawl, by virtue of the parallelized processing of input sequence.
Dynamic random-access memoryDynamic random-access memory (dynamic RAM or DRAM) is a type of random-access semiconductor memory that stores each bit of data in a memory cell, usually consisting of a tiny capacitor and a transistor, both typically based on metal–oxide–semiconductor (MOS) technology. While most DRAM memory cell designs use a capacitor and transistor, some only use two transistors. In the designs where a capacitor is used, the capacitor can either be charged or discharged; these two states are taken to represent the two values of a bit, conventionally called 0 and 1.
Honorary degreeAn honorary degree is an academic degree for which a university (or other degree-awarding institution) has waived all of the usual requirements. It is also known by the Latin phrases honoris causa ("for the sake of the honour") or ad honorem ("to the honour"). The degree is typically a doctorate or, less commonly, a master's degree, and may be awarded to someone who has no prior connection with the academic institution or no previous postsecondary education.
Non-volatile memoryNon-volatile memory (NVM) or non-volatile storage is a type of computer memory that can retain stored information even after power is removed. In contrast, volatile memory needs constant power in order to retain data. Non-volatile memory typically refers to storage in semiconductor memory chips, which store data in floating-gate memory cells consisting of floating-gate MOSFETs (metal–oxide–semiconductor field-effect transistors), including flash memory storage such as NAND flash and solid-state drives (SSD).
Activation functionActivation function of a node in an artificial neural network is a function that calculates the output of the node (based on its inputs and the weights on individual inputs). Nontrivial problems can be solved only using a nonlinear activation function. Modern activation functions include the smooth version of the ReLU, the GELU, which was used in the 2018 BERT model, the logistic (sigmoid) function used in the 2012 speech recognition model developed by Hinton et al, the ReLU used in the 2012 AlexNet computer vision model and in the 2015 ResNet model.
Flash memoryFlash memory is an electronic non-volatile computer memory storage medium that can be electrically erased and reprogrammed. The two main types of flash memory, NOR flash and NAND flash, are named for the NOR and NAND logic gates. Both use the same cell design, consisting of floating gate MOSFETs. They differ at the circuit level depending on whether the state of the bit line or word lines is pulled high or low: in NAND flash, the relationship between the bit line and the word lines resembles a NAND gate; in NOR flash, it resembles a NOR gate.
Memory cell (computing)The memory cell is the fundamental building block of computer memory. The memory cell is an electronic circuit that stores one bit of binary information and it must be set to store a logic 1 (high voltage level) and reset to store a logic 0 (low voltage level). Its value is maintained/stored until it is changed by the set/reset process. The value in the memory cell can be accessed by reading it. Over the history of computing, different memory cell architectures have been used, including core memory and bubble memory.
3D XPoint3D XPoint (pronounced three-D cross point) is a discontinued non-volatile memory (NVM) technology developed jointly by Intel and Micron Technology. It was announced in July 2015 and was available on the open market under the brand name Optane (Intel) from April 2017 to July 2022. Bit storage is based on a change of bulk resistance, in conjunction with a stackable cross-grid data access array. Initial prices are less than dynamic random-access memory (DRAM) but more than flash memory.
