Data cleansingData cleansing or data cleaning is the process of detecting and correcting (or removing) corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data. Data cleansing may be performed interactively with data wrangling tools, or as batch processing through scripting or a data quality firewall. After cleansing, a data set should be consistent with other similar data sets in the system.
Data transformation (computing)In computing, data transformation is the process of converting data from one format or structure into another format or structure. It is a fundamental aspect of most data integration and data management tasks such as data wrangling, data warehousing, data integration and application integration. Data transformation can be simple or complex based on the required changes to the data between the source (initial) data and the target (final) data. Data transformation is typically performed via a mixture of manual and automated steps.
EngineAn engine or motor is a machine designed to convert one or more forms of energy into mechanical energy. Available energy sources include potential energy (e.g. energy of the Earth's gravitational field as exploited in hydroelectric power generation), heat energy (e.g. geothermal), chemical energy, electric potential and nuclear energy (from nuclear fission or nuclear fusion). Many of these processes generate heat as an intermediate energy form, so heat engines have special importance.
RengaRenga (連歌, linked poem) is a genre of Japanese collaborative poetry in which alternating stanzas, or ku (句), of 5-7-5 and 7-7 mora (sound units, not to be confused with syllables) per line are linked in succession by multiple poets. Known as tsukuba no michi (筑波の道 The Way of Tsukuba) after the famous Tsukuba Mountain in the Kantō region, the form of poetry is said to have originated in a two-verse poetry exchange by Yamato Takeru and later gave birth to the genres haikai (俳諧) and haiku (俳句).
HaikuHaiku is a type of short form poetry that originated in Japan. Traditional Japanese haiku consist of three phrases composed of 17 phonetic units (called on in Japanese, which are similar to syllables) in a 5, 7, 5 pattern; that include a kireji, or "cutting word"; and a kigo, or seasonal reference. Similar poems that do not adhere to these rules are generally classified as senryū. Haiku originated as an opening part of a larger Japanese poem called renga.
Data integrationData integration involves combining data residing in different sources and providing users with a unified view of them. This process becomes significant in a variety of situations, which include both commercial (such as when two similar companies need to merge their databases) and scientific (combining research results from different bioinformatics repositories, for example) domains. Data integration appears with increasing frequency as the volume (that is, big data) and the need to share existing data explodes.
HaikaiHaikai (Japanese 俳諧 comic, unorthodox) may refer in both Japanese and English to haikai no renga (renku), a popular genre of Japanese linked verse, which developed in the sixteenth century out of the earlier aristocratic renga. It meant "vulgar" or "earthy", and often derived its effect from satire and puns, though "under the influence of [Matsuo] Bashō (1644–1694) the tone of haikai no renga became more serious".
File systemIn computing, a file system or filesystem (often abbreviated to fs) is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stopped and the next began, or where any piece of data was located when it was time to retrieve it. By separating the data into pieces and giving each piece a name, the data are easily isolated and identified.
Data mappingIn computing and data management, data mapping is the process of creating data element mappings between two distinct data models.
Query languageA query language, also known as data query language or database query language (DQL), is a computer language used to make queries in databases and information systems. A well known example is the Structured Query Language (SQL). Broadly, query languages can be classified according to whether they are database query languages or information retrieval query languages. The difference is that a database query language attempts to give factual answers to factual questions, while an information retrieval query language attempts to find documents containing information that is relevant to an area of inquiry.
Shard (database architecture)A database shard, or simply a shard, is a horizontal partition of data in a database or search engine. Each shard is held on a separate database server instance, to spread load. Some data within a database remains present in all shards, but some appear only in a single shard. Each shard (or server) acts as the single source for this subset of data. Horizontal partitioning is a database design principle whereby rows of a database table are held separately, rather than being split into columns (which is what normalization and vertical partitioning do, to differing extents).
Data processingData processing is the collection and manipulation of digital data to produce meaningful information. Data processing is a form of information processing, which is the modification (processing) of information in any manner detectable by an observer. The term "Data Processing", or "DP" has also been used to refer to a department within an organization responsible for the operation of data processing programs. Data processing may involve various processes, including: Validation – Ensuring that supplied data is correct and relevant.
