Apertium is a free/open-source rule-based machine translation platform. It is free software and released under the terms of the GNU General Public License. == Overview == Apertium is a transfer-based machine translation system, which uses finite state transducers for all of its lexical transformations, and Constraint Grammar taggers as well as hidden Markov models or Perceptrons for part-of-speech tagging / word category disambiguation. A structural transfer component is responsible for word movement and agreement; most Apertium language pairs up until now have used "chunking" or shallow transfer rules, though newer pairs use (possibly recursive) rules defined in a Context-free grammar. Many existing machine translation systems available at present are commercial or use proprietary technologies, which makes them very hard to adapt to new usages. Apertium code and data is free software and uses a language-independent specification, to allow for the ease of contributing to Apertium, more efficient development, and enhancing the project's overall growth. At present (December 2020), Apertium has released 51 stable language pairs, delivering fast translation with reasonably intelligible results (errors are easily corrected). Being an open-source project, Apertium provides tools for potential developers to build their own language pair and contribute to the project. == History == Apertium originated as one of the machine translation engines in the project OpenTrad, which was funded by the Spanish government, and developed by the Transducens research group at the Universitat d'Alacant. It was originally designed to translate between closely related languages, although it has recently been expanded to treat more divergent language pairs. To create a new machine translation system, one just has to develop linguistic data (dictionaries, rules) in well-specified XML formats. Language data developed for it (in collaboration with the Universidade de Vigo, the Universitat Politècnica de Catalunya and the Universitat Pompeu Fabra) currently support (in stable version) the Arabic, Aragonese, Asturian, Basque, Belarusian, Breton, Bulgarian, Catalan, Crimean Tatar, Danish, English, Esperanto, French, Galician, Hindi, Icelandic, Indonesian, Italian, Kazakh, Macedonian, Malaysian, Maltese, Northern Sami, Norwegian (Bokmål and Nynorsk), Occitan, Polish, Portuguese, Romanian, Russian, Sardinian, Serbo-Croatian, Silesian, Slovene, Spanish, Swedish, Tatar, Ukrainian, Urdu, and Welsh languages. A full list is available below. Several companies are also involved in the development of Apertium, including Prompsit Language Engineering, Imaxin Software and Eleka Ingeniaritza Linguistikoa. The project has taken part in the 2009, 2010, 2011, 2012, 2013 and 2014 editions of Google Summer of Code and the 2010, 2011, 2012, 2013, 2014, 2015, 2016 and 2017 editions of Google Code-In. == Translation methodology == This is an overall, step-by-step view how Apertium works. The diagram displays the steps that Apertium takes to translate a source-language text (the text we want to translate) into a target-language text (the translated text). Source language text is passed into Apertium for translation. The deformatter removes formatting markup (HTML, RTF, etc.) that should be kept in place but not translated. The morphological analyser segments the text (expanding elisions, marking set phrases, etc.), and looks up segments in the language dictionaries, returning dictionary forms and tags for all matches. In pairs that involve agglutinative morphology, including a number of Turkic languages, a Helsinki Finite State Transducer (HFST) is used. Otherwise, an Apertium-specific finite state transducer system called lttoolbox, is used. The morphological disambiguator (the morphological analyser and the morphological disambiguator together form the part of speech tagger) resolves ambiguous segments (i.e., when there is more than one match) by choosing one match. Apertium uses Constraint Grammar rules (with the vislcg3 parser) for most of its language pairs. Retokenisation uses a finite state transducer to match sequences of lexical units and may reorder or translate tags (often used for translating idiomatic expressions into something that more approaches the target language grammar) Lexical transfer looks up disambiguated source-language basewords to find their target-language equivalents (i.e., mapping source language to target language). For lexical transfer, Apertium uses an XML-based dictionary format called bidix. Lexical selection chooses between alternative translations when the source text word has alternative meanings. Apertium uses a specific XML-based technology, apertium-lex-tools, to perform lexical selection. Structural transfer (i.e., it is an XML format that allows writing complex structural transfer rules) can consist of one-step chunking transfer, three-step chunking transfer or a CFG-based transfer module. The chunking modules flag grammatical differences between the source language and target language (e.g. gender or number agreement) by creating a sequence of chunks containing markers for this. They then reorder or modify chunks in order to produce a grammatical translation in the target-language. The newer CFG-based module matches input sequences into possible parse trees, selecting the best-ranking one and applying transformation rules on the tree. The morphological generator uses the tags to deliver the correct target language surface form. The morphological generator is a morphological transducer, just like the morphological analyser. A morphological transducer both analyses and generates forms. The post-generator makes any necessary orthographic changes due to the contact of words (e.g. elisions). The reformatter replaces formatting markup (HTML, RTF, etc.) that was removed by the deformatter in the first step. Apertium delivers the target-language translation. == Supported languages == As of June 2026, the following 108 pairs and 51 languages and languages varieties are supported by Apertium.
Spherical basis
In pure and applied mathematics, particularly quantum mechanics and computer graphics and their applications, a spherical basis is the basis used to express spherical tensors. The spherical basis closely relates to the description of angular momentum in quantum mechanics and spherical harmonic functions. While spherical polar coordinates are one orthogonal coordinate system for expressing vectors and tensors using polar and azimuthal angles and radial distance, the spherical basis are constructed from the standard basis and use complex numbers. == In three dimensions == A vector A in 3D Euclidean space R3 can be expressed in the familiar Cartesian coordinate system in the standard basis ex, ey, ez, and coordinates Ax, Ay, Az: or any other coordinate system with associated basis set of vectors. From this extend the scalars to allow multiplication by complex numbers, so that we are now working in C 3 {\displaystyle \mathbb {C} ^{3}} rather than R 3 {\displaystyle \mathbb {R} ^{3}} . === Basis definition === In the spherical bases denoted e+, e−, e0, and associated coordinates with respect to this basis, denoted A+, A−, A0, the vector A is: where the spherical basis vectors can be defined in terms of the Cartesian basis using complex-valued coefficients in the xy plane: in which i {\displaystyle i} denotes the imaginary unit, and one normal to the plane in the z direction: e 0 = e z {\displaystyle \mathbf {e} _{0}=\mathbf {e} _{z}} The inverse relations are: === Commutator definition === While giving a basis in a 3-dimensional space is a valid definition for a spherical tensor, it only covers the case for when the rank k {\displaystyle k} is 1. For higher ranks, one may use either the commutator, or rotation definition of a spherical tensor. The commutator definition is given below, any operator T q ( k ) {\displaystyle T_{q}^{(k)}} that satisfies the following relations is a spherical tensor: [ J ± , T q ( k ) ] = ℏ ( k ∓ q ) ( k ± q + 1 ) T q ± 1 ( k ) {\displaystyle [J_{\pm },T_{q}^{(k)}]=\hbar {\sqrt {(k\mp q)(k\pm q+1)}}T_{q\pm 1}^{(k)}} [ J z , T q ( k ) ] = ℏ q T q ( k ) {\displaystyle [J_{z},T_{q}^{(k)}]=\hbar qT_{q}^{(k)}} === Rotation definition === Analogously to how the spherical harmonics transform under a rotation, a general spherical tensor transforms as follows, when the states transform under the unitary Wigner D-matrix D ( R ) {\displaystyle {\mathcal {D}}(R)} , where R is a (3×3 rotation) group element in SO(3). That is, these matrices represent the rotation group elements. With the help of its Lie algebra, one can show these two definitions are equivalent. D ( R ) T q ( k ) D † ( R ) = ∑ q ′ = − k k T q ′ ( k ) D q ′ q ( k ) {\displaystyle {\mathcal {D}}(R)T_{q}^{(k)}{\mathcal {D}}^{\dagger }(R)=\sum _{q'=-k}^{k}T_{q'}^{(k)}{\mathcal {D}}_{q'q}^{(k)}} === Coordinate vectors === For the spherical basis, the coordinates are complex-valued numbers A+, A0, A−, and can be found by substitution of (3B) into (1), or directly calculated from the inner product ⟨, ⟩ (5): A 0 = ⟨ e 0 , A ⟩ = ⟨ e z , A ⟩ = A z {\displaystyle A_{0}=\left\langle \mathbf {e} _{0},\mathbf {A} \right\rangle =\left\langle \mathbf {e} _{z},\mathbf {A} \right\rangle =A_{z}} with inverse relations: In general, for two vectors with complex coefficients in the same real-valued orthonormal basis ei, with the property ei·ej = δij, the inner product is: where · is the usual dot product and the complex conjugate must be used to keep the magnitude (or "norm") of the vector positive definite. == Properties (three dimensions) == === Orthonormality === The spherical basis is an orthonormal basis, since the inner product ⟨, ⟩ (5) of every pair vanishes meaning the basis vectors are all mutually orthogonal: ⟨ e + , e − ⟩ = ⟨ e − , e 0 ⟩ = ⟨ e 0 , e + ⟩ = 0 {\displaystyle \left\langle \mathbf {e} _{+},\mathbf {e} _{-}\right\rangle =\left\langle \mathbf {e} _{-},\mathbf {e} _{0}\right\rangle =\left\langle \mathbf {e} _{0},\mathbf {e} _{+}\right\rangle =0} and each basis vector is a unit vector: ⟨ e + , e + ⟩ = ⟨ e − , e − ⟩ = ⟨ e 0 , e 0 ⟩ = 1 {\displaystyle \left\langle \mathbf {e} _{+},\mathbf {e} _{+}\right\rangle =\left\langle \mathbf {e} _{-},\mathbf {e} _{-}\right\rangle =\left\langle \mathbf {e} _{0},\mathbf {e} _{0}\right\rangle =1} hence the need for the normalizing factors of 1 / 2 {\displaystyle 1/\!{\sqrt {2}}} . === Change of basis matrix === The defining relations (3A) can be summarized by a transformation matrix U: ( e + e − e 0 ) = U ( e x e y e z ) , U = ( − 1 2 − i 2 0 + 1 2 − i 2 0 0 0 1 ) , {\displaystyle {\begin{pmatrix}\mathbf {e} _{+}\\\mathbf {e} _{-}\\\mathbf {e} _{0}\end{pmatrix}}=\mathbf {U} {\begin{pmatrix}\mathbf {e} _{x}\\\mathbf {e} _{y}\\\mathbf {e} _{z}\end{pmatrix}}\,,\quad \mathbf {U} ={\begin{pmatrix}-{\frac {1}{\sqrt {2}}}&-{\frac {i}{\sqrt {2}}}&0\\+{\frac {1}{\sqrt {2}}}&-{\frac {i}{\sqrt {2}}}&0\\0&0&1\end{pmatrix}}\,,} with inverse: ( e x e y e z ) = U − 1 ( e + e − e 0 ) , U − 1 = ( − 1 2 + 1 2 0 + i 2 + i 2 0 0 0 1 ) . {\displaystyle {\begin{pmatrix}\mathbf {e} _{x}\\\mathbf {e} _{y}\\\mathbf {e} _{z}\end{pmatrix}}=\mathbf {U} ^{-1}{\begin{pmatrix}\mathbf {e} _{+}\\\mathbf {e} _{-}\\\mathbf {e} _{0}\end{pmatrix}}\,,\quad \mathbf {U} ^{-1}={\begin{pmatrix}-{\frac {1}{\sqrt {2}}}&+{\frac {1}{\sqrt {2}}}&0\\+{\frac {i}{\sqrt {2}}}&+{\frac {i}{\sqrt {2}}}&0\\0&0&1\end{pmatrix}}\,.} It can be seen that U is a unitary matrix, in other words its Hermitian conjugate U† (complex conjugate and matrix transpose) is also the inverse matrix U−1. For the coordinates: ( A + A − A 0 ) = U ∗ ( A x A y A z ) , U ∗ = ( − 1 2 + i 2 0 + 1 2 + i 2 0 0 0 1 ) , {\displaystyle {\begin{pmatrix}A_{+}\\A_{-}\\A_{0}\end{pmatrix}}=\mathbf {U} ^{\mathrm {} }{\begin{pmatrix}A_{x}\\A_{y}\\A_{z}\end{pmatrix}}\,,\quad \mathbf {U} ^{\mathrm {} }={\begin{pmatrix}-{\frac {1}{\sqrt {2}}}&+{\frac {i}{\sqrt {2}}}&0\\+{\frac {1}{\sqrt {2}}}&+{\frac {i}{\sqrt {2}}}&0\\0&0&1\end{pmatrix}}\,,} and inverse: ( A x A y A z ) = ( U ∗ ) − 1 ( A + A − A 0 ) , ( U ∗ ) − 1 = ( − 1 2 + 1 2 0 − i 2 − i 2 0 0 0 1 ) . {\displaystyle {\begin{pmatrix}A_{x}\\A_{y}\\A_{z}\end{pmatrix}}=(\mathbf {U} ^{\mathrm {} })^{-1}{\begin{pmatrix}A_{+}\\A_{-}\\A_{0}\end{pmatrix}}\,,\quad (\mathbf {U} ^{\mathrm {} })^{-1}={\begin{pmatrix}-{\frac {1}{\sqrt {2}}}&+{\frac {1}{\sqrt {2}}}&0\\-{\frac {i}{\sqrt {2}}}&-{\frac {i}{\sqrt {2}}}&0\\0&0&1\end{pmatrix}}\,.} === Cross products === Taking cross products of the spherical basis vectors, we find an obvious relation: e q × e q = 0 {\displaystyle \mathbf {e} _{q}\times \mathbf {e} _{q}={\boldsymbol {0}}} where q is a placeholder for +, −, 0, and two less obvious relations: e ± × e ∓ = ± i e 0 {\displaystyle \mathbf {e} _{\pm }\times \mathbf {e} _{\mp }=\pm i\mathbf {e} _{0}} e ± × e 0 = ± i e ± {\displaystyle \mathbf {e} _{\pm }\times \mathbf {e} _{0}=\pm i\mathbf {e} _{\pm }} === Inner product in the spherical basis === The inner product between two vectors A and B in the spherical basis follows from the above definition of the inner product: ⟨ A , B ⟩ = A + B + ⋆ + A − B − ⋆ + A 0 B 0 ⋆ {\displaystyle \left\langle \mathbf {A} ,\mathbf {B} \right\rangle =A_{+}B_{+}^{\star }+A_{-}B_{-}^{\star }+A_{0}B_{0}^{\star }}
Psychology of reasoning
The psychology of reasoning (also known as the cognitive science of reasoning) is the study of how people reason, often broadly defined as the process of drawing conclusions to inform how people solve problems and make decisions. It overlaps with psychology, philosophy, linguistics, cognitive science, artificial intelligence, logic, and probability theory. Psychological experiments on how humans and other animals reason have been carried out for over 100 years. An enduring question is whether or not people have the capacity to be rational. Current research in this area addresses various questions about reasoning, rationality, judgments, intelligence, relationships between emotion and reasoning, and development. == Everyday reasoning == One of the most obvious areas in which people employ reasoning is with sentences in everyday language. Most experimentation on deduction has been carried out on hypothetical thought, in particular, examining how people reason about conditionals, e.g., If A then B. Participants in experiments make the modus ponens inference, given the indicative conditional If A then B, and given the premise A, they conclude B. However, given the indicative conditional and the minor premise for the modus tollens inference, not-B, about half of the participants in experiments conclude not-A and the remainder concludes that nothing follows. The ease with which people make conditional inferences is affected by context, as demonstrated in the well-known selection task developed by Peter Wason. Participants are better able to test a conditional in an ecologically relevant context, e.g., if the envelope is sealed then it must have a 50 cent stamp on it compared to one that contains symbolic content, e.g., if the letter is a vowel then the number is even. Background knowledge can also lead to the suppression of even the simple modus ponens inference Participants given the conditional if Lisa has an essay to write then she studies late in the library and the premise Lisa has an essay to write make the modus ponens inference 'she studies late in the library', but the inference is suppressed when they are also given a second conditional if the library stays open then she studies late in the library. Interpretations of the suppression effect are controversial Other investigations of propositional inference examine how people think about disjunctive alternatives, e.g., A or else B, and how they reason about negation, e.g., It is not the case that A and B. Many experiments have been carried out to examine how people make relational inferences, including comparisons, e.g., A is better than B. Such investigations also concern spatial inferences, e.g. A is in front of B and temporal inferences, e.g. A occurs before B. Other common tasks include categorical syllogisms, used to examine how people reason about quantifiers such as All or Some, e.g., Some of the A are not B. For example if all A are B and some B are C, what (if anything) follows? == Theories of reasoning == There are several alternative theories of the cognitive processes that human reasoning is based on. One view is that people rely on a mental logic consisting of formal (abstract or syntactic) inference rules similar to those developed by logicians in the propositional calculus. Another view is that people rely on domain-specific or content-sensitive rules of inference. A third view is that people rely on mental models, that is, mental representations that correspond to imagined possibilities. A fourth view is that people compute probabilities. One controversial theoretical issue is the identification of an appropriate competence model, or a standard against which to compare human reasoning. Initially classical logic was chosen as a competence model. Subsequently, some researchers opted for non-monotonic logic and Bayesian probability. Research on mental models and reasoning has led to the suggestion that people are rational in principle but err in practice. Connectionist approaches towards reasoning have also been proposed. Despite the ongoing debate about the cognitive processes involved in human reasoning, recent research has shown that multiple approaches can be useful in modeling human thinking. For instance, studies have found that people's reasoning is often influenced by their prior beliefs, which can be modeled using Bayesian probability theory. Additionally, research on mental models has shown that people tend to reason about problems by constructing multiple mental representations of the situation, which can help them to identify relevant features and make inferences based on their understanding of the problem. Moreover, connectionist approaches to reasoning have also gained attention, which focus on the neural network models that can learn from data and generalize to new situations. == Development of reasoning == It is an active question in psychology how, why, and when the ability to reason develops from infancy to adulthood. Jean Piaget's theory of cognitive development posited general mechanisms and stages in the development of reasoning from infancy to adulthood. According to the neo-Piagetian theories of cognitive development, changes in reasoning with development come from increasing working memory capacity, increasing speed of processing, and enhanced executive functions and control. Increasing self-awareness is also an important factor. In their book The Enigma of Reason, the cognitive scientists Hugo Mercier and Dan Sperber put forward an "argumentative" theory of reasoning, claiming that humans evolved to reason primarily to justify our beliefs and actions and to convince others in a social environment. Key evidence for their theory includes the errors in reasoning that solitary individuals are prone to when their arguments are not criticized, such as logical fallacies, and how groups become much better at performing cognitive reasoning tasks when they communicate with one another and can evaluate each other's arguments. Sperber and Mercier offer one attempt to resolve the apparent paradox that the confirmation bias is so strong despite the function of reasoning naively appearing to be to come to veridical conclusions about the world. The study of the development of reasoning abilities is an ongoing area of research in psychology, and multiple factors have been proposed to explain how, why, and when reasoning develops from infancy to adulthood. Recent research has suggested that early experiences and social interactions play a critical role in the development of reasoning abilities. For example, studies have shown that infants as young as six months old can engage in basic logical reasoning, such as reasoning about the relationship between objects and their properties. Furthermore, research has highlighted the importance of parental interaction and cognitive stimulation in the development of children's reasoning abilities. Additionally, studies have suggested that cultural factors, such as educational practices and the emphasis on critical thinking, can also influence the development of reasoning skills across different populations. == Different sorts of reasoning == Philip Johnson-Laird trying to taxonomize thought, distinguished between goal-directed thinking and thinking without goal, noting that association was involved in unrelated reading. He argues that goal directed reasoning can be classified based on the problem space involved in a solution, citing Allen Newell and Herbert A. Simon. Inductive reasoning makes broad generalizations from specific cases or observations. In this process of reasoning, general assertions are made based on past specific pieces of evidence. This kind of reasoning allows the conclusion to be false even if the original statement is true. For example, if one observes a college athlete, one makes predictions and assumptions about other college athletes based on that one observation. Scientists use inductive reasoning to create theories and hypotheses. Philip Johnson-Laird distinguished inductive from deductive reasoning, in that the former creates semantic information while the later does not . In opposition, deductive reasoning is a basic form of valid reasoning. In this reasoning process a person starts with a known claim or a general belief and from there asks what follows from these foundations or how will these premises influence other beliefs. In other words, deduction starts with a hypothesis and examines the possibilities to reach a conclusion. Deduction helps people understand why their predictions are wrong and indicates that their prior knowledge or beliefs are off track. An example of deduction can be seen in the scientific method when testing hypotheses and theories. Although the conclusion usually corresponds and therefore proves the hypothesis, there are some cases where the conclusion is logical, but the generalization is not. For example, the argument, "All young girls wear skirts; Julie is a young
Data augmentation
Data augmentation is a statistical technique which allows maximum likelihood estimation from incomplete data. Data augmentation has important applications in Bayesian analysis, and the technique is widely used in machine learning to reduce overfitting when training machine learning models, achieved by training models on several slightly-modified copies of existing data. == Synthetic oversampling techniques for traditional machine learning == Synthetic Minority Over-sampling Technique (SMOTE) is a method used to address imbalanced datasets in machine learning. In such datasets, the number of samples in different classes varies significantly, leading to biased model performance. For example, in a medical diagnosis dataset with 90 samples representing healthy individuals and only 10 samples representing individuals with a particular disease, traditional algorithms may struggle to accurately classify the minority class. SMOTE rebalances the dataset by generating synthetic samples for the minority class. For instance, if there are 100 samples in the majority class and 10 in the minority class, SMOTE can create synthetic samples by randomly selecting a minority class sample and its nearest neighbors, then generating new samples along the line segments joining these neighbors. This process helps increase the representation of the minority class, improving model performance. == Data augmentation for image classification == When convolutional neural networks grew larger in mid-1990s, there was a lack of data to use, especially considering that some part of the overall dataset should be spared for later testing. It was proposed to perturb existing data with affine transformations to create new examples with the same labels, which were complemented by so-called elastic distortions in 2003, and the technique was widely used as of 2010s. Data augmentation can enhance CNN performance and acts as a countermeasure against CNN profiling attacks. Data augmentation has become fundamental in image classification, enriching training dataset diversity to improve model generalization and performance. The evolution of this practice has introduced a broad spectrum of techniques, including geometric transformations, color space adjustments, and noise injection. === Geometric Transformations === Geometric transformations alter the spatial properties of images to simulate different perspectives, orientations, and scales. Common techniques include: Affine Transformation Rotation: Rotating images by a specified degree to help models recognize objects at various angles. Reflection: Reflecting images horizontally or vertically to introduce variability in orientation. Translation: Shifting images in different directions to teach models positional invariance. Scaling Shear Mapping Cropping: Removing sections of the image to focus on particular features or simulate closer views. Elastic Distortion Morphing within the same class: Generating new samples by applying morphing techniques between two images belonging to the same class, thereby increasing intra-class diversity. === Color Space Transformations === Color space transformations modify the color properties of images, addressing variations in lighting, color saturation, and contrast. Techniques include: Brightness Adjustment: Varying the image's brightness to simulate different lighting conditions. Contrast Adjustment: Changing the contrast to help models recognize objects under various clarity levels. Saturation Adjustment: Altering saturation to prepare models for images with diverse color intensities. Color Jittering: Randomly adjusting brightness, contrast, saturation, and hue to introduce color variability. === Noise Injection === Injecting noise into images simulates real-world imperfections, teaching models to ignore irrelevant variations. Techniques involve: Gaussian Noise: Adding Gaussian noise mimics sensor noise or graininess. Salt and Pepper Noise: Introducing black or white pixels at random simulates sensor dust or dead pixels. == Data augmentation for signal processing == Residual or block bootstrap can be used for time series augmentation. === Biological signals === Synthetic data augmentation is of paramount importance for machine learning classification, particularly for biological data, which tend to be high dimensional and scarce. The applications of robotic control and augmentation in disabled and able-bodied subjects still rely mainly on subject-specific analyses. Data scarcity is notable in signal processing problems such as for Parkinson's Disease Electromyography signals, which are difficult to source - Zanini, et al. noted that it is possible to use a generative adversarial network (in particular, a DCGAN) to perform style transfer in order to generate synthetic electromyographic signals that corresponded to those exhibited by sufferers of Parkinson's Disease. The approaches are also important in electroencephalography (brainwaves). Wang, et al. explored the idea of using deep convolutional neural networks for EEG-Based Emotion Recognition, results show that emotion recognition was improved when data augmentation was used. A common approach is to generate synthetic signals by re-arranging components of real data. Lotte proposed a method of "Artificial Trial Generation Based on Analogy" where three data examples x 1 , x 2 , x 3 {\displaystyle x_{1},x_{2},x_{3}} provide examples and an artificial x s y n t h e t i c {\displaystyle x_{synthetic}} is formed which is to x 3 {\displaystyle x_{3}} what x 2 {\displaystyle x_{2}} is to x 1 {\displaystyle x_{1}} . A transformation is applied to x 1 {\displaystyle x_{1}} to make it more similar to x 2 {\displaystyle x_{2}} , the same transformation is then applied to x 3 {\displaystyle x_{3}} which generates x s y n t h e t i c {\displaystyle x_{synthetic}} . This approach was shown to improve performance of a Linear Discriminant Analysis classifier on three different datasets. Current research shows great impact can be derived from relatively simple techniques. For example, Freer observed that introducing noise into gathered data to form additional data points improved the learning ability of several models which otherwise performed relatively poorly. Tsinganos et al. studied the approaches of magnitude warping, wavelet decomposition, and synthetic surface EMG models (generative approaches) for hand gesture recognition, finding classification performance increases of up to +16% when augmented data was introduced during training. More recently, data augmentation studies have begun to focus on the field of deep learning, more specifically on the ability of generative models to create artificial data which is then introduced during the classification model training process. In 2018, Luo et al. observed that useful EEG signal data could be generated by Conditional Wasserstein Generative Adversarial Networks (GANs) which was then introduced to the training set in a classical train-test learning framework. The authors found classification performance was improved when such techniques were introduced. === Mechanical signals === The prediction of mechanical signals based on data augmentation brings a new generation of technological innovations, such as new energy dispatch, 5G communication field, and robotics control engineering. In 2022, Yang et al. integrate constraints, optimization and control into a deep network framework based on data augmentation and data pruning with spatio-temporal data correlation, and improve the interpretability, safety and controllability of deep learning in real industrial projects through explicit mathematical programming equations and analytical solutions.
Lynda Soderholm
Lynda Soderholm is a physical chemist at the U.S. Department of Energy's (DOE) Argonne National Laboratory with a specialty in f-block elements. She is a senior scientist and the lead of the Actinide, Geochemistry & Separation Sciences Theme within Argonne's Chemical Sciences and Engineering Division. Her specific role is the Separation Science group leader within Heavy Element Chemistry and Separation Science (HESS), directing basic research focused on low-energy methods for isolating lanthanide and actinide elements from complex mixtures. She has made fundamental contributions to understanding f-block chemistry and characterizing f-block elements. Soderholm became a Fellow of the American Association for the Advancement of Science (AAAS) in 2013, and is also an Argonne Distinguished Fellow. == Early life and education == Soderholm was awarded her PhD in 1982 by McMaster University under the direction of Prof John Greedan. Her dissertation focused on characterizing the structural and magnetic properties of a series of ternary f-ion oxides. After graduating, she was awarded a NATO postdoctoral fellow at the Centre national de la recherche scientifique in France from 1982 until 1985. After a short postdoctoral appointment as an Argonne postdoctoral fellow she was promoted to staff scientist the same year. Over several years, she moved up the ranks, becoming a senior chemist in 2001. She was also an adjunct professor at the University of Notre Dame from 2003 until 2007. In 2021, Soderholm was appointed interim Division Director for the Chemical Sciences and Engineering Division. == Career and research == === Uncovering structure of Yttrium-123 Superconductor === Early in her career, Soderholm focused on the characterizing the magnetic and electronic behavior of compounds containing f-ions (lanthanides and actinides) with a focus on high-Tc materials, compounds that are superconducting under usually high temperatures. She was part of the research group that first determined the structure of YBa2Cu3O7. Their discovery formed the foundation for the further developments in the broad field of superconductivity. === Understanding f-ion speciation in solution === Continuing her interest in the f-elements, Soderholm shifted her focus from solid-state materials to nanoparticles and solutions, taking advantage of advances in X-ray structural probes made available by synchrotron facilities. Building on her earlier work using neutron scattering, her team became the first to discover that plutonium exists in solution as tiny, well-defined nanoparticles. This work solved a longstanding problem in understanding transport of plutonium in the environment and resulted in the development of a new, patented approach to separating plutonium during nuclear reprocessing. === Using machine learning to evaluate molecular structures === Soderholm's more recent projects use machine learning to understand the influence of complex molecular structuring in solutions, in connection with low-energy processes for separation of f-block elements from complex mixtures. == Awards and honors == University of Chicago Board of Governors' Distinguished Performance Award, 2009. Fellow of the American Association for the Advancement of Science, 2013. Argonne Distinguished Fellow, 2016 DOE materials sciences research competition for Outstanding Scientific Accomplishments in Solid State Physics, 1987. == Select publications == Beno, M. A.; Soderholm, L.; Capone, D. W., II; Hinks, D. G.; Jorgensen, J. D.; Grace, J. D.; Schuller, I. K.; Segre, C. U.; Zhang, K., Structure of the single-phase high-temperature superconductor yttrium barium copper oxide (YBa2Cu3O7−δ). Appl. Phys. Lett. 1987, 51 (1), 57–9. Soderholm, L.; Zhang, K.; Hinks, D. G.; Beno, M. A.; Jorgensen, J. D.; Segre, C. U.; Schuller, I. K., Incorporation of praseodymium in YBa2Cu3O7−δ: electronic effects on superconductivity. Nature (London) 1987, 328 (6131), 604–5. Antonio, M. R.; Williams, C. W.; Soderholm, L., Berkelium redox speciation. Radiochim. Acta 2002, 90 (12), 851–856. Soderholm, L.; Skanthakumar, S.; Neuefeind, J., Determination of actinide speciation in solution using high-energy X-ray scattering. Anal. Bioanal. Chem. 2005, 383 (1), 48–55. Forbes, T. Z.; Burns, P. C.; Skanthakumar, S.; Soderholm, L., Synthesis, structure, and magnetism of Np2O5. J. Am. Chem. Soc. 2007, 129 (10), 2760–2761. Soderholm, L.; Almond, P. M.; Skanthakumar, S.; Wilson, R. E.; Burns, P. C., The structure of the plutonium oxide nanocluster [Pu38O56Cl54(H2O)8]14-. Angew. Chem., Int. Ed. 2008, 47 (2), 298–302. Jensen, M. P.; Gorman-Lewis, D.; Aryal, B.; Paunesku, T.; Vogt, S.; Rickert, P. G.; Seifert, S.; Lai, B.; Woloschak, G. E.; Soderholm, L., An iron-dependent and transferrin-mediated cellular uptake pathway for plutonium. Nat. Chem. Biol. 2011, 7 (8), 560–565. Wilson, R. E.; Skanthakumar, S.; Soderholm, L., Separation of Plutonium Oxide Nanoparticles and Colloids. Angew. Chem., Int. Ed. 2011, 50 (47), 11234–11237. Knope, K. E.; Soderholm, L., Solution and solid-state structural chemistry of actinide hydrates and their hydrolysis and condensation products. Chem. Rev. 2013, 113 (2), 944–994. Luo, G.; Bu, W.; Mihaylov, M.; Kuzmenko, I.; Schlossman, M. L.; Soderholm, L., X-ray reflectivity reveals a nonmonotonic ion-density profile perpendicular to the surface of ErCl3 aqueous solutions. J. Phys. Chem. C 2013, 117 (37), 19082–19090. Jin, G. B.; Lin, J.; Estes, S. L.; Skanthakumar, S.; Soderholm, L., Influence of countercation hydration enthalpies on the formation of molecular complexes: A thorium-nitrate example. J. Am. Chem. Soc. 2017, 139 (49), 18003–18008. == Patents == Solvent extraction system for plutonium colloids and other oxide nano-particles, (2016).
Situational application
In computing, a situational application is "good enough" software created for a narrow group of users with a unique set of needs. The application typically (but not always) has a short life span, and is often created within the group where it is used, sometimes by the users themselves. As the requirements of a small team using the application change, the situational application often also continues to evolve to accommodate these changes. Although situational applications are specifically designed to embrace change, significant changes in requirements may lead to an abandonment of the situational application altogether – in some cases it is just easier to develop a new one than to evolve the one in use. == Characteristics == Situational applications are developed fast, easy to use, uncomplicated, and serve a unique set of requirements. They have a narrow focus on a specific business problem, and they are written in a way where if the business problem changes rapidly, so can the situational application. This contrasts with more common enterprise applications, which are designed to address a large set of business problems, require meticulous planning, and impose a sometimes-slow and often-meticulous change process. == Origination == Clay Shirky in his essay entitled "Situated Software" described a type of software that "...is designed for use by a specific social group, rather than for a generic set of "users"." IBM later morphed the term into "situational applications". == Evolution == The successful large-scale implementation of a situational application environment in an organization requires a strategy, mindset, methodology and support structure quite different from traditional application development. This is now evolving as more companies learn how to best leverage the ideas behind situational applications. In addition, the advent of cloud-based application development and deployment platforms makes the implementation of a comprehensive situational application environment much more feasible. == Examples == A structured wiki that can host wiki applications lends itself to creation of situational applications. Some mashups can also be considered situational applications. A forms application such as a Microsoft Access Database (MDB file) can be considered a situational application. The latest implementations of situational application environments include Longjump, Force.com and WorkXpress.
Attention (machine learning)
In machine learning, attention is a method that determines the importance of each component in a sequence relative to the other components in that sequence. In natural language processing, importance is represented by "soft" weights assigned to each word in a sentence. More generally, attention encodes vectors called token embeddings across a fixed-width sequence that can range from tens to millions of tokens in size. Unlike "hard" weights, which are computed during the backwards training pass, "soft" weights exist only in the forward pass and therefore change with every step of the input. Earlier designs implemented the attention mechanism in a serial recurrent neural network (RNN) language translation system, but a more recent design, namely the transformer, removed the slower sequential RNN and relied more heavily on the faster parallel attention scheme. Inspired by ideas about attention in humans, the attention mechanism was developed to address the weaknesses of using information from the hidden layers of recurrent neural networks. Recurrent neural networks favor information contained in words at the end of a sentence and thus deemed more recent, thereby tending to attenuate the significance and associated predictive weight assigned to information earlier in the sentence. Attention allows a token equal access to any part of a sentence directly, rather than only through the previous state. == History == Additional surveys of the attention mechanism in deep learning are provided by Niu et al. and Soydaner. The major breakthrough came with self-attention, where each element in the input sequence attends to all others, enabling the model to capture global dependencies. This idea was central to the Transformer architecture, which replaced recurrence with attention mechanisms. As a result, Transformers became the foundation for models like BERT, T5 and generative pre-trained transformers (GPT). == Overview == The modern era of machine attention was revitalized by grafting an attention mechanism (Fig 1. orange) to an Encoder-Decoder. Figure 2 shows the internal step-by-step operation of the attention block (A) in Fig 1. === Interpreting attention weights === In translating between languages, alignment is the process of matching words from the source sentence to words of the translated sentence. Networks that perform verbatim translation without regard to word order would show the highest scores along the (dominant) diagonal of the matrix. The off-diagonal dominance shows that the attention mechanism is more nuanced. Consider an example of translating I love you to French. On the first pass through the decoder, 94% of the attention weight is on the first English word I, so the network offers the word je. On the second pass of the decoder, 88% of the attention weight is on the third English word you, so it offers t'. On the last pass, 95% of the attention weight is on the second English word love, so it offers aime. In the I love you example, the second word love is aligned with the third word aime. Stacking soft row vectors together for je, t', and aime yields an alignment matrix: Sometimes, alignment can be multiple-to-multiple. For example, the English phrase look it up corresponds to cherchez-le. Thus, "soft" attention weights work better than "hard" attention weights (setting one attention weight to 1, and the others to 0), as we would like the model to make a context vector consisting of a weighted sum of the hidden vectors, rather than "the best one", as there may not be a best hidden vector. == Variants == Many variants of attention implement soft weights, such as fast weight programmers, or fast weight controllers (1992). A "slow" neural network outputs the "fast" weights of another neural network through outer products. The slow network learns by gradient descent. It was later renamed as "linearized self-attention". Bahdanau-style attention, also referred to as additive attention, Luong-style attention, which is known as multiplicative attention, Early attention mechanisms similar to modern self-attention were proposed using recurrent neural networks. However, the highly parallelizable self-attention was introduced in 2017 and successfully used in the Transformer model, positional attention and factorized positional attention. For convolutional neural networks, attention mechanisms can be distinguished by the dimension on which they operate, namely: spatial attention, channel attention, or combinations. These variants recombine the encoder-side inputs to redistribute those effects to each target output. Often, a correlation-style matrix of dot products provides the re-weighting coefficients. In the figures below, W is the matrix of context attention weights, similar to the formula in Overview section above. == Optimizations == === Flash attention === The size of the attention matrix is proportional to the square of the number of input tokens. Therefore, when the input is long, calculating the attention matrix requires a lot of GPU memory. Flash attention is an implementation that reduces the memory needs and increases efficiency without sacrificing accuracy. It achieves this by partitioning the attention computation into smaller blocks that fit into the GPU's faster on-chip memory, reducing the need to store large intermediate matrices and thus lowering memory usage while increasing computational efficiency. === FlexAttention === FlexAttention is an attention kernel developed by Meta that allows users to modify attention scores prior to softmax and dynamically chooses the optimal attention algorithm. == Applications == Attention is widely used in natural language processing, computer vision, and speech recognition. In NLP, it improves context understanding in tasks like question answering and summarization. In vision, visual attention helps models focus on relevant image regions, enhancing object detection and image captioning. === Attention maps as explanations for vision transformers === From the original paper on vision transformers (ViT), visualizing attention scores as a heat map (called saliency maps or attention maps) has become an important and routine way to inspect the decision making process of ViT models. One can compute the attention maps with respect to any attention head at any layer, while the deeper layers tend to show more semantically meaningful visualization. Attention rollout is a recursive algorithm to combine attention scores across all layers, by computing the dot product of successive attention maps. Because vision transformers are typically trained in a self-supervised manner, attention maps are generally not class-sensitive. When a classification head is attached to the ViT backbone, class-discriminative attention maps (CDAM) combines attention maps and gradients with respect to the class [CLS] token. Some class-sensitive interpretability methods originally developed for convolutional neural networks can be also applied to ViT, such as GradCAM, which back-propagates the gradients to the outputs of the final attention layer. Using attention as basis of explanation for the transformers in language and vision is not without debate. While some pioneering papers analyzed and framed attention scores as explanations, higher attention scores do not always correlate with greater impact on model performances. == Mathematical representation == === Standard scaled dot-product attention === For matrices: Q ∈ R m × d k , K ∈ R n × d k {\displaystyle Q\in \mathbb {R} ^{m\times d_{k}},K\in \mathbb {R} ^{n\times d_{k}}} and V ∈ R n × d v {\displaystyle V\in \mathbb {R} ^{n\times d_{v}}} , the scaled dot-product, or QKV attention, is defined as: Attention ( Q , K , V ) = softmax ( Q K T d k ) V ∈ R m × d v {\displaystyle {\text{Attention}}(Q,K,V)={\text{softmax}}\left({\frac {QK^{T}}{\sqrt {d_{k}}}}\right)V\in \mathbb {R} ^{m\times d_{v}}} where T {\displaystyle {}^{T}} denotes transpose and the softmax function is applied independently to every row of its argument. The matrix Q {\displaystyle Q} contains m {\displaystyle m} queries, while matrices K , V {\displaystyle K,V} jointly contain an unordered set of n {\displaystyle n} key-value pairs. Value vectors in matrix V {\displaystyle V} are weighted using the weights resulting from the softmax operation, so that the rows of the m {\displaystyle m} -by- d v {\displaystyle d_{v}} output matrix are confined to the convex hull of the points in R d v {\displaystyle \mathbb {R} ^{d_{v}}} given by the rows of V {\displaystyle V} . To understand the permutation invariance and permutation equivariance properties of QKV attention, let A ∈ R m × m {\displaystyle A\in \mathbb {R} ^{m\times m}} and B ∈ R n × n {\displaystyle B\in \mathbb {R} ^{n\times n}} be permutation matrices; and D ∈ R m × n {\displaystyle D\in \mathbb {R} ^{m\times n}} an arbitrary matrix. The softmax function is permutation equivariant in the sense that: softmax ( A D B ) = A softmax ( D ) B {\displays