Multi-Task Learning

Named Entity Recognition is a challenging task, specially for low resource languages, such as Persian, due to the lack of massive gold data. As developing manually-annotated datasets is time consuming and expensive, we use a multitask learning (MTL) framework to exploit different datasets to enrich the extracted features and improve the accuracy of recognizing named entities in Persian news articles. Highly motivated auxiliary tasks are chosen to be included in a deep learning based structure. Additionally, we investigate the effect of chosen datasets on performance of the model. Our best model significantly outperformed the state of the art model by. %, according to F1 score in the phrase level.

License plate recognition is an important task applied to a myriad of important scenarios. Even though there are several methods for performing license plate recognition, our approach is designed to work not only on high-resolution license plates but also when the license plate characters are not recognizable by humans. Early approaches divided the task into several subtasks that are executed in sequence. However, since each task has its own accuracy, the errors of each are propagated to the next step. This is critical in the last two steps of the pipeline known as segmentation and recognition of the characters. Thus, we employ a technique to perform these two steps at once. The approach is based on a multi-task network where each task represents the recognition of an entire license plate character. We do not address the license plate detection problem in this paper. We also propose the use of a so called generative model for data augmentation of low-resolution images simulating images as if they were acquired farther away from where they actually are. We are able to achieve very promising results with improvements of more than 30 percentage points of accuracy on images with multiple resolutions and a character recognition accuracy on low-resolution images higher than 87%.

We study the problem of learning many related tasks simultaneously using kernel methods and regularization. The standard single-task kernel methods, such as support vector machines and regularization networks, are extended to the case of multi-task learning. Our analysis shows that the problem of estimating many task functions with regularization can be cast as a single task learning problem if a family of multi-task kernel functions we define is used. These kernels model relations among the tasks and are ...

Parkinson’s disease (PD) is a long-term degenerative disorder of the central nervous system, with symptoms generally appearing slowly over time. Predicting the PD disease is critical as motor and non-motor manifestations occur many years after the onset of neurodegeneration, hence its early management
of disease is a significant challenge in the field of PD
therapeutics. While part of previous studies with respect to the prediction of Parkinson’s Disease has been based mainly on brain images, dependencies between additional patients’ information have not been taken into account. This observation suggests that prediction of Parkinson’s Disease along with additional patients’ data with a unified framework should outperform Machine Learning (ML) algorithms that treat different
sources of patients’ information separately. Our presented framework relies on Multi-Task Learning (MTL) implemented with Deep Neural Networks (DNNs) with shared hidden layers. Our preliminary experimental results confirm the benefits of MTL over Single-Task Learning (STL), underlying the capability of our proposed system to achieve an increased Area Under the Curve (AUC) as high as 92% and helping at the same time to reduce human error.

Wellness is a widely popular concept that is commonly applied to fitness and self-help products or services. Inference of personal wellness-related attributes, such as body mass index or diseases tendency, as well as understanding of global dependencies between wellness attributes and users' behavior is of crucial importance to various applications in personal and public wellness domains. Meanwhile, the emergence of social media platforms and wearable sensors makes it feasible to perform wellness profiling for users from multiple perspectives. However, research efforts on wellness profiling and integration of social media and sensor data are relatively sparse, and this study represents one of the first attempts in this direction. Specifically, to infer personal wellness attributes, we proposed multi-source individual user profile learning framework named " TweetFit ". "TweetFit" can handle data incom-pleteness and perform wellness attributes inference from sensor and social media data simultaneously. Our experimental results show that the integration of the data from sensors and multiple social media sources can substantially boost the wellness profiling performance.

... Page 9. 280 J. Madrid-Sánchez, M. Lázaro-Gredilla, and AR Figueiras-Vidal Table 3. Test error rates (in %) of single-layer perceptron approaches in Character Recognition. ... Heskes, T.: Empirical Bayes for learning to learn. In Langley, P. ed.: ICML-2000. ...

Cognitive modeling with neural networks unrealistically ignores the role of knowledge in learning by starting from random weights. It is likely that effective use of knowledge by neural networks could significantly speed learning. A new algorithm, knowledge-based cascadecorrelation (KBCC), finds and adapts its relevant knowledge in new learning. Comparison to multi-task learning (MTL) reveals that KBCC uses its knowledge more effectively to learn faster. 1. Existing Knowledge and New Learning Neural networks typically learn de novo without the benefit of existing knowledge. However, when people learn, they routinely use their knowledge (Pazzani, 1991; Wisniewski, 1995). Such use of prior knowledge in learning is likely responsible for the ease and speed with which people learn, and for interference with new learning. The technical reason that neural networks fail to use knowledge is that they begin learning from initially random connection weights. This implements a tabula rasa view...

We investigate multi-task learning from an output space regularization perspective. Most multi-task approaches tie together related tasks by constraining them to share input spaces and function classes. In contrast to this, we propose a multi-task paradigm which we call output space regularization, in which the only constraint is that the output spaces of the multiple tasks are related. We focus on a specic instance of output space regularization, multi-task averaging, that is both widely applicable and amenable to analysis. The multi-task averaging estimator improves on the single-task sample average under certain conditions, which we detail. Our analysis shows that for a simple case the optimal similarity depends on the ratio of the task variance to the task dierences, but that for more complicated cases the optimal similarity behaves nonlinearly. Further, we show that the estimates produced are a convex combination of the tasks’ sample averages. We discuss the Bayesian viewpoint....

Abstract Epitaxy is a process strongly dependent on wafer temperature. Unfortunately, the performance of the pyrometers in charge of sensing wafer temperature deteriorate with the usage. This represents the major maintenance issue for epitaxy process engineers who have to frequently calibrate pyrometers emissivity coefficient. At the present state the change of the emissivity coefficient is heuristically based on fab tradition and process engineers experience. We present a statistical tool to map the relationship between change in the ...

Background Gene silencing using exogenous small interfering RNAs (siRNAs) is now a widespread molecular tool for gene functional study and new-drug target identification. The key mechanism in this technique is to design efficient siRNAs that incorporated into the RNA-induced silencing complexes (RISC) to bind and interact with the mRNA targets to repress their translations to proteins. Although considerable progress has been made in the computational analysis of siRNA binding efficacy, few joint analysis of different RNAi experiments conducted under different experimental scenarios has been done in research so far, while the joint analysis is an important issue in cross-platform siRNA efficacy prediction. A collective analysis of RNAi mechanisms for different datasets and experimental conditions can often provide new clues on the design of potent siRNAs. Results An elegant multi-task learning paradigm for cross-platform siRNA efficacy prediction is proposed. Experimental studies were performed on a large dataset of siRNA sequences which encompass several RNAi experiments recently conducted by different research groups. By using our multi-task learning method, the synergy among different experiments is exploited and an efficient multi-task predictor for siRNA efficacy prediction is obtained. The 19 most popular biological features for siRNA according to their jointly importance in multi-task learning were ranked. Furthermore, the hypothesis is validated out that the siRNA binding efficacy on different messenger RNAs(mRNAs) have different conditional distribution, thus the multi-task learning can be conducted by viewing tasks at an "mRNA"-level rather than at the "experiment"-level. Such distribution diversity derived from siRNAs bound to different mRNAs help indicate that the properties of target mRNA have important implications on the siRNA binding efficacy. Conclusions The knowledge gained from our study provides useful insights on how to analyze various cross-platform RNAi data for uncovering of their complex mechanism.

Coordinate gradient learning is motivated by the problem of variable selection and determining variable covariation. In this paper we propose a novel unifying framework for coordinate gradient learning (MGL) from the perspective of multi-task learning. Our approach relies on multi-task kernels to simulate the structure of gradient learning. This has several appealing properties. Firstly, it allows us to introduce a novel algorithm which appropriately captures the inherent structure of coordinate gradient learning. Secondly, this approach gives rise to a clear algorithmic process: a computational optimization algorithm which is memory and time efficient. Finally, a statistical error analysis ensures convergence of the estimated function and its gradient to the true function and true gradient. We report some preliminary experiments to validate MGL for variable selection as well as determining variable covariation.

Wellness is a widely popular concept that is commonly applied to fitness and self-help products or services. Inference of personal wellness-related attributes, such as Body Mass Index (BMI) category or diseases tendency , as well as understanding of global dependencies between wellness attributes and users' behavior is of crucial importance to various applications in personal and public wellness domains. At the same time, the emergence of social media platforms and wearable sensors makes it feasible to perform wellness profiling for users from multiple perspectives. However, research efforts on wellness profiling and integration of social media and sensor data are relatively sparse, and this study represents one of the first attempts in this direction. Specifically, we infer personal wellness attributes by utilizing our proposed multi-source multi-task wellness profile learning framework — "WellMTL" , which can handle data incompleteness and perform wellness attributes inference from sensor and social media data simultaneously. To gain insights into the data at a global level, we also examine correlations between first-order data representations and personal wellness attributes. Our experimental results show that the integration of sensor data and multiple social media sources can substantially boost the performance of individual wellness profiling.

Background Since the high dimensionality of gene expression microarray data sets degrades the generalization performance of classifiers, feature selection, which selects relevant features and discards irrelevant and redundant features, has been widely used in the bioinformatics field. Multi-task learning is a novel technique to improve prediction accuracy of tumor classification by using information contained in such discarded redundant features, but which features should be discarded or used as input or output remains an open issue. Results We demonstrate a framework for automatically selecting features to be input, output, and discarded by using a genetic algorithm, and propose two algorithms: GA-MTL (Genetic algorithm based multi-task learning) and e-GA-MTL (an enhanced version of GA-MTL). Experimental results demonstrate that this framework is effective at selecting features for multi-task learning, and that GA-MTL and e-GA-MTL perform better than other heuristic methods. Conclusions Genetic algorithms are a powerful technique to select features for multi-task learning automatically; GA-MTL and e-GA-MTL are shown to to improve generalization performance of classifiers on microarray data sets.

In multi-task learning, there are roughly two approaches to discovering representations. The first is to discover task relevant representations, i.e., those that compactly
represent solutions to particular tasks. The second is to discover domain relevant representations, i.e., those that compactly represent knowledge that remains invariant across many
tasks. In this article, we propose a new approach to multi-task learning that captures domainrelevant knowledge by learning potential-based shaping functions, which augment a task’s
reward function with artificial rewards. We address two key issues that arise when deriving
potential functions. The first is what kind of target function the potential function should
approximate; we propose three such targets and show empirically that which one is best
depends critically on the domain and learning parameters. The second issue is the representation for the potential function. This article introduces the notion of k-relevance, the expected relevance of a representation on a sample sequence of k tasks, and argues that this is
a unifying definition of relevance of which both task and domain relevance are special cases.
We prove formally that, under certain assumptions, k-relevance converges monotonically to
a fixed point as k increases, and use this property to derive Feature Selection Through Extrapolation of k-relevance (FS-TEK), a novel feature-selection algorithm. We demonstrate
empirically the benefit of FS-TEK on artificial domains.