Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem and their predictions are combined. It contains base learners that are usually weak learners. Ensembles reduce bias and variance by combining multiple trained models. Boosting is an ensemble meta-algorithm that focuses new learners on examples others got wrong to convert weak learners into strong ones like Adaboost. Decision lists are a type of logical expression used in some ensemble methods.
Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem and their predictions are combined. It contains base learners that are usually weak learners. Ensembles reduce bias and variance by combining multiple trained models. Boosting is an ensemble meta-algorithm that focuses new learners on examples others got wrong to convert weak learners into strong ones like Adaboost. Decision lists are a type of logical expression used in some ensemble methods.
Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem and their predictions are combined. It contains base learners that are usually weak learners. Ensembles reduce bias and variance by combining multiple trained models. Boosting is an ensemble meta-algorithm that focuses new learners on examples others got wrong to convert weak learners into strong ones like Adaboost. Decision lists are a type of logical expression used in some ensemble methods.
Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem and their predictions are combined. It contains base learners that are usually weak learners. Ensembles reduce bias and variance by combining multiple trained models. Boosting is an ensemble meta-algorithm that focuses new learners on examples others got wrong to convert weak learners into strong ones like Adaboost. Decision lists are a type of logical expression used in some ensemble methods.
Download as PPTX, PDF, TXT or read online from Scribd
Download as pptx, pdf, or txt
You are on page 1/ 22
ENSEMBLE
LEARNING ENSEMBLE LEARNING
• machine learning paradigm where multiple
learners are trained to solve the same problem. • the idea is to select a whole collection, or ensemble of hypotheses from hypothesis space and combine their predictions. • contains a number of learners which are usually called base learners or also referred as weak learners. • Ensembles use multiple trained models. ENSEMBLE LEARNING
• BIAS - measure of how flexible the model is.
- error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (under fitting). • VARIANCE - error from sensitivity to small fluctuations in the training set. High variance can cause an algorithm to model the random noise in the training data, rather than the intended outputs (over fitting). ENSEMBLE LEARNING APPLICATION OF ENSEMBLE LEARNING Weak learning Weak learning
• consists of a learning algorithm that classifies the
input data better than random. • an error is weak if its error is high, but the error cannot be higher than 0.5 • ε < 0.5 • if an error is not very love, it’s a weak learner. Weighted training set Weighted training set
• In such a training set, each example has
an associated weight wj > 0. • The higher the weight, the higher is the importance attached during the learning of a hypothesis. Weighted training set boosting boosting
• machine learning ensemble meta-
algorithm for primarily reducing bias, and also variance in supervised learning, and a family of machine learning algorithms that convert weak learners to strong ones. boosting
• Focus new experts on examples that others get
wrong. • Train experts sequentially. • Errors of early experts indicate the “hard” examples. • Focus later classifier on getting these examples. • Combine the whole set in the end. • Convert many “weak” learners into complex classifier. Adaboost (Adaptive boosting) Adaboost • Any learning algorithm that returns hypotheses that are probably approximately correct is called a PAC- learning algorithm. Decision list Decision list • Logical expression of a restricted form. • Consists of a series of tests, each of which is a conjunction of literal. • If a test succeeds when applied to an example description, the decision list specifies the value to be returned. • If the test fails, processing continues with the next test in the list. • Decision lists resemble decision trees, but their overall structure is simpler. In contrast, the individual tests are more complex. Decision list Decision list Decision list references
• https://www.youtube.com/watch?v=nelJ3svz0_o • https://www.youtube.com/watch?v=UHBmv7qCey4 • https://en.wikipedia.org/wiki/Boosting_(machine_learning) • Artificial Intelligence A Modern Approach, Second Edition (Stuart J. Russell and Peter Norvig )
