diff --git a/doc/modules/ensemble.rst b/doc/modules/ensemble.rst index 57a50d325497e..f4f31a0a65254 100644 --- a/doc/modules/ensemble.rst +++ b/doc/modules/ensemble.rst @@ -218,7 +218,7 @@ setting ``oob_score=True``. The size of the model with the default parameters is :math:`O( M * N * log (N) )`, where :math:`M` is the number of trees and :math:`N` is the number of samples. In order to reduce the size of the model, you can change these parameters: - ``min_samples_split``, ``min_samples_leaf``, ``max_leaf_nodes`` and ``max_depth``. + ``min_samples_split``, ``max_leaf_nodes`` and ``max_depth``. Parallelization --------------- @@ -393,9 +393,7 @@ The number of weak learners is controlled by the parameter ``n_estimators``. The the final combination. By default, weak learners are decision stumps. Different weak learners can be specified through the ``base_estimator`` parameter. The main parameters to tune to obtain good results are ``n_estimators`` and -the complexity of the base estimators (e.g., its depth ``max_depth`` or -minimum required number of samples at a leaf ``min_samples_leaf`` in case of -decision trees). +the complexity of the base estimators (e.g., its depth ``max_depth``). .. topic:: Examples: diff --git a/doc/modules/tree.rst b/doc/modules/tree.rst index 91b58b031b418..5d448f86a3f11 100644 --- a/doc/modules/tree.rst +++ b/doc/modules/tree.rst @@ -330,29 +330,18 @@ Tips on practical use for each additional level the tree grows to. Use ``max_depth`` to control the size of the tree to prevent overfitting. - * Use ``min_samples_split`` or ``min_samples_leaf`` to control the number of - samples at a leaf node. A very small number will usually mean the tree - will overfit, whereas a large number will prevent the tree from learning - the data. Try ``min_samples_leaf=5`` as an initial value. If the sample size - varies greatly, a float number can be used as percentage in these two parameters. - The main difference between the two is that ``min_samples_leaf`` guarantees - a minimum number of samples in a leaf, while ``min_samples_split`` can - create arbitrary small leaves, though ``min_samples_split`` is more common - in the literature. + * Use ``min_samples_split`` to control the number of samples at a leaf node. + A very small number will usually mean the tree will overfit, whereas a + large number will prevent the tree from learning the data. If the sample + size varies greatly, a float number can be used as percentage in this + parameter. Note that ``min_samples_split`` can create arbitrarily + small leaves. * Balance your dataset before training to prevent the tree from being biased toward the classes that are dominant. Class balancing can be done by sampling an equal number of samples from each class, or preferably by normalizing the sum of the sample weights (``sample_weight``) for each - class to the same value. Also note that weight-based pre-pruning criteria, - such as ``min_weight_fraction_leaf``, will then be less biased toward - dominant classes than criteria that are not aware of the sample weights, - like ``min_samples_leaf``. - - * If the samples are weighted, it will be easier to optimize the tree - structure using weight-based pre-pruning criterion such as - ``min_weight_fraction_leaf``, which ensure that leaf nodes contain at least - a fraction of the overall sum of the sample weights. + class to the same value. * All decision trees use ``np.float32`` arrays internally. If training data is not in this format, a copy of the dataset will be made. diff --git a/doc/whats_new/v0.20.rst b/doc/whats_new/v0.20.rst index 3a9e68801afa3..05fbdd1d4389a 100644 --- a/doc/whats_new/v0.20.rst +++ b/doc/whats_new/v0.20.rst @@ -325,6 +325,12 @@ Support for Python 3.3 has been officially dropped. while mask does not allow this functionality. :issue:`9524` by :user:`Guillaume Lemaitre `. +- |API| The parameters ``min_samples_leaf`` and ``min_weight_fraction_leaf`` in + tree-based ensembles are deprecated and will be removed (fixed to 1 and 0 + respectively) in version 0.22. These parameters were not effective for + regularization and at worst would produce bad splits. :issue:`10773` by + :user:`Bob Chen ` and `Joel Nothman`_. + - |Fix| :class:`ensemble.BaseBagging` where one could not deterministically reproduce ``fit`` result using the object attributes when ``random_state`` is set. :issue:`9723` by :user:`Guillaume Lemaitre `. @@ -1005,6 +1011,13 @@ Support for Python 3.3 has been officially dropped. considered all samples to be of equal weight importance. :issue:`11464` by :user:`John Stott `. +- |API| The parameters ``min_samples_leaf`` and ``min_weight_fraction_leaf`` in + :class:`tree.DecisionTreeClassifier` and :class:`tree.DecisionTreeRegressor` + are deprecated and will be removed (fixed to 1 and 0 respectively) in version + 0.22. These parameters were not effective for regularization and at worst + would produce bad splits. :issue:`10773` by :user:`Bob Chen ` + and `Joel Nothman`_. + :mod:`sklearn.utils` .................... diff --git a/examples/ensemble/plot_adaboost_hastie_10_2.py b/examples/ensemble/plot_adaboost_hastie_10_2.py index 4d48d13dd24f2..7fc00a77e3eab 100644 --- a/examples/ensemble/plot_adaboost_hastie_10_2.py +++ b/examples/ensemble/plot_adaboost_hastie_10_2.py @@ -43,11 +43,11 @@ X_test, y_test = X[2000:], y[2000:] X_train, y_train = X[:2000], y[:2000] -dt_stump = DecisionTreeClassifier(max_depth=1, min_samples_leaf=1) +dt_stump = DecisionTreeClassifier(max_depth=1) dt_stump.fit(X_train, y_train) dt_stump_err = 1.0 - dt_stump.score(X_test, y_test) -dt = DecisionTreeClassifier(max_depth=9, min_samples_leaf=1) +dt = DecisionTreeClassifier(max_depth=9) dt.fit(X_train, y_train) dt_err = 1.0 - dt.score(X_test, y_test) diff --git a/examples/ensemble/plot_gradient_boosting_oob.py b/examples/ensemble/plot_gradient_boosting_oob.py index ea38b326ce5c9..99f30e750b7ed 100644 --- a/examples/ensemble/plot_gradient_boosting_oob.py +++ b/examples/ensemble/plot_gradient_boosting_oob.py @@ -55,7 +55,7 @@ # Fit classifier with out-of-bag estimates params = {'n_estimators': 1200, 'max_depth': 3, 'subsample': 0.5, - 'learning_rate': 0.01, 'min_samples_leaf': 1, 'random_state': 3} + 'learning_rate': 0.01, 'random_state': 3} clf = ensemble.GradientBoostingClassifier(**params) clf.fit(X_train, y_train) diff --git a/examples/ensemble/plot_gradient_boosting_quantile.py b/examples/ensemble/plot_gradient_boosting_quantile.py index 6fb2731a513ec..99e7289710e35 100644 --- a/examples/ensemble/plot_gradient_boosting_quantile.py +++ b/examples/ensemble/plot_gradient_boosting_quantile.py @@ -41,8 +41,7 @@ def f(x): clf = GradientBoostingRegressor(loss='quantile', alpha=alpha, n_estimators=250, max_depth=3, - learning_rate=.1, min_samples_leaf=9, - min_samples_split=9) + learning_rate=.1, min_samples_split=9) clf.fit(X, y) diff --git a/examples/model_selection/plot_randomized_search.py b/examples/model_selection/plot_randomized_search.py index bac6495090e6b..a1c408e510b09 100644 --- a/examples/model_selection/plot_randomized_search.py +++ b/examples/model_selection/plot_randomized_search.py @@ -55,7 +55,6 @@ def report(results, n_top=3): param_dist = {"max_depth": [3, None], "max_features": sp_randint(1, 11), "min_samples_split": sp_randint(2, 11), - "min_samples_leaf": sp_randint(1, 11), "bootstrap": [True, False], "criterion": ["gini", "entropy"]} @@ -74,7 +73,6 @@ def report(results, n_top=3): param_grid = {"max_depth": [3, None], "max_features": [1, 3, 10], "min_samples_split": [2, 3, 10], - "min_samples_leaf": [1, 3, 10], "bootstrap": [True, False], "criterion": ["gini", "entropy"]} diff --git a/sklearn/ensemble/forest.py b/sklearn/ensemble/forest.py index 0f517166d202f..b276bf5513c90 100644 --- a/sklearn/ensemble/forest.py +++ b/sklearn/ensemble/forest.py @@ -784,8 +784,8 @@ class RandomForestClassifier(ForestClassifier): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -795,19 +795,28 @@ class RandomForestClassifier(ForestClassifier): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default="auto") The number of features to consider when looking for the best split: @@ -954,9 +963,10 @@ class labels (multi-output problem). RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini', max_depth=2, max_features='auto', max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, - min_samples_leaf=1, min_samples_split=2, - min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=None, - oob_score=False, random_state=0, verbose=0, warm_start=False) + min_samples_leaf='deprecated', min_samples_split=2, + min_weight_fraction_leaf='deprecated', n_estimators=100, + n_jobs=None, oob_score=False, random_state=0, verbose=0, + warm_start=False) >>> print(clf.feature_importances_) [0.14205973 0.76664038 0.0282433 0.06305659] >>> print(clf.predict([[0, 0, 0, 0]])) @@ -965,7 +975,7 @@ class labels (multi-output problem). Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -991,8 +1001,8 @@ def __init__(self, criterion="gini", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features="auto", max_leaf_nodes=None, min_impurity_decrease=0., @@ -1069,8 +1079,8 @@ class RandomForestRegressor(ForestRegressor): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -1080,19 +1090,28 @@ class RandomForestRegressor(ForestRegressor): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default="auto") The number of features to consider when looking for the best split: @@ -1201,9 +1220,10 @@ class RandomForestRegressor(ForestRegressor): RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=2, max_features='auto', max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, - min_samples_leaf=1, min_samples_split=2, - min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=None, - oob_score=False, random_state=0, verbose=0, warm_start=False) + min_samples_leaf='deprecated', min_samples_split=2, + min_weight_fraction_leaf='deprecated', n_estimators=100, + n_jobs=None, oob_score=False, random_state=0, verbose=0, + warm_start=False) >>> print(regr.feature_importances_) [0.18146984 0.81473937 0.00145312 0.00233767] >>> print(regr.predict([[0, 0, 0, 0]])) @@ -1212,7 +1232,7 @@ class RandomForestRegressor(ForestRegressor): Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -1238,8 +1258,8 @@ def __init__(self, criterion="mse", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features="auto", max_leaf_nodes=None, min_impurity_decrease=0., @@ -1307,8 +1327,8 @@ class ExtraTreesClassifier(ForestClassifier): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -1318,19 +1338,28 @@ class ExtraTreesClassifier(ForestClassifier): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default="auto") The number of features to consider when looking for the best split: @@ -1465,7 +1494,7 @@ class labels (multi-output problem). Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -1487,8 +1516,8 @@ def __init__(self, criterion="gini", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features="auto", max_leaf_nodes=None, min_impurity_decrease=0., @@ -1563,8 +1592,8 @@ class ExtraTreesRegressor(ForestRegressor): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -1574,19 +1603,28 @@ class ExtraTreesRegressor(ForestRegressor): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default="auto") The number of features to consider when looking for the best split: @@ -1684,7 +1722,7 @@ class ExtraTreesRegressor(ForestRegressor): Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -1705,8 +1743,8 @@ def __init__(self, criterion="mse", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features="auto", max_leaf_nodes=None, min_impurity_decrease=0., @@ -1775,8 +1813,8 @@ class RandomTreesEmbedding(BaseForest): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` is the minimum number of samples for each split. @@ -1786,19 +1824,28 @@ class RandomTreesEmbedding(BaseForest): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` is the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_leaf_nodes : int or None, optional (default=None) Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. @@ -1874,8 +1921,8 @@ def __init__(self, n_estimators='warn', max_depth=5, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_leaf_nodes=None, min_impurity_decrease=0., min_impurity_split=None, diff --git a/sklearn/ensemble/gradient_boosting.py b/sklearn/ensemble/gradient_boosting.py index 2b6165d403b57..6e9cd843d59b9 100644 --- a/sklearn/ensemble/gradient_boosting.py +++ b/sklearn/ensemble/gradient_boosting.py @@ -25,6 +25,7 @@ from abc import ABCMeta from abc import abstractmethod +import warnings from .base import BaseEnsemble from ..base import ClassifierMixin @@ -1124,13 +1125,13 @@ class BaseGradientBoosting(six.with_metaclass(ABCMeta, BaseEnsemble)): @abstractmethod def __init__(self, loss, learning_rate, n_estimators, criterion, - min_samples_split, min_samples_leaf, min_weight_fraction_leaf, + min_samples_split, min_weight_fraction_leaf, max_depth, min_impurity_decrease, min_impurity_split, init, subsample, max_features, random_state, alpha=0.9, verbose=0, max_leaf_nodes=None, - warm_start=False, presort='auto', - validation_fraction=0.1, n_iter_no_change=None, - tol=1e-4): + min_samples_leaf='deprecated', warm_start=False, + presort='auto', validation_fraction=0.1, + n_iter_no_change=None, tol=1e-4): self.n_estimators = n_estimators self.learning_rate = learning_rate @@ -1497,9 +1498,17 @@ def _fit_stages(self, X, y, y_pred, sample_weight, random_state, n_inbag = max(1, int(self.subsample * n_samples)) loss_ = self.loss_ + if self.min_weight_fraction_leaf != 'deprecated': + warnings.warn("'min_weight_fraction_leaf' is deprecated in 0.20 " + "and will be fixed to a value of 0 in 0.22.", + DeprecationWarning) + min_weight_fraction_leaf = self.min_weight_fraction_leaf + else: + min_weight_fraction_leaf = 0. + # Set min_weight_leaf from min_weight_fraction_leaf - if self.min_weight_fraction_leaf != 0. and sample_weight is not None: - min_weight_leaf = (self.min_weight_fraction_leaf * + if min_weight_fraction_leaf != 0. and sample_weight is not None: + min_weight_leaf = (min_weight_fraction_leaf * np.sum(sample_weight)) else: min_weight_leaf = 0. @@ -1737,8 +1746,8 @@ class GradientBoostingClassifier(BaseGradientBoosting, ClassifierMixin): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -1748,19 +1757,28 @@ class GradientBoostingClassifier(BaseGradientBoosting, ClassifierMixin): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_depth : integer, optional (default=3) maximum depth of the individual regression estimators. The maximum depth limits the number of nodes in the tree. Tune this parameter @@ -1937,7 +1955,8 @@ class GradientBoostingClassifier(BaseGradientBoosting, ClassifierMixin): def __init__(self, loss='deviance', learning_rate=0.1, n_estimators=100, subsample=1.0, criterion='friedman_mse', min_samples_split=2, - min_samples_leaf=1, min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_depth=3, min_impurity_decrease=0., min_impurity_split=None, init=None, random_state=None, max_features=None, verbose=0, @@ -2192,8 +2211,8 @@ class GradientBoostingRegressor(BaseGradientBoosting, RegressorMixin): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -2203,13 +2222,17 @@ class GradientBoostingRegressor(BaseGradientBoosting, RegressorMixin): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all @@ -2387,7 +2410,8 @@ class GradientBoostingRegressor(BaseGradientBoosting, RegressorMixin): def __init__(self, loss='ls', learning_rate=0.1, n_estimators=100, subsample=1.0, criterion='friedman_mse', min_samples_split=2, - min_samples_leaf=1, min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_depth=3, min_impurity_decrease=0., min_impurity_split=None, init=None, random_state=None, max_features=None, alpha=0.9, verbose=0, max_leaf_nodes=None, diff --git a/sklearn/ensemble/tests/test_forest.py b/sklearn/ensemble/tests/test_forest.py index d7586c2866571..a470913f5f327 100644 --- a/sklearn/ensemble/tests/test_forest.py +++ b/sklearn/ensemble/tests/test_forest.py @@ -762,13 +762,16 @@ def check_min_samples_leaf(name): ForestEstimator = FOREST_ESTIMATORS[name] # test boundary value - assert_raises(ValueError, - ForestEstimator(min_samples_leaf=-1).fit, X, y) - assert_raises(ValueError, - ForestEstimator(min_samples_leaf=0).fit, X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises(ValueError, + ForestEstimator(min_samples_leaf=-1).fit, X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises(ValueError, + ForestEstimator(min_samples_leaf=0).fit, X, y) est = ForestEstimator(min_samples_leaf=5, n_estimators=1, random_state=0) - est.fit(X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + est.fit(X, y) out = est.estimators_[0].tree_.apply(X) node_counts = np.bincount(out) # drop inner nodes @@ -778,7 +781,8 @@ def check_min_samples_leaf(name): est = ForestEstimator(min_samples_leaf=0.25, n_estimators=1, random_state=0) - est.fit(X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + est.fit(X, y) out = est.estimators_[0].tree_.apply(X) node_counts = np.bincount(out) # drop inner nodes @@ -811,7 +815,9 @@ def check_min_weight_fraction_leaf(name): if "RandomForest" in name: est.bootstrap = False - est.fit(X, y, sample_weight=weights) + with pytest.warns(DeprecationWarning, + match='min_weight_fraction_leaf'): + est.fit(X, y, sample_weight=weights) out = est.estimators_[0].tree_.apply(X) node_weights = np.bincount(out, weights=weights) # drop inner nodes diff --git a/sklearn/ensemble/tests/test_gradient_boosting.py b/sklearn/ensemble/tests/test_gradient_boosting.py index 6f7654c7d6061..332ab89317e1c 100644 --- a/sklearn/ensemble/tests/test_gradient_boosting.py +++ b/sklearn/ensemble/tests/test_gradient_boosting.py @@ -106,17 +106,29 @@ def test_classifier_parameter_checks(): assert_raises(ValueError, GradientBoostingClassifier(min_samples_split=1.1).fit, X, y) - assert_raises(ValueError, - GradientBoostingClassifier(min_samples_leaf=0).fit, X, y) - assert_raises(ValueError, - GradientBoostingClassifier(min_samples_leaf=-1.0).fit, X, y) - - assert_raises(ValueError, - GradientBoostingClassifier(min_weight_fraction_leaf=-1.).fit, - X, y) - assert_raises(ValueError, - GradientBoostingClassifier(min_weight_fraction_leaf=0.6).fit, - X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises( + ValueError, + GradientBoostingClassifier(min_samples_leaf=0).fit, + X, y + ) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises( + ValueError, + GradientBoostingClassifier(min_samples_leaf=-1.0).fit, + X, y + ) + + with pytest.warns(DeprecationWarning, match='min_weight_fraction_leaf'): + assert_raises(ValueError, + GradientBoostingClassifier( + min_weight_fraction_leaf=-1.).fit, + X, y) + with pytest.warns(DeprecationWarning, match='min_weight_fraction_leaf'): + assert_raises(ValueError, + GradientBoostingClassifier( + min_weight_fraction_leaf=0.6).fit, + X, y) assert_raises(ValueError, GradientBoostingClassifier(subsample=0.0).fit, X, y) diff --git a/sklearn/tree/tests/test_tree.py b/sklearn/tree/tests/test_tree.py index 37eb6582c7023..68b5040374290 100644 --- a/sklearn/tree/tests/test_tree.py +++ b/sklearn/tree/tests/test_tree.py @@ -507,16 +507,28 @@ def test_error(): assert_raises(ValueError, est.predict_proba, X2) for name, TreeEstimator in ALL_TREES.items(): - assert_raises(ValueError, TreeEstimator(min_samples_leaf=-1).fit, X, y) - assert_raises(ValueError, TreeEstimator(min_samples_leaf=.6).fit, X, y) - assert_raises(ValueError, TreeEstimator(min_samples_leaf=0.).fit, X, y) - assert_raises(ValueError, TreeEstimator(min_samples_leaf=3.).fit, X, y) - assert_raises(ValueError, - TreeEstimator(min_weight_fraction_leaf=-1).fit, - X, y) - assert_raises(ValueError, - TreeEstimator(min_weight_fraction_leaf=0.51).fit, - X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises(ValueError, + TreeEstimator(min_samples_leaf=-1).fit, X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises(ValueError, + TreeEstimator(min_samples_leaf=.6).fit, X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises(ValueError, + TreeEstimator(min_samples_leaf=0.).fit, X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + assert_raises(ValueError, + TreeEstimator(min_samples_leaf=3.).fit, X, y) + with pytest.warns(DeprecationWarning, + match='min_weight_fraction_leaf'): + assert_raises(ValueError, + TreeEstimator(min_weight_fraction_leaf=-1).fit, + X, y) + with pytest.warns(DeprecationWarning, + match='min_weight_fraction_leaf'): + assert_raises(ValueError, + TreeEstimator(min_weight_fraction_leaf=0.51).fit, + X, y) assert_raises(ValueError, TreeEstimator(min_samples_split=-1).fit, X, y) assert_raises(ValueError, TreeEstimator(min_samples_split=0.0).fit, @@ -619,7 +631,8 @@ def test_min_samples_leaf(): est = TreeEstimator(min_samples_leaf=5, max_leaf_nodes=max_leaf_nodes, random_state=0) - est.fit(X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + est.fit(X, y) out = est.tree_.apply(X) node_counts = np.bincount(out) # drop inner nodes @@ -631,7 +644,8 @@ def test_min_samples_leaf(): est = TreeEstimator(min_samples_leaf=0.1, max_leaf_nodes=max_leaf_nodes, random_state=0) - est.fit(X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + est.fit(X, y) out = est.tree_.apply(X) node_counts = np.bincount(out) # drop inner nodes @@ -660,7 +674,9 @@ def check_min_weight_fraction_leaf(name, datasets, sparse=False): est = TreeEstimator(min_weight_fraction_leaf=frac, max_leaf_nodes=max_leaf_nodes, random_state=0) - est.fit(X, y, sample_weight=weights) + with pytest.warns(DeprecationWarning, + match='min_weight_fraction_leaf'): + est.fit(X, y, sample_weight=weights) if sparse: out = est.tree_.apply(X.tocsr()) @@ -685,7 +701,9 @@ def check_min_weight_fraction_leaf(name, datasets, sparse=False): est = TreeEstimator(min_weight_fraction_leaf=frac, max_leaf_nodes=max_leaf_nodes, random_state=0) - est.fit(X, y) + with pytest.warns(DeprecationWarning, + match='min_weight_fraction_leaf'): + est.fit(X, y) if sparse: out = est.tree_.apply(X.tocsr()) @@ -731,7 +749,8 @@ def check_min_weight_fraction_leaf_with_min_samples_leaf(name, datasets, max_leaf_nodes=max_leaf_nodes, min_samples_leaf=5, random_state=0) - est.fit(X, y) + with pytest.warns(DeprecationWarning): + est.fit(X, y) if sparse: out = est.tree_.apply(X.tocsr()) @@ -756,7 +775,8 @@ def check_min_weight_fraction_leaf_with_min_samples_leaf(name, datasets, max_leaf_nodes=max_leaf_nodes, min_samples_leaf=.1, random_state=0) - est.fit(X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + est.fit(X, y) if sparse: out = est.tree_.apply(X.tocsr()) @@ -1412,10 +1432,16 @@ def check_sparse_parameters(tree, dataset): assert_array_almost_equal(s.predict(X), d.predict(X)) # Check min_samples_leaf - d = TreeEstimator(random_state=0, - min_samples_leaf=X_sparse.shape[0] // 2).fit(X, y) - s = TreeEstimator(random_state=0, - min_samples_leaf=X_sparse.shape[0] // 2).fit(X_sparse, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + d = TreeEstimator( + random_state=0, + min_samples_leaf=X_sparse.shape[0] // 2 + ).fit(X, y) + with pytest.warns(DeprecationWarning, match='min_samples_leaf'): + s = TreeEstimator( + random_state=0, + min_samples_leaf=X_sparse.shape[0] // 2 + ).fit(X_sparse, y) assert_tree_equal(d.tree_, s.tree_, "{0} with dense and sparse format gave different " "trees".format(tree)) @@ -1560,7 +1586,8 @@ def _check_min_weight_leaf_split_level(TreeEstimator, X, y, sample_weight): assert_equal(est.tree_.max_depth, 1) est = TreeEstimator(random_state=0, min_weight_fraction_leaf=0.4) - est.fit(X, y, sample_weight=sample_weight) + with pytest.warns(DeprecationWarning, match='min_weight_fraction_leaf'): + est.fit(X, y, sample_weight=sample_weight) assert_equal(est.tree_.max_depth, 0) diff --git a/sklearn/tree/tree.py b/sklearn/tree/tree.py index 7105a86ce05fa..437dc197c7a04 100644 --- a/sklearn/tree/tree.py +++ b/sklearn/tree/tree.py @@ -85,26 +85,26 @@ def __init__(self, splitter, max_depth, min_samples_split, - min_samples_leaf, min_weight_fraction_leaf, max_features, max_leaf_nodes, random_state, min_impurity_decrease, min_impurity_split, + min_samples_leaf='deprecated', class_weight=None, presort=False): self.criterion = criterion self.splitter = splitter self.max_depth = max_depth self.min_samples_split = min_samples_split - self.min_samples_leaf = min_samples_leaf self.min_weight_fraction_leaf = min_weight_fraction_leaf self.max_features = max_features self.random_state = random_state self.max_leaf_nodes = max_leaf_nodes self.min_impurity_decrease = min_impurity_decrease self.min_impurity_split = min_impurity_split + self.min_samples_leaf = min_samples_leaf self.class_weight = class_weight self.presort = presort @@ -173,18 +173,24 @@ def fit(self, X, y, sample_weight=None, check_input=True, max_leaf_nodes = (-1 if self.max_leaf_nodes is None else self.max_leaf_nodes) - if isinstance(self.min_samples_leaf, (numbers.Integral, np.integer)): - if not 1 <= self.min_samples_leaf: + if self.min_samples_leaf != 'deprecated': + warnings.warn("'min_samples_leaf' is deprecated in 0.20 and " + "will be fixed to a value of 1 in 0.22.", + DeprecationWarning) + min_samples_leaf = self.min_samples_leaf + else: + min_samples_leaf = 1 + if isinstance(min_samples_leaf, (numbers.Integral, np.integer)): + if not 1 <= min_samples_leaf: raise ValueError("min_samples_leaf must be at least 1 " "or in (0, 0.5], got %s" - % self.min_samples_leaf) - min_samples_leaf = self.min_samples_leaf + % min_samples_leaf) else: # float - if not 0. < self.min_samples_leaf <= 0.5: + if not 0. < min_samples_leaf <= 0.5: raise ValueError("min_samples_leaf must be at least 1 " "or in (0, 0.5], got %s" - % self.min_samples_leaf) - min_samples_leaf = int(ceil(self.min_samples_leaf * n_samples)) + % min_samples_leaf) + min_samples_leaf = int(ceil(min_samples_leaf * n_samples)) if isinstance(self.min_samples_split, (numbers.Integral, np.integer)): if not 2 <= self.min_samples_split: @@ -234,7 +240,15 @@ def fit(self, X, y, sample_weight=None, check_input=True, if len(y) != n_samples: raise ValueError("Number of labels=%d does not match " "number of samples=%d" % (len(y), n_samples)) - if not 0 <= self.min_weight_fraction_leaf <= 0.5: + + if self.min_weight_fraction_leaf != 'deprecated': + warnings.warn("'min_weight_fraction_leaf' is deprecated in 0.20 " + "and will be fixed to a value of 0 in 0.22.", + DeprecationWarning) + min_weight_fraction_leaf = self.min_weight_fraction_leaf + else: + min_weight_fraction_leaf = 0 + if not 0 <= min_weight_fraction_leaf <= 0.5: raise ValueError("min_weight_fraction_leaf must in [0, 0.5]") if max_depth <= 0: raise ValueError("max_depth must be greater than zero. ") @@ -269,10 +283,10 @@ def fit(self, X, y, sample_weight=None, check_input=True, # Set min_weight_leaf from min_weight_fraction_leaf if sample_weight is None: - min_weight_leaf = (self.min_weight_fraction_leaf * + min_weight_leaf = (min_weight_fraction_leaf * n_samples) else: - min_weight_leaf = (self.min_weight_fraction_leaf * + min_weight_leaf = (min_weight_fraction_leaf * np.sum(sample_weight)) if self.min_impurity_split is not None: @@ -539,8 +553,8 @@ class DecisionTreeClassifier(BaseDecisionTree, ClassifierMixin): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -550,19 +564,28 @@ class DecisionTreeClassifier(BaseDecisionTree, ClassifierMixin): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default=None) The number of features to consider when looking for the best split: @@ -680,7 +703,7 @@ class DecisionTreeClassifier(BaseDecisionTree, ClassifierMixin): Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -728,8 +751,8 @@ def __init__(self, splitter="best", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features=None, random_state=None, max_leaf_nodes=None, @@ -907,8 +930,8 @@ class DecisionTreeRegressor(BaseDecisionTree, RegressorMixin): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -918,19 +941,28 @@ class DecisionTreeRegressor(BaseDecisionTree, RegressorMixin): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default=None) The number of features to consider when looking for the best split: @@ -1019,7 +1051,7 @@ class DecisionTreeRegressor(BaseDecisionTree, RegressorMixin): Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -1067,8 +1099,8 @@ def __init__(self, splitter="best", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features=None, random_state=None, max_leaf_nodes=None, @@ -1165,8 +1197,8 @@ class ExtraTreeClassifier(DecisionTreeClassifier): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -1176,19 +1208,28 @@ class ExtraTreeClassifier(DecisionTreeClassifier): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default="auto") The number of features to consider when looking for the best split: @@ -1272,7 +1313,7 @@ class ExtraTreeClassifier(DecisionTreeClassifier): Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -1288,8 +1329,8 @@ def __init__(self, splitter="random", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features="auto", random_state=None, max_leaf_nodes=None, @@ -1349,8 +1390,8 @@ class ExtraTreeRegressor(DecisionTreeRegressor): min_samples_split : int, float, optional (default=2) The minimum number of samples required to split an internal node: - - If int, then consider `min_samples_split` as the minimum number. - - If float, then `min_samples_split` is a fraction and + - If int, then consider ``min_samples_split`` as the minimum number. + - If float, then ``min_samples_split`` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. @@ -1360,19 +1401,28 @@ class ExtraTreeRegressor(DecisionTreeRegressor): min_samples_leaf : int, float, optional (default=1) The minimum number of samples required to be at a leaf node: - - If int, then consider `min_samples_leaf` as the minimum number. - - If float, then `min_samples_leaf` is a fraction and + - If int, then consider ``min_samples_leaf`` as the minimum number. + - If float, then ``min_samples_leaf`` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. + .. deprecated:: 0.20 + The parameter ``min_samples_leaf`` is deprecated in version 0.20 and + will be fixed to a value of 1 in version 0.22. It was not effective + for regularization and empirically, 1 is the best value. min_weight_fraction_leaf : float, optional (default=0.) The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. + .. deprecated:: 0.20 + The parameter ``min_weight_fraction_leaf`` is deprecated in version + 0.20. Its implementation, like ``min_samples_leaf``, is ineffective + for regularization. + max_features : int, float, string or None, optional (default="auto") The number of features to consider when looking for the best split: @@ -1436,7 +1486,7 @@ class ExtraTreeRegressor(DecisionTreeRegressor): Notes ----- The default values for the parameters controlling the size of the trees - (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and + (e.g. ``max_depth``, ``min_samples_split``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. @@ -1452,8 +1502,8 @@ def __init__(self, splitter="random", max_depth=None, min_samples_split=2, - min_samples_leaf=1, - min_weight_fraction_leaf=0., + min_samples_leaf='deprecated', + min_weight_fraction_leaf='deprecated', max_features="auto", random_state=None, min_impurity_decrease=0.,