API split "model" parameters from "config" params. #17566
Description
We can think of our estimators' parameters as two different types:
- model hyper-parameters which influence the end result, accuracy, complexity, etc., e.g. SVM's
nuor the number of trees in a random forest. - config parameters which control other aspects of an estimator such as verbosity, copy_X, return_xyz, etc.
We could split these two types of parameters and have model hyperparams as __init__ args and the rest as set_params or a new configure args. This would also allow us to introduce new configuration parameters to all estimators w/o having to add another arg to __init__. An example would be the routing parameter required for sample props implementation, or potentially controlling log level at the estimator level.
My proposal would be to have a config_params estimator tag which includes the names of those parameters, and have clone, get_params and set_params understand them, and potentially introduce a new configure method to set those parameters.
This also would mean partially relaxing the set only __init__ params in __init__ requirement since __init__ would be allowed to set those extra params.
cc: @scikit-learn/core-devs
also related to: #17441 (comment)