Closed
Description
I have a few (probably stupid) suggestions to get this discussion started -
- Rename
make_scorertomake_objective, deprecate thegreater_is_better/scorerattributes and instead useloss_funcandscore_funcand document it well to note thatloss_funcis minimized whilescore_funcwill be maximized. (If both are given, throw an error)
make_objective(score_func=None, loss_func=None, needs_proba=False,... **kwargs)
- In
make_objective, when loss functions are passed toscore_funcinstead ofloss_func(score_funcis set tomean_squared_error), directly throw an error, as we are deprecating and retaining the oldmake_scorertill 0.20. - Replace all
_*Scorer*by(_?)*Objective*(_ProbabilityObjective/_PredictionObjective/_ThresholdedObjective) - Replace all
scoringattribute byobjectiveattribute. objectiveobject returned bymake_scorerhas a boolean propertyobjective_is_minimizedobjective_is_lossto denote whether theObjectivewas formed from aloss_func(FalseTrue) orscore_func(TrueFalse).
EDIT The objective_is_loss is just for introspection. objective will always return an error score which is always minimized, irrespective of whether it was formed from a loss_func or score_func.
- Split
cross_val_scoreintocross_val_scoreandcross_val_loss. Raise deprecation warning if thescore_funcandloss_funcare interchanged til 0.20 and after that throw an error. This is probably superfluous but helps separate demarcate what we mean bylossandscore.
Please feel free to point out if this seems stupid.
We need to have this discussion before deciding on multiple metric grid search.
Related issue : #2439
@jnothman @amueller @ogrisel @GaelVaroquaux @MechCoder @vene @larsmans @mblondel @jaquesgrobler @agramfort @TomDLT
Metadata
Metadata
Assignees
Labels
No labels