Description
Currently SearchLight.score() uses the scorer of the base_estimator.
Typically we would like to change the scoring, e.g. to use an AUC for a logistic regression, I was thinking of doing the following API:
sl = SearchLight(LogisticRegression(), scoring=my_scorer())
sl.fit(X, y).score(X, y)
An alternative is to change the estimator directly e.g.
clf = ChangerScorer(LogisticRegression(), my_scorer())
sl = SearchLight(clf)
sl.fit(X, y).score(X, y)
What I'm unclear is how to adapt the predict method as a function of the scoring metrics, and whether this should be done automatically.
Anyone has a clear idea how what would be the most sklearn way of doing it?