8.19.1.1. sklearn.metrics.Scorer

class sklearn.metrics.Scorer(score_func, greater_is_better=True, needs_threshold=False, **kwargs)

Flexible scores for any estimator.

This class wraps estimator scoring functions for the use in GridSearchCV and cross_val_score. It takes a score function, such as accuracy_score, mean_squared_error, adjusted_rand_index or average_precision and provides a call method.

Parameters :

score_func : callable,

Score function (or loss function) with signature score_func(y, y_pred, **kwargs).

greater_is_better : boolean, default=True

Whether score_func is a score function (default), meaning high is good, or a loss function, meaning low is good.

needs_threshold : bool, default=False

Whether score_func takes a continuous decision certainty. For example average_precision or the area under the roc curve can not be computed using predictions alone, but need the output of decision_function or predict_proba.

**kwargs : additional arguments

Additional parameters to be passed to score_func.

Examples

>>> from sklearn.metrics import fbeta_score, Scorer
>>> ftwo_scorer = Scorer(fbeta_score, beta=2)
>>> from sklearn.grid_search import GridSearchCV
>>> from sklearn.svm import LinearSVC
>>> grid = GridSearchCV(LinearSVC(), param_grid={'C': [1, 10]},
...                     scoring=ftwo_scorer)

__init__(score_func, greater_is_better=True, needs_threshold=False, **kwargs)

__call__(estimator, X, y)

Score X and y using the provided estimator.

Parameters :

estimator : object

Trained estimator to use for scoring. If needs_threshold is True, estimator needs to provide decision_function or predict_proba. Otherwise, estimator needs to provide predict.

X : array-like or sparse matrix

Test data that will be scored by the estimator.

y : array-like

True prediction for X.

Returns :

score : float

Score function applied to prediction of estimator on X.