skada.metrics.CircularValidation

class skada.metrics.CircularValidation(source_scorer=<function balanced_accuracy_score>, greater_is_better=True, kwargs=None)[source]

Score based on a circular validation strategy.

This scorer retrain the estimator, with the exact same parameters, on the predicted target domain samples. Then, the retrained estimator is used to predict the source domain labels. The score is then computed using the source_scorer between the true source labels and the predicted source labels.

See [11] for details.

Parameters:

source_scorercallable, default = sklearn.metrics.balanced_accuracy_score: Scorer used on the source domain samples. It should be a callable of the form source_scorer(y, y_pred).
greater_is_betterbool, default=False: Whether scorer is a score function (default), meaning high is good, or a loss function, meaning low is good. In the latter case, the scorer object will sign-flip the outcome of the scorer.

References

[11]

Bruzzone, L., & Marconcini, M. 'Domain adaptation problems: A DASVM classification technique and a circular validation strategy.' IEEE transactions on pattern analysis and machine intelligence, (2009).