skada.metrics.DeepEmbeddedValidation

class skada.metrics.DeepEmbeddedValidation(domain_classifier=None, loss_func=None, random_state=None, greater_is_better=False, kwargs=None)

Loss based on source data using features representation to weight samples.

See [20] for details.

Parameters:

domain_classifiersklearn classifier, optional

Classifier used to predict the domains. If None, a LogisticRegression is used.

loss_funccallable

Loss function with signature loss_func(y, y_pred, **kwargs). The loss function need not to be reduced.

random_stateint, RandomState instance or None, default=None

Determines random number generation for train_test_split. Pass an int for reproducible output across multiple function calls.

greater_is_betterbool, default=False

Whether scorer is a score function (default), meaning high is good, or a loss function, meaning low is good. In the latter case, the scorer object will sign-flip the outcome of the scorer.

References

[20]

Kaichao You et al. Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation. ICML, 2019.

cross_entropy_loss(y_true, y_pred, epsilon=1e-15)

Compute cross-entropy loss for a single sample between the true label and the predicted probability estimates.

This loss allows for a changing number of classes over the validation process.

Parameters:

- y_true: int

True label (integer label).

- y_pred: array-like

Predicted probabilities for each class.

- epsilon: float, optional (default=1e-15)

A small constant to avoid numerical instability.

Returns:

• float

  Cross-entropy loss for the single sample.