skada.metrics.DeepEmbeddedValidation

class skada.metrics.DeepEmbeddedValidation(domain_classifier=None, loss_func=None, random_state=None, greater_is_better=False)

Loss based on source data using features representation to weight samples.

See [20] for details.

Parameters:

domain_classifiersklearn classifier, optional

Classifier used to predict the domains. If None, a LogisticRegression is used.

loss_funccallable

Loss function with signature loss_func(y, y_pred, **kwargs). The loss function need not to be reduced.

random_stateint, RandomState instance or None, default=None

Determines random number generation for train_test_split. Pass an int for reproducible output across multiple function calls.

greater_is_betterbool, default=False

Whether scorer is a score function (default), meaning high is good, or a loss function, meaning low is good. In the latter case, the scorer object will sign-flip the outcome of the scorer.

References

[20]

Kaichao You et al. Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation. ICML, 2019.

cross_entropy_loss(y_true, y_pred, epsilon=1e-15)

Compute cross-entropy loss for a single sample between the true label and the predicted probability estimates.

This loss allows for a changing number of classes over the validation process.

Parameters:

- y_true: int

True label (integer label).

- y_pred: array-like

Predicted probabilities for each class.

- epsilon: float, optional (default=1e-15)

A small constant to avoid numerical instability.

Returns:

• float

  Cross-entropy loss for the single sample.

set_score_request(*, sample_domain: bool | None | str = '$UNCHANGED$') → DeepEmbeddedValidation

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

sample_domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_domain parameter in score.

Returns:

selfobject

The updated object.