skada.metrics.DeepEmbeddedValidation

class skada.metrics.DeepEmbeddedValidation(domain_classifier=None, loss_func=None, random_state=None, greater_is_better=False)

Loss based on source data using features representation to weight samples.

See [20] for details.

Parameters:

domain_classifiersklearn classifier, optional

Classifier used to predict the domains. If None, a LogisticRegression is used.

loss_funccallable

Loss function with signature loss_func(y, y_pred, **kwargs). The loss function need not to be reduced.

random_stateint, RandomState instance or None, default=None

Determines random number generation for train_test_split. Pass an int for reproducible output across multiple function calls.

greater_is_betterbool, default=False

Whether scorer is a score function (default), meaning high is good, or a loss function, meaning low is good. In the latter case, the scorer object will sign-flip the outcome of the scorer.

References

[20]

Kaichao You et al. Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation. ICML, 2019.

cross_entropy_loss(y_true, y_pred, epsilon=1e-15)

Compute cross-entropy loss for a single sample between the true label and the predicted probability estimates.

This loss allows for a changing number of classes over the validation process.

Parameters:

- y_true: int

True label (integer label).

- y_pred: array-like

Predicted probabilities for each class.

- epsilon: float, optional (default=1e-15)

A small constant to avoid numerical instability.

Returns:

• float

  Cross-entropy loss for the single sample.

set_score_request(*, sample_domain: bool | None | str = '$UNCHANGED$') → DeepEmbeddedValidation

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_domain parameter in score.

Returns:

selfobject

The updated object.