skada.metrics.MixValScorer

class skada.metrics.MixValScorer(alpha=0.55, ice_type='both', scoring=None, greater_is_better=True, random_state=None)

MixVal scorer for unsupervised domain adaptation.

This scorer uses mixup to create mixed samples from the target domain, and evaluates the model's consistency on these mixed samples.

See [32] for details.

Parameters:

alphafloat, default=0.55

Mixing parameter for mixup.

ice_type{'both', 'intra', 'inter'}, default='both'

Type of ICE score to compute: - 'both': Compute both intra-cluster and inter-cluster ICE scores (average). - 'intra': Compute only intra-cluster ICE score. - 'inter': Compute only inter-cluster ICE score.

scoringstr or callable, default=None

A string (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y). If None, the provided estimator object's score method is used.

greater_is_betterbool, default=True

Whether higher scores are better.

random_stateint, RandomState instance or None, default=None

Controls the randomness of the mixing process.

Attributes:

alphafloat

Mixing parameter.

random_stateRandomState

Random number generator.

_signint

1 if greater_is_better is True, -1 otherwise.

ice_typestr

Type of ICE score to compute.

References

[32]

Dapeng Hu et al. Mixed Samples as Probes for Unsupervised Model Selection in Domain Adaptation. NeurIPS, 2023.

set_score_request(*, sample_domain: bool | None | str = '$UNCHANGED$') → MixValScorer

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

sample_domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_domain parameter in score.

Returns:

selfobject

The updated object.