skada.metrics.MixValScorer

class skada.metrics.MixValScorer(alpha=0.55, ice_type='both', scoring=None, greater_is_better=True, random_state=None)

MixVal scorer for unsupervised domain adaptation.

This scorer uses mixup to create mixed samples from the target domain, and evaluates the model's consistency on these mixed samples.

See [32] for details.

Parameters:

alphafloat, default=0.55

Mixing parameter for mixup.

ice_type{'both', 'intra', 'inter'}, default='both'

Type of ICE score to compute: - 'both': Compute both intra-cluster and inter-cluster ICE scores (average). - 'intra': Compute only intra-cluster ICE score. - 'inter': Compute only inter-cluster ICE score.

scoringstr or callable, default=None

A string (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y). If None, the provided estimator object's score method is used.

greater_is_betterbool, default=True

Whether higher scores are better.

random_stateint, RandomState instance or None, default=None

Controls the randomness of the mixing process.

Attributes:

alphafloat

Mixing parameter.

random_stateRandomState

Random number generator.

_signint

1 if greater_is_better is True, -1 otherwise.

ice_typestr

Type of ICE score to compute.

References

[32]

Dapeng Hu et al. Mixed Samples as Probes for Unsupervised Model Selection in Domain Adaptation. NeurIPS, 2023.

set_score_request(*, sample_domain: bool | None | str = '$UNCHANGED$') → MixValScorer

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_domain parameter in score.

Returns:

selfobject

The updated object.