skada.metrics.MaNoScorer
- class skada.metrics.MaNoScorer(p=4, threshold=5, greater_is_better=True, kwargs=None)[source]
MaNo scorer inspired by [37], an approach for unsupervised accuracy estimation.
This scorer used the model's predictions on target data to estimate the accuracy of the model. The original implementation in [37] is tailored to neural networks and consist of three steps: 1) Recover target logits (inference step), 2) Normalize them as probabilities (e.g., with softmax), 3) Aggregate by averaging the p-norms of the target normalized logits.
To ensure compatibility with any estimator, we adapt the original implementation. If the estimator is a neural network, follow 1) --> 2) --> 3) like in [37]. Else, directly use the probabilities predicted by the estimator and then do 3).
See [37] for details.
- Parameters:
- pint, default=4
Order for the p-norm normalization. It must be non-negative.
- thresholdint, default=5
Threshold value to determine which normalization to use. If threshold <= 0, softmax normalization is always used. See Eq.(6) of [37] for more details.
- greater_is_betterbool, default=True
Whether higher scores are better.
- Returns:
- scorefloat in [0, 1] (or [-1, 0] depending on the value of self._sign).
References