Aggregation

The three aggregation operators, plus the conventions they require.

Given indicator matrices \(Y, \hat{Y} \in \{0,1\}^{n \times m}\), the four cell matrices are formed entrywise and then reduced in one of three ways:

Operator

Reduction

Conventional name

sigma_1

over all entries

micro

sigma_m

column-wise, then averaged

macro (or weighted)

sigma_n

row-wise, then averaged

exemplar (average='samples')

Column- and row-wise aggregation divide before averaging, so an individual entry can be undefined when its denominator vanishes. Because the resolution changes the reported value, it is an explicit parameter rather than an implicit default. The package defaults to 'exclude', which averages over the defined entries only.

aggmetrics.aggregation.cells(Y, P, mask=None)

Entrywise TP, FP, FN, TN matrices.

Works for crisp indicator matrices and, unchanged, for soft ones in [0, 1] when they have already been combined by a t-norm (see aggmetrics.tnorms).

mask is an optional observation mask: entries where mask == 0 are treated as unobserved and contribute nothing, which covers partial labels, semi-supervised evaluation and masked-out abstentions.

Return type:

Tuple[ndarray, ndarray, ndarray, ndarray]

aggmetrics.aggregation.sigma_1(X)

Global aggregation: sum over all entries.

Parameters:

X (ndarray)

Return type:

float

aggmetrics.aggregation.sigma_m(X)

Column-wise aggregation: one value per class or label.

Parameters:

X (ndarray)

Return type:

ndarray

aggmetrics.aggregation.sigma_n(X)

Row-wise aggregation: one value per example.

Parameters:

X (ndarray)

Return type:

ndarray

aggmetrics.aggregation.sigma_k(cell_list)

Fold-wise aggregation across K cross-validation splits.

Sums the four cell matrices (TP, FP, FN, TN) across multiple folds.

Parameters:

cell_list (Sequence[Tuple[ndarray, ndarray, ndarray, ndarray]])

Return type:

Tuple[ndarray, ndarray, ndarray, ndarray]

aggmetrics.aggregation.aggregate(Y, P, measure, how='micro', *, convention='exclude', mask=None, return_parts=False)

Apply measure to (Y, P) under the operator how.

Parameters:

  • how (Literal['micro', 'macro', 'weighted', 'exemplar']) – 'micro' pools all entries before applying the measure; 'macro' applies it per class and averages; 'weighted' applies it per class and averages with weights proportional to the class prevalence; 'exemplar' applies it per example and averages.

  • convention (str) – Treatment of undefined per-class or per-example entries. 'exclude' averages over defined entries only. 'zero' reproduces the behaviour of sklearn’s zero_division=0.

  • return_parts (bool) – If True, also return the vector of per-class or per-example values, which is what the diagnostics use to check whether a pooled value lies inside their convex hull.

  • measure (str | BinaryMeasure)

aggmetrics.aggregation.fold_aggregate(im_list, measure, how='macro', pooling='pooled', *, convention='exclude')

Aggregate a performance measure across K cross-validation folds.

Parameters:

  • im_list (Sequence) – Sequence of IndicatorMatrix instances, one per fold.

  • measure (str | BinaryMeasure) – The performance measure to compute.

  • how (Literal['micro', 'macro', 'weighted', 'exemplar']) – Intra-fold aggregation operator: 'micro', 'macro', or 'exemplar'.

  • pooling (Literal['pooled', 'averaged']) – 'pooled' concatenates the folds’ predictions before computing the non-linear ratio, so folds may differ in size; 'averaged' computes the measure separately in each fold and returns the sample-weighted mean.

  • convention (str) – Convention for undefined entries ('exclude', 'zero', etc.).

Return type:

float