tpcp.validate: Validation helper#

Module for all helper methods to evaluate algorithms.

Classes#

DatasetSplitter([base_splitter, groupby, ...])

Wrapper around sklearn cross-validation splitters to support grouping and stratification with tpcp-Datasets.

Scoring#

Scorer(score_func, DatasetT, ScoreType], *, ...)

A scorer to score multiple data points of a dataset and average the results.

Aggregator(*[, return_raw_scores])

Base class for aggregators.

FloatAggregator(func, *[, return_raw_scores])

MacroFloatAggregator(*, groupby, group_agg, ...)

Aggregate first on the provided groupby level and then aggregate these results.

mean_agg(value)

Calculate the mean of the values.

no_agg(value)

Wrapper to wrap one or multiple output values of a scorer to prevent aggregation of these values.

Functions#

cross_validate(optimizable, dataset, *, scoring)

Evaluate a pipeline on a dataset using cross validation.

validate(pipeline, dataset, *, scoring[, ...])

Evaluate a pipeline on a dataset without any optimization.