tpcp.validate: Validation helper

Module for all helper methods to evaluate algorithms.

Classes

DatasetSplitter([base_splitter, groupby, ...])

Wrapper around sklearn cross-validation splitters to support grouping and stratification with tpcp-Datasets.

Scoring

Scorer(score_func, ~tpcp._dataset.DatasetT], ...)	A scorer to score multiple data points of a dataset and average the results.
Aggregator(*[, return_raw_scores])	Base class for aggregators.
FloatAggregator(func, *[, return_raw_scores])
MacroFloatAggregator(*, groupby, group_agg, ...)	Aggregate first on the provided groupby level and then aggregate these results.

mean_agg(value)	Calculate the mean of the values.
no_agg(value)	Wrapper to wrap one or multiple output values of a scorer to prevent aggregation of these values.

Functions

cross_validate(optimizable, dataset, *[, ...])	Evaluate a pipeline on a dataset using cross validation.
validate(pipeline, dataset, *[, scoring, ...])	Evaluate a pipeline on a dataset without any optimization.