tpcp.validate
.Scorer#
- class tpcp.validate.Scorer(score_func: ~typing.Callable[[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT], ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any], ~typing.Dict[str, ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any]]], ~typing.Dict[str, ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any], ~typing.Dict[str, ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any]]]]]]], *, default_aggregator: ~typing.Type[~tpcp.validate._scorer.Aggregator[~tpcp.validate._scorer.T]] = <class 'tpcp.validate._scorer.MeanAggregator'>, single_score_callback: ~typing.Optional[~tpcp.validate._scorer.ScoreCallback[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT, ~tpcp.validate._scorer.T]] = None, **kwargs: ~typing.Any)[source]#
A scorer to score multiple data points of a dataset and average the results.
- Parameters:
- score_func
The callable that is used to score each data point
- single_score_callback
Callback function that is called after each datapoint that is scored. It should have the following call signature:
>>> def callback( ... *, ... step: int, ... scores: Tuple[_SCORE_TYPE, ...], ... scorer: "Scorer", ... pipeline: Pipeline, ... dataset: Dataset, ... **_ ... ) -> None: ... ...
All parameters will be passed as keyword arguments. This means, if your callback only needs a subset of the defined parameters, you can ignore them by using unused kwargs:
>>> def callback(*, step: int, pipeline: Pipeline, **_): ... ...
- kwargs
Additional arguments that might be used by the scorer. These are ignored for the base scorer.
Methods
__call__
(pipeline, dataset)Score the pipeline with the provided data.
- __init__(score_func: ~typing.Callable[[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT], ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any], ~typing.Dict[str, ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any]]], ~typing.Dict[str, ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any], ~typing.Dict[str, ~typing.Union[~tpcp.validate._scorer.T, Aggregator[Any]]]]]]], *, default_aggregator: ~typing.Type[~tpcp.validate._scorer.Aggregator[~tpcp.validate._scorer.T]] = <class 'tpcp.validate._scorer.MeanAggregator'>, single_score_callback: ~typing.Optional[~tpcp.validate._scorer.ScoreCallback[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT, ~tpcp.validate._scorer.T]] = None, **kwargs: ~typing.Any)[source]#
Examples using tpcp.validate.Scorer
#
Custom Optuna Optimizer
Custom Scorer