tpcp.validate
.Scorer#
- class tpcp.validate.Scorer(score_func: ~typing.Callable[[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT], ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any] | ~typing.Dict[str, ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any]] | ~typing.Dict[str, ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any] | ~typing.Dict[str, ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any]]]], *, default_aggregator: ~typing.Type[~tpcp.validate._scorer.Aggregator[~tpcp.validate._scorer.T]] = <class 'tpcp.validate._scorer.MeanAggregator'>, single_score_callback: ~tpcp.validate._scorer.ScoreCallback[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT, ~tpcp.validate._scorer.T] | None = None, n_jobs: int | None = None, verbose: int = 0, pre_dispatch: str | int = '2*n_jobs', progress_bar: bool = True)[source]#
A scorer to score multiple data points of a dataset and average the results.
- Parameters:
- score_func
The callable that is used to score each data point
- single_score_callback
Callback function that is called after each datapoint that is scored. It should have the following call signature:
>>> def callback( ... *, ... step: int, ... scores: Tuple[_SCORE_TYPE, ...], ... scorer: "Scorer", ... pipeline: Pipeline, ... dataset: Dataset, ... **_ ... ) -> None: ... ...
All parameters will be passed as keyword arguments. This means, if your callback only needs a subset of the defined parameters, you can ignore them by using unused kwargs:
>>> def callback(*, step: int, pipeline: Pipeline, **_): ... ...
- n_jobs
The number of parallel jobs to run. Each job will run on a single data point. Note, that the single_score_callback will still be called in the main thread, after a job is finished. However, it could be that multiple jobs are finished before the callback is called. The callback is still gurateed to be called in the order of the data points. If None, no parallelization is used.
- verbose
Controls the verbosity of the parallelization. See
joblib.Parallel
for more details.- pre_dispatch
Controls the number of jobs that get dispatched during parallelization. See
joblib.Parallel
for more details.- progress_bar
True/False to enable/disable a
tqdm
progress bar.
Methods
__call__
(pipeline, dataset)Score the pipeline with the provided data.
clone
()Create a new instance of the class with all parameters copied over.
get_params
([deep])Get parameters for this algorithm.
set_params
(**params)Set the parameters of this Algorithm.
- __init__(score_func: ~typing.Callable[[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT], ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any] | ~typing.Dict[str, ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any]] | ~typing.Dict[str, ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any] | ~typing.Dict[str, ~tpcp.validate._scorer.T | ~tpcp.validate._scorer.Aggregator[~typing.Any]]]], *, default_aggregator: ~typing.Type[~tpcp.validate._scorer.Aggregator[~tpcp.validate._scorer.T]] = <class 'tpcp.validate._scorer.MeanAggregator'>, single_score_callback: ~tpcp.validate._scorer.ScoreCallback[~tpcp._pipeline.PipelineT, ~tpcp._dataset.DatasetT, ~tpcp.validate._scorer.T] | None = None, n_jobs: int | None = None, verbose: int = 0, pre_dispatch: str | int = '2*n_jobs', progress_bar: bool = True) None [source]#
- clone() Self [source]#
Create a new instance of the class with all parameters copied over.
This will create a new instance of the class itself and all nested objects
- get_params(deep: bool = True) Dict[str, Any] [source]#
Get parameters for this algorithm.
- Parameters:
- deep
Only relevant if object contains nested algorithm objects. If this is the case and deep is True, the params of these nested objects are included in the output using a prefix like
nested_object_name__
(Note the two “_” at the end)
- Returns:
- params
Parameter names mapped to their values.