Optimization Info#

Tpcp is focused on “running” pipelines and less on the “optimization” step. This is great for traditional algorithms and algorithms will complex return values, as you can easily store multiple parameters as attributes on the object during run.

However, for optimization, you are limited to modifying input parameters. This works well in many cases, but sometimes, you need additional information from the optimization. For example, you might want to extract the loss decay of an iterative learning algorithms. This information is something that you wouldn’t want to store in the input parameters (usually).

For these cases tpcp provides the self_optimize_with_info method. This is basically identical to self_optimize, but is expected to provide two return values: the optimized instance AND an arbitrary additional object containing any information you like. Methods that get optimizable pipelines as input (e.g. Optimize are aware of these method and will call self_optimize_with_info if available and store the additional info as result objects.

The OptimizablePipeline base-class is implemented in a way that you only need to worry about implementing either the self_optimize_with_info or the self_optimize method. The other will be available automatically (the additional info will be NOTHING, if the method is not implemented).

If you are implementing a new Algorithm (instead of a pipeline), we don’t provide this additional support, but it is relatively simple to implement yourself.

In the following we will show how all of this works by expanding the QRS detection algorithm implemented in the other examples to return additional information from the optimization.

from typing import Dict, List, Tuple

import numpy as np
import pandas as pd
from sklearn.metrics import roc_curve
from typing_extensions import Self

from examples.algorithms.algorithms_qrs_detection_final import (
    OptimizableQrsDetector,
    QRSDetector,
    match_events_with_reference,
)
from examples.datasets.datasets_final_ecg import ECGExampleData
from tpcp import HyperParameter, OptimizableParameter, OptimizablePipeline, Parameter, cf, make_optimize_safe
from tpcp.optimize import Optimize

In the algorithm class below, we basically reimplemented the OptimizableQrsDetector from the algorithm example. However, instead of the self_optimize method, we implemented the self_optimize_with_info method and added additional information from the threshold selection process to the output of the optimization.

To ensure interface compatibility with other algorithms, we also provided a self_optimize method, that simply calls self_optimize_with_info under the hood.

class OptimizableQrsDetectorWithInfo(QRSDetector):
    min_r_peak_height_over_baseline: OptimizableParameter[float]
    r_peak_match_tolerance_s: HyperParameter[float]

    def __init__(
        self,
        max_heart_rate_bpm: float = 200.0,
        min_r_peak_height_over_baseline: float = 1.0,
        r_peak_match_tolerance_s: float = 0.01,
        high_pass_filter_cutoff_hz: float = 1,
    ):
        self.r_peak_match_tolerance_s = r_peak_match_tolerance_s
        super().__init__(
            max_heart_rate_bpm=max_heart_rate_bpm,
            min_r_peak_height_over_baseline=min_r_peak_height_over_baseline,
            high_pass_filter_cutoff_hz=high_pass_filter_cutoff_hz,
        )

    @make_optimize_safe
    def self_optimize_with_info(
        self, ecg_data: List[pd.Series], r_peaks: List[pd.Series], sampling_rate_hz: float
    ) -> Tuple[Self, Dict[str, np.ndarray]]:
        all_labels = []
        all_peak_heights = []
        for d, p in zip(ecg_data, r_peaks):
            filtered = self._filter(d.to_numpy().flatten(), sampling_rate_hz)
            # Find all potential peaks without the height threshold
            potential_peaks = self._search_strategy(filtered, sampling_rate_hz, use_height=False)
            # Determine the label for each peak, by matching them with our ground truth
            labels = np.zeros(potential_peaks.shape)
            matches = match_events_with_reference(
                events=np.atleast_2d(potential_peaks).T,
                reference=np.atleast_2d(p.to_numpy().astype(int)).T,
                tolerance=self.r_peak_match_tolerance_s * sampling_rate_hz,
            )
            tp_matches = matches[(~np.isnan(matches)).all(axis=1), 0].astype(int)
            labels[tp_matches] = 1
            labels = labels.astype(bool)
            all_labels.append(labels)
            all_peak_heights.append(filtered[potential_peaks])
        all_labels = np.hstack(all_labels)
        all_peak_heights = np.hstack(all_peak_heights)
        # We "brute-force" a good cutoff by testing a bunch of thresholds and then calculating the Youden Index for
        # each.
        fpr, tpr, thresholds = roc_curve(all_labels, all_peak_heights)
        youden_index = tpr - fpr
        # The best Youden index gives us a balance between sensitivity and specificity.
        self.min_r_peak_height_over_baseline = thresholds[np.argmax(youden_index)]

        # Here we create the additional infor object:
        additional_info = {"all_youden_index": youden_index, "all_thresholds": thresholds}
        return self, additional_info

    def self_optimize(self, ecg_data: List[pd.Series], r_peaks: List[pd.Series], sampling_rate_hz: float) -> Self:
        return self.self_optimize_with_info(ecg_data=ecg_data, r_peaks=r_peaks, sampling_rate_hz=sampling_rate_hz)[0]

To use this algorithm in an optimization, we need a pipeline to wrap it. Below we can find a reimplementation of the pipline from the “Optimizable Pipeline” example.

However, instead of implementing self_optimize method, we implemented the self_optimize_with_info method and also called the self_optimize_with_info of our algorithm under the hood.

Note, that for pipelines, we don’t need to implement a dummy self_optimize method. Our baseclass already takes care of that.

class MyPipeline(OptimizablePipeline[ECGExampleData]):
    algorithm: Parameter[OptimizableQrsDetectorWithInfo]
    algorithm__min_r_peak_height_over_baseline: OptimizableParameter[float]

    r_peak_positions_: pd.Series

    def __init__(self, algorithm: OptimizableQrsDetectorWithInfo = cf(OptimizableQrsDetectorWithInfo())):
        self.algorithm = algorithm

    @make_optimize_safe
    def self_optimize_with_info(self, dataset: ECGExampleData, **kwargs):
        ecg_data = [d.data["ecg"] for d in dataset]
        r_peaks = [d.r_peak_positions_["r_peak_position"] for d in dataset]
        # Note: We need to clone the algorithm instance, to make sure we don't leak any data between runs.
        algo = self.algorithm.clone()
        # Here we call the `self_optimize_with_info` method!
        self.algorithm, additional_data = algo.self_optimize_with_info(ecg_data, r_peaks, dataset.sampling_rate_hz)
        return self, additional_data

    def run(self, datapoint: ECGExampleData):
        # Note: We need to clone the algorithm instance, to make sure we don't leak any data between runs.
        algo = self.algorithm.clone()
        algo.detect(datapoint.data["ecg"], datapoint.sampling_rate_hz)

        self.r_peak_positions_ = algo.r_peak_positions_
        return self

Let’s test this class!

However, first we need some test data

from pathlib import Path

from sklearn.model_selection import train_test_split

try:
    HERE = Path(__file__).parent
except NameError:
    HERE = Path(".").resolve()
data_path = HERE.parent.parent / "example_data/ecg_mit_bih_arrhythmia/data"
example_data = ECGExampleData(data_path)

train_set, test_set = train_test_split(example_data, train_size=0.7, random_state=0)
# We only want a single dataset in the test set
test_set = test_set[0]

With the train data we can try out the optimization

optimized_pipe, info = MyPipeline().self_optimize_with_info(train_set)
info
{'all_youden_index': array([0.00000000e+00, 6.19540301e-05, 1.52512466e-05, 8.86714683e-02,
       8.86247656e-02, 1.01263388e-01, 1.01216685e-01, 1.07040364e-01,
       1.06993661e-01, 1.07489293e-01, 1.07442590e-01, 1.07752361e-01,
       1.07705658e-01, 1.23751752e-01, 1.23705049e-01, 1.24076773e-01,
       1.24030070e-01, 1.29915703e-01, 1.29869000e-01, 1.30302678e-01,
       1.30255976e-01, 1.31185286e-01, 1.31138583e-01, 1.36714446e-01,
       1.36667743e-01, 1.36915559e-01, 1.36868857e-01, 1.39161156e-01,
       1.39114453e-01, 1.40973074e-01, 1.40926371e-01, 1.42227406e-01,
       1.42180703e-01, 1.43729554e-01, 1.43682851e-01, 1.57932278e-01,
       1.57885575e-01, 2.48152597e-01, 2.48105894e-01, 2.48477618e-01,
       2.48430916e-01, 2.49917812e-01, 2.49871109e-01, 2.50180880e-01,
       2.50134177e-01, 2.51744982e-01, 2.51698279e-01, 2.52441727e-01,
       2.52395024e-01, 2.53324335e-01, 2.53230929e-01, 2.53850470e-01,
       2.53803767e-01, 2.55290664e-01, 2.55243961e-01, 2.57969938e-01,
       2.57923235e-01, 2.59410132e-01, 2.59363429e-01, 2.59797107e-01,
       2.59750405e-01, 2.60493853e-01, 2.60447150e-01, 2.60633012e-01,
       2.60586310e-01, 2.61949298e-01, 2.61902595e-01, 2.63017768e-01,
       2.62971065e-01, 2.63838422e-01, 2.63791719e-01, 2.65278616e-01,
       2.65185210e-01, 2.68035095e-01, 2.67988393e-01, 2.70652416e-01,
       2.70605713e-01, 2.71473070e-01, 2.71426367e-01, 2.72665447e-01,
       2.72618745e-01, 2.72742653e-01, 2.72695950e-01, 2.75917559e-01,
       2.75870857e-01, 2.76924075e-01, 2.76877372e-01, 2.77806683e-01,
       2.77759980e-01, 2.77883888e-01, 2.77837185e-01, 2.78456726e-01,
       2.78410023e-01, 2.79339333e-01, 2.79292630e-01, 2.79478493e-01,
       2.79431790e-01, 2.88353170e-01, 2.88306467e-01, 2.92023709e-01,
       2.91977006e-01, 2.95880110e-01, 2.95833407e-01, 3.01223408e-01,
       3.01176705e-01, 3.02415786e-01, 3.02369083e-01, 3.02740807e-01,
       3.02694105e-01, 3.13164336e-01, 3.13117633e-01, 3.42297981e-01,
       3.42251278e-01, 3.46588060e-01, 3.46541358e-01, 3.47470668e-01,
       3.47423965e-01, 3.51265115e-01, 3.51218412e-01, 3.59644160e-01,
       3.59597458e-01, 3.66350447e-01, 3.66303744e-01, 3.85571447e-01,
       3.85524745e-01, 3.94941757e-01, 3.94895054e-01, 4.04374021e-01,
       4.04327318e-01, 4.12691112e-01, 4.12644410e-01, 4.30982802e-01,
       4.30936100e-01, 4.36264146e-01, 4.36217444e-01, 4.46006180e-01,
       4.45959477e-01, 4.49243041e-01, 4.49196338e-01, 4.52851626e-01,
       4.52804923e-01, 4.54167912e-01, 4.54121209e-01, 4.55793968e-01,
       4.55747265e-01, 4.55995081e-01, 4.55948379e-01, 4.63073092e-01,
       4.63026389e-01, 4.63212251e-01, 4.63165549e-01, 4.63599227e-01,
       4.63552524e-01, 4.64667696e-01, 4.64620994e-01, 4.64744902e-01,
       4.64698199e-01, 4.74177166e-01, 4.74130463e-01, 4.74378279e-01,
       4.74331576e-01, 4.78172726e-01, 4.78126023e-01, 4.78187977e-01,
       4.78141274e-01, 4.86071390e-01, 4.86024688e-01, 5.16320208e-01,
       5.16273505e-01, 5.22840633e-01, 5.22793930e-01, 5.23165654e-01,
       5.23118951e-01, 5.24729756e-01, 5.24683053e-01, 5.27966617e-01,
       5.27919914e-01, 5.28911179e-01, 5.28864476e-01, 5.29917694e-01,
       5.29870992e-01, 5.31357888e-01, 5.31311185e-01, 5.31744864e-01,
       5.31698161e-01, 5.41796668e-01, 5.41749965e-01, 5.47511690e-01,
       5.47464987e-01, 5.47526941e-01, 5.47480238e-01, 5.52622423e-01,
       5.52575720e-01, 5.56231008e-01, 5.56184305e-01, 5.59343961e-01,
       5.59297258e-01, 5.62394959e-01, 5.62348256e-01, 5.64206877e-01,
       5.64160175e-01, 5.71470750e-01, 5.71424047e-01, 5.73344622e-01,
       5.73297920e-01, 5.89158151e-01, 5.89111448e-01, 5.89235356e-01,
       5.89188654e-01, 5.89808194e-01, 5.89761491e-01, 5.97257929e-01,
       5.97211226e-01, 5.98760077e-01, 5.98713374e-01, 6.00571995e-01,
       6.00478589e-01, 6.05001234e-01, 6.04954531e-01, 6.14805222e-01,
       6.14758519e-01, 6.15440013e-01, 6.15393310e-01, 6.16384575e-01,
       6.16337872e-01, 6.19373620e-01, 6.19326917e-01, 6.20689905e-01,
       6.20643203e-01, 6.22563778e-01, 6.22517075e-01, 6.23508339e-01,
       6.23461636e-01, 6.26063706e-01, 6.26017003e-01, 6.28866888e-01,
       6.28820186e-01, 6.41210992e-01, 6.41164289e-01, 6.44200036e-01,
       6.44153333e-01, 6.45454368e-01, 6.45407665e-01, 6.46027206e-01,
       6.45980503e-01, 6.47777170e-01, 6.47730467e-01, 6.51943341e-01,
       6.51896638e-01, 6.52392270e-01, 6.52345568e-01, 6.52841200e-01,
       6.52794497e-01, 6.59733348e-01, 6.59686646e-01, 6.64147336e-01,
       6.64100633e-01, 6.65277760e-01, 6.65231057e-01, 6.66779908e-01,
       6.66733205e-01, 6.70326539e-01, 6.70279836e-01, 6.72757997e-01,
       6.72711294e-01, 6.77233938e-01, 6.77187236e-01, 6.80532753e-01,
       6.80486050e-01, 6.82096855e-01, 6.82050152e-01, 6.82236015e-01,
       6.82189312e-01, 6.82313220e-01, 6.82266517e-01, 6.83195828e-01,
       6.83149125e-01, 6.90335792e-01, 6.90289089e-01, 7.03547252e-01,
       7.03500549e-01, 7.05173308e-01, 7.05126605e-01, 7.07480858e-01,
       7.07434155e-01, 7.11151397e-01, 7.11104695e-01, 7.11662281e-01,
       7.11615578e-01, 7.11677532e-01, 7.11630829e-01, 7.12126461e-01,
       7.12079759e-01, 7.15673092e-01, 7.15579687e-01, 7.18057848e-01,
       7.18011145e-01, 7.21418617e-01, 7.21371914e-01, 7.22734903e-01,
       7.22688200e-01, 7.24422913e-01, 7.24376210e-01, 7.26234831e-01,
       7.26188128e-01, 7.26250082e-01, 7.26203379e-01, 7.27132690e-01,
       7.27085987e-01, 7.27395757e-01, 7.27349055e-01, 7.28464227e-01,
       7.28417524e-01, 7.33745571e-01, 7.33698868e-01, 7.39336685e-01,
       7.39289982e-01, 7.41644235e-01, 7.41597532e-01, 7.42464889e-01,
       7.42418186e-01, 7.45330025e-01, 7.45283323e-01, 7.46646311e-01,
       7.46599609e-01, 7.47095241e-01, 7.47048538e-01, 7.49154975e-01,
       7.49108272e-01, 7.49294134e-01, 7.49247432e-01, 7.49371340e-01,
       7.49324637e-01, 7.50068085e-01, 7.50021382e-01, 7.53986440e-01,
       7.53939738e-01, 7.55054910e-01, 7.55008207e-01, 7.55194069e-01,
       7.55147367e-01, 7.56200585e-01, 7.56153882e-01, 7.57083193e-01,
       7.57036490e-01, 7.57408214e-01, 7.57361511e-01, 7.57485419e-01,
       7.57438717e-01, 7.59854924e-01, 7.59808221e-01, 7.61790750e-01,
       7.61744047e-01, 7.61867955e-01, 7.61821253e-01, 7.61883207e-01,
       7.61836504e-01, 7.62394090e-01, 7.62347387e-01, 7.63090836e-01,
       7.63044133e-01, 7.63539765e-01, 7.63493062e-01, 7.67829844e-01,
       7.67783142e-01, 7.69765671e-01, 7.69718968e-01, 7.70772186e-01,
       7.70725484e-01, 7.70973300e-01, 7.70926597e-01, 7.74457977e-01,
       7.74364571e-01, 7.76347100e-01, 7.76300397e-01, 7.76424305e-01,
       7.76377602e-01, 7.76935189e-01, 7.76888486e-01, 7.79862279e-01,
       7.79815577e-01, 7.82169830e-01, 7.82123127e-01, 7.83857840e-01,
       7.83811137e-01, 7.87032747e-01, 7.86986044e-01, 7.88410987e-01,
       7.88364284e-01, 7.89541410e-01, 7.89494708e-01, 7.89556662e-01,
       7.89509959e-01, 7.89633867e-01, 7.89587164e-01, 7.89834980e-01,
       7.89788277e-01, 7.90160002e-01, 7.90113299e-01, 7.90485023e-01,
       7.90438320e-01, 7.91801309e-01, 7.91754606e-01, 7.92436100e-01,
       7.92389398e-01, 7.93008938e-01, 7.92962235e-01, 7.93581775e-01,
       7.93535073e-01, 7.93906797e-01, 7.93860094e-01, 7.95285037e-01,
       7.95238334e-01, 7.96725231e-01, 7.96678528e-01, 8.00333816e-01,
       8.00287113e-01, 8.00349067e-01, 8.00302364e-01, 8.03090295e-01,
       8.03043593e-01, 8.03291409e-01, 8.03198003e-01, 8.03259957e-01,
       8.03213255e-01, 8.04204519e-01, 8.04157816e-01, 8.05087127e-01,
       8.05040424e-01, 8.05598010e-01, 8.05551307e-01, 8.06542572e-01,
       8.06495869e-01, 8.07177363e-01, 8.07130661e-01, 8.07564339e-01,
       8.07517636e-01, 8.10119705e-01, 8.10073003e-01, 8.11435991e-01,
       8.11389288e-01, 8.11451242e-01, 8.11404540e-01, 8.11652356e-01,
       8.11558950e-01, 8.11930674e-01, 8.11883972e-01, 8.11945926e-01,
       8.11899223e-01, 8.13571982e-01, 8.13525279e-01, 8.13587233e-01,
       8.13540530e-01, 8.14098116e-01, 8.14051414e-01, 8.14175322e-01,
       8.14128619e-01, 8.14190573e-01, 8.14143870e-01, 8.15073181e-01,
       8.14979775e-01, 8.15165637e-01, 8.15118934e-01, 8.15180888e-01,
       8.15134186e-01, 8.16806944e-01, 8.16760242e-01, 8.17193920e-01,
       8.17147217e-01, 8.18758022e-01, 8.18711319e-01, 8.22490515e-01,
       8.22443812e-01, 8.23620939e-01, 8.23574236e-01, 8.23760098e-01,
       8.23713395e-01, 8.23837303e-01, 8.23790600e-01, 8.24038417e-01,
       8.23991714e-01, 8.27770910e-01, 8.27724207e-01, 8.27786161e-01,
       8.27739458e-01, 8.28792677e-01, 8.28745974e-01, 8.29737238e-01,
       8.29690536e-01, 8.30433984e-01, 8.30387281e-01, 8.31936132e-01,
       8.31889429e-01, 8.32880694e-01, 8.32833991e-01, 8.41321693e-01,
       8.41274990e-01, 8.42204301e-01, 8.42157598e-01, 8.42405414e-01,
       8.42358711e-01, 8.45580321e-01, 8.45533618e-01, 8.48631319e-01,
       8.48584617e-01, 8.49575881e-01, 8.49529178e-01, 8.51573661e-01,
       8.51526959e-01, 8.54067074e-01, 8.54020371e-01, 8.57551751e-01,
       8.57505048e-01, 8.60912520e-01, 8.60865817e-01, 8.61795127e-01,
       8.61748424e-01, 8.62182103e-01, 8.62135400e-01, 8.65604826e-01,
       8.65558123e-01, 8.67230882e-01, 8.67184179e-01, 8.72388317e-01,
       8.72341615e-01, 8.75872994e-01, 8.75826291e-01, 8.83012959e-01,
       8.82966256e-01, 8.89471429e-01, 8.89424727e-01, 8.90168175e-01,
       8.90121472e-01, 8.90431242e-01, 8.90384540e-01, 8.96146264e-01,
       8.96099562e-01, 8.96595194e-01, 8.96548491e-01, 9.04106883e-01,
       9.04060180e-01, 9.07963284e-01, 9.07916581e-01, 9.12005547e-01,
       9.11958844e-01, 9.13569649e-01, 9.13522946e-01, 9.18789039e-01,
       9.18742336e-01, 9.21716129e-01, 9.21669427e-01, 9.21979197e-01,
       9.21932494e-01, 9.25525828e-01, 9.25479125e-01, 9.26780160e-01,
       9.26733457e-01, 9.29645296e-01, 9.29598593e-01, 9.33997330e-01,
       9.33950627e-01, 9.35437524e-01, 9.35390821e-01, 9.35948407e-01,
       9.35901704e-01, 9.36149520e-01, 9.36102818e-01, 9.37094082e-01,
       9.37047379e-01, 9.37171287e-01, 9.37077882e-01, 9.39679951e-01,
       9.39633248e-01, 9.40934283e-01, 9.40887580e-01, 9.41321258e-01,
       9.41274556e-01, 9.41584326e-01, 9.41537623e-01, 9.41723485e-01,
       9.41676782e-01, 9.45332070e-01, 9.45285367e-01, 9.47515712e-01,
       9.47469009e-01, 9.48770044e-01, 9.48723341e-01, 9.48785295e-01,
       9.48738593e-01, 9.49543995e-01, 9.49497292e-01, 9.50116832e-01,
       9.50070130e-01, 9.50503808e-01, 9.50457105e-01, 9.50828829e-01,
       9.50782127e-01, 9.51153851e-01, 9.51107148e-01, 9.51231056e-01,
       9.51184353e-01, 9.55706997e-01, 9.55660295e-01, 9.56837421e-01,
       9.56790718e-01, 9.57472213e-01, 9.57425510e-01, 9.58107004e-01,
       9.58060301e-01, 9.58617888e-01, 9.58571185e-01, 9.58633139e-01,
       9.58586436e-01, 9.59205977e-01, 9.59159274e-01, 9.59469044e-01,
       9.59422341e-01, 9.59732111e-01, 9.59685408e-01, 9.59809317e-01,
       9.59762614e-01, 9.60382154e-01, 9.60335451e-01, 9.60521313e-01,
       9.60474611e-01, 9.60598519e-01, 9.60551816e-01, 9.60613770e-01,
       9.60567067e-01, 9.60629021e-01, 9.60582318e-01, 9.61449675e-01,
       9.61402972e-01, 9.61712742e-01, 9.61666039e-01, 9.63462706e-01,
       9.63369301e-01, 9.63431255e-01, 9.63384552e-01, 9.63446506e-01,
       9.63353100e-01, 9.63477008e-01, 9.63430306e-01, 9.63863984e-01,
       9.63817281e-01, 9.64127051e-01, 9.64080348e-01, 9.64328165e-01,
       9.64281462e-01, 9.64591232e-01, 9.64544529e-01, 9.64916253e-01,
       9.64869551e-01, 9.64993459e-01, 9.64900053e-01, 9.65085915e-01,
       9.65039212e-01, 9.65410937e-01, 9.65364234e-01, 9.65859866e-01,
       9.65813163e-01, 9.65999025e-01, 9.65952323e-01, 9.66262093e-01,
       9.66215390e-01, 9.66587114e-01, 9.66540411e-01, 9.66602365e-01,
       9.66555663e-01, 9.66803479e-01, 9.66756776e-01, 9.67004592e-01,
       9.66957889e-01, 9.67143751e-01, 9.67097048e-01, 9.67159003e-01,
       9.67112300e-01, 9.67422070e-01, 9.67328664e-01, 9.67452572e-01,
       9.67359167e-01, 9.67421121e-01, 9.67374418e-01, 9.68241774e-01,
       9.68195072e-01, 9.68318980e-01, 9.68272277e-01, 9.68767909e-01,
       9.68721206e-01, 9.68907069e-01, 9.68813663e-01, 9.68875617e-01,
       9.68828914e-01, 9.68890868e-01, 9.68844165e-01, 9.69091982e-01,
       9.69045279e-01, 9.69355049e-01, 9.69261643e-01, 9.69323597e-01,
       9.69276895e-01, 9.69772527e-01, 9.69725824e-01, 9.69787778e-01,
       9.69741075e-01, 9.69988891e-01, 9.69942189e-01, 9.70251959e-01,
       9.70205256e-01, 9.70267210e-01, 9.70220507e-01, 9.70530277e-01,
       9.70483575e-01, 9.70545529e-01, 9.70498826e-01, 9.70622734e-01,
       9.70576031e-01, 9.70699939e-01, 9.70653236e-01, 9.70839099e-01,
       9.70792396e-01, 9.70854350e-01, 9.70807647e-01, 9.70993509e-01,
       9.70946806e-01, 9.71008760e-01, 9.70868652e-01, 9.70930606e-01,
       9.70883903e-01, 9.71007811e-01, 9.70961109e-01, 9.71085017e-01,
       9.71038314e-01, 9.71224176e-01, 9.71177473e-01, 9.71301381e-01,
       9.71254678e-01, 9.71626403e-01, 9.71579700e-01, 9.71703608e-01,
       9.71656905e-01, 9.71780813e-01, 9.71734110e-01, 9.71858018e-01,
       9.71624504e-01, 9.71686458e-01, 9.71639756e-01, 9.71949526e-01,
       9.71809418e-01, 9.71871372e-01, 9.71824669e-01, 9.71948577e-01,
       9.71901874e-01, 9.71963828e-01, 9.71917125e-01, 9.71979079e-01,
       9.71932377e-01, 9.71994331e-01, 9.71900925e-01, 9.72954143e-01,
       9.72907441e-01, 9.73031349e-01, 9.72984646e-01, 9.73046600e-01,
       9.72906492e-01, 9.73030400e-01, 9.72936994e-01, 9.73184810e-01,
       9.73138108e-01, 9.73447878e-01, 9.73401175e-01, 9.73648991e-01,
       9.73462180e-01, 9.73771950e-01, 9.73398328e-01, 9.73460282e-01,
       9.73413579e-01, 9.73537487e-01, 9.73444081e-01, 9.73691898e-01,
       9.73505086e-01, 9.73628995e-01, 9.73582292e-01, 9.73706200e-01,
       9.73659497e-01, 9.73845359e-01, 9.73705251e-01, 9.73829159e-01,
       9.73689050e-01, 9.73812959e-01, 9.73766256e-01, 9.73828210e-01,
       9.73781507e-01, 9.73905415e-01, 9.73858712e-01, 9.73982620e-01,
       9.73935918e-01, 9.73997872e-01, 9.73951169e-01, 9.74013123e-01,
       9.73919717e-01, 9.73981671e-01, 9.73794860e-01, 9.73918768e-01,
       9.73872065e-01, 9.73995973e-01, 9.73902568e-01, 9.73964522e-01,
       9.73917819e-01, 9.74041727e-01, 9.73948322e-01, 9.74010276e-01,
       9.73823465e-01, 9.73885419e-01, 9.73792013e-01, 9.73853967e-01,
       9.73807264e-01, 9.73931172e-01, 9.73791064e-01, 9.73853018e-01,
       9.73806315e-01, 9.73868269e-01, 9.73447944e-01, 9.73633806e-01,
       9.73353590e-01, 9.73415544e-01, 9.73368841e-01, 9.73492749e-01,
       9.73352641e-01, 9.73414595e-01, 9.73367892e-01, 9.73553754e-01,
       9.73507051e-01, 9.73630959e-01, 9.73397445e-01, 9.73645261e-01,
       9.73598559e-01, 9.73660513e-01, 9.73613810e-01, 9.73675764e-01,
       9.73348844e-01, 9.73410798e-01, 9.73270690e-01, 9.73332644e-01,
       9.73099130e-01, 9.73223038e-01, 9.73176335e-01, 9.73238289e-01,
       9.73098181e-01, 9.73160135e-01, 9.72973324e-01, 9.73035278e-01,
       9.72988575e-01, 9.73050529e-01, 9.72770313e-01, 9.72832267e-01,
       9.72738861e-01, 9.72800815e-01, 9.72707410e-01, 9.72769364e-01,
       9.72489147e-01, 9.72551101e-01, 9.72270884e-01, 9.72332838e-01,
       9.72146027e-01, 9.72207981e-01, 9.71180520e-01, 9.71304428e-01,
       9.71117617e-01, 9.71179571e-01, 9.69965298e-01, 9.70027252e-01,
       9.69980550e-01, 9.70042504e-01, 9.68921637e-01, 9.69045545e-01,
       9.68998842e-01, 9.69060796e-01, 9.68967391e-01, 9.69029345e-01,
       9.68982642e-01, 9.69044596e-01, 9.68997893e-01, 9.69059847e-01,
       9.68826333e-01, 9.68888287e-01, 9.68841585e-01, 9.68903539e-01,
       9.68763430e-01, 9.68825384e-01, 9.68638573e-01, 9.68700527e-01,
       9.67626363e-01, 9.67688317e-01, 9.65960314e-01, 9.66084222e-01,
       9.64496328e-01, 9.64558282e-01, 9.58160000e-01, 9.58221954e-01,
       9.52197295e-01, 9.52259249e-01, 9.51231788e-01, 9.51293742e-01,
       9.51200336e-01, 9.51262291e-01, 9.47899690e-01, 9.47961644e-01,
       9.41423254e-01, 9.41485208e-01, 9.39523692e-01, 9.39585646e-01,
       9.38184562e-01, 9.38246516e-01, 9.33249318e-01, 9.33311272e-01,
       9.25978935e-01, 9.26040889e-01, 9.24406292e-01, 9.24468246e-01,
       9.18910615e-01, 9.18972569e-01, 9.17898405e-01, 9.17960359e-01,
       9.12916458e-01, 9.12978412e-01, 8.85703987e-01, 8.85765941e-01,
       8.01374011e-01, 8.01435965e-01, 7.82568040e-01, 7.82629994e-01,
       5.75222933e-01, 5.75284887e-01, 0.00000000e+00]), 'all_thresholds': array([ 4.82557838,  3.82557838,  3.70343786,  2.94218565,  2.942074  ,
        2.8203026 ,  2.82026647,  2.78245009,  2.782324  ,  2.77971057,
        2.7793436 ,  2.77901207,  2.77884858,  2.6966796 ,  2.69646795,
        2.69351429,  2.69293252,  2.65556322,  2.65497563,  2.65093531,
        2.65079644,  2.64342359,  2.6427749 ,  2.57653608,  2.57650732,
        2.57292632,  2.57236881,  2.51270735,  2.51209908,  2.45747118,
        2.45537237,  2.38290977,  2.38233765,  2.26898769,  2.26876282,
        2.11383814,  2.11369168,  1.85102953,  1.84930445,  1.84516188,
        1.84442315,  1.8191622 ,  1.8183331 ,  1.81245478,  1.81042573,
        1.77732863,  1.77693616,  1.76610969,  1.7641405 ,  1.74665293,
        1.74600351,  1.73458204,  1.73448568,  1.71982794,  1.7186537 ,
        1.68146866,  1.67828692,  1.6674466 ,  1.66723499,  1.66381917,
        1.66265832,  1.65184295,  1.65137835,  1.6504861 ,  1.64939589,
        1.63631954,  1.63608634,  1.62667492,  1.6265638 ,  1.61984373,
        1.61953213,  1.61034214,  1.60841267,  1.59141332,  1.59135861,
        1.57477517,  1.57311128,  1.56784863,  1.56781236,  1.56222299,
        1.56177292,  1.5615277 ,  1.56112938,  1.54836107,  1.54834868,
        1.54586096,  1.54535768,  1.54166149,  1.5415238 ,  1.54115035,
        1.5411388 ,  1.53939579,  1.53928026,  1.53737686,  1.53720324,
        1.53696605,  1.5369137 ,  1.51737166,  1.51733414,  1.51156244,
        1.51151847,  1.5058482 ,  1.50560667,  1.4992319 ,  1.49895466,
        1.4973381 ,  1.49728848,  1.49680808,  1.49678793,  1.48685761,
        1.4868404 ,  1.46332823,  1.46330189,  1.46039387,  1.46034866,
        1.45976768,  1.45964813,  1.45711255,  1.4570676 ,  1.45084292,
        1.45081534,  1.44635881,  1.44634333,  1.43252731,  1.43235989,
        1.4249837 ,  1.42491962,  1.41716628,  1.41703196,  1.40956213,
        1.40945521,  1.39358169,  1.39352731,  1.38868577,  1.38856911,
        1.37897231,  1.37894062,  1.37544923,  1.37530062,  1.371638  ,
        1.37159405,  1.37012799,  1.36987876,  1.36785537,  1.36780377,
        1.36762469,  1.36750989,  1.35921131,  1.35918614,  1.35892724,
        1.35879641,  1.35834045,  1.3581641 ,  1.35689246,  1.35683036,
        1.35675603,  1.35658806,  1.3449695 ,  1.34492062,  1.34454382,
        1.34446322,  1.33962769,  1.33957819,  1.33956742,  1.33950089,
        1.32910081,  1.32908027,  1.28451325,  1.2843395 ,  1.27323641,
        1.27321097,  1.27238728,  1.27224228,  1.26970784,  1.26945415,
        1.26370913,  1.26365353,  1.26196129,  1.26175542,  1.25988863,
        1.25981328,  1.25754058,  1.25751213,  1.25640799,  1.25613685,
        1.24065691,  1.24047563,  1.23175455,  1.23168287,  1.2313269 ,
        1.23110531,  1.22269257,  1.22262297,  1.21646045,  1.21642907,
        1.21159062,  1.21157299,  1.20654606,  1.20618608,  1.20328851,
        1.20327127,  1.19307087,  1.19304479,  1.19039625,  1.1903346 ,
        1.1694793 ,  1.16937474,  1.16931868,  1.16919917,  1.16852717,
        1.1685237 ,  1.16048866,  1.16048261,  1.15888975,  1.15888371,
        1.15719859,  1.15686527,  1.1520799 ,  1.15203302,  1.14303115,
        1.14298737,  1.14253623,  1.14250571,  1.14159804,  1.1415949 ,
        1.13871869,  1.13870954,  1.13653078,  1.13649647,  1.13512337,
        1.13511381,  1.13421743,  1.1341007 ,  1.13184295,  1.13181107,
        1.12932203,  1.12929242,  1.12035945,  1.12035425,  1.11784788,
        1.11776482,  1.11632325,  1.11619945,  1.11590985,  1.11582998,
        1.1144199 ,  1.11439797,  1.11134006,  1.11122845,  1.11103901,
        1.11098802,  1.11044349,  1.11043379,  1.10510776,  1.10503208,
        1.10239814,  1.10233051,  1.1017279 ,  1.10161173,  1.10064277,
        1.10063671,  1.0980769 ,  1.09801448,  1.09647131,  1.09645743,
        1.09290406,  1.09279491,  1.09051218,  1.09045126,  1.08885437,
        1.08875789,  1.08855991,  1.08855664,  1.08851413,  1.08848593,
        1.08769603,  1.08765913,  1.08263579,  1.08249446,  1.07375172,
        1.07373002,  1.07275922,  1.07270919,  1.0713031 ,  1.07124633,
        1.06855012,  1.06854607,  1.06814642,  1.06813638,  1.06813321,
        1.06801285,  1.06741648,  1.06741356,  1.06514284,  1.0650941 ,
        1.06329835,  1.06324538,  1.06106925,  1.06105992,  1.06031355,
        1.0603068 ,  1.05941026,  1.05932492,  1.05832529,  1.05831017,
        1.05824411,  1.05821427,  1.05767542,  1.05765743,  1.05748705,
        1.0574214 ,  1.05668666,  1.05666404,  1.05275835,  1.05274152,
        1.04813048,  1.04809599,  1.04618247,  1.0460922 ,  1.04546659,
        1.04538985,  1.04300093,  1.04294721,  1.04187907,  1.04182971,
        1.04154616,  1.04144491,  1.03969332,  1.03969224,  1.03956863,
        1.0394514 ,  1.03937604,  1.0393111 ,  1.03862453,  1.03856745,
        1.03456277,  1.0345567 ,  1.03393262,  1.03389013,  1.03371654,
        1.03365245,  1.03275111,  1.03270477,  1.03203226,  1.032031  ,
        1.03181089,  1.03176566,  1.03168584,  1.03165066,  1.03019743,
        1.03015088,  1.02882299,  1.02866972,  1.02855439,  1.02852727,
        1.02851534,  1.02844424,  1.02785266,  1.02783572,  1.02713685,
        1.02710104,  1.02673965,  1.02673711,  1.02336653,  1.02332079,
        1.02159723,  1.021587  ,  1.02068661,  1.0206236 ,  1.0203852 ,
        1.02030698,  1.0173309 ,  1.01720208,  1.01500154,  1.01491965,
        1.01484806,  1.01483593,  1.01399961,  1.01392336,  1.01075976,
        1.01070478,  1.00845234,  1.00842688,  1.00634991,  1.00602003,
        1.00229234,  1.00205662,  1.00020597,  1.00017986,  0.99891037,
        0.99887411,  0.99884444,  0.99873595,  0.99861254,  0.99856464,
        0.99821125,  0.99818945,  0.99794761,  0.9978216 ,  0.99727441,
        0.99725162,  0.99580181,  0.99567839,  0.99509029,  0.99504531,
        0.99458661,  0.9945848 ,  0.99432725,  0.99412351,  0.9936942 ,
        0.99368563,  0.99177834,  0.99173382,  0.9899686 ,  0.9899361 ,
        0.98464518,  0.98462125,  0.98454289,  0.98452069,  0.97951525,
        0.97938638,  0.97915821,  0.97872533,  0.97872281,  0.97860977,
        0.97704656,  0.97702392,  0.97541824,  0.97525394,  0.97408005,
        0.97382923,  0.97236646,  0.97232604,  0.97161007,  0.9713012 ,
        0.9710974 ,  0.97082728,  0.96651514,  0.96632545,  0.96345172,
        0.96344564,  0.96340662,  0.96336658,  0.96279613,  0.96254359,
        0.96175349,  0.96170284,  0.9616607 ,  0.96153871,  0.9579177 ,
        0.95778477,  0.95767818,  0.95761271,  0.95614508,  0.95613962,
        0.95584829,  0.95575495,  0.95543679,  0.95537128,  0.95370547,
        0.95328636,  0.95268223,  0.95265827,  0.95261964,  0.95236633,
        0.94847639,  0.94838579,  0.94656382,  0.94653863,  0.94204673,
        0.94202674,  0.93405545,  0.93402455,  0.93054319,  0.93046595,
        0.92988758,  0.92984985,  0.92982241,  0.92959247,  0.92918463,
        0.92915776,  0.92155603,  0.92120089,  0.92118698,  0.92115569,
        0.91851667,  0.91850813,  0.9160845 ,  0.91587852,  0.91407055,
        0.91376373,  0.91141424,  0.91138031,  0.90885957,  0.9087869 ,
        0.89192196,  0.89185769,  0.89011777,  0.89011382,  0.88981681,
        0.88964674,  0.88223642,  0.88217023,  0.87608721,  0.87585445,
        0.87458083,  0.87451373,  0.87082195,  0.87065471,  0.86633224,
        0.86617382,  0.86041355,  0.86035668,  0.85566435,  0.85553359,
        0.85400699,  0.85382926,  0.85297244,  0.85295241,  0.84783928,
        0.84775332,  0.84527466,  0.8452058 ,  0.83736825,  0.83730929,
        0.83145018,  0.83140077,  0.81958078,  0.81949127,  0.80841038,
        0.80828759,  0.80696615,  0.80695282,  0.80637223,  0.8063226 ,
        0.79746988,  0.79736956,  0.79645878,  0.79610352,  0.78487544,
        0.78483116,  0.77803143,  0.7779771 ,  0.77073243,  0.77052098,
        0.76815736,  0.76814805,  0.75952639,  0.75933744,  0.75377991,
        0.7537638 ,  0.75349873,  0.75344439,  0.74899447,  0.74893122,
        0.74752407,  0.74740191,  0.74275142,  0.74248021,  0.7359401 ,
        0.73590682,  0.73294   ,  0.73287211,  0.73188959,  0.731845  ,
        0.73109145,  0.73088291,  0.72934919,  0.72927018,  0.72913392,
        0.72904428,  0.72613004,  0.72596188,  0.72366922,  0.72347317,
        0.72253149,  0.7224706 ,  0.7221855 ,  0.72197678,  0.72176066,
        0.72173089,  0.71558356,  0.71555074,  0.71100395,  0.71095497,
        0.70901417,  0.70897256,  0.70878428,  0.70850139,  0.70738402,
        0.70721433,  0.70578037,  0.70577725,  0.70505623,  0.70505126,
        0.70442012,  0.70433963,  0.70360878,  0.70360087,  0.70348527,
        0.70344195,  0.69658778,  0.69656318,  0.69391896,  0.69385435,
        0.69181593,  0.69146007,  0.69061568,  0.69055202,  0.68924453,
        0.68882348,  0.68802755,  0.68799173,  0.68747934,  0.68746464,
        0.68639005,  0.68625035,  0.68601778,  0.6859202 ,  0.68553113,
        0.68552224,  0.68364926,  0.68356245,  0.6828158 ,  0.68272716,
        0.68241768,  0.68182262,  0.68134824,  0.68119533,  0.68111281,
        0.68089411,  0.67901509,  0.67881768,  0.67715718,  0.67676617,
        0.67200944,  0.67186711,  0.6717779 ,  0.67127497,  0.67075445,
        0.67023658,  0.67018498,  0.67010033,  0.66885714,  0.66861262,
        0.66798426,  0.667942  ,  0.66671985,  0.66662631,  0.66594216,
        0.66551798,  0.66452187,  0.66428268,  0.66416434,  0.66391563,
        0.66331347,  0.66324723,  0.66132322,  0.66127354,  0.65977049,
        0.65947404,  0.65865743,  0.65857733,  0.65823229,  0.65805309,
        0.65656019,  0.6563066 ,  0.65626032,  0.65612456,  0.65563928,
        0.65553178,  0.65464861,  0.65452534,  0.65396304,  0.65382149,
        0.65369281,  0.653397  ,  0.65184375,  0.65130698,  0.65082814,
        0.65054604,  0.65053239,  0.6499699 ,  0.64747889,  0.64732049,
        0.64671599,  0.64666635,  0.64548275,  0.6454153 ,  0.64448791,
        0.64416142,  0.64371965,  0.64354348,  0.64344275,  0.64300468,
        0.64193997,  0.64171255,  0.64133969,  0.64116234,  0.64096289,
        0.6409129 ,  0.63952081,  0.63950827,  0.63902306,  0.6389233 ,
        0.63810876,  0.63783852,  0.63612388,  0.63547062,  0.63525282,
        0.6352358 ,  0.63379831,  0.63365277,  0.63328088,  0.63308378,
        0.63295313,  0.63290852,  0.63287906,  0.63286552,  0.63227   ,
        0.63222855,  0.63221683,  0.63184702,  0.63122582,  0.63087174,
        0.630557  ,  0.63022362,  0.63011146,  0.63009744,  0.62971778,
        0.62968969,  0.62932408,  0.62889603,  0.62797538,  0.62746714,
        0.62719731,  0.62652844,  0.62395336,  0.62332224,  0.62276692,
        0.62275731,  0.62205803,  0.62201743,  0.62137472,  0.62055958,
        0.62018453,  0.62012921,  0.61888304,  0.61813255,  0.61765367,
        0.61726459,  0.61717313,  0.61716811,  0.61702895,  0.61659888,
        0.61659286,  0.61639966,  0.61627868,  0.61600207,  0.61276081,
        0.61251471,  0.61229583,  0.61227858,  0.61224416,  0.61170973,
        0.61081375,  0.61007074,  0.60960521,  0.60893997,  0.60807332,
        0.60793793,  0.60691953,  0.60579938,  0.60138818,  0.5993801 ,
        0.59928863,  0.59908586,  0.59791037,  0.5977756 ,  0.59737943,
        0.59573503,  0.59489916,  0.59489023,  0.59487381,  0.59473522,
        0.59385356,  0.59284896,  0.5926476 ,  0.59146026,  0.58956582,
        0.58946917,  0.58939275,  0.58901627,  0.58871067,  0.58866832,
        0.5881326 ,  0.58799827,  0.58791072,  0.58785609,  0.58708307,
        0.58648895,  0.58586351,  0.58498751,  0.58414999,  0.58376531,
        0.58313602,  0.58264483,  0.58242003,  0.58207685,  0.58164475,
        0.58117116,  0.58109385,  0.58020233,  0.58007321,  0.58001233,
        0.57955434,  0.57878865,  0.5781294 ,  0.57724563,  0.5771014 ,
        0.57698526,  0.57695894,  0.57481346,  0.57309526,  0.5707037 ,
        0.5703109 ,  0.56986351,  0.56927792,  0.56755315,  0.56728743,
        0.56726834,  0.56684715,  0.56684128,  0.56659215,  0.56464794,
        0.56284241,  0.56283656,  0.56226535,  0.56224435,  0.56194802,
        0.55939182,  0.55933155,  0.55828076,  0.55817749,  0.55459145,
        0.55137933,  0.55071159,  0.55052289,  0.54967242,  0.54913999,
        0.54759925,  0.54735771,  0.54720498,  0.54577005,  0.54359384,
        0.54345838,  0.54165384,  0.54090694,  0.53980642,  0.53939542,
        0.5384282 ,  0.53820027,  0.53604474,  0.53531947,  0.53360005,
        0.53271364,  0.52357718,  0.52207081,  0.51976737,  0.51943839,
        0.50831462,  0.50815075,  0.50784908,  0.50754449,  0.49730647,
        0.49701034,  0.49661627,  0.49645763,  0.49530115,  0.49518421,
        0.49509224,  0.49497211,  0.49466723,  0.49448543,  0.49353372,
        0.49333923,  0.49187409,  0.49169893,  0.48957056,  0.48943141,
        0.488741  ,  0.48860717,  0.48436627,  0.48351431,  0.4759083 ,
        0.47570301,  0.46690634,  0.46679363,  0.44257288,  0.44252377,
        0.42786575,  0.42765668,  0.42551195,  0.42546797,  0.42501634,
        0.42499752,  0.41733453,  0.41724043,  0.40333304,  0.40315878,
        0.39878492,  0.3987665 ,  0.39650699,  0.39632832,  0.38491504,
        0.38486632,  0.36865236,  0.36863028,  0.36407871,  0.36402918,
        0.35090496,  0.35080663,  0.34914529,  0.3491028 ,  0.33322126,
        0.33313405,  0.27414564,  0.27394971,  0.19306236,  0.19293001,
        0.17693419,  0.17684902,  0.08041446,  0.08037123, -2.03591419])}
optimized_pipe
MyPipeline(algorithm=OptimizableQrsDetectorWithInfo(high_pass_filter_cutoff_hz=1, max_heart_rate_bpm=200.0, min_r_peak_height_over_baseline=0.5816447455722318, r_peak_match_tolerance_s=0.01))

But we can also just call the auto-generated self_optimize method and don’t get the info:

optimized_pipe = MyPipeline().self_optimize(train_set)
optimized_pipe
MyPipeline(algorithm=OptimizableQrsDetectorWithInfo(high_pass_filter_cutoff_hz=1, max_heart_rate_bpm=200.0, min_r_peak_height_over_baseline=0.5816447455722318, r_peak_match_tolerance_s=0.01))

However, in most cases, we should just use the Optimize wrapper. It will call the self_optimize_with_info method if available (you can force it to use self_optimize using the optimize_with_info parameter) and then provide the additional info as attribute

{'all_youden_index': array([0.00000000e+00, 6.19540301e-05, 1.52512466e-05, 8.86714683e-02,
       8.86247656e-02, 1.01263388e-01, 1.01216685e-01, 1.07040364e-01,
       1.06993661e-01, 1.07489293e-01, 1.07442590e-01, 1.07752361e-01,
       1.07705658e-01, 1.23751752e-01, 1.23705049e-01, 1.24076773e-01,
       1.24030070e-01, 1.29915703e-01, 1.29869000e-01, 1.30302678e-01,
       1.30255976e-01, 1.31185286e-01, 1.31138583e-01, 1.36714446e-01,
       1.36667743e-01, 1.36915559e-01, 1.36868857e-01, 1.39161156e-01,
       1.39114453e-01, 1.40973074e-01, 1.40926371e-01, 1.42227406e-01,
       1.42180703e-01, 1.43729554e-01, 1.43682851e-01, 1.57932278e-01,
       1.57885575e-01, 2.48152597e-01, 2.48105894e-01, 2.48477618e-01,
       2.48430916e-01, 2.49917812e-01, 2.49871109e-01, 2.50180880e-01,
       2.50134177e-01, 2.51744982e-01, 2.51698279e-01, 2.52441727e-01,
       2.52395024e-01, 2.53324335e-01, 2.53230929e-01, 2.53850470e-01,
       2.53803767e-01, 2.55290664e-01, 2.55243961e-01, 2.57969938e-01,
       2.57923235e-01, 2.59410132e-01, 2.59363429e-01, 2.59797107e-01,
       2.59750405e-01, 2.60493853e-01, 2.60447150e-01, 2.60633012e-01,
       2.60586310e-01, 2.61949298e-01, 2.61902595e-01, 2.63017768e-01,
       2.62971065e-01, 2.63838422e-01, 2.63791719e-01, 2.65278616e-01,
       2.65185210e-01, 2.68035095e-01, 2.67988393e-01, 2.70652416e-01,
       2.70605713e-01, 2.71473070e-01, 2.71426367e-01, 2.72665447e-01,
       2.72618745e-01, 2.72742653e-01, 2.72695950e-01, 2.75917559e-01,
       2.75870857e-01, 2.76924075e-01, 2.76877372e-01, 2.77806683e-01,
       2.77759980e-01, 2.77883888e-01, 2.77837185e-01, 2.78456726e-01,
       2.78410023e-01, 2.79339333e-01, 2.79292630e-01, 2.79478493e-01,
       2.79431790e-01, 2.88353170e-01, 2.88306467e-01, 2.92023709e-01,
       2.91977006e-01, 2.95880110e-01, 2.95833407e-01, 3.01223408e-01,
       3.01176705e-01, 3.02415786e-01, 3.02369083e-01, 3.02740807e-01,
       3.02694105e-01, 3.13164336e-01, 3.13117633e-01, 3.42297981e-01,
       3.42251278e-01, 3.46588060e-01, 3.46541358e-01, 3.47470668e-01,
       3.47423965e-01, 3.51265115e-01, 3.51218412e-01, 3.59644160e-01,
       3.59597458e-01, 3.66350447e-01, 3.66303744e-01, 3.85571447e-01,
       3.85524745e-01, 3.94941757e-01, 3.94895054e-01, 4.04374021e-01,
       4.04327318e-01, 4.12691112e-01, 4.12644410e-01, 4.30982802e-01,
       4.30936100e-01, 4.36264146e-01, 4.36217444e-01, 4.46006180e-01,
       4.45959477e-01, 4.49243041e-01, 4.49196338e-01, 4.52851626e-01,
       4.52804923e-01, 4.54167912e-01, 4.54121209e-01, 4.55793968e-01,
       4.55747265e-01, 4.55995081e-01, 4.55948379e-01, 4.63073092e-01,
       4.63026389e-01, 4.63212251e-01, 4.63165549e-01, 4.63599227e-01,
       4.63552524e-01, 4.64667696e-01, 4.64620994e-01, 4.64744902e-01,
       4.64698199e-01, 4.74177166e-01, 4.74130463e-01, 4.74378279e-01,
       4.74331576e-01, 4.78172726e-01, 4.78126023e-01, 4.78187977e-01,
       4.78141274e-01, 4.86071390e-01, 4.86024688e-01, 5.16320208e-01,
       5.16273505e-01, 5.22840633e-01, 5.22793930e-01, 5.23165654e-01,
       5.23118951e-01, 5.24729756e-01, 5.24683053e-01, 5.27966617e-01,
       5.27919914e-01, 5.28911179e-01, 5.28864476e-01, 5.29917694e-01,
       5.29870992e-01, 5.31357888e-01, 5.31311185e-01, 5.31744864e-01,
       5.31698161e-01, 5.41796668e-01, 5.41749965e-01, 5.47511690e-01,
       5.47464987e-01, 5.47526941e-01, 5.47480238e-01, 5.52622423e-01,
       5.52575720e-01, 5.56231008e-01, 5.56184305e-01, 5.59343961e-01,
       5.59297258e-01, 5.62394959e-01, 5.62348256e-01, 5.64206877e-01,
       5.64160175e-01, 5.71470750e-01, 5.71424047e-01, 5.73344622e-01,
       5.73297920e-01, 5.89158151e-01, 5.89111448e-01, 5.89235356e-01,
       5.89188654e-01, 5.89808194e-01, 5.89761491e-01, 5.97257929e-01,
       5.97211226e-01, 5.98760077e-01, 5.98713374e-01, 6.00571995e-01,
       6.00478589e-01, 6.05001234e-01, 6.04954531e-01, 6.14805222e-01,
       6.14758519e-01, 6.15440013e-01, 6.15393310e-01, 6.16384575e-01,
       6.16337872e-01, 6.19373620e-01, 6.19326917e-01, 6.20689905e-01,
       6.20643203e-01, 6.22563778e-01, 6.22517075e-01, 6.23508339e-01,
       6.23461636e-01, 6.26063706e-01, 6.26017003e-01, 6.28866888e-01,
       6.28820186e-01, 6.41210992e-01, 6.41164289e-01, 6.44200036e-01,
       6.44153333e-01, 6.45454368e-01, 6.45407665e-01, 6.46027206e-01,
       6.45980503e-01, 6.47777170e-01, 6.47730467e-01, 6.51943341e-01,
       6.51896638e-01, 6.52392270e-01, 6.52345568e-01, 6.52841200e-01,
       6.52794497e-01, 6.59733348e-01, 6.59686646e-01, 6.64147336e-01,
       6.64100633e-01, 6.65277760e-01, 6.65231057e-01, 6.66779908e-01,
       6.66733205e-01, 6.70326539e-01, 6.70279836e-01, 6.72757997e-01,
       6.72711294e-01, 6.77233938e-01, 6.77187236e-01, 6.80532753e-01,
       6.80486050e-01, 6.82096855e-01, 6.82050152e-01, 6.82236015e-01,
       6.82189312e-01, 6.82313220e-01, 6.82266517e-01, 6.83195828e-01,
       6.83149125e-01, 6.90335792e-01, 6.90289089e-01, 7.03547252e-01,
       7.03500549e-01, 7.05173308e-01, 7.05126605e-01, 7.07480858e-01,
       7.07434155e-01, 7.11151397e-01, 7.11104695e-01, 7.11662281e-01,
       7.11615578e-01, 7.11677532e-01, 7.11630829e-01, 7.12126461e-01,
       7.12079759e-01, 7.15673092e-01, 7.15579687e-01, 7.18057848e-01,
       7.18011145e-01, 7.21418617e-01, 7.21371914e-01, 7.22734903e-01,
       7.22688200e-01, 7.24422913e-01, 7.24376210e-01, 7.26234831e-01,
       7.26188128e-01, 7.26250082e-01, 7.26203379e-01, 7.27132690e-01,
       7.27085987e-01, 7.27395757e-01, 7.27349055e-01, 7.28464227e-01,
       7.28417524e-01, 7.33745571e-01, 7.33698868e-01, 7.39336685e-01,
       7.39289982e-01, 7.41644235e-01, 7.41597532e-01, 7.42464889e-01,
       7.42418186e-01, 7.45330025e-01, 7.45283323e-01, 7.46646311e-01,
       7.46599609e-01, 7.47095241e-01, 7.47048538e-01, 7.49154975e-01,
       7.49108272e-01, 7.49294134e-01, 7.49247432e-01, 7.49371340e-01,
       7.49324637e-01, 7.50068085e-01, 7.50021382e-01, 7.53986440e-01,
       7.53939738e-01, 7.55054910e-01, 7.55008207e-01, 7.55194069e-01,
       7.55147367e-01, 7.56200585e-01, 7.56153882e-01, 7.57083193e-01,
       7.57036490e-01, 7.57408214e-01, 7.57361511e-01, 7.57485419e-01,
       7.57438717e-01, 7.59854924e-01, 7.59808221e-01, 7.61790750e-01,
       7.61744047e-01, 7.61867955e-01, 7.61821253e-01, 7.61883207e-01,
       7.61836504e-01, 7.62394090e-01, 7.62347387e-01, 7.63090836e-01,
       7.63044133e-01, 7.63539765e-01, 7.63493062e-01, 7.67829844e-01,
       7.67783142e-01, 7.69765671e-01, 7.69718968e-01, 7.70772186e-01,
       7.70725484e-01, 7.70973300e-01, 7.70926597e-01, 7.74457977e-01,
       7.74364571e-01, 7.76347100e-01, 7.76300397e-01, 7.76424305e-01,
       7.76377602e-01, 7.76935189e-01, 7.76888486e-01, 7.79862279e-01,
       7.79815577e-01, 7.82169830e-01, 7.82123127e-01, 7.83857840e-01,
       7.83811137e-01, 7.87032747e-01, 7.86986044e-01, 7.88410987e-01,
       7.88364284e-01, 7.89541410e-01, 7.89494708e-01, 7.89556662e-01,
       7.89509959e-01, 7.89633867e-01, 7.89587164e-01, 7.89834980e-01,
       7.89788277e-01, 7.90160002e-01, 7.90113299e-01, 7.90485023e-01,
       7.90438320e-01, 7.91801309e-01, 7.91754606e-01, 7.92436100e-01,
       7.92389398e-01, 7.93008938e-01, 7.92962235e-01, 7.93581775e-01,
       7.93535073e-01, 7.93906797e-01, 7.93860094e-01, 7.95285037e-01,
       7.95238334e-01, 7.96725231e-01, 7.96678528e-01, 8.00333816e-01,
       8.00287113e-01, 8.00349067e-01, 8.00302364e-01, 8.03090295e-01,
       8.03043593e-01, 8.03291409e-01, 8.03198003e-01, 8.03259957e-01,
       8.03213255e-01, 8.04204519e-01, 8.04157816e-01, 8.05087127e-01,
       8.05040424e-01, 8.05598010e-01, 8.05551307e-01, 8.06542572e-01,
       8.06495869e-01, 8.07177363e-01, 8.07130661e-01, 8.07564339e-01,
       8.07517636e-01, 8.10119705e-01, 8.10073003e-01, 8.11435991e-01,
       8.11389288e-01, 8.11451242e-01, 8.11404540e-01, 8.11652356e-01,
       8.11558950e-01, 8.11930674e-01, 8.11883972e-01, 8.11945926e-01,
       8.11899223e-01, 8.13571982e-01, 8.13525279e-01, 8.13587233e-01,
       8.13540530e-01, 8.14098116e-01, 8.14051414e-01, 8.14175322e-01,
       8.14128619e-01, 8.14190573e-01, 8.14143870e-01, 8.15073181e-01,
       8.14979775e-01, 8.15165637e-01, 8.15118934e-01, 8.15180888e-01,
       8.15134186e-01, 8.16806944e-01, 8.16760242e-01, 8.17193920e-01,
       8.17147217e-01, 8.18758022e-01, 8.18711319e-01, 8.22490515e-01,
       8.22443812e-01, 8.23620939e-01, 8.23574236e-01, 8.23760098e-01,
       8.23713395e-01, 8.23837303e-01, 8.23790600e-01, 8.24038417e-01,
       8.23991714e-01, 8.27770910e-01, 8.27724207e-01, 8.27786161e-01,
       8.27739458e-01, 8.28792677e-01, 8.28745974e-01, 8.29737238e-01,
       8.29690536e-01, 8.30433984e-01, 8.30387281e-01, 8.31936132e-01,
       8.31889429e-01, 8.32880694e-01, 8.32833991e-01, 8.41321693e-01,
       8.41274990e-01, 8.42204301e-01, 8.42157598e-01, 8.42405414e-01,
       8.42358711e-01, 8.45580321e-01, 8.45533618e-01, 8.48631319e-01,
       8.48584617e-01, 8.49575881e-01, 8.49529178e-01, 8.51573661e-01,
       8.51526959e-01, 8.54067074e-01, 8.54020371e-01, 8.57551751e-01,
       8.57505048e-01, 8.60912520e-01, 8.60865817e-01, 8.61795127e-01,
       8.61748424e-01, 8.62182103e-01, 8.62135400e-01, 8.65604826e-01,
       8.65558123e-01, 8.67230882e-01, 8.67184179e-01, 8.72388317e-01,
       8.72341615e-01, 8.75872994e-01, 8.75826291e-01, 8.83012959e-01,
       8.82966256e-01, 8.89471429e-01, 8.89424727e-01, 8.90168175e-01,
       8.90121472e-01, 8.90431242e-01, 8.90384540e-01, 8.96146264e-01,
       8.96099562e-01, 8.96595194e-01, 8.96548491e-01, 9.04106883e-01,
       9.04060180e-01, 9.07963284e-01, 9.07916581e-01, 9.12005547e-01,
       9.11958844e-01, 9.13569649e-01, 9.13522946e-01, 9.18789039e-01,
       9.18742336e-01, 9.21716129e-01, 9.21669427e-01, 9.21979197e-01,
       9.21932494e-01, 9.25525828e-01, 9.25479125e-01, 9.26780160e-01,
       9.26733457e-01, 9.29645296e-01, 9.29598593e-01, 9.33997330e-01,
       9.33950627e-01, 9.35437524e-01, 9.35390821e-01, 9.35948407e-01,
       9.35901704e-01, 9.36149520e-01, 9.36102818e-01, 9.37094082e-01,
       9.37047379e-01, 9.37171287e-01, 9.37077882e-01, 9.39679951e-01,
       9.39633248e-01, 9.40934283e-01, 9.40887580e-01, 9.41321258e-01,
       9.41274556e-01, 9.41584326e-01, 9.41537623e-01, 9.41723485e-01,
       9.41676782e-01, 9.45332070e-01, 9.45285367e-01, 9.47515712e-01,
       9.47469009e-01, 9.48770044e-01, 9.48723341e-01, 9.48785295e-01,
       9.48738593e-01, 9.49543995e-01, 9.49497292e-01, 9.50116832e-01,
       9.50070130e-01, 9.50503808e-01, 9.50457105e-01, 9.50828829e-01,
       9.50782127e-01, 9.51153851e-01, 9.51107148e-01, 9.51231056e-01,
       9.51184353e-01, 9.55706997e-01, 9.55660295e-01, 9.56837421e-01,
       9.56790718e-01, 9.57472213e-01, 9.57425510e-01, 9.58107004e-01,
       9.58060301e-01, 9.58617888e-01, 9.58571185e-01, 9.58633139e-01,
       9.58586436e-01, 9.59205977e-01, 9.59159274e-01, 9.59469044e-01,
       9.59422341e-01, 9.59732111e-01, 9.59685408e-01, 9.59809317e-01,
       9.59762614e-01, 9.60382154e-01, 9.60335451e-01, 9.60521313e-01,
       9.60474611e-01, 9.60598519e-01, 9.60551816e-01, 9.60613770e-01,
       9.60567067e-01, 9.60629021e-01, 9.60582318e-01, 9.61449675e-01,
       9.61402972e-01, 9.61712742e-01, 9.61666039e-01, 9.63462706e-01,
       9.63369301e-01, 9.63431255e-01, 9.63384552e-01, 9.63446506e-01,
       9.63353100e-01, 9.63477008e-01, 9.63430306e-01, 9.63863984e-01,
       9.63817281e-01, 9.64127051e-01, 9.64080348e-01, 9.64328165e-01,
       9.64281462e-01, 9.64591232e-01, 9.64544529e-01, 9.64916253e-01,
       9.64869551e-01, 9.64993459e-01, 9.64900053e-01, 9.65085915e-01,
       9.65039212e-01, 9.65410937e-01, 9.65364234e-01, 9.65859866e-01,
       9.65813163e-01, 9.65999025e-01, 9.65952323e-01, 9.66262093e-01,
       9.66215390e-01, 9.66587114e-01, 9.66540411e-01, 9.66602365e-01,
       9.66555663e-01, 9.66803479e-01, 9.66756776e-01, 9.67004592e-01,
       9.66957889e-01, 9.67143751e-01, 9.67097048e-01, 9.67159003e-01,
       9.67112300e-01, 9.67422070e-01, 9.67328664e-01, 9.67452572e-01,
       9.67359167e-01, 9.67421121e-01, 9.67374418e-01, 9.68241774e-01,
       9.68195072e-01, 9.68318980e-01, 9.68272277e-01, 9.68767909e-01,
       9.68721206e-01, 9.68907069e-01, 9.68813663e-01, 9.68875617e-01,
       9.68828914e-01, 9.68890868e-01, 9.68844165e-01, 9.69091982e-01,
       9.69045279e-01, 9.69355049e-01, 9.69261643e-01, 9.69323597e-01,
       9.69276895e-01, 9.69772527e-01, 9.69725824e-01, 9.69787778e-01,
       9.69741075e-01, 9.69988891e-01, 9.69942189e-01, 9.70251959e-01,
       9.70205256e-01, 9.70267210e-01, 9.70220507e-01, 9.70530277e-01,
       9.70483575e-01, 9.70545529e-01, 9.70498826e-01, 9.70622734e-01,
       9.70576031e-01, 9.70699939e-01, 9.70653236e-01, 9.70839099e-01,
       9.70792396e-01, 9.70854350e-01, 9.70807647e-01, 9.70993509e-01,
       9.70946806e-01, 9.71008760e-01, 9.70868652e-01, 9.70930606e-01,
       9.70883903e-01, 9.71007811e-01, 9.70961109e-01, 9.71085017e-01,
       9.71038314e-01, 9.71224176e-01, 9.71177473e-01, 9.71301381e-01,
       9.71254678e-01, 9.71626403e-01, 9.71579700e-01, 9.71703608e-01,
       9.71656905e-01, 9.71780813e-01, 9.71734110e-01, 9.71858018e-01,
       9.71624504e-01, 9.71686458e-01, 9.71639756e-01, 9.71949526e-01,
       9.71809418e-01, 9.71871372e-01, 9.71824669e-01, 9.71948577e-01,
       9.71901874e-01, 9.71963828e-01, 9.71917125e-01, 9.71979079e-01,
       9.71932377e-01, 9.71994331e-01, 9.71900925e-01, 9.72954143e-01,
       9.72907441e-01, 9.73031349e-01, 9.72984646e-01, 9.73046600e-01,
       9.72906492e-01, 9.73030400e-01, 9.72936994e-01, 9.73184810e-01,
       9.73138108e-01, 9.73447878e-01, 9.73401175e-01, 9.73648991e-01,
       9.73462180e-01, 9.73771950e-01, 9.73398328e-01, 9.73460282e-01,
       9.73413579e-01, 9.73537487e-01, 9.73444081e-01, 9.73691898e-01,
       9.73505086e-01, 9.73628995e-01, 9.73582292e-01, 9.73706200e-01,
       9.73659497e-01, 9.73845359e-01, 9.73705251e-01, 9.73829159e-01,
       9.73689050e-01, 9.73812959e-01, 9.73766256e-01, 9.73828210e-01,
       9.73781507e-01, 9.73905415e-01, 9.73858712e-01, 9.73982620e-01,
       9.73935918e-01, 9.73997872e-01, 9.73951169e-01, 9.74013123e-01,
       9.73919717e-01, 9.73981671e-01, 9.73794860e-01, 9.73918768e-01,
       9.73872065e-01, 9.73995973e-01, 9.73902568e-01, 9.73964522e-01,
       9.73917819e-01, 9.74041727e-01, 9.73948322e-01, 9.74010276e-01,
       9.73823465e-01, 9.73885419e-01, 9.73792013e-01, 9.73853967e-01,
       9.73807264e-01, 9.73931172e-01, 9.73791064e-01, 9.73853018e-01,
       9.73806315e-01, 9.73868269e-01, 9.73447944e-01, 9.73633806e-01,
       9.73353590e-01, 9.73415544e-01, 9.73368841e-01, 9.73492749e-01,
       9.73352641e-01, 9.73414595e-01, 9.73367892e-01, 9.73553754e-01,
       9.73507051e-01, 9.73630959e-01, 9.73397445e-01, 9.73645261e-01,
       9.73598559e-01, 9.73660513e-01, 9.73613810e-01, 9.73675764e-01,
       9.73348844e-01, 9.73410798e-01, 9.73270690e-01, 9.73332644e-01,
       9.73099130e-01, 9.73223038e-01, 9.73176335e-01, 9.73238289e-01,
       9.73098181e-01, 9.73160135e-01, 9.72973324e-01, 9.73035278e-01,
       9.72988575e-01, 9.73050529e-01, 9.72770313e-01, 9.72832267e-01,
       9.72738861e-01, 9.72800815e-01, 9.72707410e-01, 9.72769364e-01,
       9.72489147e-01, 9.72551101e-01, 9.72270884e-01, 9.72332838e-01,
       9.72146027e-01, 9.72207981e-01, 9.71180520e-01, 9.71304428e-01,
       9.71117617e-01, 9.71179571e-01, 9.69965298e-01, 9.70027252e-01,
       9.69980550e-01, 9.70042504e-01, 9.68921637e-01, 9.69045545e-01,
       9.68998842e-01, 9.69060796e-01, 9.68967391e-01, 9.69029345e-01,
       9.68982642e-01, 9.69044596e-01, 9.68997893e-01, 9.69059847e-01,
       9.68826333e-01, 9.68888287e-01, 9.68841585e-01, 9.68903539e-01,
       9.68763430e-01, 9.68825384e-01, 9.68638573e-01, 9.68700527e-01,
       9.67626363e-01, 9.67688317e-01, 9.65960314e-01, 9.66084222e-01,
       9.64496328e-01, 9.64558282e-01, 9.58160000e-01, 9.58221954e-01,
       9.52197295e-01, 9.52259249e-01, 9.51231788e-01, 9.51293742e-01,
       9.51200336e-01, 9.51262291e-01, 9.47899690e-01, 9.47961644e-01,
       9.41423254e-01, 9.41485208e-01, 9.39523692e-01, 9.39585646e-01,
       9.38184562e-01, 9.38246516e-01, 9.33249318e-01, 9.33311272e-01,
       9.25978935e-01, 9.26040889e-01, 9.24406292e-01, 9.24468246e-01,
       9.18910615e-01, 9.18972569e-01, 9.17898405e-01, 9.17960359e-01,
       9.12916458e-01, 9.12978412e-01, 8.85703987e-01, 8.85765941e-01,
       8.01374011e-01, 8.01435965e-01, 7.82568040e-01, 7.82629994e-01,
       5.75222933e-01, 5.75284887e-01, 0.00000000e+00]), 'all_thresholds': array([ 4.82557838,  3.82557838,  3.70343786,  2.94218565,  2.942074  ,
        2.8203026 ,  2.82026647,  2.78245009,  2.782324  ,  2.77971057,
        2.7793436 ,  2.77901207,  2.77884858,  2.6966796 ,  2.69646795,
        2.69351429,  2.69293252,  2.65556322,  2.65497563,  2.65093531,
        2.65079644,  2.64342359,  2.6427749 ,  2.57653608,  2.57650732,
        2.57292632,  2.57236881,  2.51270735,  2.51209908,  2.45747118,
        2.45537237,  2.38290977,  2.38233765,  2.26898769,  2.26876282,
        2.11383814,  2.11369168,  1.85102953,  1.84930445,  1.84516188,
        1.84442315,  1.8191622 ,  1.8183331 ,  1.81245478,  1.81042573,
        1.77732863,  1.77693616,  1.76610969,  1.7641405 ,  1.74665293,
        1.74600351,  1.73458204,  1.73448568,  1.71982794,  1.7186537 ,
        1.68146866,  1.67828692,  1.6674466 ,  1.66723499,  1.66381917,
        1.66265832,  1.65184295,  1.65137835,  1.6504861 ,  1.64939589,
        1.63631954,  1.63608634,  1.62667492,  1.6265638 ,  1.61984373,
        1.61953213,  1.61034214,  1.60841267,  1.59141332,  1.59135861,
        1.57477517,  1.57311128,  1.56784863,  1.56781236,  1.56222299,
        1.56177292,  1.5615277 ,  1.56112938,  1.54836107,  1.54834868,
        1.54586096,  1.54535768,  1.54166149,  1.5415238 ,  1.54115035,
        1.5411388 ,  1.53939579,  1.53928026,  1.53737686,  1.53720324,
        1.53696605,  1.5369137 ,  1.51737166,  1.51733414,  1.51156244,
        1.51151847,  1.5058482 ,  1.50560667,  1.4992319 ,  1.49895466,
        1.4973381 ,  1.49728848,  1.49680808,  1.49678793,  1.48685761,
        1.4868404 ,  1.46332823,  1.46330189,  1.46039387,  1.46034866,
        1.45976768,  1.45964813,  1.45711255,  1.4570676 ,  1.45084292,
        1.45081534,  1.44635881,  1.44634333,  1.43252731,  1.43235989,
        1.4249837 ,  1.42491962,  1.41716628,  1.41703196,  1.40956213,
        1.40945521,  1.39358169,  1.39352731,  1.38868577,  1.38856911,
        1.37897231,  1.37894062,  1.37544923,  1.37530062,  1.371638  ,
        1.37159405,  1.37012799,  1.36987876,  1.36785537,  1.36780377,
        1.36762469,  1.36750989,  1.35921131,  1.35918614,  1.35892724,
        1.35879641,  1.35834045,  1.3581641 ,  1.35689246,  1.35683036,
        1.35675603,  1.35658806,  1.3449695 ,  1.34492062,  1.34454382,
        1.34446322,  1.33962769,  1.33957819,  1.33956742,  1.33950089,
        1.32910081,  1.32908027,  1.28451325,  1.2843395 ,  1.27323641,
        1.27321097,  1.27238728,  1.27224228,  1.26970784,  1.26945415,
        1.26370913,  1.26365353,  1.26196129,  1.26175542,  1.25988863,
        1.25981328,  1.25754058,  1.25751213,  1.25640799,  1.25613685,
        1.24065691,  1.24047563,  1.23175455,  1.23168287,  1.2313269 ,
        1.23110531,  1.22269257,  1.22262297,  1.21646045,  1.21642907,
        1.21159062,  1.21157299,  1.20654606,  1.20618608,  1.20328851,
        1.20327127,  1.19307087,  1.19304479,  1.19039625,  1.1903346 ,
        1.1694793 ,  1.16937474,  1.16931868,  1.16919917,  1.16852717,
        1.1685237 ,  1.16048866,  1.16048261,  1.15888975,  1.15888371,
        1.15719859,  1.15686527,  1.1520799 ,  1.15203302,  1.14303115,
        1.14298737,  1.14253623,  1.14250571,  1.14159804,  1.1415949 ,
        1.13871869,  1.13870954,  1.13653078,  1.13649647,  1.13512337,
        1.13511381,  1.13421743,  1.1341007 ,  1.13184295,  1.13181107,
        1.12932203,  1.12929242,  1.12035945,  1.12035425,  1.11784788,
        1.11776482,  1.11632325,  1.11619945,  1.11590985,  1.11582998,
        1.1144199 ,  1.11439797,  1.11134006,  1.11122845,  1.11103901,
        1.11098802,  1.11044349,  1.11043379,  1.10510776,  1.10503208,
        1.10239814,  1.10233051,  1.1017279 ,  1.10161173,  1.10064277,
        1.10063671,  1.0980769 ,  1.09801448,  1.09647131,  1.09645743,
        1.09290406,  1.09279491,  1.09051218,  1.09045126,  1.08885437,
        1.08875789,  1.08855991,  1.08855664,  1.08851413,  1.08848593,
        1.08769603,  1.08765913,  1.08263579,  1.08249446,  1.07375172,
        1.07373002,  1.07275922,  1.07270919,  1.0713031 ,  1.07124633,
        1.06855012,  1.06854607,  1.06814642,  1.06813638,  1.06813321,
        1.06801285,  1.06741648,  1.06741356,  1.06514284,  1.0650941 ,
        1.06329835,  1.06324538,  1.06106925,  1.06105992,  1.06031355,
        1.0603068 ,  1.05941026,  1.05932492,  1.05832529,  1.05831017,
        1.05824411,  1.05821427,  1.05767542,  1.05765743,  1.05748705,
        1.0574214 ,  1.05668666,  1.05666404,  1.05275835,  1.05274152,
        1.04813048,  1.04809599,  1.04618247,  1.0460922 ,  1.04546659,
        1.04538985,  1.04300093,  1.04294721,  1.04187907,  1.04182971,
        1.04154616,  1.04144491,  1.03969332,  1.03969224,  1.03956863,
        1.0394514 ,  1.03937604,  1.0393111 ,  1.03862453,  1.03856745,
        1.03456277,  1.0345567 ,  1.03393262,  1.03389013,  1.03371654,
        1.03365245,  1.03275111,  1.03270477,  1.03203226,  1.032031  ,
        1.03181089,  1.03176566,  1.03168584,  1.03165066,  1.03019743,
        1.03015088,  1.02882299,  1.02866972,  1.02855439,  1.02852727,
        1.02851534,  1.02844424,  1.02785266,  1.02783572,  1.02713685,
        1.02710104,  1.02673965,  1.02673711,  1.02336653,  1.02332079,
        1.02159723,  1.021587  ,  1.02068661,  1.0206236 ,  1.0203852 ,
        1.02030698,  1.0173309 ,  1.01720208,  1.01500154,  1.01491965,
        1.01484806,  1.01483593,  1.01399961,  1.01392336,  1.01075976,
        1.01070478,  1.00845234,  1.00842688,  1.00634991,  1.00602003,
        1.00229234,  1.00205662,  1.00020597,  1.00017986,  0.99891037,
        0.99887411,  0.99884444,  0.99873595,  0.99861254,  0.99856464,
        0.99821125,  0.99818945,  0.99794761,  0.9978216 ,  0.99727441,
        0.99725162,  0.99580181,  0.99567839,  0.99509029,  0.99504531,
        0.99458661,  0.9945848 ,  0.99432725,  0.99412351,  0.9936942 ,
        0.99368563,  0.99177834,  0.99173382,  0.9899686 ,  0.9899361 ,
        0.98464518,  0.98462125,  0.98454289,  0.98452069,  0.97951525,
        0.97938638,  0.97915821,  0.97872533,  0.97872281,  0.97860977,
        0.97704656,  0.97702392,  0.97541824,  0.97525394,  0.97408005,
        0.97382923,  0.97236646,  0.97232604,  0.97161007,  0.9713012 ,
        0.9710974 ,  0.97082728,  0.96651514,  0.96632545,  0.96345172,
        0.96344564,  0.96340662,  0.96336658,  0.96279613,  0.96254359,
        0.96175349,  0.96170284,  0.9616607 ,  0.96153871,  0.9579177 ,
        0.95778477,  0.95767818,  0.95761271,  0.95614508,  0.95613962,
        0.95584829,  0.95575495,  0.95543679,  0.95537128,  0.95370547,
        0.95328636,  0.95268223,  0.95265827,  0.95261964,  0.95236633,
        0.94847639,  0.94838579,  0.94656382,  0.94653863,  0.94204673,
        0.94202674,  0.93405545,  0.93402455,  0.93054319,  0.93046595,
        0.92988758,  0.92984985,  0.92982241,  0.92959247,  0.92918463,
        0.92915776,  0.92155603,  0.92120089,  0.92118698,  0.92115569,
        0.91851667,  0.91850813,  0.9160845 ,  0.91587852,  0.91407055,
        0.91376373,  0.91141424,  0.91138031,  0.90885957,  0.9087869 ,
        0.89192196,  0.89185769,  0.89011777,  0.89011382,  0.88981681,
        0.88964674,  0.88223642,  0.88217023,  0.87608721,  0.87585445,
        0.87458083,  0.87451373,  0.87082195,  0.87065471,  0.86633224,
        0.86617382,  0.86041355,  0.86035668,  0.85566435,  0.85553359,
        0.85400699,  0.85382926,  0.85297244,  0.85295241,  0.84783928,
        0.84775332,  0.84527466,  0.8452058 ,  0.83736825,  0.83730929,
        0.83145018,  0.83140077,  0.81958078,  0.81949127,  0.80841038,
        0.80828759,  0.80696615,  0.80695282,  0.80637223,  0.8063226 ,
        0.79746988,  0.79736956,  0.79645878,  0.79610352,  0.78487544,
        0.78483116,  0.77803143,  0.7779771 ,  0.77073243,  0.77052098,
        0.76815736,  0.76814805,  0.75952639,  0.75933744,  0.75377991,
        0.7537638 ,  0.75349873,  0.75344439,  0.74899447,  0.74893122,
        0.74752407,  0.74740191,  0.74275142,  0.74248021,  0.7359401 ,
        0.73590682,  0.73294   ,  0.73287211,  0.73188959,  0.731845  ,
        0.73109145,  0.73088291,  0.72934919,  0.72927018,  0.72913392,
        0.72904428,  0.72613004,  0.72596188,  0.72366922,  0.72347317,
        0.72253149,  0.7224706 ,  0.7221855 ,  0.72197678,  0.72176066,
        0.72173089,  0.71558356,  0.71555074,  0.71100395,  0.71095497,
        0.70901417,  0.70897256,  0.70878428,  0.70850139,  0.70738402,
        0.70721433,  0.70578037,  0.70577725,  0.70505623,  0.70505126,
        0.70442012,  0.70433963,  0.70360878,  0.70360087,  0.70348527,
        0.70344195,  0.69658778,  0.69656318,  0.69391896,  0.69385435,
        0.69181593,  0.69146007,  0.69061568,  0.69055202,  0.68924453,
        0.68882348,  0.68802755,  0.68799173,  0.68747934,  0.68746464,
        0.68639005,  0.68625035,  0.68601778,  0.6859202 ,  0.68553113,
        0.68552224,  0.68364926,  0.68356245,  0.6828158 ,  0.68272716,
        0.68241768,  0.68182262,  0.68134824,  0.68119533,  0.68111281,
        0.68089411,  0.67901509,  0.67881768,  0.67715718,  0.67676617,
        0.67200944,  0.67186711,  0.6717779 ,  0.67127497,  0.67075445,
        0.67023658,  0.67018498,  0.67010033,  0.66885714,  0.66861262,
        0.66798426,  0.667942  ,  0.66671985,  0.66662631,  0.66594216,
        0.66551798,  0.66452187,  0.66428268,  0.66416434,  0.66391563,
        0.66331347,  0.66324723,  0.66132322,  0.66127354,  0.65977049,
        0.65947404,  0.65865743,  0.65857733,  0.65823229,  0.65805309,
        0.65656019,  0.6563066 ,  0.65626032,  0.65612456,  0.65563928,
        0.65553178,  0.65464861,  0.65452534,  0.65396304,  0.65382149,
        0.65369281,  0.653397  ,  0.65184375,  0.65130698,  0.65082814,
        0.65054604,  0.65053239,  0.6499699 ,  0.64747889,  0.64732049,
        0.64671599,  0.64666635,  0.64548275,  0.6454153 ,  0.64448791,
        0.64416142,  0.64371965,  0.64354348,  0.64344275,  0.64300468,
        0.64193997,  0.64171255,  0.64133969,  0.64116234,  0.64096289,
        0.6409129 ,  0.63952081,  0.63950827,  0.63902306,  0.6389233 ,
        0.63810876,  0.63783852,  0.63612388,  0.63547062,  0.63525282,
        0.6352358 ,  0.63379831,  0.63365277,  0.63328088,  0.63308378,
        0.63295313,  0.63290852,  0.63287906,  0.63286552,  0.63227   ,
        0.63222855,  0.63221683,  0.63184702,  0.63122582,  0.63087174,
        0.630557  ,  0.63022362,  0.63011146,  0.63009744,  0.62971778,
        0.62968969,  0.62932408,  0.62889603,  0.62797538,  0.62746714,
        0.62719731,  0.62652844,  0.62395336,  0.62332224,  0.62276692,
        0.62275731,  0.62205803,  0.62201743,  0.62137472,  0.62055958,
        0.62018453,  0.62012921,  0.61888304,  0.61813255,  0.61765367,
        0.61726459,  0.61717313,  0.61716811,  0.61702895,  0.61659888,
        0.61659286,  0.61639966,  0.61627868,  0.61600207,  0.61276081,
        0.61251471,  0.61229583,  0.61227858,  0.61224416,  0.61170973,
        0.61081375,  0.61007074,  0.60960521,  0.60893997,  0.60807332,
        0.60793793,  0.60691953,  0.60579938,  0.60138818,  0.5993801 ,
        0.59928863,  0.59908586,  0.59791037,  0.5977756 ,  0.59737943,
        0.59573503,  0.59489916,  0.59489023,  0.59487381,  0.59473522,
        0.59385356,  0.59284896,  0.5926476 ,  0.59146026,  0.58956582,
        0.58946917,  0.58939275,  0.58901627,  0.58871067,  0.58866832,
        0.5881326 ,  0.58799827,  0.58791072,  0.58785609,  0.58708307,
        0.58648895,  0.58586351,  0.58498751,  0.58414999,  0.58376531,
        0.58313602,  0.58264483,  0.58242003,  0.58207685,  0.58164475,
        0.58117116,  0.58109385,  0.58020233,  0.58007321,  0.58001233,
        0.57955434,  0.57878865,  0.5781294 ,  0.57724563,  0.5771014 ,
        0.57698526,  0.57695894,  0.57481346,  0.57309526,  0.5707037 ,
        0.5703109 ,  0.56986351,  0.56927792,  0.56755315,  0.56728743,
        0.56726834,  0.56684715,  0.56684128,  0.56659215,  0.56464794,
        0.56284241,  0.56283656,  0.56226535,  0.56224435,  0.56194802,
        0.55939182,  0.55933155,  0.55828076,  0.55817749,  0.55459145,
        0.55137933,  0.55071159,  0.55052289,  0.54967242,  0.54913999,
        0.54759925,  0.54735771,  0.54720498,  0.54577005,  0.54359384,
        0.54345838,  0.54165384,  0.54090694,  0.53980642,  0.53939542,
        0.5384282 ,  0.53820027,  0.53604474,  0.53531947,  0.53360005,
        0.53271364,  0.52357718,  0.52207081,  0.51976737,  0.51943839,
        0.50831462,  0.50815075,  0.50784908,  0.50754449,  0.49730647,
        0.49701034,  0.49661627,  0.49645763,  0.49530115,  0.49518421,
        0.49509224,  0.49497211,  0.49466723,  0.49448543,  0.49353372,
        0.49333923,  0.49187409,  0.49169893,  0.48957056,  0.48943141,
        0.488741  ,  0.48860717,  0.48436627,  0.48351431,  0.4759083 ,
        0.47570301,  0.46690634,  0.46679363,  0.44257288,  0.44252377,
        0.42786575,  0.42765668,  0.42551195,  0.42546797,  0.42501634,
        0.42499752,  0.41733453,  0.41724043,  0.40333304,  0.40315878,
        0.39878492,  0.3987665 ,  0.39650699,  0.39632832,  0.38491504,
        0.38486632,  0.36865236,  0.36863028,  0.36407871,  0.36402918,
        0.35090496,  0.35080663,  0.34914529,  0.3491028 ,  0.33322126,
        0.33313405,  0.27414564,  0.27394971,  0.19306236,  0.19293001,
        0.17693419,  0.17684902,  0.08041446,  0.08037123, -2.03591419])}
optimizer.optimized_pipeline_
MyPipeline(algorithm=OptimizableQrsDetectorWithInfo(high_pass_filter_cutoff_hz=1, max_heart_rate_bpm=200.0, min_r_peak_height_over_baseline=0.5816447455722318, r_peak_match_tolerance_s=0.01))

As Optimize is aware of this and stores the info as a result attribute, the information is also available in the output of a cross validation. TODO: Make GridSearchCV aware of it!

Further Notes#

Sometimes it might be a good idea to provide separate implementation of self_optimize and self_optimize_with_info. This might be required, when collecting and calculating the additional info creates a relevant computational overhead. However, you should make sure, that the two methods return the same optimization result otherwise.

Total running time of the script: ( 0 minutes 2.600 seconds)

Estimated memory usage: 14 MB

Gallery generated by Sphinx-Gallery