.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/integrations/_01_tensorflow.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_integrations__01_tensorflow.py: .. _tensorflow: Tensorflow/Keras ================ .. note:: This example requires the `tensorflow` package to be installed. Theoretically, tpcp is framework agnostic and can be used with any framework. However, due to the way some frameworks handle their objects, some special handling internally is required. Hence, this example does not only serve as example on how to use tensorflow with tpcp, but also as a test case for these special cases. When using tpcp with any machine learning framework, you either want to use a pretrained model with a normal pipeline or a train your own model as part of an Optimizable Pipeline. Here we show the second case, as it is more complex, and you are likely able to figure out the first case yourself. This means, we are planning to perform the following steps: 1. Create a pipeline that creates and trains a model. 2. Allow the modification of model hyperparameters. 3. Run a simple cross-validation to demonstrate the functionality. This example reimplements the basic MNIST example from the [tensorflow documentation](https://www.tensorflow.org/tutorials/keras/classification). Some Notes ---------- In this example we show how to implement a Pipeline that uses tensorflow. You could implement an Algorithm in a similar way. This would actually be easier, as no specific handling of the input data would be required. For a pipeline, we need to create a custom Dataset class, as this is the expected input for a pipeline. .. GENERATED FROM PYTHON SOURCE LINES 39-54 The Dataset ----------- We are using the normal fashion MNIST dataset for this example It consists of 60.000 images of 28x28 pixels, each with a label. We will ignore the typical train-test split, as we want to do our own cross-validation. In addition, we will simulate an additional "index level". In this (and most typical deep learning datasets), each datapoint is one vector for which we can make one prediction. In tpcp, we usually deal with datasets, where you might have multiple pieces of information for each datapoint. For example, one datapoint could be a patient, for which we have an entire time series of measurements. We will simulate this here, by creating the index of our dataset as 1000 groups each containing 60 images. Other than that, the dataset is pretty standard. Besides the `create_index` method, we only need to implement the `input_as_array` and `labels_as_array` methods that allow us to easily access the data once we selected a single group. .. GENERATED FROM PYTHON SOURCE LINES 54-93 .. code-block:: default from functools import lru_cache import numpy as np import pandas as pd import tensorflow as tf from tpcp import Dataset tf.keras.utils.set_random_seed(812) tf.config.experimental.enable_op_determinism() @lru_cache(maxsize=1) def get_fashion_mnist_data(): # Note: We throw train and test sets together, as we don't care about the official split here. # We will create our own split later. (train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.fashion_mnist.load_data() return np.array(list(train_images) + list(test_images)), list(train_labels) + list(test_labels) class FashionMNIST(Dataset): def input_as_array(self) -> np.ndarray: self.assert_is_single(None, "input_as_array") group_id = int(self.group_label.group_id) images, _ = get_fashion_mnist_data() return images[group_id * 60 : (group_id + 1) * 60].reshape((60, 28, 28)) / 255 def labels_as_array(self) -> np.ndarray: self.assert_is_single(None, "labels_as_array") group_id = int(self.group_label.group_id) _, labels = get_fashion_mnist_data() return np.array(labels[group_id * 60 : (group_id + 1) * 60]) def create_index(self) -> pd.DataFrame: # There are 60.000 images in total. # We simulate 1000 groups of 60 images each. return pd.DataFrame({"group_id": list(range(1000))}) .. GENERATED FROM PYTHON SOURCE LINES 94-95 We can see our Dataset works as expected: .. GENERATED FROM PYTHON SOURCE LINES 95-98 .. code-block:: default dataset = FashionMNIST() dataset[0].input_as_array().shape .. rst-class:: sphx-glr-script-out .. code-block:: none Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz 8192/29515 [=======>......................] - ETA: 0s 29515/29515 [==============================] - 0s 1us/step Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz 8192/26421880 [..............................] - ETA: 0s 131072/26421880 [..............................] - ETA: 10s 999424/26421880 [>.............................] - ETA: 2s  5242880/26421880 [====>.........................] - ETA: 0s 10059776/26421880 [==========>...................] - ETA: 0s 14417920/26421880 [===============>..............] - ETA: 0s 19234816/26421880 [====================>.........] - ETA: 0s 23658496/26421880 [=========================>....] - ETA: 0s 26421880/26421880 [==============================] - 0s 0us/step Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz 5148/5148 [==============================] - 0s 0us/step Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz 8192/4422102 [..............................] - ETA: 0s 139264/4422102 [..............................] - ETA: 1s 974848/4422102 [=====>........................] - ETA: 0s 2138112/4422102 [=============>................] - ETA: 0s 4422102/4422102 [==============================] - 0s 0us/step (60, 28, 28) .. GENERATED FROM PYTHON SOURCE LINES 99-101 .. code-block:: default dataset[0].labels_as_array().shape .. rst-class:: sphx-glr-script-out .. code-block:: none (60,) .. GENERATED FROM PYTHON SOURCE LINES 102-139 The Pipeline ------------ We will create a pipeline that uses a simple neural network to classify the images. In tpcp, all "things" that should be optimized need to be parameters. This means our model itself needs to be a parameter of the pipeline. However, as we don't have the model yet, as its creation depends on other hyperparameters, we add it as an optional parameter initialized with `None`. Further, we prefix the parameter name with an underscore, to signify, that this is not a parameter that should be modified manually by the user. This is just convention, and it is up to you to decide how you want to name your parameters. We further introduce a hyperparameter `n_dense_layer_nodes` to show how we can influence the model creation. The optimize method +++++++++++++++++++ To make our pipeline optimizable, it needs to inherit from `OptimizablePipeline`. Further we need to mark at least one of the parameters as `OptiPara` using the type annotation. We do this for our `_model` parameter. Finally, we need to implement the `self_optimize` method. This method will get the entire training dataset as input and should update the `_model` parameter with the trained model. Hence, we first extract the relevant data (remember, each datapoint is 60 images), by concatinating all images over all groups in the dataset. Then we create the Keras model based on the hyperparameters. Finally, we train the model and update the `_model` parameter. Here we chose to wrap the method with `make_optimize_safe`. This decorator will perform some runtime checks to ensure that the method is implemented correctly. The run method ++++++++++++++ The run method expects that the `_model` parameter is already set (i.e. the pipeline was already optimized). It gets a single datapoint as input (remember, a datapoint is a single group of 60 images). We then extract the data from the datapoint and let the model make a prediction. We store the prediction on our output attribute `predictions_`. The trailing underscore is a convention to signify, that this is an "result" attribute. .. GENERATED FROM PYTHON SOURCE LINES 139-200 .. code-block:: default import warnings from typing import Optional from typing_extensions import Self from tpcp import OptimizablePipeline, OptiPara, make_action_safe, make_optimize_safe class KerasPipeline(OptimizablePipeline): n_dense_layer_nodes: int n_train_epochs: int _model: OptiPara[Optional[tf.keras.Sequential]] predictions_: np.ndarray def __init__(self, n_dense_layer_nodes=128, n_train_epochs=5, _model: Optional[tf.keras.Sequential] = None): self.n_dense_layer_nodes = n_dense_layer_nodes self.n_train_epochs = n_train_epochs self._model = _model @property def predicted_labels_(self): return np.argmax(self.predictions_, axis=1) @make_optimize_safe def self_optimize(self, dataset, **_) -> Self: data = np.vstack([d.input_as_array() for d in dataset]) labels = np.hstack([d.labels_as_array() for d in dataset]) print(data.shape) if self._model is not None: warnings.warn("Overwriting existing model!") self._model = tf.keras.Sequential( [ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(self.n_dense_layer_nodes, activation="relu"), tf.keras.layers.Dense(10), ] ) self._model.compile( optimizer="adam", loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=["accuracy"], ) self._model.fit(data, labels, epochs=self.n_train_epochs) return self @make_action_safe def run(self, datapoint) -> Self: if self._model is None: raise RuntimeError("Model not trained yet!") data = datapoint.input_as_array() self.predictions_ = self._model.predict(data) return self .. GENERATED FROM PYTHON SOURCE LINES 201-206 Testing the pipeline -------------------- We can now test our pipeline. We will run the optimization using a couple of datapoints (to keep everything fast) and then use `run` to get the predictions for a single unseen datapoint. .. GENERATED FROM PYTHON SOURCE LINES 206-211 .. code-block:: default pipeline = KerasPipeline().self_optimize(FashionMNIST()[:10]) p1 = pipeline.run(FashionMNIST()[11]) print(p1.predicted_labels_) print(FashionMNIST()[11].labels_as_array()) .. rst-class:: sphx-glr-script-out .. code-block:: none (600, 28, 28) Epoch 1/5 1/19 [>.............................] - ETA: 11s - loss: 2.3593 - accuracy: 0.1250 19/19 [==============================] - 1s 3ms/step - loss: 1.5112 - accuracy: 0.4933 Epoch 2/5 1/19 [>.............................] - ETA: 0s - loss: 0.9220 - accuracy: 0.7188 19/19 [==============================] - 0s 2ms/step - loss: 0.8705 - accuracy: 0.7083 Epoch 3/5 1/19 [>.............................] - ETA: 0s - loss: 1.0179 - accuracy: 0.6875 19/19 [==============================] - 0s 2ms/step - loss: 0.6799 - accuracy: 0.7850 Epoch 4/5 1/19 [>.............................] - ETA: 0s - loss: 0.4521 - accuracy: 0.8750 19/19 [==============================] - 0s 2ms/step - loss: 0.5962 - accuracy: 0.8133 Epoch 5/5 1/19 [>.............................] - ETA: 0s - loss: 0.4230 - accuracy: 0.8438 19/19 [==============================] - 0s 2ms/step - loss: 0.5158 - accuracy: 0.8400 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step [8 8 0 9 6 0 7 3 7 9 3 8 6 3 7 8 1 4 0 7 9 8 5 5 2 1 3 3 1 9 7 5 9 9 7 8 2 7 2 7 2 6 7 1 1 7 5 4 8 3 5 9 0 7 3 0 0 9 1 9] [8 8 0 9 2 0 7 3 7 9 3 8 4 3 7 8 1 4 0 7 9 8 5 5 2 1 3 4 6 7 7 5 9 9 7 8 2 7 4 7 0 3 5 1 1 5 5 2 8 3 5 9 0 7 3 0 0 7 1 9] .. GENERATED FROM PYTHON SOURCE LINES 212-214 We can see that even with just 5 epochs, the model already performs quite well. To quantify we can calculate the accuracy for this datapoint: .. GENERATED FROM PYTHON SOURCE LINES 214-218 .. code-block:: default from sklearn.metrics import accuracy_score accuracy_score(p1.predicted_labels_, FashionMNIST()[11].labels_as_array()) .. rst-class:: sphx-glr-script-out .. code-block:: none 0.8 .. GENERATED FROM PYTHON SOURCE LINES 219-225 Cross Validation ---------------- If we want to run a cross validation, we need to formalize the scoring into a function. We will calculate two types of accuracy: First, the accuracy per group and second, the accuracy over all images across all groups. For more information about how this works, check the :ref:`Custom Scorer ` example. .. GENERATED FROM PYTHON SOURCE LINES 225-248 .. code-block:: default from collections.abc import Sequence from tpcp.validate import Aggregator class SingleValueAccuracy(Aggregator[np.ndarray]): RETURN_RAW_SCORES = False @classmethod def aggregate(cls, /, values: Sequence[tuple[np.ndarray, np.ndarray]], **_) -> dict[str, float]: return {"accuracy": accuracy_score(np.hstack([v[0] for v in values]), np.hstack([v[1] for v in values]))} def scoring(pipeline, datapoint): result: np.ndarray = pipeline.safe_run(datapoint).predicted_labels_ reference = datapoint.labels_as_array() return { "accuracy": accuracy_score(result, reference), "per_sample": SingleValueAccuracy((result, reference)), } .. GENERATED FROM PYTHON SOURCE LINES 249-256 Now we can run a cross validation. We will only run it on a subset of the data, to keep the runtime manageable. .. note:: You might see warnings about retracing of the model. This is because we clone the pipeline before each call to the run method. This is a good idea to ensure that all pipelines are independent of each other, however, might result in some performance overhead. .. GENERATED FROM PYTHON SOURCE LINES 256-262 .. code-block:: default from tpcp.optimize import Optimize from tpcp.validate import cross_validate pipeline = KerasPipeline(n_train_epochs=10) cv_results = cross_validate(Optimize(pipeline), FashionMNIST()[:100], scoring=scoring, cv=3) .. rst-class:: sphx-glr-script-out .. code-block:: none CV Folds: 0%| | 0/3 [00:00.........................] - ETA: 0s - loss: 1.5346 - accuracy: 0.4389  45/124 [=========>....................] - ETA: 0s - loss: 1.2047 - accuracy: 0.5660 68/124 [===============>..............] - ETA: 0s - loss: 1.0559 - accuracy: 0.6245 91/124 [=====================>........] - ETA: 0s - loss: 0.9814 - accuracy: 0.6552 114/124 [==========================>...] - ETA: 0s - loss: 0.9195 - accuracy: 0.6806 124/124 [==============================] - 1s 2ms/step - loss: 0.8968 - accuracy: 0.6919 Epoch 2/10 1/124 [..............................] - ETA: 0s - loss: 1.0481 - accuracy: 0.7500 24/124 [====>.........................] - ETA: 0s - loss: 0.5817 - accuracy: 0.8112 46/124 [==========>...................] - ETA: 0s - loss: 0.5914 - accuracy: 0.8023 68/124 [===============>..............] - ETA: 0s - loss: 0.5709 - accuracy: 0.8065 90/124 [====================>.........] - ETA: 0s - loss: 0.5670 - accuracy: 0.8087 113/124 [==========================>...] - ETA: 0s - loss: 0.5562 - accuracy: 0.8125 124/124 [==============================] - 0s 2ms/step - loss: 0.5606 - accuracy: 0.8104 Epoch 3/10 1/124 [..............................] - ETA: 0s - loss: 0.7180 - accuracy: 0.6250 23/124 [====>.........................] - ETA: 0s - loss: 0.5287 - accuracy: 0.8084 45/124 [=========>....................] - ETA: 0s - loss: 0.5210 - accuracy: 0.8188 68/124 [===============>..............] - ETA: 0s - loss: 0.5074 - accuracy: 0.8222 91/124 [=====================>........] - ETA: 0s - loss: 0.4976 - accuracy: 0.8252 114/124 [==========================>...] - ETA: 0s - loss: 0.4983 - accuracy: 0.8246 124/124 [==============================] - 0s 2ms/step - loss: 0.5009 - accuracy: 0.8232 Epoch 4/10 1/124 [..............................] - ETA: 0s - loss: 0.4630 - accuracy: 0.8125 24/124 [====>.........................] - ETA: 0s - loss: 0.4142 - accuracy: 0.8568 47/124 [==========>...................] - ETA: 0s - loss: 0.4267 - accuracy: 0.8524 70/124 [===============>..............] - ETA: 0s - loss: 0.4397 - accuracy: 0.8487 93/124 [=====================>........] - ETA: 0s - loss: 0.4434 - accuracy: 0.8464 116/124 [===========================>..] - ETA: 0s - loss: 0.4432 - accuracy: 0.8443 124/124 [==============================] - 0s 2ms/step - loss: 0.4448 - accuracy: 0.8437 Epoch 5/10 1/124 [..............................] - ETA: 0s - loss: 0.6924 - accuracy: 0.6250 24/124 [====>.........................] - ETA: 0s - loss: 0.4531 - accuracy: 0.8216 47/124 [==========>...................] - ETA: 0s - loss: 0.4399 - accuracy: 0.8384 70/124 [===============>..............] - ETA: 0s - loss: 0.4194 - accuracy: 0.8473 93/124 [=====================>........] - ETA: 0s - loss: 0.4076 - accuracy: 0.8548 116/124 [===========================>..] - ETA: 0s - loss: 0.4123 - accuracy: 0.8548 124/124 [==============================] - 0s 2ms/step - loss: 0.4119 - accuracy: 0.8540 Epoch 6/10 1/124 [..............................] - ETA: 0s - loss: 0.5976 - accuracy: 0.7812 24/124 [====>.........................] - ETA: 0s - loss: 0.4102 - accuracy: 0.8594 47/124 [==========>...................] - ETA: 0s - loss: 0.3838 - accuracy: 0.8664 70/124 [===============>..............] - ETA: 0s - loss: 0.3852 - accuracy: 0.8647 92/124 [=====================>........] - ETA: 0s - loss: 0.3858 - accuracy: 0.8662 115/124 [==========================>...] - ETA: 0s - loss: 0.3816 - accuracy: 0.8677 124/124 [==============================] - 0s 2ms/step - loss: 0.3792 - accuracy: 0.8687 Epoch 7/10 1/124 [..............................] - ETA: 0s - loss: 0.4009 - accuracy: 0.8438 24/124 [====>.........................] - ETA: 0s - loss: 0.3615 - accuracy: 0.8672 47/124 [==========>...................] - ETA: 0s - loss: 0.3492 - accuracy: 0.8737 70/124 [===============>..............] - ETA: 0s - loss: 0.3589 - accuracy: 0.8750 93/124 [=====================>........] - ETA: 0s - loss: 0.3573 - accuracy: 0.8770 116/124 [===========================>..] - ETA: 0s - loss: 0.3569 - accuracy: 0.8772 124/124 [==============================] - 0s 2ms/step - loss: 0.3555 - accuracy: 0.8768 Epoch 8/10 1/124 [..............................] - ETA: 0s - loss: 0.2810 - accuracy: 0.9062 24/124 [====>.........................] - ETA: 0s - loss: 0.2964 - accuracy: 0.9128 47/124 [==========>...................] - ETA: 0s - loss: 0.2937 - accuracy: 0.9049 70/124 [===============>..............] - ETA: 0s - loss: 0.3136 - accuracy: 0.8955 93/124 [=====================>........] - ETA: 0s - loss: 0.3299 - accuracy: 0.8901 115/124 [==========================>...] - ETA: 0s - loss: 0.3410 - accuracy: 0.8848 124/124 [==============================] - 0s 2ms/step - loss: 0.3464 - accuracy: 0.8826 Epoch 9/10 1/124 [..............................] - ETA: 0s - loss: 0.2351 - accuracy: 0.9375 24/124 [====>.........................] - ETA: 0s - loss: 0.3400 - accuracy: 0.8867 47/124 [==========>...................] - ETA: 0s - loss: 0.3378 - accuracy: 0.8810 70/124 [===============>..............] - ETA: 0s - loss: 0.3258 - accuracy: 0.8862 92/124 [=====================>........] - ETA: 0s - loss: 0.3217 - accuracy: 0.8906 115/124 [==========================>...] - ETA: 0s - loss: 0.3208 - accuracy: 0.8891 124/124 [==============================] - 0s 2ms/step - loss: 0.3196 - accuracy: 0.8886 Epoch 10/10 1/124 [..............................] - ETA: 0s - loss: 0.2527 - accuracy: 0.9688 24/124 [====>.........................] - ETA: 0s - loss: 0.2444 - accuracy: 0.9128 46/124 [==========>...................] - ETA: 0s - loss: 0.2720 - accuracy: 0.9035 69/124 [===============>..............] - ETA: 0s - loss: 0.2922 - accuracy: 0.8981 92/124 [=====================>........] - ETA: 0s - loss: 0.2951 - accuracy: 0.8981 115/124 [==========================>...] - ETA: 0s - loss: 0.2948 - accuracy: 0.8992 124/124 [==============================] - 0s 2ms/step - loss: 0.2921 - accuracy: 0.8997 Datapoints: 0%| | 0/34 [00:00...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 3%|▎ | 1/34 [00:00<00:06, 5.09it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 6%|▌ | 2/34 [00:00<00:06, 5.05it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 9%|▉ | 3/34 [00:00<00:05, 5.22it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 12%|█▏ | 4/34 [00:00<00:05, 5.37it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 15%|█▍ | 5/34 [00:00<00:05, 5.42it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 18%|█▊ | 6/34 [00:01<00:05, 5.36it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 21%|██ | 7/34 [00:01<00:05, 5.39it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 24%|██▎ | 8/34 [00:01<00:04, 5.30it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 26%|██▋ | 9/34 [00:01<00:04, 5.19it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 29%|██▉ | 10/34 [00:01<00:04, 5.16it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 32%|███▏ | 11/34 [00:02<00:04, 5.27it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 35%|███▌ | 12/34 [00:02<00:04, 5.32it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 38%|███▊ | 13/34 [00:02<00:03, 5.39it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 41%|████ | 14/34 [00:02<00:03, 5.45it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 44%|████▍ | 15/34 [00:02<00:03, 5.38it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 47%|████▋ | 16/34 [00:03<00:03, 5.28it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 50%|█████ | 17/34 [00:03<00:03, 5.18it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 53%|█████▎ | 18/34 [00:03<00:03, 5.07it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 56%|█████▌ | 19/34 [00:03<00:02, 5.19it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 59%|█████▉ | 20/34 [00:03<00:02, 5.31it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 62%|██████▏ | 21/34 [00:03<00:02, 5.22it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 65%|██████▍ | 22/34 [00:04<00:02, 5.33it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 68%|██████▊ | 23/34 [00:04<00:02, 5.27it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 71%|███████ | 24/34 [00:04<00:01, 5.15it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 74%|███████▎ | 25/34 [00:04<00:01, 5.13it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 76%|███████▋ | 26/34 [00:04<00:01, 5.24it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 79%|███████▉ | 27/34 [00:05<00:01, 5.13it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 82%|████████▏ | 28/34 [00:05<00:01, 5.22it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 85%|████████▌ | 29/34 [00:05<00:00, 5.31it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 88%|████████▊ | 30/34 [00:05<00:00, 5.16it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 91%|█████████ | 31/34 [00:05<00:00, 5.15it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 94%|█████████▍| 32/34 [00:06<00:00, 5.13it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 97%|█████████▋| 33/34 [00:06<00:00, 5.08it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 100%|██████████| 34/34 [00:06<00:00, 5.09it/s] Datapoints: 100%|██████████| 34/34 [00:06<00:00, 5.23it/s] CV Folds: 33%|███▎ | 1/3 [00:13<00:26, 13.08s/it](4020, 28, 28) Epoch 1/10 1/126 [..............................] - ETA: 1:09 - loss: 2.3656 - accuracy: 0.1250 23/126 [====>.........................] - ETA: 0s - loss: 1.5016 - accuracy: 0.5014  46/126 [=========>....................] - ETA: 0s - loss: 1.2168 - accuracy: 0.5951 69/126 [===============>..............] - ETA: 0s - loss: 1.0854 - accuracy: 0.6318 92/126 [====================>.........] - ETA: 0s - loss: 0.9872 - accuracy: 0.6675 115/126 [==========================>...] - ETA: 0s - loss: 0.9161 - accuracy: 0.6902 126/126 [==============================] - 1s 2ms/step - loss: 0.8913 - accuracy: 0.7000 Epoch 2/10 1/126 [..............................] - ETA: 0s - loss: 0.4984 - accuracy: 0.8125 23/126 [====>.........................] - ETA: 0s - loss: 0.5577 - accuracy: 0.8043 46/126 [=========>....................] - ETA: 0s - loss: 0.5677 - accuracy: 0.7989 69/126 [===============>..............] - ETA: 0s - loss: 0.5895 - accuracy: 0.7912 92/126 [====================>.........] - ETA: 0s - loss: 0.5754 - accuracy: 0.7959 115/126 [==========================>...] - ETA: 0s - loss: 0.5706 - accuracy: 0.7970 126/126 [==============================] - 0s 2ms/step - loss: 0.5662 - accuracy: 0.8002 Epoch 3/10 1/126 [..............................] - ETA: 0s - loss: 0.5249 - accuracy: 0.8125 24/126 [====>.........................] - ETA: 0s - loss: 0.5165 - accuracy: 0.8281 47/126 [==========>...................] - ETA: 0s - loss: 0.4722 - accuracy: 0.8344 70/126 [===============>..............] - ETA: 0s - loss: 0.4697 - accuracy: 0.8375 93/126 [=====================>........] - ETA: 0s - loss: 0.4784 - accuracy: 0.8350 116/126 [==========================>...] - ETA: 0s - loss: 0.4861 - accuracy: 0.8300 126/126 [==============================] - 0s 2ms/step - loss: 0.4893 - accuracy: 0.8279 Epoch 4/10 1/126 [..............................] - ETA: 0s - loss: 0.2082 - accuracy: 0.9688 24/126 [====>.........................] - ETA: 0s - loss: 0.4214 - accuracy: 0.8581 47/126 [==========>...................] - ETA: 0s - loss: 0.4409 - accuracy: 0.8457 70/126 [===============>..............] - ETA: 0s - loss: 0.4286 - accuracy: 0.8513 93/126 [=====================>........] - ETA: 0s - loss: 0.4376 - accuracy: 0.8501 116/126 [==========================>...] - ETA: 0s - loss: 0.4469 - accuracy: 0.8443 126/126 [==============================] - 0s 2ms/step - loss: 0.4431 - accuracy: 0.8453 Epoch 5/10 1/126 [..............................] - ETA: 0s - loss: 0.1979 - accuracy: 0.9375 24/126 [====>.........................] - ETA: 0s - loss: 0.3937 - accuracy: 0.8620 47/126 [==========>...................] - ETA: 0s - loss: 0.4028 - accuracy: 0.8531 70/126 [===============>..............] - ETA: 0s - loss: 0.4030 - accuracy: 0.8567 93/126 [=====================>........] - ETA: 0s - loss: 0.4077 - accuracy: 0.8558 116/126 [==========================>...] - ETA: 0s - loss: 0.4028 - accuracy: 0.8575 126/126 [==============================] - 0s 2ms/step - loss: 0.4046 - accuracy: 0.8580 Epoch 6/10 1/126 [..............................] - ETA: 0s - loss: 0.4598 - accuracy: 0.8125 24/126 [====>.........................] - ETA: 0s - loss: 0.3701 - accuracy: 0.8594 47/126 [==========>...................] - ETA: 0s - loss: 0.3768 - accuracy: 0.8610 70/126 [===============>..............] - ETA: 0s - loss: 0.3763 - accuracy: 0.8647 93/126 [=====================>........] - ETA: 0s - loss: 0.3766 - accuracy: 0.8663 116/126 [==========================>...] - ETA: 0s - loss: 0.3728 - accuracy: 0.8672 126/126 [==============================] - 0s 2ms/step - loss: 0.3772 - accuracy: 0.8667 Epoch 7/10 1/126 [..............................] - ETA: 0s - loss: 0.2244 - accuracy: 0.9062 24/126 [====>.........................] - ETA: 0s - loss: 0.3537 - accuracy: 0.8802 47/126 [==========>...................] - ETA: 0s - loss: 0.3262 - accuracy: 0.8876 70/126 [===============>..............] - ETA: 0s - loss: 0.3386 - accuracy: 0.8821 93/126 [=====================>........] - ETA: 0s - loss: 0.3481 - accuracy: 0.8807 116/126 [==========================>...] - ETA: 0s - loss: 0.3472 - accuracy: 0.8790 126/126 [==============================] - 0s 2ms/step - loss: 0.3479 - accuracy: 0.8786 Epoch 8/10 1/126 [..............................] - ETA: 0s - loss: 0.2092 - accuracy: 0.9375 24/126 [====>.........................] - ETA: 0s - loss: 0.2914 - accuracy: 0.9049 47/126 [==========>...................] - ETA: 0s - loss: 0.3042 - accuracy: 0.8956 70/126 [===============>..............] - ETA: 0s - loss: 0.3183 - accuracy: 0.8879 93/126 [=====================>........] - ETA: 0s - loss: 0.3303 - accuracy: 0.8864 116/126 [==========================>...] - ETA: 0s - loss: 0.3276 - accuracy: 0.8869 126/126 [==============================] - 0s 2ms/step - loss: 0.3285 - accuracy: 0.8863 Epoch 9/10 1/126 [..............................] - ETA: 0s - loss: 0.1886 - accuracy: 0.9062 24/126 [====>.........................] - ETA: 0s - loss: 0.2912 - accuracy: 0.9023 47/126 [==========>...................] - ETA: 0s - loss: 0.3027 - accuracy: 0.8943 70/126 [===============>..............] - ETA: 0s - loss: 0.3121 - accuracy: 0.8808 93/126 [=====================>........] - ETA: 0s - loss: 0.3105 - accuracy: 0.8841 116/126 [==========================>...] - ETA: 0s - loss: 0.3145 - accuracy: 0.8879 126/126 [==============================] - 0s 2ms/step - loss: 0.3120 - accuracy: 0.8900 Epoch 10/10 1/126 [..............................] - ETA: 0s - loss: 0.2327 - accuracy: 0.9062 24/126 [====>.........................] - ETA: 0s - loss: 0.2974 - accuracy: 0.8789 47/126 [==========>...................] - ETA: 0s - loss: 0.3049 - accuracy: 0.8863 70/126 [===============>..............] - ETA: 0s - loss: 0.3075 - accuracy: 0.8866 93/126 [=====================>........] - ETA: 0s - loss: 0.3012 - accuracy: 0.8884 116/126 [==========================>...] - ETA: 0s - loss: 0.3039 - accuracy: 0.8887 126/126 [==============================] - 0s 2ms/step - loss: 0.3065 - accuracy: 0.8871 Datapoints: 0%| | 0/33 [00:00...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 3%|▎ | 1/33 [00:00<00:06, 5.15it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 6%|▌ | 2/33 [00:00<00:06, 5.05it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 9%|▉ | 3/33 [00:00<00:05, 5.04it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 12%|█▏ | 4/33 [00:00<00:05, 5.21it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 15%|█▌ | 5/33 [00:00<00:05, 5.33it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 18%|█▊ | 6/33 [00:01<00:05, 5.27it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 21%|██ | 7/33 [00:01<00:05, 5.17it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 24%|██▍ | 8/33 [00:01<00:04, 5.12it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 27%|██▋ | 9/33 [00:01<00:04, 5.24it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 30%|███ | 10/33 [00:01<00:04, 5.12it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 33%|███▎ | 11/33 [00:02<00:04, 5.09it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 36%|███▋ | 12/33 [00:02<00:04, 5.06it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 39%|███▉ | 13/33 [00:02<00:03, 5.06it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 42%|████▏ | 14/33 [00:02<00:03, 5.05it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 45%|████▌ | 15/33 [00:02<00:03, 5.04it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 48%|████▊ | 16/33 [00:03<00:03, 5.00it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 52%|█████▏ | 17/33 [00:03<00:03, 5.13it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 55%|█████▍ | 18/33 [00:03<00:02, 5.24it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 58%|█████▊ | 19/33 [00:03<00:02, 5.27it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 61%|██████ | 20/33 [00:03<00:02, 5.21it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 64%|██████▎ | 21/33 [00:04<00:02, 5.17it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 67%|██████▋ | 22/33 [00:04<00:02, 5.06it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 70%|██████▉ | 23/33 [00:04<00:01, 5.15it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 73%|███████▎ | 24/33 [00:04<00:01, 5.23it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 76%|███████▌ | 25/33 [00:04<00:01, 5.28it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 79%|███████▉ | 26/33 [00:05<00:01, 5.38it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 82%|████████▏ | 27/33 [00:05<00:01, 5.29it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 85%|████████▍ | 28/33 [00:05<00:00, 5.19it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 88%|████████▊ | 29/33 [00:05<00:00, 5.16it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 91%|█████████ | 30/33 [00:05<00:00, 5.16it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 94%|█████████▍| 31/33 [00:06<00:00, 5.11it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 97%|█████████▋| 32/33 [00:06<00:00, 5.23it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 100%|██████████| 33/33 [00:06<00:00, 5.32it/s] Datapoints: 100%|██████████| 33/33 [00:06<00:00, 5.18it/s] CV Folds: 67%|██████▋ | 2/3 [00:23<00:11, 11.65s/it](4020, 28, 28) Epoch 1/10 1/126 [..............................] - ETA: 1:08 - loss: 2.3129 - accuracy: 0.0938 24/126 [====>.........................] - ETA: 0s - loss: 1.4718 - accuracy: 0.5026  47/126 [==========>...................] - ETA: 0s - loss: 1.2165 - accuracy: 0.5904 69/126 [===============>..............] - ETA: 0s - loss: 1.0876 - accuracy: 0.6322 92/126 [====================>.........] - ETA: 0s - loss: 0.9881 - accuracy: 0.6678 115/126 [==========================>...] - ETA: 0s - loss: 0.9221 - accuracy: 0.6894 126/126 [==============================] - 1s 2ms/step - loss: 0.8965 - accuracy: 0.6983 Epoch 2/10 1/126 [..............................] - ETA: 0s - loss: 0.4769 - accuracy: 0.8125 24/126 [====>.........................] - ETA: 0s - loss: 0.5971 - accuracy: 0.7930 47/126 [==========>...................] - ETA: 0s - loss: 0.5900 - accuracy: 0.7992 70/126 [===============>..............] - ETA: 0s - loss: 0.5994 - accuracy: 0.7929 93/126 [=====================>........] - ETA: 0s - loss: 0.5830 - accuracy: 0.8017 116/126 [==========================>...] - ETA: 0s - loss: 0.5656 - accuracy: 0.8060 126/126 [==============================] - 0s 2ms/step - loss: 0.5750 - accuracy: 0.8040 Epoch 3/10 1/126 [..............................] - ETA: 0s - loss: 0.6893 - accuracy: 0.7812 24/126 [====>.........................] - ETA: 0s - loss: 0.5626 - accuracy: 0.8060 47/126 [==========>...................] - ETA: 0s - loss: 0.5343 - accuracy: 0.8145 69/126 [===============>..............] - ETA: 0s - loss: 0.5346 - accuracy: 0.8134 91/126 [====================>.........] - ETA: 0s - loss: 0.5219 - accuracy: 0.8218 113/126 [=========================>....] - ETA: 0s - loss: 0.5104 - accuracy: 0.8252 126/126 [==============================] - 0s 2ms/step - loss: 0.5140 - accuracy: 0.8224 Epoch 4/10 1/126 [..............................] - ETA: 0s - loss: 0.4519 - accuracy: 0.9062 24/126 [====>.........................] - ETA: 0s - loss: 0.4256 - accuracy: 0.8581 47/126 [==========>...................] - ETA: 0s - loss: 0.4503 - accuracy: 0.8424 70/126 [===============>..............] - ETA: 0s - loss: 0.4453 - accuracy: 0.8415 93/126 [=====================>........] - ETA: 0s - loss: 0.4420 - accuracy: 0.8444 116/126 [==========================>...] - ETA: 0s - loss: 0.4631 - accuracy: 0.8384 126/126 [==============================] - 0s 2ms/step - loss: 0.4667 - accuracy: 0.8358 Epoch 5/10 1/126 [..............................] - ETA: 0s - loss: 0.4192 - accuracy: 0.8125 24/126 [====>.........................] - ETA: 0s - loss: 0.4185 - accuracy: 0.8633 47/126 [==========>...................] - ETA: 0s - loss: 0.4139 - accuracy: 0.8630 70/126 [===============>..............] - ETA: 0s - loss: 0.4153 - accuracy: 0.8643 93/126 [=====================>........] - ETA: 0s - loss: 0.4228 - accuracy: 0.8599 116/126 [==========================>...] - ETA: 0s - loss: 0.4171 - accuracy: 0.8610 126/126 [==============================] - 0s 2ms/step - loss: 0.4173 - accuracy: 0.8600 Epoch 6/10 1/126 [..............................] - ETA: 0s - loss: 0.8799 - accuracy: 0.7500 24/126 [====>.........................] - ETA: 0s - loss: 0.3761 - accuracy: 0.8672 47/126 [==========>...................] - ETA: 0s - loss: 0.3851 - accuracy: 0.8677 70/126 [===============>..............] - ETA: 0s - loss: 0.3909 - accuracy: 0.8661 93/126 [=====================>........] - ETA: 0s - loss: 0.3953 - accuracy: 0.8616 116/126 [==========================>...] - ETA: 0s - loss: 0.3941 - accuracy: 0.8631 126/126 [==============================] - 0s 2ms/step - loss: 0.3949 - accuracy: 0.8624 Epoch 7/10 1/126 [..............................] - ETA: 0s - loss: 0.2022 - accuracy: 0.9375 24/126 [====>.........................] - ETA: 0s - loss: 0.3764 - accuracy: 0.8776 47/126 [==========>...................] - ETA: 0s - loss: 0.3475 - accuracy: 0.8816 70/126 [===============>..............] - ETA: 0s - loss: 0.3481 - accuracy: 0.8781 93/126 [=====================>........] - ETA: 0s - loss: 0.3624 - accuracy: 0.8774 115/126 [==========================>...] - ETA: 0s - loss: 0.3652 - accuracy: 0.8764 126/126 [==============================] - 0s 2ms/step - loss: 0.3635 - accuracy: 0.8774 Epoch 8/10 1/126 [..............................] - ETA: 0s - loss: 0.3186 - accuracy: 0.8750 24/126 [====>.........................] - ETA: 0s - loss: 0.3329 - accuracy: 0.8841 47/126 [==========>...................] - ETA: 0s - loss: 0.3320 - accuracy: 0.8856 70/126 [===============>..............] - ETA: 0s - loss: 0.3330 - accuracy: 0.8862 93/126 [=====================>........] - ETA: 0s - loss: 0.3369 - accuracy: 0.8858 116/126 [==========================>...] - ETA: 0s - loss: 0.3402 - accuracy: 0.8825 126/126 [==============================] - 0s 2ms/step - loss: 0.3457 - accuracy: 0.8826 Epoch 9/10 1/126 [..............................] - ETA: 0s - loss: 0.2176 - accuracy: 0.9375 24/126 [====>.........................] - ETA: 0s - loss: 0.3504 - accuracy: 0.8828 47/126 [==========>...................] - ETA: 0s - loss: 0.3412 - accuracy: 0.8763 70/126 [===============>..............] - ETA: 0s - loss: 0.3244 - accuracy: 0.8844 93/126 [=====================>........] - ETA: 0s - loss: 0.3284 - accuracy: 0.8837 116/126 [==========================>...] - ETA: 0s - loss: 0.3302 - accuracy: 0.8885 126/126 [==============================] - 0s 2ms/step - loss: 0.3285 - accuracy: 0.8878 Epoch 10/10 1/126 [..............................] - ETA: 0s - loss: 0.3079 - accuracy: 0.8125 23/126 [====>.........................] - ETA: 0s - loss: 0.2855 - accuracy: 0.9049 46/126 [=========>....................] - ETA: 0s - loss: 0.2977 - accuracy: 0.9029 68/126 [===============>..............] - ETA: 0s - loss: 0.2887 - accuracy: 0.9053 90/126 [====================>.........] - ETA: 0s - loss: 0.2920 - accuracy: 0.9049 112/126 [=========================>....] - ETA: 0s - loss: 0.2976 - accuracy: 0.9004 126/126 [==============================] - 0s 2ms/step - loss: 0.3008 - accuracy: 0.8993 Datapoints: 0%| | 0/33 [00:00...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 3%|▎ | 1/33 [00:00<00:05, 5.61it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 6%|▌ | 2/33 [00:00<00:05, 5.60it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 9%|▉ | 3/33 [00:00<00:05, 5.55it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 12%|█▏ | 4/33 [00:00<00:05, 5.48it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 15%|█▌ | 5/33 [00:00<00:05, 5.52it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 18%|█▊ | 6/33 [00:01<00:04, 5.55it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 21%|██ | 7/33 [00:01<00:04, 5.35it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 24%|██▍ | 8/33 [00:01<00:04, 5.26it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 27%|██▋ | 9/33 [00:01<00:04, 5.31it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 30%|███ | 10/33 [00:01<00:04, 5.37it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 33%|███▎ | 11/33 [00:02<00:04, 5.45it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 36%|███▋ | 12/33 [00:02<00:03, 5.34it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 39%|███▉ | 13/33 [00:02<00:03, 5.24it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 42%|████▏ | 14/33 [00:02<00:03, 5.20it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 45%|████▌ | 15/33 [00:02<00:03, 5.15it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 48%|████▊ | 16/33 [00:03<00:03, 5.11it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 52%|█████▏ | 17/33 [00:03<00:03, 5.12it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 55%|█████▍ | 18/33 [00:03<00:02, 5.11it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 58%|█████▊ | 19/33 [00:03<00:02, 5.18it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 61%|██████ | 20/33 [00:03<00:02, 5.30it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 64%|██████▎ | 21/33 [00:03<00:02, 5.38it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 67%|██████▋ | 22/33 [00:04<00:02, 5.26it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 70%|██████▉ | 23/33 [00:04<00:01, 5.37it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 73%|███████▎ | 24/33 [00:04<00:01, 5.30it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 76%|███████▌ | 25/33 [00:04<00:01, 5.34it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 79%|███████▉ | 26/33 [00:04<00:01, 5.40it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 82%|████████▏ | 27/33 [00:05<00:01, 5.24it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 85%|████████▍ | 28/33 [00:05<00:00, 5.11it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 88%|████████▊ | 29/33 [00:05<00:00, 5.21it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 91%|█████████ | 30/33 [00:05<00:00, 5.30it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 94%|█████████▍| 31/33 [00:06<00:00, 3.23it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 97%|█████████▋| 32/33 [00:06<00:00, 3.62it/s] 1/2 [==============>...............] - ETA: 0s 2/2 [==============================] - 0s 2ms/step Datapoints: 100%|██████████| 33/33 [00:06<00:00, 4.00it/s] Datapoints: 100%|██████████| 33/33 [00:06<00:00, 4.97it/s] CV Folds: 100%|██████████| 3/3 [00:36<00:00, 12.34s/it] CV Folds: 100%|██████████| 3/3 [00:36<00:00, 12.30s/it] .. GENERATED FROM PYTHON SOURCE LINES 263-264 We can now look at the results per group: .. GENERATED FROM PYTHON SOURCE LINES 264-266 .. code-block:: default cv_results["test_single_accuracy"] .. rst-class:: sphx-glr-script-out .. code-block:: none [[0.85, 0.8666666666666667, 0.9333333333333333, 0.8, 0.85, 0.85, 0.85, 0.8833333333333333, 0.85, 0.85, 0.8166666666666667, 0.8666666666666667, 0.9333333333333333, 0.8, 0.8833333333333333, 0.8, 0.8, 0.8833333333333333, 0.7833333333333333, 0.8166666666666667, 0.8, 0.85, 0.8, 0.95, 0.8, 0.8833333333333333, 0.8833333333333333, 0.85, 0.85, 0.7833333333333333, 0.75, 0.8666666666666667, 0.8333333333333334, 0.9333333333333333], [0.7833333333333333, 0.8166666666666667, 0.7666666666666667, 0.7833333333333333, 0.85, 0.8, 0.8, 0.8166666666666667, 0.8, 0.8333333333333334, 0.8333333333333334, 0.8, 0.8166666666666667, 0.85, 0.75, 0.7, 0.8333333333333334, 0.8166666666666667, 0.8666666666666667, 0.7666666666666667, 0.7833333333333333, 0.7833333333333333, 0.7666666666666667, 0.8833333333333333, 0.8333333333333334, 0.7833333333333333, 0.8, 0.85, 0.85, 0.8166666666666667, 0.7666666666666667, 0.8, 0.8833333333333333], [0.7666666666666667, 0.9166666666666666, 0.8, 0.8, 0.7833333333333333, 0.85, 0.8833333333333333, 0.85, 0.9, 0.8833333333333333, 0.9, 0.85, 0.8833333333333333, 0.8166666666666667, 0.8333333333333334, 0.8333333333333334, 0.85, 0.7833333333333333, 0.7166666666666667, 0.7333333333333333, 0.75, 0.7833333333333333, 0.85, 0.7666666666666667, 0.7833333333333333, 0.9, 0.8666666666666667, 0.8333333333333334, 0.8333333333333334, 0.8166666666666667, 0.85, 0.85, 0.9]] .. GENERATED FROM PYTHON SOURCE LINES 267-268 And the overall accuracy as the average over all samples of all groups within a fold: .. GENERATED FROM PYTHON SOURCE LINES 268-269 .. code-block:: default cv_results["test_per_sample__accuracy"] .. rst-class:: sphx-glr-script-out .. code-block:: none array([0.84705882, 0.80858586, 0.83080808]) .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 41.496 seconds) **Estimated memory usage:** 284 MB .. _sphx_glr_download_auto_examples_integrations__01_tensorflow.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: _01_tensorflow.py <_01_tensorflow.py>` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: _01_tensorflow.ipynb <_01_tensorflow.ipynb>` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_