Demonstration of Pipeline for Training/Evaluation Landmark Detection (ISBI 2015 Cephalometric X-ray Dataset)#
In this tutorial, we will demonstrate the use of the Pipline Module in landmarker library, which
allows for training and evaluating landmark detection algorithms in a few lines of code. We will use
the ISBI 2015 Cephalometric X-ray dataset to train and evaluate the SpatialConfiguration model.
The ISBI 2015 Cephalometric X-ray dataset is a dataset of 2D cephalometric X-rays. The dataset
contains 400 images, each with 19 landmarks annotated. The dataset is split into a training set of
150 images and two test sets of 150 and 100 images respectively.
# import everyting from data module which is located ../src/data.py
import sys
sys.path.append('../src')
# !python -c "import landmarker" || pip install landmarker
Downloading the dataset#
The landmarker library provides a utility function to download the ISBI 2015 Cephalometric X-ray
dataset. The dataset will be downloaded to the data folder in the current working directory.
from landmarker.datasets.cepha400 import get_cepha_dataset
path_dir = "data/"
(image_paths_train, image_paths_test1, image_paths_test2,
landmarks_train, landmarks_test1, landmarks_test2,
pixel_spacings_train, pixel_spacings_test1, pixel_spacings_test2,
) = get_cepha_dataset(path_dir=path_dir, junior=False, cv=False)
Pipelines in landmarker#
The landmarker library provides a Pipeline class that allows for training and evaluating landmark
detection algorithms in a few lines of code. The Pipeline class is a wrapper around different
components of the library. There are five main types of pipelines, which are distinguished by our
own categorizion of landmark detection algorithms. See the figure below for categorization of
landmark detection algorithms.
We can divide landmark detection algorithms into two main categories: Heatmap-based and Coordinate-based. Heatmap-based algorithms predict a heatmap for each landmark, this heatmap is than decoded to obtain the landmark location. Coordinate-based algorithms directly predict the landmark location. Heatmap-based algorithms can be further divided into Direct and Indirect approaches. In direct heatmap-based approaches, a heatmap is generated for each landmark following a probability distrubion (e.g., the bivariate gaussian distribution), this heatmap is than used as target for the network. In indirect heatmap-based approaches, the model predicts an intermediate heatmap, which is than decoded (by a differentiable function) to obtain the landmarks. The Direct Heatmap-based algorithms can be further divided into Static and Adaptive approaches. In static heatmap-based approaches, the heatmap is generated using a fixed probability distribution, i.e., the parameters of the probability distribution are constant the entire learning procdeure. In adaptive heatmap-based approaches, the parameters of the probability distribution are learned or are adapted by a scheduler algorithm. The Indirect Heatmap-based algorithms can be further divided into Uncertainty aware and Uncertainty unaware approaches. In Uncertainty aware approaches, the algorithm decodes from the heamtmap the uncertainty besides the landmark location. In Uncertainty unaware approaches, the algorithm only decodes the landmark location from the heatmap.
The landmarker library provides a Pipeline class for each of the five categories of landmark
detection algorithms. Each Pipeline class has a fit, predict, and evaluate method.
The fit method is used to train the model, the predict method is used to predict landmarks on
new images, and the evaluate method is used to evaluate the model on a test set and give additional
information about the model performance.
The five Pipeline classes are:
StaticHeatmapRegressionPipeline: Static Heatmap-based algorithmsAdaptiveHeatmapRegressionPipeline: Adaptive Heatmap-based algorithmsIndirectUncertaintyAwareHeatmapRegressionPipeline: Uncertainty aware Indirect Heatmap-based algorithmsIndirectUncertaintyUnawareHeatmapRegressionPipeline: Uncertainty unaware Indirect Heatmap-based algorithmsCoordinateRegressionPipeline: Coordinate-based algorithms
import torch
from landmarker.heatmap.decoder import coord_argmax
from monai.transforms import (Compose, RandAffined, RandGaussianNoised, RandStdShiftIntensityd,
RandScaleIntensityd, RandAdjustContrastd, RandHistogramShiftd)
import numpy as np
device = 'cuda' if torch.cuda.is_available() else 'cpu'
epochs = 1000
lr = 0.0001
batch_size = 4
cache_ds = True
dim_img = (512, 512)
Static Heatmap Regression#
from landmarker.pipeline import StaticHeatmapRegressionPipeline
train_config = {
"criterion": "cross_entropy",
"optimizer": "adam",
"lr": lr,
"batch_size": batch_size,
"epochs": epochs,
"lr_scheduler": True,
}
staticHeatmapRegressionPipeline = StaticHeatmapRegressionPipeline(
sigmas=3.0, model="ProbSpatialConfigurationNet", in_channels_img=1, nb_landmarks=19, train_config=train_config,
heatmap_decoder=coord_argmax, verbose=True, device=device)
fn_keys = ('image', 'seg')
spatial_transformd = [RandAffined(fn_keys, prob=1,
rotate_range=(-np.pi/12, np.pi/12),
translate_range=(-10, 10),
scale_range=(-0.1, 0.1),
shear_range=(-0.1, 0.1)
)]
composed_transformd = Compose(spatial_transformd +
[RandGaussianNoised(('image', ), prob=0.1, mean=0, std=0.005), # Add gaussian noise
RandStdShiftIntensityd(('image', ), prob=0.1, factors=0.1), # Add random intensity shift
RandScaleIntensityd(('image', ), factors=0.25, prob=0.1), # Add random intensity scaling
RandAdjustContrastd(('image', ), prob=0.1, gamma=(0.5,2)), # Randomly adjust contrast
RandHistogramShiftd(('image', ), prob=0.1), # Randomly shift histogram
])
staticHeatmapRegressionPipeline.fit(image_paths_train, landmarks_train,
transform=composed_transformd, cache_ds=cache_ds)
Reading, normalizing and resizing 150 images and landmarks...
0%| | 0/150 [00:00<?, ?it/s]/opt/conda/lib/python3.11/site-packages/torchvision/transforms/functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
warnings.warn(
100%|██████████| 150/150 [00:03<00:00, 44.88it/s]
Creating heatmaps for 150 images... (batch size: 32)
100%|██████████| 4/4 [00:19<00:00, 4.90s/it]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.28it/s]
Epoch 1/1000 - Train loss: 0.48456608 - Val loss: 0.07820681
100%|██████████| 30/30 [00:14<00:00, 2.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.00it/s]
Epoch 2/1000 - Train loss: 0.06119466 - Val loss: 0.04535827
INFO: Val_Loss improved (0.07820681 --> 0.04535827). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.64it/s]
Epoch 3/1000 - Train loss: 0.04011767 - Val loss: 0.03184384
INFO: Val_Loss improved (0.04535827 --> 0.03184384). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 11.75it/s]
Epoch 4/1000 - Train loss: 0.03116542 - Val loss: 0.02627548
INFO: Val_Loss improved (0.03184384 --> 0.02627548). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.20it/s]
Epoch 5/1000 - Train loss: 0.02801667 - Val loss: 0.02402726
INFO: Val_Loss improved (0.02627548 --> 0.02402726). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.16it/s]
Epoch 6/1000 - Train loss: 0.02563009 - Val loss: 0.02298292
INFO: Val_Loss improved (0.02402726 --> 0.02298292). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 11.74it/s]
Epoch 7/1000 - Train loss: 0.02558527 - Val loss: 0.02249208
INFO: Val_Loss improved (0.02298292 --> 0.02249208). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.09it/s]
Epoch 8/1000 - Train loss: 0.02278032 - Val loss: 0.02075039
INFO: Val_Loss improved (0.02249208 --> 0.02075039). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.18it/s]
Epoch 9/1000 - Train loss: 0.02396483 - Val loss: 0.02115519
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.02075039, Val Loss: 0.02115519]
100%|██████████| 30/30 [00:14<00:00, 2.10it/s]
100%|██████████| 8/8 [00:00<00:00, 15.19it/s]
Epoch 10/1000 - Train loss: 0.02247344 - Val loss: 0.01972099
INFO: Val_Loss improved (0.02075039 --> 0.01972099). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.19it/s]
Epoch 11/1000 - Train loss: 0.02023788 - Val loss: 0.01821169
INFO: Val_Loss improved (0.01972099 --> 0.01821169). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.20it/s]
Epoch 12/1000 - Train loss: 0.01985775 - Val loss: 0.01788033
INFO: Val_Loss improved (0.01821169 --> 0.01788033). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.86it/s]
Epoch 13/1000 - Train loss: 0.01908606 - Val loss: 0.01773746
INFO: Val_Loss improved (0.01788033 --> 0.01773746). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.76it/s]
Epoch 14/1000 - Train loss: 0.01770712 - Val loss: 0.01606670
INFO: Val_Loss improved (0.01773746 --> 0.01606670). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.56it/s]
Epoch 15/1000 - Train loss: 0.01755407 - Val loss: 0.01699400
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01606670, Val Loss: 0.01699400]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.24it/s]
Epoch 16/1000 - Train loss: 0.01730135 - Val loss: 0.01563357
INFO: Val_Loss improved (0.01606670 --> 0.01563357). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.11it/s]
Epoch 17/1000 - Train loss: 0.01651349 - Val loss: 0.01468884
INFO: Val_Loss improved (0.01563357 --> 0.01468884). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.99it/s]
Epoch 18/1000 - Train loss: 0.01547971 - Val loss: 0.01435053
INFO: Val_Loss improved (0.01468884 --> 0.01435053). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 15.07it/s]
Epoch 19/1000 - Train loss: 0.01581260 - Val loss: 0.01384214
INFO: Val_Loss improved (0.01435053 --> 0.01384214). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.24it/s]
Epoch 20/1000 - Train loss: 0.01496048 - Val loss: 0.01364034
INFO: Val_Loss improved (0.01384214 --> 0.01364034). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.12it/s]
Epoch 21/1000 - Train loss: 0.01448723 - Val loss: 0.01320895
INFO: Val_Loss improved (0.01364034 --> 0.01320895). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.07it/s]
Epoch 22/1000 - Train loss: 0.01408443 - Val loss: 0.01307917
INFO: Val_Loss improved (0.01320895 --> 0.01307917). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.12it/s]
Epoch 23/1000 - Train loss: 0.01417712 - Val loss: 0.01315840
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01307917, Val Loss: 0.01315840]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.07it/s]
Epoch 24/1000 - Train loss: 0.01374987 - Val loss: 0.01357075
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01307917, Val Loss: 0.01357075]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.18it/s]
Epoch 25/1000 - Train loss: 0.01382210 - Val loss: 0.01278199
INFO: Val_Loss improved (0.01307917 --> 0.01278199). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.82it/s]
Epoch 26/1000 - Train loss: 0.01328331 - Val loss: 0.01363062
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01278199, Val Loss: 0.01363062]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 13.02it/s]
Epoch 27/1000 - Train loss: 0.01309800 - Val loss: 0.01254932
INFO: Val_Loss improved (0.01278199 --> 0.01254932). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.67it/s]
Epoch 28/1000 - Train loss: 0.01313648 - Val loss: 0.01185074
INFO: Val_Loss improved (0.01254932 --> 0.01185074). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.16it/s]
Epoch 29/1000 - Train loss: 0.01284352 - Val loss: 0.01273465
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01185074, Val Loss: 0.01273465]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.74it/s]
Epoch 30/1000 - Train loss: 0.01283666 - Val loss: 0.01191444
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01185074, Val Loss: 0.01191444]
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.16it/s]
Epoch 31/1000 - Train loss: 0.01240972 - Val loss: 0.01151623
INFO: Val_Loss improved (0.01185074 --> 0.01151623). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.17it/s]
Epoch 32/1000 - Train loss: 0.01237445 - Val loss: 0.01144978
INFO: Val_Loss improved (0.01151623 --> 0.01144978). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 11.43it/s]
Epoch 33/1000 - Train loss: 0.01221325 - Val loss: 0.01126695
INFO: Val_Loss improved (0.01144978 --> 0.01126695). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.71it/s]
Epoch 34/1000 - Train loss: 0.01202571 - Val loss: 0.01173629
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01126695, Val Loss: 0.01173629]
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.81it/s]
Epoch 35/1000 - Train loss: 0.01226049 - Val loss: 0.01173733
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01126695, Val Loss: 0.01173733]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 11.94it/s]
Epoch 36/1000 - Train loss: 0.01216949 - Val loss: 0.01171309
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.01126695, Val Loss: 0.01171309]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.14it/s]
Epoch 37/1000 - Train loss: 0.01187495 - Val loss: 0.01150904
INFO: Early stopping counter 4 of 20 [Best Val Loss: 0.01126695, Val Loss: 0.01150904]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.01it/s]
Epoch 38/1000 - Train loss: 0.01159003 - Val loss: 0.01118320
INFO: Val_Loss improved (0.01126695 --> 0.01118320). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.19it/s]
Epoch 39/1000 - Train loss: 0.01154289 - Val loss: 0.01159309
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01118320, Val Loss: 0.01159309]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.05it/s]
Epoch 40/1000 - Train loss: 0.01180986 - Val loss: 0.01124836
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01118320, Val Loss: 0.01124836]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.89it/s]
Epoch 41/1000 - Train loss: 0.01144714 - Val loss: 0.01101789
INFO: Val_Loss improved (0.01118320 --> 0.01101789). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.20it/s]
Epoch 42/1000 - Train loss: 0.01138962 - Val loss: 0.01070085
INFO: Val_Loss improved (0.01101789 --> 0.01070085). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.86it/s]
Epoch 43/1000 - Train loss: 0.01122832 - Val loss: 0.01081460
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01070085, Val Loss: 0.01081460]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.80it/s]
Epoch 44/1000 - Train loss: 0.01134688 - Val loss: 0.01078831
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01070085, Val Loss: 0.01078831]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.72it/s]
Epoch 45/1000 - Train loss: 0.01126763 - Val loss: 0.01073931
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.01070085, Val Loss: 0.01073931]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 11.58it/s]
Epoch 46/1000 - Train loss: 0.01099794 - Val loss: 0.01052524
INFO: Val_Loss improved (0.01070085 --> 0.01052524). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.79it/s]
Epoch 47/1000 - Train loss: 0.01091585 - Val loss: 0.01084511
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01052524, Val Loss: 0.01084511]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.28it/s]
Epoch 48/1000 - Train loss: 0.01087643 - Val loss: 0.01071133
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01052524, Val Loss: 0.01071133]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.00it/s]
Epoch 49/1000 - Train loss: 0.01087959 - Val loss: 0.01118732
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.01052524, Val Loss: 0.01118732]
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.78it/s]
Epoch 50/1000 - Train loss: 0.01087003 - Val loss: 0.01067984
INFO: Early stopping counter 4 of 20 [Best Val Loss: 0.01052524, Val Loss: 0.01067984]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.89it/s]
Epoch 51/1000 - Train loss: 0.01087018 - Val loss: 0.01067530
INFO: Early stopping counter 5 of 20 [Best Val Loss: 0.01052524, Val Loss: 0.01067530]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.80it/s]
Epoch 52/1000 - Train loss: 0.01075756 - Val loss: 0.01051296
INFO: Val_Loss improved (0.01052524 --> 0.01051296). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.89it/s]
Epoch 53/1000 - Train loss: 0.01055805 - Val loss: 0.01061273
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01051296, Val Loss: 0.01061273]
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 11.52it/s]
Epoch 54/1000 - Train loss: 0.01055324 - Val loss: 0.01046159
INFO: Val_Loss improved (0.01051296 --> 0.01046159). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.15it/s]
Epoch 55/1000 - Train loss: 0.01065379 - Val loss: 0.01053972
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01046159, Val Loss: 0.01053972]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.86it/s]
Epoch 56/1000 - Train loss: 0.01060829 - Val loss: 0.01046817
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01046159, Val Loss: 0.01046817]
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 14.70it/s]
Epoch 57/1000 - Train loss: 0.01055616 - Val loss: 0.01094156
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.01046159, Val Loss: 0.01094156]
100%|██████████| 30/30 [00:14<00:00, 2.01it/s]
100%|██████████| 8/8 [00:00<00:00, 14.44it/s]
Epoch 58/1000 - Train loss: 0.01062112 - Val loss: 0.01039732
INFO: Val_Loss improved (0.01046159 --> 0.01039732). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.04it/s]
Epoch 59/1000 - Train loss: 0.01041449 - Val loss: 0.01037637
INFO: Val_Loss improved (0.01039732 --> 0.01037637). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.59it/s]
Epoch 60/1000 - Train loss: 0.01037190 - Val loss: 0.01039623
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01037637, Val Loss: 0.01039623]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.87it/s]
Epoch 61/1000 - Train loss: 0.01044462 - Val loss: 0.01073627
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01037637, Val Loss: 0.01073627]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 12.35it/s]
Epoch 62/1000 - Train loss: 0.01018149 - Val loss: 0.01039113
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.01037637, Val Loss: 0.01039113]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.87it/s]
Epoch 63/1000 - Train loss: 0.01021082 - Val loss: 0.01021087
INFO: Val_Loss improved (0.01037637 --> 0.01021087). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.88it/s]
Epoch 64/1000 - Train loss: 0.01037503 - Val loss: 0.01026311
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01021087, Val Loss: 0.01026311]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 11.83it/s]
Epoch 65/1000 - Train loss: 0.01032077 - Val loss: 0.01028056
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01021087, Val Loss: 0.01028056]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.12it/s]
Epoch 66/1000 - Train loss: 0.01021410 - Val loss: 0.01017674
INFO: Val_Loss improved (0.01021087 --> 0.01017674). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.83it/s]
Epoch 67/1000 - Train loss: 0.01008175 - Val loss: 0.01016441
INFO: Val_Loss improved (0.01017674 --> 0.01016441). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 14.82it/s]
Epoch 68/1000 - Train loss: 0.01004059 - Val loss: 0.01012874
INFO: Val_Loss improved (0.01016441 --> 0.01012874). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 14.86it/s]
Epoch 69/1000 - Train loss: 0.01014923 - Val loss: 0.01023215
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01012874, Val Loss: 0.01023215]
100%|██████████| 30/30 [00:14<00:00, 2.10it/s]
100%|██████████| 8/8 [00:00<00:00, 12.83it/s]
Epoch 70/1000 - Train loss: 0.01018373 - Val loss: 0.01011526
INFO: Val_Loss improved (0.01012874 --> 0.01011526). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.11it/s]
Epoch 71/1000 - Train loss: 0.00996894 - Val loss: 0.01010693
INFO: Val_Loss improved (0.01011526 --> 0.01010693). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.14it/s]
Epoch 72/1000 - Train loss: 0.00993563 - Val loss: 0.01030059
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01030059]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 11.53it/s]
Epoch 73/1000 - Train loss: 0.01014246 - Val loss: 0.01027180
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01027180]
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.12it/s]
Epoch 74/1000 - Train loss: 0.01009416 - Val loss: 0.01022595
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01022595]
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.89it/s]
Epoch 75/1000 - Train loss: 0.00987914 - Val loss: 0.01031398
INFO: Early stopping counter 4 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01031398]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 11.12it/s]
Epoch 76/1000 - Train loss: 0.00990222 - Val loss: 0.01020577
INFO: Early stopping counter 5 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01020577]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.11it/s]
Epoch 77/1000 - Train loss: 0.00980769 - Val loss: 0.01022438
INFO: Early stopping counter 6 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01022438]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.85it/s]
Epoch 78/1000 - Train loss: 0.00987293 - Val loss: 0.01018776
INFO: Early stopping counter 7 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01018776]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.90it/s]
Epoch 79/1000 - Train loss: 0.00991806 - Val loss: 0.01011349
INFO: Early stopping counter 8 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01011349]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.79it/s]
Epoch 80/1000 - Train loss: 0.00996851 - Val loss: 0.01016685
INFO: Early stopping counter 9 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01016685]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 11.91it/s]
Epoch 81/1000 - Train loss: 0.00977897 - Val loss: 0.01013217
INFO: Early stopping counter 10 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01013217]
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 15.00it/s]
Epoch 82/1000 - Train loss: 0.00975014 - Val loss: 0.01028129
INFO: Early stopping counter 11 of 20 [Best Val Loss: 0.01010693, Val Loss: 0.01028129]
Epoch 00082: reducing learning rate of group 0 to 1.0000e-05.
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.11it/s]
Epoch 83/1000 - Train loss: 0.00949251 - Val loss: 0.00985738
INFO: Val_Loss improved (0.01010693 --> 0.00985738). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.00it/s]
Epoch 84/1000 - Train loss: 0.00938011 - Val loss: 0.00983365
INFO: Val_Loss improved (0.00985738 --> 0.00983365). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.63it/s]
Epoch 85/1000 - Train loss: 0.00940332 - Val loss: 0.00981839
INFO: Val_Loss improved (0.00983365 --> 0.00981839). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.68it/s]
Epoch 86/1000 - Train loss: 0.00922132 - Val loss: 0.00983816
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.00981839, Val Loss: 0.00983816]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.79it/s]
Epoch 87/1000 - Train loss: 0.00924544 - Val loss: 0.00985642
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.00981839, Val Loss: 0.00985642]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.14it/s]
Epoch 88/1000 - Train loss: 0.00927064 - Val loss: 0.00982420
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.00981839, Val Loss: 0.00982420]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 10.92it/s]
Epoch 89/1000 - Train loss: 0.00933471 - Val loss: 0.00980317
INFO: Val_Loss improved (0.00981839 --> 0.00980317). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.17it/s]
Epoch 90/1000 - Train loss: 0.00911487 - Val loss: 0.00981695
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.00980317, Val Loss: 0.00981695]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.16it/s]
Epoch 91/1000 - Train loss: 0.00932538 - Val loss: 0.00980277
INFO: Val_Loss improved (0.00980317 --> 0.00980277). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.11it/s]
100%|██████████| 8/8 [00:00<00:00, 14.76it/s]
Epoch 92/1000 - Train loss: 0.00927360 - Val loss: 0.00981150
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.00980277, Val Loss: 0.00981150]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.58it/s]
Epoch 93/1000 - Train loss: 0.00924085 - Val loss: 0.00978998
INFO: Val_Loss improved (0.00980277 --> 0.00978998). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.15it/s]
Epoch 94/1000 - Train loss: 0.00904858 - Val loss: 0.00980902
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.00978998, Val Loss: 0.00980902]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.05it/s]
Epoch 95/1000 - Train loss: 0.00901913 - Val loss: 0.00984321
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.00978998, Val Loss: 0.00984321]
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 15.10it/s]
Epoch 96/1000 - Train loss: 0.00919902 - Val loss: 0.00983568
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.00978998, Val Loss: 0.00983568]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.88it/s]
Epoch 97/1000 - Train loss: 0.00925518 - Val loss: 0.00982571
INFO: Early stopping counter 4 of 20 [Best Val Loss: 0.00978998, Val Loss: 0.00982571]
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.74it/s]
Epoch 98/1000 - Train loss: 0.00906396 - Val loss: 0.00984604
INFO: Early stopping counter 5 of 20 [Best Val Loss: 0.00978998, Val Loss: 0.00984604]
100%|██████████| 30/30 [00:14<00:00, 2.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.89it/s]
Epoch 99/1000 - Train loss: 0.00920320 - Val loss: 0.00981750
INFO: Early stopping counter 6 of 20 [Best Val Loss: 0.00978998, Val Loss: 0.00981750]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 14.82it/s]
Epoch 100/1000 - Train loss: 0.00902839 - Val loss: 0.00981296
INFO: Early stopping counter 7 of 20 [Best Val Loss: 0.00978998, Val Loss: 0.00981296]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.82it/s]
Epoch 101/1000 - Train loss: 0.00931925 - Val loss: 0.00978774
INFO: Val_Loss improved (0.00978998 --> 0.00978774). Saving model ...
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 11.74it/s]
Epoch 102/1000 - Train loss: 0.00907969 - Val loss: 0.00985042
INFO: Early stopping counter 1 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00985042]
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 14.73it/s]
Epoch 103/1000 - Train loss: 0.00910060 - Val loss: 0.00983125
INFO: Early stopping counter 2 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00983125]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.06it/s]
Epoch 104/1000 - Train loss: 0.00925249 - Val loss: 0.00981529
INFO: Early stopping counter 3 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00981529]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.89it/s]
Epoch 105/1000 - Train loss: 0.00917958 - Val loss: 0.00984119
INFO: Early stopping counter 4 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984119]
100%|██████████| 30/30 [00:14<00:00, 2.01it/s]
100%|██████████| 8/8 [00:00<00:00, 14.58it/s]
Epoch 106/1000 - Train loss: 0.00904605 - Val loss: 0.00982259
INFO: Early stopping counter 5 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00982259]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 11.91it/s]
Epoch 107/1000 - Train loss: 0.00909474 - Val loss: 0.00983578
INFO: Early stopping counter 6 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00983578]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.97it/s]
Epoch 108/1000 - Train loss: 0.00902998 - Val loss: 0.00984329
INFO: Early stopping counter 7 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984329]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.01it/s]
Epoch 109/1000 - Train loss: 0.00898055 - Val loss: 0.00984011
INFO: Early stopping counter 8 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984011]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.88it/s]
Epoch 110/1000 - Train loss: 0.00895875 - Val loss: 0.00985817
INFO: Early stopping counter 9 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00985817]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.17it/s]
Epoch 111/1000 - Train loss: 0.00907765 - Val loss: 0.00984268
INFO: Early stopping counter 10 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984268]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 12.55it/s]
Epoch 112/1000 - Train loss: 0.00907301 - Val loss: 0.00987383
INFO: Early stopping counter 11 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00987383]
Epoch 00112: reducing learning rate of group 0 to 1.0000e-06.
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.81it/s]
Epoch 113/1000 - Train loss: 0.00904370 - Val loss: 0.00985202
INFO: Early stopping counter 12 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00985202]
100%|██████████| 30/30 [00:14<00:00, 2.08it/s]
100%|██████████| 8/8 [00:00<00:00, 14.08it/s]
Epoch 114/1000 - Train loss: 0.00906327 - Val loss: 0.00984335
INFO: Early stopping counter 13 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984335]
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 14.64it/s]
Epoch 115/1000 - Train loss: 0.00902286 - Val loss: 0.00983205
INFO: Early stopping counter 14 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00983205]
100%|██████████| 30/30 [00:14<00:00, 2.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.07it/s]
Epoch 116/1000 - Train loss: 0.00887368 - Val loss: 0.00984072
INFO: Early stopping counter 15 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984072]
100%|██████████| 30/30 [00:14<00:00, 2.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.15it/s]
Epoch 117/1000 - Train loss: 0.00893007 - Val loss: 0.00983728
INFO: Early stopping counter 16 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00983728]
100%|██████████| 30/30 [00:14<00:00, 2.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.12it/s]
Epoch 118/1000 - Train loss: 0.00899721 - Val loss: 0.00983874
INFO: Early stopping counter 17 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00983874]
100%|██████████| 30/30 [00:14<00:00, 2.11it/s]
100%|██████████| 8/8 [00:00<00:00, 11.48it/s]
Epoch 119/1000 - Train loss: 0.00886878 - Val loss: 0.00983945
INFO: Early stopping counter 18 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00983945]
100%|██████████| 30/30 [00:14<00:00, 2.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.05it/s]
Epoch 120/1000 - Train loss: 0.00896552 - Val loss: 0.00984456
INFO: Early stopping counter 19 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984456]
100%|██████████| 30/30 [00:14<00:00, 2.09it/s]
100%|██████████| 8/8 [00:00<00:00, 13.23it/s]
Epoch 121/1000 - Train loss: 0.00898887 - Val loss: 0.00984438
INFO: Early stopping counter 20 of 20 [Best Val Loss: 0.00978774, Val Loss: 0.00984438]
INFO: Early stopping DONE!
INFO: Best Val Loss: 0.00978774
Loading best model...
results_static = staticHeatmapRegressionPipeline.evaluate(image_paths_test1, landmarks_test1, pixel_spacings_test1)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:23<00:00, 6.30it/s]
100%|██████████| 150/150 [00:02<00:00, 58.71it/s]
Detection report:
1# Point-to-point error (PE) statistics:
======================================================================
Class Mean Median Std Min Max
----------------------------------------------------------------------
Landmark 0 1.15 1.02 1.00 0.02 9.68
Landmark 1 1.98 1.28 1.94 0.13 10.14
Landmark 2 2.09 1.75 1.46 0.33 8.32
Landmark 3 2.88 1.76 2.53 0.13 11.37
Landmark 4 2.52 2.07 1.75 0.20 10.08
Landmark 5 1.73 1.35 1.30 0.10 6.46
Landmark 6 1.28 1.12 0.83 0.06 4.11
Landmark 7 1.12 0.88 0.90 0.06 6.62
Landmark 8 1.07 0.87 0.84 0.05 8.02
Landmark 9 3.88 3.78 2.22 0.29 10.38
Landmark 10 1.31 1.11 0.97 0.03 6.26
Landmark 11 1.07 0.85 0.94 0.02 8.45
Landmark 12 1.08 0.95 0.65 0.05 3.19
Landmark 13 1.01 0.93 0.59 0.02 3.20
Landmark 14 1.46 1.26 1.07 0.04 6.13
Landmark 15 1.85 1.53 1.39 0.13 7.99
Landmark 16 1.26 1.05 0.82 0.10 4.47
Landmark 17 2.11 1.67 1.80 0.06 13.14
Landmark 18 2.76 1.86 2.36 0.03 10.23
======================================================================
2# Success detection rate (SDR):
================================================================================
Class SDR (PE≤2mm) SDR (PE≤2.5mm) SDR (PE≤3mm) SDR (PE≤4mm)
--------------------------------------------------------------------------------
Landmark 0 89.33 96.00 97.33 98.67
Landmark 1 68.67 74.00 80.67 89.33
Landmark 2 58.67 68.67 81.33 90.67
Landmark 3 56.00 58.67 62.67 72.00
Landmark 4 47.33 60.00 69.33 84.00
Landmark 5 71.33 81.33 85.33 92.00
Landmark 6 83.33 90.67 95.33 99.33
Landmark 7 86.00 90.67 96.67 98.67
Landmark 8 90.67 97.33 98.67 99.33
Landmark 9 24.67 31.33 38.00 55.33
Landmark 10 84.67 92.00 94.67 98.00
Landmark 11 88.67 95.33 98.00 98.67
Landmark 12 91.33 95.33 97.33 100.00
Landmark 13 94.67 98.00 99.33 100.00
Landmark 14 79.33 88.00 91.33 97.33
Landmark 15 68.00 76.00 83.33 93.33
Landmark 16 82.67 90.00 96.00 98.67
Landmark 17 61.33 74.67 79.33 87.33
Landmark 18 53.33 59.33 62.00 75.33
================================================================================
Adaptive Heatmap Regression#
from landmarker.pipeline import AdaptiveHeatmapRegressionPipeline
train_config = {
"criterion": "l2",
"optimizer": "sgd",
"lr": 1e-6,
"batch_size": 1,
"epochs": epochs,
"lr_scheduler": True,
}
fn_keys = ('image', )
spatial_transformd = [RandAffined(fn_keys, prob=1,
rotate_range=(-np.pi/12, np.pi/12),
translate_range=(-10, 10),
scale_range=(-0.1, 0.1),
shear_range=(-0.1, 0.1)
)]
composed_transformd = Compose(spatial_transformd +
[RandGaussianNoised(('image', ), prob=0.1, mean=0, std=0.005), # Add gaussian noise
RandStdShiftIntensityd(('image', ), prob=0.1, factors=0.1), # Add random intensity shift
RandScaleIntensityd(('image', ), factors=0.25, prob=0.1), # Add random intensity scaling
RandAdjustContrastd(('image', ), prob=0.1, gamma=(0.5,2)), # Randomly adjust contrast
RandHistogramShiftd(('image', ), prob=0.1), # Randomly shift histogram
])
adaptiveHeatmapRegressionPipeline = AdaptiveHeatmapRegressionPipeline(
sigmas=3.0, alpha=5.0, model="OriginalSpatialConfigurationNet", in_channels_img=1, nb_landmarks=19,
train_config=train_config, heatmap_decoder=coord_argmax, verbose=True, device=device, dim_img=dim_img,
gamma=100
)
adaptiveHeatmapRegressionPipeline.fit(image_paths_train, landmarks_train,
transform=composed_transformd, cache_ds=cache_ds)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:03<00:00, 46.21it/s]
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 25.07it/s]
Epoch 1/1000 - Train loss: 2520.17437948 - Val loss: 2497.31059570
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 22.60it/s]
Epoch 2/1000 - Train loss: 2471.79089559 - Val loss: 2446.74260254
INFO: Val_Loss improved (2497.31059570 --> 2446.74260254). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 24.92it/s]
Epoch 3/1000 - Train loss: 2425.61185303 - Val loss: 2398.70942383
INFO: Val_Loss improved (2446.74260254 --> 2398.70942383). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 22.49it/s]
Epoch 4/1000 - Train loss: 2377.50608724 - Val loss: 2335.66014811
INFO: Val_Loss improved (2398.70942383 --> 2335.66014811). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 24.90it/s]
Epoch 5/1000 - Train loss: 2318.95544637 - Val loss: 2269.54754232
INFO: Val_Loss improved (2335.66014811 --> 2269.54754232). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 22.17it/s]
Epoch 6/1000 - Train loss: 2267.15118612 - Val loss: 2230.64373372
INFO: Val_Loss improved (2269.54754232 --> 2230.64373372). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.18it/s]
100%|██████████| 30/30 [00:01<00:00, 24.94it/s]
Epoch 7/1000 - Train loss: 2230.22536214 - Val loss: 2171.42314453
INFO: Val_Loss improved (2230.64373372 --> 2171.42314453). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.16it/s]
100%|██████████| 30/30 [00:01<00:00, 22.43it/s]
Epoch 8/1000 - Train loss: 2189.79413656 - Val loss: 2145.96385905
INFO: Val_Loss improved (2171.42314453 --> 2145.96385905). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 25.16it/s]
Epoch 9/1000 - Train loss: 2191.05345662 - Val loss: 2118.52312419
INFO: Val_Loss improved (2145.96385905 --> 2118.52312419). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.18it/s]
100%|██████████| 30/30 [00:01<00:00, 23.45it/s]
Epoch 10/1000 - Train loss: 2139.15714213 - Val loss: 2088.05939535
INFO: Val_Loss improved (2118.52312419 --> 2088.05939535). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.28it/s]
100%|██████████| 30/30 [00:01<00:00, 25.18it/s]
Epoch 11/1000 - Train loss: 2117.69013977 - Val loss: 2081.55734863
INFO: Val_Loss improved (2088.05939535 --> 2081.55734863). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 21.62it/s]
Epoch 12/1000 - Train loss: 2094.45719604 - Val loss: 2046.68827311
INFO: Val_Loss improved (2081.55734863 --> 2046.68827311). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 25.20it/s]
Epoch 13/1000 - Train loss: 2074.38592631 - Val loss: 2042.18785807
INFO: Val_Loss improved (2046.68827311 --> 2042.18785807). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.23it/s]
100%|██████████| 30/30 [00:01<00:00, 21.67it/s]
Epoch 14/1000 - Train loss: 2040.59126485 - Val loss: 1971.26418050
INFO: Val_Loss improved (2042.18785807 --> 1971.26418050). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 24.70it/s]
Epoch 15/1000 - Train loss: 2003.98301900 - Val loss: 1937.48043620
INFO: Val_Loss improved (1971.26418050 --> 1937.48043620). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.42it/s]
100%|██████████| 30/30 [00:01<00:00, 21.93it/s]
Epoch 16/1000 - Train loss: 1958.74205017 - Val loss: 1882.55478109
INFO: Val_Loss improved (1937.48043620 --> 1882.55478109). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.17it/s]
100%|██████████| 30/30 [00:01<00:00, 24.74it/s]
Epoch 17/1000 - Train loss: 1931.64278361 - Val loss: 1867.43637288
INFO: Val_Loss improved (1882.55478109 --> 1867.43637288). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 21.54it/s]
Epoch 18/1000 - Train loss: 1896.80345357 - Val loss: 1843.63395589
INFO: Val_Loss improved (1867.43637288 --> 1843.63395589). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 25.12it/s]
Epoch 19/1000 - Train loss: 1863.52335002 - Val loss: 1792.70079753
INFO: Val_Loss improved (1843.63395589 --> 1792.70079753). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 22.11it/s]
Epoch 20/1000 - Train loss: 1828.09517822 - Val loss: 1774.48603109
INFO: Val_Loss improved (1792.70079753 --> 1774.48603109). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 24.83it/s]
Epoch 21/1000 - Train loss: 1817.35934041 - Val loss: 1745.99462891
INFO: Val_Loss improved (1774.48603109 --> 1745.99462891). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.15it/s]
100%|██████████| 30/30 [00:01<00:00, 22.07it/s]
Epoch 22/1000 - Train loss: 1790.72115784 - Val loss: 1730.47307129
INFO: Val_Loss improved (1745.99462891 --> 1730.47307129). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.30it/s]
100%|██████████| 30/30 [00:01<00:00, 24.83it/s]
Epoch 23/1000 - Train loss: 1768.49018453 - Val loss: 1728.91549479
INFO: Val_Loss improved (1730.47307129 --> 1728.91549479). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 22.44it/s]
Epoch 24/1000 - Train loss: 1746.91979167 - Val loss: 1698.89884033
INFO: Val_Loss improved (1728.91549479 --> 1698.89884033). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 24.43it/s]
Epoch 25/1000 - Train loss: 1735.38847453 - Val loss: 1687.31722819
INFO: Val_Loss improved (1698.89884033 --> 1687.31722819). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 21.97it/s]
Epoch 26/1000 - Train loss: 1714.79725342 - Val loss: 1672.73883464
INFO: Val_Loss improved (1687.31722819 --> 1672.73883464). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.28it/s]
100%|██████████| 30/30 [00:01<00:00, 24.69it/s]
Epoch 27/1000 - Train loss: 1690.90497335 - Val loss: 1669.00861816
INFO: Val_Loss improved (1672.73883464 --> 1669.00861816). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 21.82it/s]
Epoch 28/1000 - Train loss: 1669.69746501 - Val loss: 1641.60055339
INFO: Val_Loss improved (1669.00861816 --> 1641.60055339). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.18it/s]
100%|██████████| 30/30 [00:01<00:00, 24.72it/s]
Epoch 29/1000 - Train loss: 1663.16037089 - Val loss: 1626.07932129
INFO: Val_Loss improved (1641.60055339 --> 1626.07932129). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.23it/s]
100%|██████████| 30/30 [00:01<00:00, 22.51it/s]
Epoch 30/1000 - Train loss: 1650.19652100 - Val loss: 1612.13771159
INFO: Val_Loss improved (1626.07932129 --> 1612.13771159). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.30it/s]
100%|██████████| 30/30 [00:01<00:00, 24.93it/s]
Epoch 31/1000 - Train loss: 1633.42124837 - Val loss: 1611.48265381
INFO: Val_Loss improved (1612.13771159 --> 1611.48265381). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 21.89it/s]
Epoch 32/1000 - Train loss: 1635.30846761 - Val loss: 1602.32178955
INFO: Val_Loss improved (1611.48265381 --> 1602.32178955). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 25.14it/s]
Epoch 33/1000 - Train loss: 1617.08901265 - Val loss: 1601.25444336
INFO: Val_Loss improved (1602.32178955 --> 1601.25444336). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 22.47it/s]
Epoch 34/1000 - Train loss: 1592.58872274 - Val loss: 1578.11588542
INFO: Val_Loss improved (1601.25444336 --> 1578.11588542). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 24.37it/s]
Epoch 35/1000 - Train loss: 1596.56266479 - Val loss: 1571.92644043
INFO: Val_Loss improved (1578.11588542 --> 1571.92644043). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 21.21it/s]
Epoch 36/1000 - Train loss: 1582.85530904 - Val loss: 1564.39441732
INFO: Val_Loss improved (1571.92644043 --> 1564.39441732). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.29it/s]
100%|██████████| 30/30 [00:01<00:00, 24.80it/s]
Epoch 37/1000 - Train loss: 1579.91471761 - Val loss: 1604.42547607
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1564.39441732, Val Loss: 1604.42547607]
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 21.87it/s]
Epoch 38/1000 - Train loss: 1564.89337260 - Val loss: 1581.56490885
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1564.39441732, Val Loss: 1581.56490885]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 24.84it/s]
Epoch 39/1000 - Train loss: 1562.31970825 - Val loss: 1576.18032227
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1564.39441732, Val Loss: 1576.18032227]
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 22.22it/s]
Epoch 40/1000 - Train loss: 1561.82283529 - Val loss: 1543.27392171
INFO: Val_Loss improved (1564.39441732 --> 1543.27392171). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 25.00it/s]
Epoch 41/1000 - Train loss: 1552.04312541 - Val loss: 1579.95044352
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1543.27392171, Val Loss: 1579.95044352]
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 22.40it/s]
Epoch 42/1000 - Train loss: 1548.36367086 - Val loss: 1547.22502848
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1543.27392171, Val Loss: 1547.22502848]
100%|██████████| 120/120 [00:16<00:00, 7.23it/s]
100%|██████████| 30/30 [00:01<00:00, 25.18it/s]
Epoch 43/1000 - Train loss: 1521.50076701 - Val loss: 1551.93543701
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1543.27392171, Val Loss: 1551.93543701]
100%|██████████| 120/120 [00:16<00:00, 7.30it/s]
100%|██████████| 30/30 [00:01<00:00, 25.16it/s]
Epoch 44/1000 - Train loss: 1533.50863444 - Val loss: 1551.95876058
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1543.27392171, Val Loss: 1551.95876058]
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 21.80it/s]
Epoch 45/1000 - Train loss: 1517.34419759 - Val loss: 1524.78242187
INFO: Val_Loss improved (1543.27392171 --> 1524.78242187). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 25.06it/s]
Epoch 46/1000 - Train loss: 1510.43135274 - Val loss: 1521.74549561
INFO: Val_Loss improved (1524.78242187 --> 1521.74549561). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 22.04it/s]
Epoch 47/1000 - Train loss: 1499.74609172 - Val loss: 1534.25233154
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1521.74549561, Val Loss: 1534.25233154]
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 25.28it/s]
Epoch 48/1000 - Train loss: 1494.28978678 - Val loss: 1535.73928630
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1521.74549561, Val Loss: 1535.73928630]
100%|██████████| 120/120 [00:16<00:00, 7.18it/s]
100%|██████████| 30/30 [00:01<00:00, 21.80it/s]
Epoch 49/1000 - Train loss: 1485.46776733 - Val loss: 1529.94301758
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1521.74549561, Val Loss: 1529.94301758]
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 25.25it/s]
Epoch 50/1000 - Train loss: 1497.27207235 - Val loss: 1533.38873291
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1521.74549561, Val Loss: 1533.38873291]
100%|██████████| 120/120 [00:16<00:00, 7.23it/s]
100%|██████████| 30/30 [00:01<00:00, 25.35it/s]
Epoch 51/1000 - Train loss: 1480.23327942 - Val loss: 1537.81709798
INFO: Early stopping counter 5 of 20 [Best Val Loss: 1521.74549561, Val Loss: 1537.81709798]
100%|██████████| 120/120 [00:16<00:00, 7.32it/s]
100%|██████████| 30/30 [00:01<00:00, 21.39it/s]
Epoch 52/1000 - Train loss: 1465.88573100 - Val loss: 1520.88831380
INFO: Val_Loss improved (1521.74549561 --> 1520.88831380). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 25.39it/s]
Epoch 53/1000 - Train loss: 1483.87281087 - Val loss: 1503.93408203
INFO: Val_Loss improved (1520.88831380 --> 1503.93408203). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 22.48it/s]
Epoch 54/1000 - Train loss: 1480.88431702 - Val loss: 1514.97798258
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1503.93408203, Val Loss: 1514.97798258]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 24.71it/s]
Epoch 55/1000 - Train loss: 1479.31743266 - Val loss: 1503.74938965
INFO: Val_Loss improved (1503.93408203 --> 1503.74938965). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 22.11it/s]
Epoch 56/1000 - Train loss: 1466.65839132 - Val loss: 1506.43861898
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1503.74938965, Val Loss: 1506.43861898]
100%|██████████| 120/120 [00:16<00:00, 7.23it/s]
100%|██████████| 30/30 [00:01<00:00, 24.78it/s]
Epoch 57/1000 - Train loss: 1462.62595317 - Val loss: 1524.08071696
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1503.74938965, Val Loss: 1524.08071696]
100%|██████████| 120/120 [00:16<00:00, 7.17it/s]
100%|██████████| 30/30 [00:01<00:00, 22.08it/s]
Epoch 58/1000 - Train loss: 1461.18101705 - Val loss: 1506.32097575
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1503.74938965, Val Loss: 1506.32097575]
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 24.90it/s]
Epoch 59/1000 - Train loss: 1448.37948303 - Val loss: 1504.56092529
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1503.74938965, Val Loss: 1504.56092529]
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 25.13it/s]
Epoch 60/1000 - Train loss: 1436.34779154 - Val loss: 1485.80478923
INFO: Val_Loss improved (1503.74938965 --> 1485.80478923). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.34it/s]
100%|██████████| 30/30 [00:01<00:00, 21.97it/s]
Epoch 61/1000 - Train loss: 1440.86187439 - Val loss: 1498.22763672
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1485.80478923, Val Loss: 1498.22763672]
100%|██████████| 120/120 [00:16<00:00, 7.15it/s]
100%|██████████| 30/30 [00:01<00:00, 24.60it/s]
Epoch 62/1000 - Train loss: 1434.67921346 - Val loss: 1499.84899089
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1485.80478923, Val Loss: 1499.84899089]
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 22.10it/s]
Epoch 63/1000 - Train loss: 1429.57323710 - Val loss: 1532.10005290
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1485.80478923, Val Loss: 1532.10005290]
100%|██████████| 120/120 [00:16<00:00, 7.31it/s]
100%|██████████| 30/30 [00:01<00:00, 24.61it/s]
Epoch 64/1000 - Train loss: 1429.44683228 - Val loss: 1510.61236165
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1485.80478923, Val Loss: 1510.61236165]
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 25.11it/s]
Epoch 65/1000 - Train loss: 1414.42365417 - Val loss: 1489.06867269
INFO: Early stopping counter 5 of 20 [Best Val Loss: 1485.80478923, Val Loss: 1489.06867269]
100%|██████████| 120/120 [00:16<00:00, 7.28it/s]
100%|██████████| 30/30 [00:01<00:00, 21.77it/s]
Epoch 66/1000 - Train loss: 1400.64874878 - Val loss: 1491.51576742
INFO: Early stopping counter 6 of 20 [Best Val Loss: 1485.80478923, Val Loss: 1491.51576742]
100%|██████████| 120/120 [00:16<00:00, 7.32it/s]
100%|██████████| 30/30 [00:01<00:00, 24.78it/s]
Epoch 67/1000 - Train loss: 1392.29015503 - Val loss: 1482.02675374
INFO: Val_Loss improved (1485.80478923 --> 1482.02675374). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 22.00it/s]
Epoch 68/1000 - Train loss: 1417.37634583 - Val loss: 1486.24620768
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1482.02675374, Val Loss: 1486.24620768]
100%|██████████| 120/120 [00:16<00:00, 7.34it/s]
100%|██████████| 30/30 [00:01<00:00, 24.60it/s]
Epoch 69/1000 - Train loss: 1416.52066752 - Val loss: 1489.25712484
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1482.02675374, Val Loss: 1489.25712484]
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 25.25it/s]
Epoch 70/1000 - Train loss: 1409.18207499 - Val loss: 1496.23450114
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1482.02675374, Val Loss: 1496.23450114]
100%|██████████| 120/120 [00:16<00:00, 7.10it/s]
100%|██████████| 30/30 [00:01<00:00, 24.54it/s]
Epoch 71/1000 - Train loss: 1401.15682882 - Val loss: 1475.28702393
INFO: Val_Loss improved (1482.02675374 --> 1475.28702393). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.23it/s]
100%|██████████| 30/30 [00:01<00:00, 25.03it/s]
Epoch 72/1000 - Train loss: 1400.40001221 - Val loss: 1470.05228678
INFO: Val_Loss improved (1475.28702393 --> 1470.05228678). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.34it/s]
100%|██████████| 30/30 [00:01<00:00, 22.47it/s]
Epoch 73/1000 - Train loss: 1384.28774719 - Val loss: 1487.32732747
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1470.05228678, Val Loss: 1487.32732747]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 25.16it/s]
Epoch 74/1000 - Train loss: 1401.00240987 - Val loss: 1482.39081624
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1470.05228678, Val Loss: 1482.39081624]
100%|██████████| 120/120 [00:16<00:00, 7.34it/s]
100%|██████████| 30/30 [00:01<00:00, 22.19it/s]
Epoch 75/1000 - Train loss: 1385.18510437 - Val loss: 1509.66620687
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1470.05228678, Val Loss: 1509.66620687]
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 25.17it/s]
Epoch 76/1000 - Train loss: 1386.03049113 - Val loss: 1466.41359456
INFO: Val_Loss improved (1470.05228678 --> 1466.41359456). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.32it/s]
100%|██████████| 30/30 [00:01<00:00, 21.78it/s]
Epoch 77/1000 - Train loss: 1372.62018636 - Val loss: 1474.41927897
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1474.41927897]
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 24.78it/s]
Epoch 78/1000 - Train loss: 1381.02753906 - Val loss: 1475.12618408
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1475.12618408]
100%|██████████| 120/120 [00:16<00:00, 7.17it/s]
100%|██████████| 30/30 [00:01<00:00, 24.95it/s]
Epoch 79/1000 - Train loss: 1373.49257202 - Val loss: 1492.77657064
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1492.77657064]
100%|██████████| 120/120 [00:16<00:00, 7.29it/s]
100%|██████████| 30/30 [00:01<00:00, 22.62it/s]
Epoch 80/1000 - Train loss: 1366.05015767 - Val loss: 1481.34626465
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1481.34626465]
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 24.44it/s]
Epoch 81/1000 - Train loss: 1376.96361593 - Val loss: 1478.53971761
INFO: Early stopping counter 5 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1478.53971761]
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 21.55it/s]
Epoch 82/1000 - Train loss: 1365.36792297 - Val loss: 1483.57531738
INFO: Early stopping counter 6 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1483.57531738]
100%|██████████| 120/120 [00:16<00:00, 7.12it/s]
100%|██████████| 30/30 [00:01<00:00, 25.18it/s]
Epoch 83/1000 - Train loss: 1363.81930745 - Val loss: 1503.75220133
INFO: Early stopping counter 7 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1503.75220133]
100%|██████████| 120/120 [00:16<00:00, 7.35it/s]
100%|██████████| 30/30 [00:01<00:00, 22.31it/s]
Epoch 84/1000 - Train loss: 1375.25551147 - Val loss: 1469.17236735
INFO: Early stopping counter 8 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1469.17236735]
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 25.02it/s]
Epoch 85/1000 - Train loss: 1359.73573608 - Val loss: 1477.95154622
INFO: Early stopping counter 9 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1477.95154622]
100%|██████████| 120/120 [00:16<00:00, 7.31it/s]
100%|██████████| 30/30 [00:01<00:00, 24.60it/s]
Epoch 86/1000 - Train loss: 1352.54728088 - Val loss: 1487.98225505
INFO: Early stopping counter 10 of 20 [Best Val Loss: 1466.41359456, Val Loss: 1487.98225505]
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 21.92it/s]
Epoch 87/1000 - Train loss: 1359.97195841 - Val loss: 1464.42423910
INFO: Val_Loss improved (1466.41359456 --> 1464.42423910). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 25.41it/s]
Epoch 88/1000 - Train loss: 1347.97164103 - Val loss: 1462.62575277
INFO: Val_Loss improved (1464.42423910 --> 1462.62575277). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 22.48it/s]
Epoch 89/1000 - Train loss: 1341.52504272 - Val loss: 1478.61234538
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1462.62575277, Val Loss: 1478.61234538]
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 24.82it/s]
Epoch 90/1000 - Train loss: 1338.05002441 - Val loss: 1484.90279948
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1462.62575277, Val Loss: 1484.90279948]
100%|██████████| 120/120 [00:16<00:00, 7.39it/s]
100%|██████████| 30/30 [00:01<00:00, 24.29it/s]
Epoch 91/1000 - Train loss: 1333.96632080 - Val loss: 1465.91394450
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1462.62575277, Val Loss: 1465.91394450]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 21.61it/s]
Epoch 92/1000 - Train loss: 1330.47940165 - Val loss: 1479.34451090
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1462.62575277, Val Loss: 1479.34451090]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 25.08it/s]
Epoch 93/1000 - Train loss: 1343.58989868 - Val loss: 1456.79302165
INFO: Val_Loss improved (1462.62575277 --> 1456.79302165). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 21.77it/s]
Epoch 94/1000 - Train loss: 1341.85667725 - Val loss: 1482.75269368
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1482.75269368]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 24.58it/s]
Epoch 95/1000 - Train loss: 1327.94591268 - Val loss: 1468.00319417
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1468.00319417]
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 21.51it/s]
Epoch 96/1000 - Train loss: 1341.89235840 - Val loss: 1505.50149333
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1505.50149333]
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 24.80it/s]
Epoch 97/1000 - Train loss: 1320.53960571 - Val loss: 1469.53007406
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1469.53007406]
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 22.11it/s]
Epoch 98/1000 - Train loss: 1329.71967468 - Val loss: 1471.28465169
INFO: Early stopping counter 5 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1471.28465169]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 24.83it/s]
Epoch 99/1000 - Train loss: 1318.38923543 - Val loss: 1460.38648275
INFO: Early stopping counter 6 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1460.38648275]
100%|██████████| 120/120 [00:16<00:00, 7.31it/s]
100%|██████████| 30/30 [00:01<00:00, 25.11it/s]
Epoch 100/1000 - Train loss: 1308.92377726 - Val loss: 1476.00021159
INFO: Early stopping counter 7 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1476.00021159]
100%|██████████| 120/120 [00:16<00:00, 7.16it/s]
100%|██████████| 30/30 [00:01<00:00, 23.06it/s]
Epoch 101/1000 - Train loss: 1327.68236796 - Val loss: 1464.87981364
INFO: Early stopping counter 8 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1464.87981364]
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 25.06it/s]
Epoch 102/1000 - Train loss: 1321.35953776 - Val loss: 1484.84060059
INFO: Early stopping counter 9 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1484.84060059]
100%|██████████| 120/120 [00:16<00:00, 7.23it/s]
100%|██████████| 30/30 [00:01<00:00, 21.99it/s]
Epoch 103/1000 - Train loss: 1308.20714315 - Val loss: 1504.38103027
INFO: Early stopping counter 10 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1504.38103027]
100%|██████████| 120/120 [00:16<00:00, 7.18it/s]
100%|██████████| 30/30 [00:01<00:00, 24.93it/s]
Epoch 104/1000 - Train loss: 1306.83864644 - Val loss: 1491.88827718
INFO: Early stopping counter 11 of 20 [Best Val Loss: 1456.79302165, Val Loss: 1491.88827718]
Epoch 00104: reducing learning rate of group 0 to 1.0000e-07.
Epoch 00104: reducing learning rate of group 1 to 1.0000e-07.
Epoch 00104: reducing learning rate of group 2 to 1.0000e-07.
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 22.39it/s]
Epoch 105/1000 - Train loss: 1279.85932617 - Val loss: 1449.60711670
INFO: Val_Loss improved (1456.79302165 --> 1449.60711670). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 25.40it/s]
Epoch 106/1000 - Train loss: 1264.75489400 - Val loss: 1436.74384359
INFO: Val_Loss improved (1449.60711670 --> 1436.74384359). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.30it/s]
100%|██████████| 30/30 [00:01<00:00, 21.87it/s]
Epoch 107/1000 - Train loss: 1243.51479085 - Val loss: 1439.20335693
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1436.74384359, Val Loss: 1439.20335693]
100%|██████████| 120/120 [00:16<00:00, 7.27it/s]
100%|██████████| 30/30 [00:01<00:00, 24.72it/s]
Epoch 108/1000 - Train loss: 1256.19158834 - Val loss: 1448.30501302
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1436.74384359, Val Loss: 1448.30501302]
100%|██████████| 120/120 [00:16<00:00, 7.29it/s]
100%|██████████| 30/30 [00:01<00:00, 25.03it/s]
Epoch 109/1000 - Train loss: 1246.35949809 - Val loss: 1446.15256755
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1436.74384359, Val Loss: 1446.15256755]
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 22.18it/s]
Epoch 110/1000 - Train loss: 1262.11881917 - Val loss: 1436.12414958
INFO: Val_Loss improved (1436.74384359 --> 1436.12414958). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 25.18it/s]
Epoch 111/1000 - Train loss: 1254.50967509 - Val loss: 1437.80731608
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1436.12414958, Val Loss: 1437.80731608]
100%|██████████| 120/120 [00:16<00:00, 7.10it/s]
100%|██████████| 30/30 [00:01<00:00, 22.22it/s]
Epoch 112/1000 - Train loss: 1259.48186137 - Val loss: 1435.54129232
INFO: Val_Loss improved (1436.12414958 --> 1435.54129232). Saving model ...
100%|██████████| 120/120 [00:16<00:00, 7.30it/s]
100%|██████████| 30/30 [00:01<00:00, 24.89it/s]
Epoch 113/1000 - Train loss: 1236.83064779 - Val loss: 1444.17726237
INFO: Early stopping counter 1 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1444.17726237]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 21.82it/s]
Epoch 114/1000 - Train loss: 1235.66631266 - Val loss: 1442.45617676
INFO: Early stopping counter 2 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1442.45617676]
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 24.98it/s]
Epoch 115/1000 - Train loss: 1237.60505981 - Val loss: 1443.30299072
INFO: Early stopping counter 3 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1443.30299072]
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 21.98it/s]
Epoch 116/1000 - Train loss: 1238.48349915 - Val loss: 1446.86625163
INFO: Early stopping counter 4 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1446.86625163]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 25.12it/s]
Epoch 117/1000 - Train loss: 1228.81956685 - Val loss: 1438.73139242
INFO: Early stopping counter 5 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1438.73139242]
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 22.29it/s]
Epoch 118/1000 - Train loss: 1235.31768494 - Val loss: 1436.46524251
INFO: Early stopping counter 6 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1436.46524251]
100%|██████████| 120/120 [00:16<00:00, 7.22it/s]
100%|██████████| 30/30 [00:01<00:00, 24.84it/s]
Epoch 119/1000 - Train loss: 1221.74420980 - Val loss: 1438.38082275
INFO: Early stopping counter 7 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1438.38082275]
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 25.06it/s]
Epoch 120/1000 - Train loss: 1222.66671143 - Val loss: 1442.09947103
INFO: Early stopping counter 8 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1442.09947103]
100%|██████████| 120/120 [00:16<00:00, 7.18it/s]
100%|██████████| 30/30 [00:01<00:00, 22.69it/s]
Epoch 121/1000 - Train loss: 1215.57423859 - Val loss: 1446.69070231
INFO: Early stopping counter 9 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1446.69070231]
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 24.85it/s]
Epoch 122/1000 - Train loss: 1235.16498667 - Val loss: 1443.17941895
INFO: Early stopping counter 10 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1443.17941895]
100%|██████████| 120/120 [00:16<00:00, 7.25it/s]
100%|██████████| 30/30 [00:01<00:00, 21.81it/s]
Epoch 123/1000 - Train loss: 1224.58130798 - Val loss: 1438.14779053
INFO: Early stopping counter 11 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1438.14779053]
Epoch 00123: reducing learning rate of group 0 to 1.0000e-08.
Epoch 00123: reducing learning rate of group 1 to 1.0000e-08.
Epoch 00123: reducing learning rate of group 2 to 1.0000e-08.
100%|██████████| 120/120 [00:16<00:00, 7.20it/s]
100%|██████████| 30/30 [00:01<00:00, 24.20it/s]
Epoch 124/1000 - Train loss: 1217.41171468 - Val loss: 1440.67330322
INFO: Early stopping counter 12 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1440.67330322]
100%|██████████| 120/120 [00:16<00:00, 7.32it/s]
100%|██████████| 30/30 [00:01<00:00, 22.59it/s]
Epoch 125/1000 - Train loss: 1228.28821411 - Val loss: 1445.15652669
INFO: Early stopping counter 13 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1445.15652669]
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 24.57it/s]
Epoch 126/1000 - Train loss: 1224.62659912 - Val loss: 1444.86668701
INFO: Early stopping counter 14 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1444.86668701]
100%|██████████| 120/120 [00:16<00:00, 7.26it/s]
100%|██████████| 30/30 [00:01<00:00, 25.10it/s]
Epoch 127/1000 - Train loss: 1216.19667664 - Val loss: 1442.61728516
INFO: Early stopping counter 15 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1442.61728516]
100%|██████████| 120/120 [00:16<00:00, 7.29it/s]
100%|██████████| 30/30 [00:01<00:00, 22.07it/s]
Epoch 128/1000 - Train loss: 1224.08571574 - Val loss: 1441.93986003
INFO: Early stopping counter 16 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1441.93986003]
100%|██████████| 120/120 [00:16<00:00, 7.24it/s]
100%|██████████| 30/30 [00:01<00:00, 24.63it/s]
Epoch 129/1000 - Train loss: 1218.61766357 - Val loss: 1443.84890544
INFO: Early stopping counter 17 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1443.84890544]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 22.12it/s]
Epoch 130/1000 - Train loss: 1212.96040141 - Val loss: 1444.55605876
INFO: Early stopping counter 18 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1444.55605876]
100%|██████████| 120/120 [00:16<00:00, 7.19it/s]
100%|██████████| 30/30 [00:01<00:00, 24.78it/s]
Epoch 131/1000 - Train loss: 1225.11510468 - Val loss: 1443.43215332
INFO: Early stopping counter 19 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1443.43215332]
100%|██████████| 120/120 [00:16<00:00, 7.21it/s]
100%|██████████| 30/30 [00:01<00:00, 21.85it/s]
Epoch 132/1000 - Train loss: 1215.74841410 - Val loss: 1445.45094808
INFO: Early stopping counter 20 of 20 [Best Val Loss: 1435.54129232, Val Loss: 1445.45094808]
INFO: Early stopping DONE!
INFO: Best Val Loss: 1435.54129232
Loading best model...
results_adaptive = adaptiveHeatmapRegressionPipeline.evaluate(image_paths_test1, landmarks_test1, pixel_spacings_test1)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:03<00:00, 47.15it/s]
100%|██████████| 150/150 [00:04<00:00, 34.66it/s]
Detection report:
1# Point-to-point error (PE) statistics:
======================================================================
Class Mean Median Std Min Max
----------------------------------------------------------------------
Landmark 0 0.72 0.54 1.23 0.03 11.49
Landmark 1 1.71 0.93 1.97 0.03 11.26
Landmark 2 1.94 1.50 1.53 0.07 9.10
Landmark 3 2.98 1.87 2.56 0.05 10.33
Landmark 4 2.24 1.82 1.51 0.14 7.10
Landmark 5 1.70 1.23 1.49 0.03 7.45
Landmark 6 1.06 0.82 0.84 0.08 4.99
Landmark 7 1.04 0.78 1.04 0.06 8.76
Landmark 8 0.96 0.83 0.71 0.08 4.37
Landmark 9 3.64 3.34 2.14 0.06 9.09
Landmark 10 0.89 0.66 0.97 0.03 8.65
Landmark 11 0.67 0.50 0.87 0.02 8.05
Landmark 12 0.96 0.79 0.67 0.05 3.81
Landmark 13 0.82 0.69 0.55 0.04 2.88
Landmark 14 1.13 0.87 1.03 0.05 6.85
Landmark 15 2.04 1.84 1.47 0.07 8.39
Landmark 16 1.07 0.86 0.77 0.08 3.71
Landmark 17 1.74 1.20 1.68 0.10 13.14
Landmark 18 2.69 1.60 2.51 0.13 11.06
======================================================================
2# Success detection rate (SDR):
================================================================================
Class SDR (PE≤2mm) SDR (PE≤2.5mm) SDR (PE≤3mm) SDR (PE≤4mm)
--------------------------------------------------------------------------------
Landmark 0 98.00 98.00 98.00 98.67
Landmark 1 75.33 78.67 80.67 86.67
Landmark 2 66.67 73.33 79.33 88.67
Landmark 3 54.00 57.33 60.67 67.33
Landmark 4 56.67 66.00 76.00 87.33
Landmark 5 72.67 82.00 87.33 92.00
Landmark 6 87.33 93.33 96.67 98.67
Landmark 7 89.33 94.00 96.00 98.67
Landmark 8 91.33 96.67 98.00 99.33
Landmark 9 28.00 36.67 44.00 61.33
Landmark 10 90.67 94.67 97.33 98.67
Landmark 11 95.33 96.00 96.67 98.67
Landmark 12 92.00 97.33 98.67 100.00
Landmark 13 96.00 99.33 100.00 100.00
Landmark 14 87.33 91.33 95.33 97.33
Landmark 15 56.00 68.00 82.00 90.67
Landmark 16 84.67 94.67 98.00 100.00
Landmark 17 72.00 80.00 86.00 93.33
Landmark 18 56.00 59.33 60.67 73.33
================================================================================
Indirect Uncertainty Aware Heatmap Regression#
from landmarker.pipeline import IndirectUncertaintyAwareHeatmapRegressionPipeline
train_config = {
"criterion": "multivariate_gaussian_nllloss",
"optimizer": "adam",
"lr": lr,
"batch_size": batch_size,
"epochs": epochs,
"lr_scheduler": True,
}
indirectUncertaintyAwareHeatmapRegressionPipeline = IndirectUncertaintyAwareHeatmapRegressionPipeline(
differentiable_decoder="softargmax", model="CoordConvUNet", in_channels_img=1, nb_landmarks=19,
train_config=train_config, heatmap_decoder=coord_argmax, verbose=True, device=device)
indirectUncertaintyAwareHeatmapRegressionPipeline.fit(image_paths_train, landmarks_train,
transform=composed_transformd, cache_ds=cache_ds)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:03<00:00, 47.28it/s]
100%|██████████| 30/30 [00:09<00:00, 3.22it/s]
100%|██████████| 8/8 [00:00<00:00, 9.28it/s]
Epoch 1/1000 - Train loss: 8.85510486 - Val loss: 7.59643334
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.53it/s]
Epoch 2/1000 - Train loss: 6.91894143 - Val loss: 6.12938422
INFO: Val_Loss improved (7.59643334 --> 6.12938422). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.49it/s]
Epoch 3/1000 - Train loss: 6.06907252 - Val loss: 5.84842372
INFO: Val_Loss improved (6.12938422 --> 5.84842372). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.40it/s]
Epoch 4/1000 - Train loss: 5.71870116 - Val loss: 5.36497980
INFO: Val_Loss improved (5.84842372 --> 5.36497980). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.18it/s]
100%|██████████| 8/8 [00:00<00:00, 9.33it/s]
Epoch 5/1000 - Train loss: 5.25066692 - Val loss: 4.96219409
INFO: Val_Loss improved (5.36497980 --> 4.96219409). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 9.55it/s]
Epoch 6/1000 - Train loss: 5.05971494 - Val loss: 4.93065578
INFO: Val_Loss improved (4.96219409 --> 4.93065578). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 8.29it/s]
Epoch 7/1000 - Train loss: 5.15118046 - Val loss: 5.10248494
INFO: Early stopping counter 1 of 20 [Best Val Loss: 4.93065578, Val Loss: 5.10248494]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:01<00:00, 7.93it/s]
Epoch 8/1000 - Train loss: 4.98578777 - Val loss: 4.80787593
INFO: Val_Loss improved (4.93065578 --> 4.80787593). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 8.43it/s]
Epoch 9/1000 - Train loss: 4.98838374 - Val loss: 4.93277884
INFO: Early stopping counter 1 of 20 [Best Val Loss: 4.80787593, Val Loss: 4.93277884]
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 8.20it/s]
Epoch 10/1000 - Train loss: 4.99105763 - Val loss: 4.79771185
INFO: Val_Loss improved (4.80787593 --> 4.79771185). Saving model ...
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.48it/s]
Epoch 11/1000 - Train loss: 4.93627516 - Val loss: 4.78159946
INFO: Val_Loss improved (4.79771185 --> 4.78159946). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.56it/s]
Epoch 12/1000 - Train loss: 4.79021955 - Val loss: 4.70071155
INFO: Val_Loss improved (4.78159946 --> 4.70071155). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 9.40it/s]
Epoch 13/1000 - Train loss: 4.61603632 - Val loss: 4.57115924
INFO: Val_Loss improved (4.70071155 --> 4.57115924). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 9.48it/s]
Epoch 14/1000 - Train loss: 4.67758387 - Val loss: 4.50821054
INFO: Val_Loss improved (4.57115924 --> 4.50821054). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:00<00:00, 9.19it/s]
Epoch 15/1000 - Train loss: 4.72548086 - Val loss: 4.62499678
INFO: Early stopping counter 1 of 20 [Best Val Loss: 4.50821054, Val Loss: 4.62499678]
100%|██████████| 30/30 [00:09<00:00, 3.24it/s]
100%|██████████| 8/8 [00:00<00:00, 9.59it/s]
Epoch 16/1000 - Train loss: 4.64401859 - Val loss: 4.53061330
INFO: Early stopping counter 2 of 20 [Best Val Loss: 4.50821054, Val Loss: 4.53061330]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.60it/s]
Epoch 17/1000 - Train loss: 4.57761165 - Val loss: 4.57983488
INFO: Early stopping counter 3 of 20 [Best Val Loss: 4.50821054, Val Loss: 4.57983488]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.50it/s]
Epoch 18/1000 - Train loss: 4.51541526 - Val loss: 4.48269498
INFO: Val_Loss improved (4.50821054 --> 4.48269498). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.58it/s]
Epoch 19/1000 - Train loss: 4.40800363 - Val loss: 4.38752508
INFO: Val_Loss improved (4.48269498 --> 4.38752508). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.52it/s]
Epoch 20/1000 - Train loss: 4.46701659 - Val loss: 4.37847430
INFO: Val_Loss improved (4.38752508 --> 4.37847430). Saving model ...
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.64it/s]
Epoch 21/1000 - Train loss: 4.69981979 - Val loss: 4.59034610
INFO: Early stopping counter 1 of 20 [Best Val Loss: 4.37847430, Val Loss: 4.59034610]
100%|██████████| 30/30 [00:08<00:00, 3.36it/s]
100%|██████████| 8/8 [00:00<00:00, 9.42it/s]
Epoch 22/1000 - Train loss: 4.62229788 - Val loss: 4.37903506
INFO: Early stopping counter 2 of 20 [Best Val Loss: 4.37847430, Val Loss: 4.37903506]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.35it/s]
Epoch 23/1000 - Train loss: 4.61753197 - Val loss: 4.51658791
INFO: Early stopping counter 3 of 20 [Best Val Loss: 4.37847430, Val Loss: 4.51658791]
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 8.23it/s]
Epoch 24/1000 - Train loss: 4.54359821 - Val loss: 4.31195867
INFO: Val_Loss improved (4.37847430 --> 4.31195867). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:01<00:00, 7.96it/s]
Epoch 25/1000 - Train loss: 4.35595875 - Val loss: 4.42103237
INFO: Early stopping counter 1 of 20 [Best Val Loss: 4.31195867, Val Loss: 4.42103237]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 8.68it/s]
Epoch 26/1000 - Train loss: 4.35665911 - Val loss: 4.28162855
INFO: Val_Loss improved (4.31195867 --> 4.28162855). Saving model ...
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 8.59it/s]
Epoch 27/1000 - Train loss: 5.12586400 - Val loss: 4.96437174
INFO: Early stopping counter 1 of 20 [Best Val Loss: 4.28162855, Val Loss: 4.96437174]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 8.11it/s]
Epoch 28/1000 - Train loss: 4.85585357 - Val loss: 4.71523291
INFO: Early stopping counter 2 of 20 [Best Val Loss: 4.28162855, Val Loss: 4.71523291]
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 9.31it/s]
Epoch 29/1000 - Train loss: 4.59158312 - Val loss: 4.43826681
INFO: Early stopping counter 3 of 20 [Best Val Loss: 4.28162855, Val Loss: 4.43826681]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.51it/s]
Epoch 30/1000 - Train loss: 4.82431269 - Val loss: 4.52425504
INFO: Early stopping counter 4 of 20 [Best Val Loss: 4.28162855, Val Loss: 4.52425504]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.39it/s]
Epoch 31/1000 - Train loss: 4.54218276 - Val loss: 4.46837831
INFO: Early stopping counter 5 of 20 [Best Val Loss: 4.28162855, Val Loss: 4.46837831]
100%|██████████| 30/30 [00:09<00:00, 3.24it/s]
100%|██████████| 8/8 [00:00<00:00, 9.42it/s]
Epoch 32/1000 - Train loss: 4.39115094 - Val loss: 4.34298444
INFO: Early stopping counter 6 of 20 [Best Val Loss: 4.28162855, Val Loss: 4.34298444]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.47it/s]
Epoch 33/1000 - Train loss: 4.24387945 - Val loss: 4.14431262
INFO: Val_Loss improved (4.28162855 --> 4.14431262). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.69it/s]
Epoch 34/1000 - Train loss: 4.24269914 - Val loss: 4.17740303
INFO: Early stopping counter 1 of 20 [Best Val Loss: 4.14431262, Val Loss: 4.17740303]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.21it/s]
Epoch 35/1000 - Train loss: 4.26248790 - Val loss: 4.28302526
INFO: Early stopping counter 2 of 20 [Best Val Loss: 4.14431262, Val Loss: 4.28302526]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.67it/s]
Epoch 36/1000 - Train loss: 4.26652256 - Val loss: 4.16854298
INFO: Early stopping counter 3 of 20 [Best Val Loss: 4.14431262, Val Loss: 4.16854298]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.56it/s]
Epoch 37/1000 - Train loss: 4.11032185 - Val loss: 4.10340676
INFO: Val_Loss improved (4.14431262 --> 4.10340676). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.29it/s]
Epoch 38/1000 - Train loss: 4.04608590 - Val loss: 4.01403943
INFO: Val_Loss improved (4.10340676 --> 4.01403943). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.49it/s]
Epoch 39/1000 - Train loss: 4.05152966 - Val loss: 3.98341984
INFO: Val_Loss improved (4.01403943 --> 3.98341984). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:00<00:00, 9.41it/s]
Epoch 40/1000 - Train loss: 4.07697851 - Val loss: 4.11061296
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.98341984, Val Loss: 4.11061296]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.40it/s]
Epoch 41/1000 - Train loss: 4.42868789 - Val loss: 4.07130396
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.98341984, Val Loss: 4.07130396]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.62it/s]
Epoch 42/1000 - Train loss: 4.09979694 - Val loss: 4.00169739
INFO: Early stopping counter 3 of 20 [Best Val Loss: 3.98341984, Val Loss: 4.00169739]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.32it/s]
Epoch 43/1000 - Train loss: 4.02112029 - Val loss: 3.92284566
INFO: Val_Loss improved (3.98341984 --> 3.92284566). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.30it/s]
100%|██████████| 8/8 [00:00<00:00, 9.64it/s]
Epoch 44/1000 - Train loss: 3.95782094 - Val loss: 3.95465562
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.92284566, Val Loss: 3.95465562]
100%|██████████| 30/30 [00:09<00:00, 3.30it/s]
100%|██████████| 8/8 [00:00<00:00, 9.49it/s]
Epoch 45/1000 - Train loss: 3.95550180 - Val loss: 3.91840258
INFO: Val_Loss improved (3.92284566 --> 3.91840258). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.26it/s]
100%|██████████| 8/8 [00:00<00:00, 9.59it/s]
Epoch 46/1000 - Train loss: 3.91377574 - Val loss: 3.91609269
INFO: Val_Loss improved (3.91840258 --> 3.91609269). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.65it/s]
Epoch 47/1000 - Train loss: 3.90995443 - Val loss: 3.87182361
INFO: Val_Loss improved (3.91609269 --> 3.87182361). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.58it/s]
Epoch 48/1000 - Train loss: 4.07745135 - Val loss: 4.05069673
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.87182361, Val Loss: 4.05069673]
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:00<00:00, 9.66it/s]
Epoch 49/1000 - Train loss: 4.02598852 - Val loss: 3.96864817
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.87182361, Val Loss: 3.96864817]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.64it/s]
Epoch 50/1000 - Train loss: 3.96653429 - Val loss: 3.88883504
INFO: Early stopping counter 3 of 20 [Best Val Loss: 3.87182361, Val Loss: 3.88883504]
100%|██████████| 30/30 [00:08<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.21it/s]
Epoch 51/1000 - Train loss: 3.95393209 - Val loss: 3.92625776
INFO: Early stopping counter 4 of 20 [Best Val Loss: 3.87182361, Val Loss: 3.92625776]
100%|██████████| 30/30 [00:09<00:00, 3.26it/s]
100%|██████████| 8/8 [00:01<00:00, 7.69it/s]
Epoch 52/1000 - Train loss: 4.08453796 - Val loss: 3.89360470
INFO: Early stopping counter 5 of 20 [Best Val Loss: 3.87182361, Val Loss: 3.89360470]
100%|██████████| 30/30 [00:09<00:00, 3.30it/s]
100%|██████████| 8/8 [00:00<00:00, 8.34it/s]
Epoch 53/1000 - Train loss: 3.90907013 - Val loss: 3.85758355
INFO: Val_Loss improved (3.87182361 --> 3.85758355). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:00<00:00, 9.59it/s]
Epoch 54/1000 - Train loss: 4.66085027 - Val loss: 6.74637651
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.85758355, Val Loss: 6.74637651]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 8.03it/s]
Epoch 55/1000 - Train loss: 5.75211306 - Val loss: 4.86323512
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.85758355, Val Loss: 4.86323512]
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:00<00:00, 9.02it/s]
Epoch 56/1000 - Train loss: 4.75762550 - Val loss: 4.37663180
INFO: Early stopping counter 3 of 20 [Best Val Loss: 3.85758355, Val Loss: 4.37663180]
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.44it/s]
Epoch 57/1000 - Train loss: 4.38826461 - Val loss: 4.21312171
INFO: Early stopping counter 4 of 20 [Best Val Loss: 3.85758355, Val Loss: 4.21312171]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.38it/s]
Epoch 58/1000 - Train loss: 4.23661785 - Val loss: 4.25166380
INFO: Early stopping counter 5 of 20 [Best Val Loss: 3.85758355, Val Loss: 4.25166380]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.39it/s]
Epoch 59/1000 - Train loss: 4.17909139 - Val loss: 4.08822960
INFO: Early stopping counter 6 of 20 [Best Val Loss: 3.85758355, Val Loss: 4.08822960]
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:00<00:00, 9.32it/s]
Epoch 60/1000 - Train loss: 4.13937922 - Val loss: 4.09647110
INFO: Early stopping counter 7 of 20 [Best Val Loss: 3.85758355, Val Loss: 4.09647110]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.46it/s]
Epoch 61/1000 - Train loss: 4.09285215 - Val loss: 4.13667926
INFO: Early stopping counter 8 of 20 [Best Val Loss: 3.85758355, Val Loss: 4.13667926]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.57it/s]
Epoch 62/1000 - Train loss: 4.01331544 - Val loss: 3.98019740
INFO: Early stopping counter 9 of 20 [Best Val Loss: 3.85758355, Val Loss: 3.98019740]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.38it/s]
Epoch 63/1000 - Train loss: 4.10436813 - Val loss: 3.99139217
INFO: Early stopping counter 10 of 20 [Best Val Loss: 3.85758355, Val Loss: 3.99139217]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.32it/s]
Epoch 64/1000 - Train loss: 4.00684422 - Val loss: 3.96545848
INFO: Early stopping counter 11 of 20 [Best Val Loss: 3.85758355, Val Loss: 3.96545848]
Epoch 00064: reducing learning rate of group 0 to 1.0000e-05.
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.41it/s]
Epoch 65/1000 - Train loss: 3.93920770 - Val loss: 3.87765694
INFO: Early stopping counter 12 of 20 [Best Val Loss: 3.85758355, Val Loss: 3.87765694]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.44it/s]
Epoch 66/1000 - Train loss: 3.88295544 - Val loss: 3.84944502
INFO: Val_Loss improved (3.85758355 --> 3.84944502). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.26it/s]
100%|██████████| 8/8 [00:00<00:00, 9.51it/s]
Epoch 67/1000 - Train loss: 3.86767789 - Val loss: 3.82956198
INFO: Val_Loss improved (3.84944502 --> 3.82956198). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:00<00:00, 9.47it/s]
Epoch 68/1000 - Train loss: 3.84284116 - Val loss: 3.83163616
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.82956198, Val Loss: 3.83163616]
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:00<00:00, 9.44it/s]
Epoch 69/1000 - Train loss: 3.85159651 - Val loss: 3.82200968
INFO: Val_Loss improved (3.82956198 --> 3.82200968). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:00<00:00, 9.52it/s]
Epoch 70/1000 - Train loss: 3.82052618 - Val loss: 3.80827555
INFO: Val_Loss improved (3.82200968 --> 3.80827555). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.41it/s]
Epoch 71/1000 - Train loss: 3.84265261 - Val loss: 3.80577126
INFO: Val_Loss improved (3.80827555 --> 3.80577126). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.37it/s]
Epoch 72/1000 - Train loss: 3.82410820 - Val loss: 3.79334858
INFO: Val_Loss improved (3.80577126 --> 3.79334858). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.01it/s]
Epoch 73/1000 - Train loss: 3.82503133 - Val loss: 3.79687616
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.79334858, Val Loss: 3.79687616]
100%|██████████| 30/30 [00:09<00:00, 3.23it/s]
100%|██████████| 8/8 [00:00<00:00, 8.36it/s]
Epoch 74/1000 - Train loss: 3.80565243 - Val loss: 3.78701562
INFO: Val_Loss improved (3.79334858 --> 3.78701562). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.30it/s]
100%|██████████| 8/8 [00:00<00:00, 9.02it/s]
Epoch 75/1000 - Train loss: 3.80070345 - Val loss: 3.79263958
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.78701562, Val Loss: 3.79263958]
100%|██████████| 30/30 [00:08<00:00, 3.37it/s]
100%|██████████| 8/8 [00:00<00:00, 8.11it/s]
Epoch 76/1000 - Train loss: 3.81755869 - Val loss: 3.78314465
INFO: Val_Loss improved (3.78701562 --> 3.78314465). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 8.28it/s]
Epoch 77/1000 - Train loss: 3.78451836 - Val loss: 3.78510928
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.78314465, Val Loss: 3.78510928]
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.11it/s]
Epoch 78/1000 - Train loss: 3.78902260 - Val loss: 3.78523952
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.78314465, Val Loss: 3.78523952]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.51it/s]
Epoch 79/1000 - Train loss: 3.78996565 - Val loss: 3.78160250
INFO: Val_Loss improved (3.78314465 --> 3.78160250). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.64it/s]
Epoch 80/1000 - Train loss: 3.79049989 - Val loss: 3.78135321
INFO: Val_Loss improved (3.78160250 --> 3.78135321). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.37it/s]
Epoch 81/1000 - Train loss: 3.78169355 - Val loss: 3.77773052
INFO: Val_Loss improved (3.78135321 --> 3.77773052). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.26it/s]
100%|██████████| 8/8 [00:00<00:00, 9.37it/s]
Epoch 82/1000 - Train loss: 3.79986929 - Val loss: 3.77594328
INFO: Val_Loss improved (3.77773052 --> 3.77594328). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.26it/s]
100%|██████████| 8/8 [00:00<00:00, 9.38it/s]
Epoch 83/1000 - Train loss: 3.78482768 - Val loss: 3.78830889
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.77594328, Val Loss: 3.78830889]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.55it/s]
Epoch 84/1000 - Train loss: 3.77490079 - Val loss: 3.77104747
INFO: Val_Loss improved (3.77594328 --> 3.77104747). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.51it/s]
Epoch 85/1000 - Train loss: 3.78201613 - Val loss: 3.76636192
INFO: Val_Loss improved (3.77104747 --> 3.76636192). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.24it/s]
Epoch 86/1000 - Train loss: 3.79024415 - Val loss: 3.76555797
INFO: Val_Loss improved (3.76636192 --> 3.76555797). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.63it/s]
Epoch 87/1000 - Train loss: 3.78525988 - Val loss: 3.76147664
INFO: Val_Loss improved (3.76555797 --> 3.76147664). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.26it/s]
100%|██████████| 8/8 [00:00<00:00, 8.05it/s]
Epoch 88/1000 - Train loss: 3.75150569 - Val loss: 3.76784489
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.76147664, Val Loss: 3.76784489]
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 8.05it/s]
Epoch 89/1000 - Train loss: 3.77586070 - Val loss: 3.76030931
INFO: Val_Loss improved (3.76147664 --> 3.76030931). Saving model ...
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 8.42it/s]
Epoch 90/1000 - Train loss: 3.76295729 - Val loss: 3.75988063
INFO: Val_Loss improved (3.76030931 --> 3.75988063). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 9.48it/s]
Epoch 91/1000 - Train loss: 3.78959276 - Val loss: 3.75728071
INFO: Val_Loss improved (3.75988063 --> 3.75728071). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.13it/s]
Epoch 92/1000 - Train loss: 3.73473466 - Val loss: 3.75538385
INFO: Val_Loss improved (3.75728071 --> 3.75538385). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.30it/s]
100%|██████████| 8/8 [00:00<00:00, 9.53it/s]
Epoch 93/1000 - Train loss: 3.75608141 - Val loss: 3.77317765
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.77317765]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.32it/s]
Epoch 94/1000 - Train loss: 3.88994872 - Val loss: 3.81714508
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.81714508]
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.42it/s]
Epoch 95/1000 - Train loss: 3.80783256 - Val loss: 3.78769559
INFO: Early stopping counter 3 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.78769559]
100%|██████████| 30/30 [00:08<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.50it/s]
Epoch 96/1000 - Train loss: 3.83292464 - Val loss: 3.78950366
INFO: Early stopping counter 4 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.78950366]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.55it/s]
Epoch 97/1000 - Train loss: 3.76118761 - Val loss: 3.77567178
INFO: Early stopping counter 5 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.77567178]
100%|██████████| 30/30 [00:08<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.27it/s]
Epoch 98/1000 - Train loss: 3.76332304 - Val loss: 3.77460632
INFO: Early stopping counter 6 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.77460632]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.20it/s]
Epoch 99/1000 - Train loss: 3.77482249 - Val loss: 3.78226271
INFO: Early stopping counter 7 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.78226271]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.61it/s]
Epoch 100/1000 - Train loss: 3.73493481 - Val loss: 3.76429901
INFO: Early stopping counter 8 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.76429901]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.47it/s]
Epoch 101/1000 - Train loss: 3.74984626 - Val loss: 3.76292908
INFO: Early stopping counter 9 of 20 [Best Val Loss: 3.75538385, Val Loss: 3.76292908]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.28it/s]
Epoch 102/1000 - Train loss: 3.74703756 - Val loss: 3.74859375
INFO: Val_Loss improved (3.75538385 --> 3.74859375). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:00<00:00, 9.13it/s]
Epoch 103/1000 - Train loss: 3.73837921 - Val loss: 3.74923840
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.74859375, Val Loss: 3.74923840]
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:00<00:00, 9.21it/s]
Epoch 104/1000 - Train loss: 3.75119637 - Val loss: 3.76732951
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.74859375, Val Loss: 3.76732951]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.43it/s]
Epoch 105/1000 - Train loss: 3.73819882 - Val loss: 3.75505343
INFO: Early stopping counter 3 of 20 [Best Val Loss: 3.74859375, Val Loss: 3.75505343]
100%|██████████| 30/30 [00:08<00:00, 3.36it/s]
100%|██████████| 8/8 [00:00<00:00, 9.68it/s]
Epoch 106/1000 - Train loss: 3.73136776 - Val loss: 3.75483164
INFO: Early stopping counter 4 of 20 [Best Val Loss: 3.74859375, Val Loss: 3.75483164]
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:00<00:00, 9.44it/s]
Epoch 107/1000 - Train loss: 3.72989981 - Val loss: 3.75379816
INFO: Early stopping counter 5 of 20 [Best Val Loss: 3.74859375, Val Loss: 3.75379816]
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.52it/s]
Epoch 108/1000 - Train loss: 3.72725613 - Val loss: 3.74747911
INFO: Val_Loss improved (3.74859375 --> 3.74747911). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.56it/s]
Epoch 109/1000 - Train loss: 3.73011173 - Val loss: 3.73601508
INFO: Val_Loss improved (3.74747911 --> 3.73601508). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.25it/s]
100%|██████████| 8/8 [00:00<00:00, 9.55it/s]
Epoch 110/1000 - Train loss: 3.73187178 - Val loss: 3.75182942
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.73601508, Val Loss: 3.75182942]
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 9.49it/s]
Epoch 111/1000 - Train loss: 3.71066221 - Val loss: 3.74586546
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.73601508, Val Loss: 3.74586546]
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.46it/s]
Epoch 112/1000 - Train loss: 3.71336044 - Val loss: 3.73554242
INFO: Val_Loss improved (3.73601508 --> 3.73554242). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:01<00:00, 7.86it/s]
Epoch 113/1000 - Train loss: 3.72597338 - Val loss: 3.73183063
INFO: Val_Loss improved (3.73554242 --> 3.73183063). Saving model ...
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 8.23it/s]
Epoch 114/1000 - Train loss: 3.68859132 - Val loss: 3.73213685
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.73183063, Val Loss: 3.73213685]
100%|██████████| 30/30 [00:09<00:00, 3.27it/s]
100%|██████████| 8/8 [00:00<00:00, 8.76it/s]
Epoch 115/1000 - Train loss: 3.71711060 - Val loss: 3.72904569
INFO: Val_Loss improved (3.73183063 --> 3.72904569). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.29it/s]
100%|██████████| 8/8 [00:00<00:00, 9.39it/s]
Epoch 116/1000 - Train loss: 3.70393657 - Val loss: 3.72765914
INFO: Val_Loss improved (3.72904569 --> 3.72765914). Saving model ...
100%|██████████| 30/30 [00:09<00:00, 3.31it/s]
100%|██████████| 8/8 [00:00<00:00, 9.31it/s]
Epoch 117/1000 - Train loss: 3.72524166 - Val loss: 3.73544490
INFO: Early stopping counter 1 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.73544490]
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 8.94it/s]
Epoch 118/1000 - Train loss: 4.17725425 - Val loss: 3.97696140
INFO: Early stopping counter 2 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.97696140]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.28it/s]
Epoch 119/1000 - Train loss: 3.91945306 - Val loss: 3.84220195
INFO: Early stopping counter 3 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.84220195]
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.19it/s]
Epoch 120/1000 - Train loss: 3.87438979 - Val loss: 3.79740766
INFO: Early stopping counter 4 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.79740766]
100%|██████████| 30/30 [00:09<00:00, 3.28it/s]
100%|██████████| 8/8 [00:00<00:00, 9.50it/s]
Epoch 121/1000 - Train loss: 3.82391456 - Val loss: 3.77655920
INFO: Early stopping counter 5 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.77655920]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.76it/s]
Epoch 122/1000 - Train loss: 3.79587768 - Val loss: 3.76153845
INFO: Early stopping counter 6 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.76153845]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.20it/s]
Epoch 123/1000 - Train loss: 3.76190375 - Val loss: 3.75512239
INFO: Early stopping counter 7 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.75512239]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.38it/s]
Epoch 124/1000 - Train loss: 3.77141262 - Val loss: 3.74580756
INFO: Early stopping counter 8 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.74580756]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.47it/s]
Epoch 125/1000 - Train loss: 3.75412406 - Val loss: 3.74485603
INFO: Early stopping counter 9 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.74485603]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.59it/s]
Epoch 126/1000 - Train loss: 3.76170596 - Val loss: 3.73887038
INFO: Early stopping counter 10 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.73887038]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.51it/s]
Epoch 127/1000 - Train loss: 3.72874257 - Val loss: 3.72954422
INFO: Early stopping counter 11 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.72954422]
Epoch 00127: reducing learning rate of group 0 to 1.0000e-06.
100%|██████████| 30/30 [00:09<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.42it/s]
Epoch 128/1000 - Train loss: 3.73651519 - Val loss: 3.72970301
INFO: Early stopping counter 12 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.72970301]
100%|██████████| 30/30 [00:08<00:00, 3.33it/s]
100%|██████████| 8/8 [00:00<00:00, 9.40it/s]
Epoch 129/1000 - Train loss: 3.72904679 - Val loss: 3.73000795
INFO: Early stopping counter 13 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.73000795]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.47it/s]
Epoch 130/1000 - Train loss: 3.78000259 - Val loss: 3.72892934
INFO: Early stopping counter 14 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.72892934]
100%|██████████| 30/30 [00:08<00:00, 3.35it/s]
100%|██████████| 8/8 [00:00<00:00, 9.68it/s]
Epoch 131/1000 - Train loss: 3.74277909 - Val loss: 3.72932908
INFO: Early stopping counter 15 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.72932908]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.56it/s]
Epoch 132/1000 - Train loss: 3.76630864 - Val loss: 3.72931364
INFO: Early stopping counter 16 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.72931364]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.56it/s]
Epoch 133/1000 - Train loss: 3.76336833 - Val loss: 3.72902253
INFO: Early stopping counter 17 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.72902253]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.91it/s]
Epoch 134/1000 - Train loss: 3.73013325 - Val loss: 3.72983351
INFO: Early stopping counter 18 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.72983351]
100%|██████████| 30/30 [00:09<00:00, 3.32it/s]
100%|██████████| 8/8 [00:00<00:00, 9.57it/s]
Epoch 135/1000 - Train loss: 3.72845173 - Val loss: 3.73007038
INFO: Early stopping counter 19 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.73007038]
100%|██████████| 30/30 [00:08<00:00, 3.34it/s]
100%|██████████| 8/8 [00:00<00:00, 9.40it/s]
Epoch 136/1000 - Train loss: 3.75371335 - Val loss: 3.73033398
INFO: Early stopping counter 20 of 20 [Best Val Loss: 3.72765914, Val Loss: 3.73033398]
INFO: Early stopping DONE!
INFO: Best Val Loss: 3.72765914
Loading best model...
results_uncertainty_aware = indirectUncertaintyAwareHeatmapRegressionPipeline.evaluate(image_paths_test1, landmarks_test1, pixel_spacings_test1)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:03<00:00, 47.94it/s]
100%|██████████| 150/150 [00:03<00:00, 49.44it/s]
Detection report:
1# Point-to-point error (PE) statistics:
======================================================================
Class Mean Median Std Min Max
----------------------------------------------------------------------
Landmark 0 2.16 1.95 1.46 0.36 13.21
Landmark 1 2.49 1.86 2.32 0.03 13.28
Landmark 2 2.81 2.47 1.58 0.20 9.41
Landmark 3 3.18 2.80 2.03 0.15 10.20
Landmark 4 4.14 3.78 2.12 0.54 11.39
Landmark 5 3.98 3.74 2.27 0.24 11.98
Landmark 6 2.04 1.80 1.13 0.07 4.89
Landmark 7 1.88 1.72 1.17 0.19 7.28
Landmark 8 1.75 1.56 1.00 0.12 5.16
Landmark 9 4.21 3.64 2.74 0.55 19.02
Landmark 10 3.54 3.35 1.68 0.45 9.58
Landmark 11 3.38 3.27 1.57 0.48 7.74
Landmark 12 3.75 3.57 1.79 0.24 11.72
Landmark 13 3.71 3.44 2.31 0.30 20.06
Landmark 14 2.22 1.99 1.24 0.14 7.45
Landmark 15 2.96 2.43 1.82 0.21 10.11
Landmark 16 2.06 1.97 1.03 0.14 5.64
Landmark 17 3.17 2.80 1.92 0.08 10.34
Landmark 18 3.07 2.57 2.10 0.31 11.41
======================================================================
2# Success detection rate (SDR):
================================================================================
Class SDR (PE≤2mm) SDR (PE≤2.5mm) SDR (PE≤3mm) SDR (PE≤4mm)
--------------------------------------------------------------------------------
Landmark 0 52.00 68.67 80.67 94.00
Landmark 1 54.00 65.33 72.00 83.33
Landmark 2 32.67 50.67 58.00 81.33
Landmark 3 30.67 44.67 55.33 72.67
Landmark 4 14.67 19.33 36.00 54.00
Landmark 5 20.00 29.33 38.00 54.67
Landmark 6 54.67 72.00 78.00 92.67
Landmark 7 60.67 76.67 84.67 94.67
Landmark 8 66.00 80.00 87.33 96.67
Landmark 9 20.67 28.00 40.67 58.00
Landmark 10 17.33 30.00 41.33 64.00
Landmark 11 22.67 30.67 39.33 68.67
Landmark 12 16.00 24.67 36.67 62.00
Landmark 13 20.00 32.67 43.33 60.00
Landmark 14 51.33 64.00 79.33 90.67
Landmark 15 36.00 52.00 59.33 73.33
Landmark 16 53.33 69.33 83.33 94.67
Landmark 17 30.00 42.67 54.67 74.00
Landmark 18 38.00 48.67 59.33 70.67
================================================================================
Indirect Uncertainty Unaware Heatmap Regression#
from landmarker.pipeline import IndirectUncertaintyUnawareHeatmapRegressionPipeline
train_config = {
"criterion": "mse",
"optimizer": "adam",
"lr": lr,
"batch_size": batch_size,
"epochs": epochs,
"lr_scheduler": True,
}
indirectUncertaintyUnawareHeatmapRegressionPipeline = IndirectUncertaintyUnawareHeatmapRegressionPipeline(
differentiable_decoder="weighted_spatial_mean", model="CoordConvUNet", in_channels_img=1, nb_landmarks=19,
train_config=train_config, heatmap_decoder=coord_argmax, verbose=True, device=device)
indirectUncertaintyUnawareHeatmapRegressionPipeline.fit(image_paths_train, landmarks_train,
transform=composed_transformd, cache_ds=cache_ds)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:03<00:00, 47.31it/s]
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 15.27it/s]
Epoch 1/1000 - Train loss: 1432.58005371 - Val loss: 349.50663185
100%|██████████| 30/30 [00:07<00:00, 3.87it/s]
100%|██████████| 8/8 [00:00<00:00, 15.27it/s]
Epoch 2/1000 - Train loss: 441.34426168 - Val loss: 328.38031387
INFO: Val_Loss improved (349.50663185 --> 328.38031387). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.13it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 3/1000 - Train loss: 376.23353424 - Val loss: 276.64474678
INFO: Val_Loss improved (328.38031387 --> 276.64474678). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████| 8/8 [00:00<00:00, 15.64it/s]
Epoch 4/1000 - Train loss: 392.11278585 - Val loss: 292.77058029
INFO: Early stopping counter 1 of 20 [Best Val Loss: 276.64474678, Val Loss: 292.77058029]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 5/1000 - Train loss: 371.33113047 - Val loss: 310.82950592
INFO: Early stopping counter 2 of 20 [Best Val Loss: 276.64474678, Val Loss: 310.82950592]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 12.54it/s]
Epoch 6/1000 - Train loss: 363.01584117 - Val loss: 274.11997604
INFO: Val_Loss improved (276.64474678 --> 274.11997604). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 7/1000 - Train loss: 357.93456472 - Val loss: 269.37882423
INFO: Val_Loss improved (274.11997604 --> 269.37882423). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 11.77it/s]
Epoch 8/1000 - Train loss: 289.91758982 - Val loss: 257.82475281
INFO: Val_Loss improved (269.37882423 --> 257.82475281). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.12it/s]
100%|██████████| 8/8 [00:00<00:00, 15.35it/s]
Epoch 9/1000 - Train loss: 308.01807709 - Val loss: 241.84839058
INFO: Val_Loss improved (257.82475281 --> 241.84839058). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.09it/s]
100%|██████████| 8/8 [00:00<00:00, 13.06it/s]
Epoch 10/1000 - Train loss: 288.68202159 - Val loss: 254.95864105
INFO: Early stopping counter 1 of 20 [Best Val Loss: 241.84839058, Val Loss: 254.95864105]
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 11/1000 - Train loss: 282.41177114 - Val loss: 238.59182549
INFO: Val_Loss improved (241.84839058 --> 238.59182549). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.33it/s]
Epoch 12/1000 - Train loss: 290.92325134 - Val loss: 291.95289040
INFO: Early stopping counter 1 of 20 [Best Val Loss: 238.59182549, Val Loss: 291.95289040]
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.34it/s]
Epoch 13/1000 - Train loss: 315.44053268 - Val loss: 224.61744308
INFO: Val_Loss improved (238.59182549 --> 224.61744308). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.54it/s]
Epoch 14/1000 - Train loss: 290.51144842 - Val loss: 214.51088142
INFO: Val_Loss improved (224.61744308 --> 214.51088142). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.02it/s]
Epoch 15/1000 - Train loss: 223.35217056 - Val loss: 207.01332283
INFO: Val_Loss improved (214.51088142 --> 207.01332283). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.32it/s]
Epoch 16/1000 - Train loss: 199.61557083 - Val loss: 175.41614342
INFO: Val_Loss improved (207.01332283 --> 175.41614342). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.87it/s]
100%|██████████| 8/8 [00:00<00:00, 14.99it/s]
Epoch 17/1000 - Train loss: 172.55779877 - Val loss: 114.80544949
INFO: Val_Loss improved (175.41614342 --> 114.80544949). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.91it/s]
Epoch 18/1000 - Train loss: 204.76841380 - Val loss: 147.17397022
INFO: Early stopping counter 1 of 20 [Best Val Loss: 114.80544949, Val Loss: 147.17397022]
100%|██████████| 30/30 [00:07<00:00, 4.13it/s]
100%|██████████| 8/8 [00:00<00:00, 12.13it/s]
Epoch 19/1000 - Train loss: 149.39702911 - Val loss: 121.54714203
INFO: Early stopping counter 2 of 20 [Best Val Loss: 114.80544949, Val Loss: 121.54714203]
100%|██████████| 30/30 [00:07<00:00, 3.97it/s]
100%|██████████| 8/8 [00:00<00:00, 15.05it/s]
Epoch 20/1000 - Train loss: 153.00494130 - Val loss: 92.78096199
INFO: Val_Loss improved (114.80544949 --> 92.78096199). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 13.66it/s]
Epoch 21/1000 - Train loss: 138.98860906 - Val loss: 90.76975250
INFO: Val_Loss improved (92.78096199 --> 90.76975250). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 22/1000 - Train loss: 130.99720917 - Val loss: 105.83924389
INFO: Early stopping counter 1 of 20 [Best Val Loss: 90.76975250, Val Loss: 105.83924389]
100%|██████████| 30/30 [00:07<00:00, 4.09it/s]
100%|██████████| 8/8 [00:00<00:00, 15.65it/s]
Epoch 23/1000 - Train loss: 110.34088593 - Val loss: 79.80815887
INFO: Val_Loss improved (90.76975250 --> 79.80815887). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 24/1000 - Train loss: 109.50581690 - Val loss: 69.56830406
INFO: Val_Loss improved (79.80815887 --> 69.56830406). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.32it/s]
Epoch 25/1000 - Train loss: 109.02542178 - Val loss: 74.56030130
INFO: Early stopping counter 1 of 20 [Best Val Loss: 69.56830406, Val Loss: 74.56030130]
100%|██████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████| 8/8 [00:00<00:00, 15.03it/s]
Epoch 26/1000 - Train loss: 100.37559916 - Val loss: 78.38579273
INFO: Early stopping counter 2 of 20 [Best Val Loss: 69.56830406, Val Loss: 78.38579273]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 27/1000 - Train loss: 148.76856906 - Val loss: 166.49565887
INFO: Early stopping counter 3 of 20 [Best Val Loss: 69.56830406, Val Loss: 166.49565887]
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.24it/s]
Epoch 28/1000 - Train loss: 145.68456752 - Val loss: 85.85405684
INFO: Early stopping counter 4 of 20 [Best Val Loss: 69.56830406, Val Loss: 85.85405684]
100%|██████████| 30/30 [00:07<00:00, 4.12it/s]
100%|██████████| 8/8 [00:00<00:00, 15.33it/s]
Epoch 29/1000 - Train loss: 133.25662651 - Val loss: 86.95092010
INFO: Early stopping counter 5 of 20 [Best Val Loss: 69.56830406, Val Loss: 86.95092010]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 12.17it/s]
Epoch 30/1000 - Train loss: 108.84701869 - Val loss: 77.28955984
INFO: Early stopping counter 6 of 20 [Best Val Loss: 69.56830406, Val Loss: 77.28955984]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.55it/s]
Epoch 31/1000 - Train loss: 110.51684799 - Val loss: 114.48029804
INFO: Early stopping counter 7 of 20 [Best Val Loss: 69.56830406, Val Loss: 114.48029804]
100%|██████████| 30/30 [00:07<00:00, 4.14it/s]
100%|██████████| 8/8 [00:00<00:00, 15.34it/s]
Epoch 32/1000 - Train loss: 123.32030792 - Val loss: 68.40540600
INFO: Val_Loss improved (69.56830406 --> 68.40540600). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.92it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 33/1000 - Train loss: 101.17132123 - Val loss: 74.78913021
INFO: Early stopping counter 1 of 20 [Best Val Loss: 68.40540600, Val Loss: 74.78913021]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.00it/s]
Epoch 34/1000 - Train loss: 98.18336678 - Val loss: 87.81921530
INFO: Early stopping counter 2 of 20 [Best Val Loss: 68.40540600, Val Loss: 87.81921530]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.71it/s]
Epoch 35/1000 - Train loss: 95.39057045 - Val loss: 98.71096706
INFO: Early stopping counter 3 of 20 [Best Val Loss: 68.40540600, Val Loss: 98.71096706]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.16it/s]
Epoch 36/1000 - Train loss: 90.37196693 - Val loss: 89.67273140
INFO: Early stopping counter 4 of 20 [Best Val Loss: 68.40540600, Val Loss: 89.67273140]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 12.77it/s]
Epoch 37/1000 - Train loss: 93.65731392 - Val loss: 69.06077671
INFO: Early stopping counter 5 of 20 [Best Val Loss: 68.40540600, Val Loss: 69.06077671]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.00it/s]
Epoch 38/1000 - Train loss: 91.11967379 - Val loss: 69.78103781
INFO: Early stopping counter 6 of 20 [Best Val Loss: 68.40540600, Val Loss: 69.78103781]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.61it/s]
Epoch 39/1000 - Train loss: 82.24810143 - Val loss: 65.18016195
INFO: Val_Loss improved (68.40540600 --> 65.18016195). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 40/1000 - Train loss: 88.69924126 - Val loss: 69.67384768
INFO: Early stopping counter 1 of 20 [Best Val Loss: 65.18016195, Val Loss: 69.67384768]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.56it/s]
Epoch 41/1000 - Train loss: 81.55583661 - Val loss: 69.47547007
INFO: Early stopping counter 2 of 20 [Best Val Loss: 65.18016195, Val Loss: 69.47547007]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.20it/s]
Epoch 42/1000 - Train loss: 80.23750178 - Val loss: 71.28932667
INFO: Early stopping counter 3 of 20 [Best Val Loss: 65.18016195, Val Loss: 71.28932667]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.48it/s]
Epoch 43/1000 - Train loss: 72.64606870 - Val loss: 66.98516893
INFO: Early stopping counter 4 of 20 [Best Val Loss: 65.18016195, Val Loss: 66.98516893]
100%|██████████| 30/30 [00:07<00:00, 4.10it/s]
100%|██████████| 8/8 [00:00<00:00, 11.61it/s]
Epoch 44/1000 - Train loss: 77.36352310 - Val loss: 70.18756485
INFO: Early stopping counter 5 of 20 [Best Val Loss: 65.18016195, Val Loss: 70.18756485]
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.64it/s]
Epoch 45/1000 - Train loss: 85.56965205 - Val loss: 70.31513977
INFO: Early stopping counter 6 of 20 [Best Val Loss: 65.18016195, Val Loss: 70.31513977]
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 11.56it/s]
Epoch 46/1000 - Train loss: 78.27993253 - Val loss: 78.45104170
INFO: Early stopping counter 7 of 20 [Best Val Loss: 65.18016195, Val Loss: 78.45104170]
100%|██████████| 30/30 [00:07<00:00, 4.10it/s]
100%|██████████| 8/8 [00:00<00:00, 15.35it/s]
Epoch 47/1000 - Train loss: 73.50983734 - Val loss: 63.24764442
INFO: Val_Loss improved (65.18016195 --> 63.24764442). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.32it/s]
Epoch 48/1000 - Train loss: 67.33593432 - Val loss: 69.23108292
INFO: Early stopping counter 1 of 20 [Best Val Loss: 63.24764442, Val Loss: 69.23108292]
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 49/1000 - Train loss: 78.84624379 - Val loss: 66.52953577
INFO: Early stopping counter 2 of 20 [Best Val Loss: 63.24764442, Val Loss: 66.52953577]
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 50/1000 - Train loss: 68.60206337 - Val loss: 60.32505608
INFO: Val_Loss improved (63.24764442 --> 60.32505608). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 51/1000 - Train loss: 72.84241130 - Val loss: 60.93443918
INFO: Early stopping counter 1 of 20 [Best Val Loss: 60.32505608, Val Loss: 60.93443918]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.21it/s]
Epoch 52/1000 - Train loss: 69.95638072 - Val loss: 60.75983620
INFO: Early stopping counter 2 of 20 [Best Val Loss: 60.32505608, Val Loss: 60.75983620]
100%|██████████| 30/30 [00:07<00:00, 4.13it/s]
100%|██████████| 8/8 [00:00<00:00, 11.38it/s]
Epoch 53/1000 - Train loss: 76.11658656 - Val loss: 60.97965384
INFO: Early stopping counter 3 of 20 [Best Val Loss: 60.32505608, Val Loss: 60.97965384]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.95it/s]
Epoch 54/1000 - Train loss: 66.61078415 - Val loss: 63.43210602
INFO: Early stopping counter 4 of 20 [Best Val Loss: 60.32505608, Val Loss: 63.43210602]
100%|██████████| 30/30 [00:07<00:00, 4.17it/s]
100%|██████████| 8/8 [00:00<00:00, 15.46it/s]
Epoch 55/1000 - Train loss: 63.42112401 - Val loss: 58.07504606
INFO: Val_Loss improved (60.32505608 --> 58.07504606). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.12it/s]
100%|██████████| 8/8 [00:00<00:00, 15.37it/s]
Epoch 56/1000 - Train loss: 73.73526713 - Val loss: 73.32352638
INFO: Early stopping counter 1 of 20 [Best Val Loss: 58.07504606, Val Loss: 73.32352638]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.32it/s]
Epoch 57/1000 - Train loss: 77.40273018 - Val loss: 61.84420347
INFO: Early stopping counter 2 of 20 [Best Val Loss: 58.07504606, Val Loss: 61.84420347]
100%|██████████| 30/30 [00:07<00:00, 3.92it/s]
100%|██████████| 8/8 [00:00<00:00, 15.42it/s]
Epoch 58/1000 - Train loss: 81.12803154 - Val loss: 63.48475885
INFO: Early stopping counter 3 of 20 [Best Val Loss: 58.07504606, Val Loss: 63.48475885]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 14.97it/s]
Epoch 59/1000 - Train loss: 86.02595889 - Val loss: 82.55435133
INFO: Early stopping counter 4 of 20 [Best Val Loss: 58.07504606, Val Loss: 82.55435133]
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 12.55it/s]
Epoch 60/1000 - Train loss: 143.24568291 - Val loss: 79.94613171
INFO: Early stopping counter 5 of 20 [Best Val Loss: 58.07504606, Val Loss: 79.94613171]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 61/1000 - Train loss: 96.87782338 - Val loss: 66.95822811
INFO: Early stopping counter 6 of 20 [Best Val Loss: 58.07504606, Val Loss: 66.95822811]
100%|██████████| 30/30 [00:07<00:00, 4.13it/s]
100%|██████████| 8/8 [00:00<00:00, 15.20it/s]
Epoch 62/1000 - Train loss: 76.22317467 - Val loss: 60.57251215
INFO: Early stopping counter 7 of 20 [Best Val Loss: 58.07504606, Val Loss: 60.57251215]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.03it/s]
Epoch 63/1000 - Train loss: 74.11097336 - Val loss: 55.47835255
INFO: Val_Loss improved (58.07504606 --> 55.47835255). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.11it/s]
100%|██████████| 8/8 [00:00<00:00, 15.18it/s]
Epoch 64/1000 - Train loss: 65.94261805 - Val loss: 54.79224634
INFO: Val_Loss improved (55.47835255 --> 54.79224634). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.65it/s]
Epoch 65/1000 - Train loss: 70.43326836 - Val loss: 58.09678173
INFO: Early stopping counter 1 of 20 [Best Val Loss: 54.79224634, Val Loss: 58.09678173]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.61it/s]
Epoch 66/1000 - Train loss: 67.89893684 - Val loss: 52.04281616
INFO: Val_Loss improved (54.79224634 --> 52.04281616). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 12.60it/s]
Epoch 67/1000 - Train loss: 64.44804840 - Val loss: 53.40323734
INFO: Early stopping counter 1 of 20 [Best Val Loss: 52.04281616, Val Loss: 53.40323734]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.69it/s]
Epoch 68/1000 - Train loss: 67.42769814 - Val loss: 64.28558397
INFO: Early stopping counter 2 of 20 [Best Val Loss: 52.04281616, Val Loss: 64.28558397]
100%|██████████| 30/30 [00:07<00:00, 4.08it/s]
100%|██████████| 8/8 [00:00<00:00, 12.47it/s]
Epoch 69/1000 - Train loss: 61.12713178 - Val loss: 55.88816166
INFO: Early stopping counter 3 of 20 [Best Val Loss: 52.04281616, Val Loss: 55.88816166]
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.55it/s]
Epoch 70/1000 - Train loss: 69.71374563 - Val loss: 59.79438972
INFO: Early stopping counter 4 of 20 [Best Val Loss: 52.04281616, Val Loss: 59.79438972]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 71/1000 - Train loss: 64.94914627 - Val loss: 58.39906931
INFO: Early stopping counter 5 of 20 [Best Val Loss: 52.04281616, Val Loss: 58.39906931]
100%|██████████| 30/30 [00:07<00:00, 3.94it/s]
100%|██████████| 8/8 [00:00<00:00, 15.42it/s]
Epoch 72/1000 - Train loss: 61.72897911 - Val loss: 56.70644999
INFO: Early stopping counter 6 of 20 [Best Val Loss: 52.04281616, Val Loss: 56.70644999]
100%|██████████| 30/30 [00:07<00:00, 4.09it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 73/1000 - Train loss: 61.88589948 - Val loss: 53.61888790
INFO: Early stopping counter 7 of 20 [Best Val Loss: 52.04281616, Val Loss: 53.61888790]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 13.68it/s]
Epoch 74/1000 - Train loss: 65.35392011 - Val loss: 57.18182087
INFO: Early stopping counter 8 of 20 [Best Val Loss: 52.04281616, Val Loss: 57.18182087]
100%|██████████| 30/30 [00:07<00:00, 4.12it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 75/1000 - Train loss: 60.89752185 - Val loss: 54.57855415
INFO: Early stopping counter 9 of 20 [Best Val Loss: 52.04281616, Val Loss: 54.57855415]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.00it/s]
Epoch 76/1000 - Train loss: 57.33588409 - Val loss: 67.31166267
INFO: Early stopping counter 10 of 20 [Best Val Loss: 52.04281616, Val Loss: 67.31166267]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.62it/s]
Epoch 77/1000 - Train loss: 59.23038228 - Val loss: 64.37057543
INFO: Early stopping counter 11 of 20 [Best Val Loss: 52.04281616, Val Loss: 64.37057543]
Epoch 00077: reducing learning rate of group 0 to 1.0000e-05.
100%|██████████| 30/30 [00:07<00:00, 3.94it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 78/1000 - Train loss: 53.99343840 - Val loss: 49.27794266
INFO: Val_Loss improved (52.04281616 --> 49.27794266). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.49it/s]
Epoch 79/1000 - Train loss: 49.95957686 - Val loss: 45.74886751
INFO: Val_Loss improved (49.27794266 --> 45.74886751). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.10it/s]
100%|██████████| 8/8 [00:00<00:00, 15.27it/s]
Epoch 80/1000 - Train loss: 48.05936750 - Val loss: 46.11921215
INFO: Early stopping counter 1 of 20 [Best Val Loss: 45.74886751, Val Loss: 46.11921215]
100%|██████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████| 8/8 [00:00<00:00, 15.64it/s]
Epoch 81/1000 - Train loss: 47.76334972 - Val loss: 46.49041557
INFO: Early stopping counter 2 of 20 [Best Val Loss: 45.74886751, Val Loss: 46.49041557]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.29it/s]
Epoch 82/1000 - Train loss: 50.12773304 - Val loss: 45.39162493
INFO: Val_Loss improved (45.74886751 --> 45.39162493). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 83/1000 - Train loss: 47.82182096 - Val loss: 46.73012495
INFO: Early stopping counter 1 of 20 [Best Val Loss: 45.39162493, Val Loss: 46.73012495]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 84/1000 - Train loss: 49.85537402 - Val loss: 47.83369255
INFO: Early stopping counter 2 of 20 [Best Val Loss: 45.39162493, Val Loss: 47.83369255]
100%|██████████| 30/30 [00:07<00:00, 4.10it/s]
100%|██████████| 8/8 [00:00<00:00, 12.31it/s]
Epoch 85/1000 - Train loss: 47.18789330 - Val loss: 45.34372807
INFO: Val_Loss improved (45.39162493 --> 45.34372807). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.63it/s]
Epoch 86/1000 - Train loss: 47.02933458 - Val loss: 43.69222641
INFO: Val_Loss improved (45.34372807 --> 43.69222641). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 12.56it/s]
Epoch 87/1000 - Train loss: 43.39310315 - Val loss: 44.21463108
INFO: Early stopping counter 1 of 20 [Best Val Loss: 43.69222641, Val Loss: 44.21463108]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.63it/s]
Epoch 88/1000 - Train loss: 47.67752323 - Val loss: 44.29642296
INFO: Early stopping counter 2 of 20 [Best Val Loss: 43.69222641, Val Loss: 44.29642296]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 11.40it/s]
Epoch 89/1000 - Train loss: 45.24736856 - Val loss: 43.58366776
INFO: Val_Loss improved (43.69222641 --> 43.58366776). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 90/1000 - Train loss: 46.55373739 - Val loss: 44.45827913
INFO: Early stopping counter 1 of 20 [Best Val Loss: 43.58366776, Val Loss: 44.45827913]
100%|██████████| 30/30 [00:07<00:00, 4.10it/s]
100%|██████████| 8/8 [00:00<00:00, 15.65it/s]
Epoch 91/1000 - Train loss: 44.43442904 - Val loss: 44.21039343
INFO: Early stopping counter 2 of 20 [Best Val Loss: 43.58366776, Val Loss: 44.21039343]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.97it/s]
Epoch 92/1000 - Train loss: 45.61654078 - Val loss: 45.46934772
INFO: Early stopping counter 3 of 20 [Best Val Loss: 43.58366776, Val Loss: 45.46934772]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.04it/s]
Epoch 93/1000 - Train loss: 46.54712969 - Val loss: 44.98750424
INFO: Early stopping counter 4 of 20 [Best Val Loss: 43.58366776, Val Loss: 44.98750424]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 14.25it/s]
Epoch 94/1000 - Train loss: 45.66402810 - Val loss: 43.06004572
INFO: Val_Loss improved (43.58366776 --> 43.06004572). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.29it/s]
Epoch 95/1000 - Train loss: 44.35696869 - Val loss: 43.56993628
INFO: Early stopping counter 1 of 20 [Best Val Loss: 43.06004572, Val Loss: 43.56993628]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 11.85it/s]
Epoch 96/1000 - Train loss: 45.29828281 - Val loss: 43.92742300
INFO: Early stopping counter 2 of 20 [Best Val Loss: 43.06004572, Val Loss: 43.92742300]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 14.96it/s]
Epoch 97/1000 - Train loss: 47.01943436 - Val loss: 45.10442710
INFO: Early stopping counter 3 of 20 [Best Val Loss: 43.06004572, Val Loss: 45.10442710]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 11.96it/s]
Epoch 98/1000 - Train loss: 43.12115695 - Val loss: 43.46459413
INFO: Early stopping counter 4 of 20 [Best Val Loss: 43.06004572, Val Loss: 43.46459413]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.49it/s]
Epoch 99/1000 - Train loss: 45.71320915 - Val loss: 44.56985521
INFO: Early stopping counter 5 of 20 [Best Val Loss: 43.06004572, Val Loss: 44.56985521]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.23it/s]
Epoch 100/1000 - Train loss: 41.93958759 - Val loss: 43.53477526
INFO: Early stopping counter 6 of 20 [Best Val Loss: 43.06004572, Val Loss: 43.53477526]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 101/1000 - Train loss: 44.24736137 - Val loss: 43.19081521
INFO: Early stopping counter 7 of 20 [Best Val Loss: 43.06004572, Val Loss: 43.19081521]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.86it/s]
Epoch 102/1000 - Train loss: 46.67050610 - Val loss: 45.04051375
INFO: Early stopping counter 8 of 20 [Best Val Loss: 43.06004572, Val Loss: 45.04051375]
100%|██████████| 30/30 [00:07<00:00, 4.13it/s]
100%|██████████| 8/8 [00:00<00:00, 11.93it/s]
Epoch 103/1000 - Train loss: 43.33753058 - Val loss: 44.29933929
INFO: Early stopping counter 9 of 20 [Best Val Loss: 43.06004572, Val Loss: 44.29933929]
100%|██████████| 30/30 [00:07<00:00, 4.12it/s]
100%|██████████| 8/8 [00:00<00:00, 15.63it/s]
Epoch 104/1000 - Train loss: 43.07311942 - Val loss: 42.00266409
INFO: Val_Loss improved (43.06004572 --> 42.00266409). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.06it/s]
Epoch 105/1000 - Train loss: 42.01729120 - Val loss: 44.21896172
INFO: Early stopping counter 1 of 20 [Best Val Loss: 42.00266409, Val Loss: 44.21896172]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.55it/s]
Epoch 106/1000 - Train loss: 42.56461398 - Val loss: 43.00169897
INFO: Early stopping counter 2 of 20 [Best Val Loss: 42.00266409, Val Loss: 43.00169897]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.27it/s]
Epoch 107/1000 - Train loss: 45.27241758 - Val loss: 43.27138710
INFO: Early stopping counter 3 of 20 [Best Val Loss: 42.00266409, Val Loss: 43.27138710]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.00it/s]
Epoch 108/1000 - Train loss: 44.28864740 - Val loss: 48.00853682
INFO: Early stopping counter 4 of 20 [Best Val Loss: 42.00266409, Val Loss: 48.00853682]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.21it/s]
Epoch 109/1000 - Train loss: 41.55263081 - Val loss: 45.79313374
INFO: Early stopping counter 5 of 20 [Best Val Loss: 42.00266409, Val Loss: 45.79313374]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 12.23it/s]
Epoch 110/1000 - Train loss: 46.41050720 - Val loss: 43.96519136
INFO: Early stopping counter 6 of 20 [Best Val Loss: 42.00266409, Val Loss: 43.96519136]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 14.84it/s]
Epoch 111/1000 - Train loss: 44.34117285 - Val loss: 41.72570753
INFO: Val_Loss improved (42.00266409 --> 41.72570753). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 11.38it/s]
Epoch 112/1000 - Train loss: 42.30998802 - Val loss: 43.60769749
INFO: Early stopping counter 1 of 20 [Best Val Loss: 41.72570753, Val Loss: 43.60769749]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 113/1000 - Train loss: 42.62953256 - Val loss: 43.39713311
INFO: Early stopping counter 2 of 20 [Best Val Loss: 41.72570753, Val Loss: 43.39713311]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 114/1000 - Train loss: 42.67382863 - Val loss: 44.40387130
INFO: Early stopping counter 3 of 20 [Best Val Loss: 41.72570753, Val Loss: 44.40387130]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.05it/s]
Epoch 115/1000 - Train loss: 40.77759870 - Val loss: 43.19646358
INFO: Early stopping counter 4 of 20 [Best Val Loss: 41.72570753, Val Loss: 43.19646358]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.63it/s]
Epoch 116/1000 - Train loss: 41.72474073 - Val loss: 41.55093575
INFO: Val_Loss improved (41.72570753 --> 41.55093575). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.51it/s]
Epoch 117/1000 - Train loss: 43.65695718 - Val loss: 41.72132754
INFO: Early stopping counter 1 of 20 [Best Val Loss: 41.55093575, Val Loss: 41.72132754]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.59it/s]
Epoch 118/1000 - Train loss: 41.00299683 - Val loss: 42.18699288
INFO: Early stopping counter 2 of 20 [Best Val Loss: 41.55093575, Val Loss: 42.18699288]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 11.53it/s]
Epoch 119/1000 - Train loss: 41.38116411 - Val loss: 44.10482073
INFO: Early stopping counter 3 of 20 [Best Val Loss: 41.55093575, Val Loss: 44.10482073]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.19it/s]
Epoch 120/1000 - Train loss: 38.15060463 - Val loss: 43.15365267
INFO: Early stopping counter 4 of 20 [Best Val Loss: 41.55093575, Val Loss: 43.15365267]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 13.19it/s]
Epoch 121/1000 - Train loss: 43.02294884 - Val loss: 40.43577290
INFO: Val_Loss improved (41.55093575 --> 40.43577290). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.53it/s]
Epoch 122/1000 - Train loss: 42.94450041 - Val loss: 42.16876364
INFO: Early stopping counter 1 of 20 [Best Val Loss: 40.43577290, Val Loss: 42.16876364]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.61it/s]
Epoch 123/1000 - Train loss: 40.85943330 - Val loss: 40.82318568
INFO: Early stopping counter 2 of 20 [Best Val Loss: 40.43577290, Val Loss: 40.82318568]
100%|██████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████| 8/8 [00:00<00:00, 15.04it/s]
Epoch 124/1000 - Train loss: 41.09095338 - Val loss: 41.32658911
INFO: Early stopping counter 3 of 20 [Best Val Loss: 40.43577290, Val Loss: 41.32658911]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.10it/s]
Epoch 125/1000 - Train loss: 41.13162270 - Val loss: 43.14753413
INFO: Early stopping counter 4 of 20 [Best Val Loss: 40.43577290, Val Loss: 43.14753413]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.20it/s]
Epoch 126/1000 - Train loss: 40.09490668 - Val loss: 41.58460140
INFO: Early stopping counter 5 of 20 [Best Val Loss: 40.43577290, Val Loss: 41.58460140]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 127/1000 - Train loss: 41.76855564 - Val loss: 42.56486082
INFO: Early stopping counter 6 of 20 [Best Val Loss: 40.43577290, Val Loss: 42.56486082]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 12.03it/s]
Epoch 128/1000 - Train loss: 39.98443769 - Val loss: 41.33374763
INFO: Early stopping counter 7 of 20 [Best Val Loss: 40.43577290, Val Loss: 41.33374763]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.03it/s]
Epoch 129/1000 - Train loss: 41.53404433 - Val loss: 39.90520310
INFO: Val_Loss improved (40.43577290 --> 39.90520310). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 13.59it/s]
Epoch 130/1000 - Train loss: 38.86129697 - Val loss: 41.81429601
INFO: Early stopping counter 1 of 20 [Best Val Loss: 39.90520310, Val Loss: 41.81429601]
100%|██████████| 30/30 [00:07<00:00, 4.13it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 131/1000 - Train loss: 41.45553627 - Val loss: 42.05300331
INFO: Early stopping counter 2 of 20 [Best Val Loss: 39.90520310, Val Loss: 42.05300331]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 11.03it/s]
Epoch 132/1000 - Train loss: 38.07146428 - Val loss: 41.04862666
INFO: Early stopping counter 3 of 20 [Best Val Loss: 39.90520310, Val Loss: 41.04862666]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.52it/s]
Epoch 133/1000 - Train loss: 41.89619344 - Val loss: 40.60813403
INFO: Early stopping counter 4 of 20 [Best Val Loss: 39.90520310, Val Loss: 40.60813403]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.58it/s]
Epoch 134/1000 - Train loss: 38.36455383 - Val loss: 42.45879388
INFO: Early stopping counter 5 of 20 [Best Val Loss: 39.90520310, Val Loss: 42.45879388]
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 15.45it/s]
Epoch 135/1000 - Train loss: 39.22196128 - Val loss: 41.46722555
INFO: Early stopping counter 6 of 20 [Best Val Loss: 39.90520310, Val Loss: 41.46722555]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.59it/s]
Epoch 136/1000 - Train loss: 41.04954656 - Val loss: 39.83861184
INFO: Val_Loss improved (39.90520310 --> 39.83861184). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.90it/s]
100%|██████████| 8/8 [00:00<00:00, 15.64it/s]
Epoch 137/1000 - Train loss: 39.66324927 - Val loss: 41.78479123
INFO: Early stopping counter 1 of 20 [Best Val Loss: 39.83861184, Val Loss: 41.78479123]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 138/1000 - Train loss: 38.61101398 - Val loss: 39.69172168
INFO: Val_Loss improved (39.83861184 --> 39.69172168). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.88it/s]
100%|██████████| 8/8 [00:00<00:00, 15.47it/s]
Epoch 139/1000 - Train loss: 38.76593132 - Val loss: 40.63271689
INFO: Early stopping counter 1 of 20 [Best Val Loss: 39.69172168, Val Loss: 40.63271689]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.61it/s]
Epoch 140/1000 - Train loss: 37.66318372 - Val loss: 39.76094174
INFO: Early stopping counter 2 of 20 [Best Val Loss: 39.69172168, Val Loss: 39.76094174]
100%|██████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 141/1000 - Train loss: 39.09748306 - Val loss: 40.16558433
INFO: Early stopping counter 3 of 20 [Best Val Loss: 39.69172168, Val Loss: 40.16558433]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 142/1000 - Train loss: 38.61544304 - Val loss: 40.27877498
INFO: Early stopping counter 4 of 20 [Best Val Loss: 39.69172168, Val Loss: 40.27877498]
100%|██████████| 30/30 [00:07<00:00, 4.08it/s]
100%|██████████| 8/8 [00:00<00:00, 11.02it/s]
Epoch 143/1000 - Train loss: 39.46086292 - Val loss: 39.65192342
INFO: Val_Loss improved (39.69172168 --> 39.65192342). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████| 8/8 [00:00<00:00, 14.96it/s]
Epoch 144/1000 - Train loss: 38.69194209 - Val loss: 39.90435815
INFO: Early stopping counter 1 of 20 [Best Val Loss: 39.65192342, Val Loss: 39.90435815]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 12.24it/s]
Epoch 145/1000 - Train loss: 39.35000636 - Val loss: 41.41125917
INFO: Early stopping counter 2 of 20 [Best Val Loss: 39.65192342, Val Loss: 41.41125917]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.24it/s]
Epoch 146/1000 - Train loss: 38.05909424 - Val loss: 42.26799989
INFO: Early stopping counter 3 of 20 [Best Val Loss: 39.65192342, Val Loss: 42.26799989]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 11.57it/s]
Epoch 147/1000 - Train loss: 36.41190917 - Val loss: 41.96804261
INFO: Early stopping counter 4 of 20 [Best Val Loss: 39.65192342, Val Loss: 41.96804261]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.63it/s]
Epoch 148/1000 - Train loss: 37.47595558 - Val loss: 39.35655808
INFO: Val_Loss improved (39.65192342 --> 39.35655808). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 13.15it/s]
Epoch 149/1000 - Train loss: 36.11246808 - Val loss: 38.31411266
INFO: Val_Loss improved (39.35655808 --> 38.31411266). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.22it/s]
Epoch 150/1000 - Train loss: 37.98194237 - Val loss: 39.61350465
INFO: Early stopping counter 1 of 20 [Best Val Loss: 38.31411266, Val Loss: 39.61350465]
100%|██████████| 30/30 [00:07<00:00, 4.13it/s]
100%|██████████| 8/8 [00:00<00:00, 15.51it/s]
Epoch 151/1000 - Train loss: 38.55883185 - Val loss: 39.88976884
INFO: Early stopping counter 2 of 20 [Best Val Loss: 38.31411266, Val Loss: 39.88976884]
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 152/1000 - Train loss: 39.67609978 - Val loss: 39.68302035
INFO: Early stopping counter 3 of 20 [Best Val Loss: 38.31411266, Val Loss: 39.68302035]
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 15.64it/s]
Epoch 153/1000 - Train loss: 37.21382109 - Val loss: 39.16732073
INFO: Early stopping counter 4 of 20 [Best Val Loss: 38.31411266, Val Loss: 39.16732073]
100%|██████████| 30/30 [00:07<00:00, 3.92it/s]
100%|██████████| 8/8 [00:00<00:00, 15.24it/s]
Epoch 154/1000 - Train loss: 37.90754827 - Val loss: 37.09850740
INFO: Val_Loss improved (38.31411266 --> 37.09850740). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.59it/s]
Epoch 155/1000 - Train loss: 36.26669127 - Val loss: 40.05311155
INFO: Early stopping counter 1 of 20 [Best Val Loss: 37.09850740, Val Loss: 40.05311155]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 12.13it/s]
Epoch 156/1000 - Train loss: 36.66463846 - Val loss: 39.10835814
INFO: Early stopping counter 2 of 20 [Best Val Loss: 37.09850740, Val Loss: 39.10835814]
100%|██████████| 30/30 [00:07<00:00, 4.09it/s]
100%|██████████| 8/8 [00:00<00:00, 14.95it/s]
Epoch 157/1000 - Train loss: 36.74543413 - Val loss: 40.91708088
INFO: Early stopping counter 3 of 20 [Best Val Loss: 37.09850740, Val Loss: 40.91708088]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 11.75it/s]
Epoch 158/1000 - Train loss: 40.83118757 - Val loss: 39.15069962
INFO: Early stopping counter 4 of 20 [Best Val Loss: 37.09850740, Val Loss: 39.15069962]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.22it/s]
Epoch 159/1000 - Train loss: 37.43844662 - Val loss: 39.31073928
INFO: Early stopping counter 5 of 20 [Best Val Loss: 37.09850740, Val Loss: 39.31073928]
100%|██████████| 30/30 [00:07<00:00, 4.18it/s]
100%|██████████| 8/8 [00:00<00:00, 15.32it/s]
Epoch 160/1000 - Train loss: 37.32476673 - Val loss: 37.62407756
INFO: Early stopping counter 6 of 20 [Best Val Loss: 37.09850740, Val Loss: 37.62407756]
100%|██████████| 30/30 [00:07<00:00, 3.91it/s]
100%|██████████| 8/8 [00:00<00:00, 15.61it/s]
Epoch 161/1000 - Train loss: 36.71446546 - Val loss: 39.28552628
INFO: Early stopping counter 7 of 20 [Best Val Loss: 37.09850740, Val Loss: 39.28552628]
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 15.49it/s]
Epoch 162/1000 - Train loss: 36.16711082 - Val loss: 39.64702678
INFO: Early stopping counter 8 of 20 [Best Val Loss: 37.09850740, Val Loss: 39.64702678]
100%|██████████| 30/30 [00:07<00:00, 3.90it/s]
100%|██████████| 8/8 [00:00<00:00, 14.44it/s]
Epoch 163/1000 - Train loss: 37.23798148 - Val loss: 35.58124685
INFO: Val_Loss improved (37.09850740 --> 35.58124685). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 164/1000 - Train loss: 35.80358257 - Val loss: 36.53410912
INFO: Early stopping counter 1 of 20 [Best Val Loss: 35.58124685, Val Loss: 36.53410912]
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 165/1000 - Train loss: 35.37853165 - Val loss: 39.37427163
INFO: Early stopping counter 2 of 20 [Best Val Loss: 35.58124685, Val Loss: 39.37427163]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.17it/s]
Epoch 166/1000 - Train loss: 36.45082251 - Val loss: 36.49728727
INFO: Early stopping counter 3 of 20 [Best Val Loss: 35.58124685, Val Loss: 36.49728727]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 11.75it/s]
Epoch 167/1000 - Train loss: 38.67345441 - Val loss: 37.12879968
INFO: Early stopping counter 4 of 20 [Best Val Loss: 35.58124685, Val Loss: 37.12879968]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.23it/s]
Epoch 168/1000 - Train loss: 37.46104024 - Val loss: 39.44440150
INFO: Early stopping counter 5 of 20 [Best Val Loss: 35.58124685, Val Loss: 39.44440150]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 11.62it/s]
Epoch 169/1000 - Train loss: 36.76152350 - Val loss: 37.68941903
INFO: Early stopping counter 6 of 20 [Best Val Loss: 35.58124685, Val Loss: 37.68941903]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.27it/s]
Epoch 170/1000 - Train loss: 34.79674727 - Val loss: 38.79588413
INFO: Early stopping counter 7 of 20 [Best Val Loss: 35.58124685, Val Loss: 38.79588413]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.07it/s]
Epoch 171/1000 - Train loss: 34.97310549 - Val loss: 36.43840742
INFO: Early stopping counter 8 of 20 [Best Val Loss: 35.58124685, Val Loss: 36.43840742]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.25it/s]
Epoch 172/1000 - Train loss: 33.48363825 - Val loss: 37.89077759
INFO: Early stopping counter 9 of 20 [Best Val Loss: 35.58124685, Val Loss: 37.89077759]
100%|██████████| 30/30 [00:07<00:00, 4.09it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 173/1000 - Train loss: 34.97876740 - Val loss: 38.62010837
INFO: Early stopping counter 10 of 20 [Best Val Loss: 35.58124685, Val Loss: 38.62010837]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 12.17it/s]
Epoch 174/1000 - Train loss: 35.12396717 - Val loss: 36.46629333
INFO: Early stopping counter 11 of 20 [Best Val Loss: 35.58124685, Val Loss: 36.46629333]
Epoch 00174: reducing learning rate of group 0 to 1.0000e-06.
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.58it/s]
Epoch 175/1000 - Train loss: 31.76579800 - Val loss: 36.28238463
INFO: Early stopping counter 12 of 20 [Best Val Loss: 35.58124685, Val Loss: 36.28238463]
100%|██████████| 30/30 [00:07<00:00, 4.11it/s]
100%|██████████| 8/8 [00:00<00:00, 15.32it/s]
Epoch 176/1000 - Train loss: 31.95371857 - Val loss: 35.54588366
INFO: Val_Loss improved (35.58124685 --> 35.54588366). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.94it/s]
100%|██████████| 8/8 [00:00<00:00, 15.27it/s]
Epoch 177/1000 - Train loss: 32.47842356 - Val loss: 35.68616509
INFO: Early stopping counter 1 of 20 [Best Val Loss: 35.54588366, Val Loss: 35.68616509]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 14.99it/s]
Epoch 178/1000 - Train loss: 31.29053936 - Val loss: 35.28742337
INFO: Val_Loss improved (35.54588366 --> 35.28742337). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.27it/s]
Epoch 179/1000 - Train loss: 31.05234299 - Val loss: 35.73370457
INFO: Early stopping counter 1 of 20 [Best Val Loss: 35.28742337, Val Loss: 35.73370457]
100%|██████████| 30/30 [00:07<00:00, 4.12it/s]
100%|██████████| 8/8 [00:00<00:00, 15.67it/s]
Epoch 180/1000 - Train loss: 34.95473512 - Val loss: 35.00746751
INFO: Val_Loss improved (35.28742337 --> 35.00746751). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 11.64it/s]
Epoch 181/1000 - Train loss: 30.59342505 - Val loss: 34.83002424
INFO: Val_Loss improved (35.00746751 --> 34.83002424). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.15it/s]
100%|██████████| 8/8 [00:00<00:00, 15.29it/s]
Epoch 182/1000 - Train loss: 33.42124945 - Val loss: 34.81971407
INFO: Val_Loss improved (34.83002424 --> 34.81971407). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 12.53it/s]
Epoch 183/1000 - Train loss: 31.43383859 - Val loss: 34.76997685
INFO: Val_Loss improved (34.81971407 --> 34.76997685). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 184/1000 - Train loss: 31.69736621 - Val loss: 34.90452433
INFO: Early stopping counter 1 of 20 [Best Val Loss: 34.76997685, Val Loss: 34.90452433]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 13.42it/s]
Epoch 185/1000 - Train loss: 32.84410947 - Val loss: 34.76568055
INFO: Val_Loss improved (34.76997685 --> 34.76568055). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.15it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 186/1000 - Train loss: 31.34254576 - Val loss: 35.36488295
INFO: Early stopping counter 1 of 20 [Best Val Loss: 34.76568055, Val Loss: 35.36488295]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 187/1000 - Train loss: 32.18668207 - Val loss: 35.41547322
INFO: Early stopping counter 2 of 20 [Best Val Loss: 34.76568055, Val Loss: 35.41547322]
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.41it/s]
Epoch 188/1000 - Train loss: 31.42789682 - Val loss: 34.52363729
INFO: Val_Loss improved (34.76568055 --> 34.52363729). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.98it/s]
Epoch 189/1000 - Train loss: 31.70517286 - Val loss: 34.57112122
INFO: Early stopping counter 1 of 20 [Best Val Loss: 34.52363729, Val Loss: 34.57112122]
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 14.93it/s]
Epoch 190/1000 - Train loss: 31.72186998 - Val loss: 34.85233045
INFO: Early stopping counter 2 of 20 [Best Val Loss: 34.52363729, Val Loss: 34.85233045]
100%|██████████| 30/30 [00:07<00:00, 4.11it/s]
100%|██████████| 8/8 [00:00<00:00, 15.53it/s]
Epoch 191/1000 - Train loss: 31.21776683 - Val loss: 34.34136701
INFO: Val_Loss improved (34.52363729 --> 34.34136701). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.94it/s]
100%|██████████| 8/8 [00:00<00:00, 13.06it/s]
Epoch 192/1000 - Train loss: 31.55443357 - Val loss: 34.69532824
INFO: Early stopping counter 1 of 20 [Best Val Loss: 34.34136701, Val Loss: 34.69532824]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.21it/s]
Epoch 193/1000 - Train loss: 30.45799491 - Val loss: 35.04775357
INFO: Early stopping counter 2 of 20 [Best Val Loss: 34.34136701, Val Loss: 35.04775357]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 12.56it/s]
Epoch 194/1000 - Train loss: 32.25894457 - Val loss: 34.99770403
INFO: Early stopping counter 3 of 20 [Best Val Loss: 34.34136701, Val Loss: 34.99770403]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.54it/s]
Epoch 195/1000 - Train loss: 32.50859782 - Val loss: 34.33271933
INFO: Val_Loss improved (34.34136701 --> 34.33271933). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 196/1000 - Train loss: 32.06724478 - Val loss: 34.70734739
INFO: Early stopping counter 1 of 20 [Best Val Loss: 34.33271933, Val Loss: 34.70734739]
100%|██████████| 30/30 [00:07<00:00, 3.94it/s]
100%|██████████| 8/8 [00:00<00:00, 15.60it/s]
Epoch 197/1000 - Train loss: 31.56831258 - Val loss: 34.28079510
INFO: Val_Loss improved (34.33271933 --> 34.28079510). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.20it/s]
Epoch 198/1000 - Train loss: 33.02170887 - Val loss: 34.85013628
INFO: Early stopping counter 1 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.85013628]
100%|██████████| 30/30 [00:07<00:00, 3.92it/s]
100%|██████████| 8/8 [00:00<00:00, 15.29it/s]
Epoch 199/1000 - Train loss: 33.77564812 - Val loss: 34.78754306
INFO: Early stopping counter 2 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.78754306]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.21it/s]
Epoch 200/1000 - Train loss: 30.78012212 - Val loss: 34.89586806
INFO: Early stopping counter 3 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.89586806]
100%|██████████| 30/30 [00:07<00:00, 3.92it/s]
100%|██████████| 8/8 [00:00<00:00, 15.64it/s]
Epoch 201/1000 - Train loss: 30.32124557 - Val loss: 35.03154564
INFO: Early stopping counter 4 of 20 [Best Val Loss: 34.28079510, Val Loss: 35.03154564]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 14.53it/s]
Epoch 202/1000 - Train loss: 30.50468527 - Val loss: 34.67246771
INFO: Early stopping counter 5 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.67246771]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 12.73it/s]
Epoch 203/1000 - Train loss: 30.45641524 - Val loss: 34.82704782
INFO: Early stopping counter 6 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.82704782]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.05it/s]
Epoch 204/1000 - Train loss: 31.48709227 - Val loss: 34.71329093
INFO: Early stopping counter 7 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.71329093]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 12.39it/s]
Epoch 205/1000 - Train loss: 30.88077679 - Val loss: 34.33069372
INFO: Early stopping counter 8 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.33069372]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.19it/s]
Epoch 206/1000 - Train loss: 31.39557756 - Val loss: 34.42279840
INFO: Early stopping counter 9 of 20 [Best Val Loss: 34.28079510, Val Loss: 34.42279840]
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 14.72it/s]
Epoch 207/1000 - Train loss: 31.61685689 - Val loss: 34.19430447
INFO: Val_Loss improved (34.28079510 --> 34.19430447). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.90it/s]
100%|██████████| 8/8 [00:00<00:00, 15.54it/s]
Epoch 208/1000 - Train loss: 31.05229143 - Val loss: 33.81500292
INFO: Val_Loss improved (34.19430447 --> 33.81500292). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████| 8/8 [00:00<00:00, 11.65it/s]
Epoch 209/1000 - Train loss: 30.89002991 - Val loss: 34.12603807
INFO: Early stopping counter 1 of 20 [Best Val Loss: 33.81500292, Val Loss: 34.12603807]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.18it/s]
Epoch 210/1000 - Train loss: 30.69245472 - Val loss: 35.21102190
INFO: Early stopping counter 2 of 20 [Best Val Loss: 33.81500292, Val Loss: 35.21102190]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.19it/s]
Epoch 211/1000 - Train loss: 30.36144002 - Val loss: 34.03814721
INFO: Early stopping counter 3 of 20 [Best Val Loss: 33.81500292, Val Loss: 34.03814721]
100%|██████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 212/1000 - Train loss: 32.14047769 - Val loss: 34.05796480
INFO: Early stopping counter 4 of 20 [Best Val Loss: 33.81500292, Val Loss: 34.05796480]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.35it/s]
Epoch 213/1000 - Train loss: 30.52763443 - Val loss: 33.86782050
INFO: Early stopping counter 5 of 20 [Best Val Loss: 33.81500292, Val Loss: 33.86782050]
100%|██████████| 30/30 [00:07<00:00, 4.16it/s]
100%|██████████| 8/8 [00:00<00:00, 14.41it/s]
Epoch 214/1000 - Train loss: 32.10073465 - Val loss: 34.00179696
INFO: Early stopping counter 6 of 20 [Best Val Loss: 33.81500292, Val Loss: 34.00179696]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.61it/s]
Epoch 215/1000 - Train loss: 31.13212051 - Val loss: 34.05943942
INFO: Early stopping counter 7 of 20 [Best Val Loss: 33.81500292, Val Loss: 34.05943942]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 12.45it/s]
Epoch 216/1000 - Train loss: 29.96085243 - Val loss: 33.97352171
INFO: Early stopping counter 8 of 20 [Best Val Loss: 33.81500292, Val Loss: 33.97352171]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.62it/s]
Epoch 217/1000 - Train loss: 29.42524077 - Val loss: 34.48810673
INFO: Early stopping counter 9 of 20 [Best Val Loss: 33.81500292, Val Loss: 34.48810673]
100%|██████████| 30/30 [00:07<00:00, 4.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.03it/s]
Epoch 218/1000 - Train loss: 31.56156489 - Val loss: 34.53911352
INFO: Early stopping counter 10 of 20 [Best Val Loss: 33.81500292, Val Loss: 34.53911352]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 219/1000 - Train loss: 29.88689810 - Val loss: 33.49059629
INFO: Val_Loss improved (33.81500292 --> 33.49059629). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.23it/s]
Epoch 220/1000 - Train loss: 30.47460880 - Val loss: 34.25500298
INFO: Early stopping counter 1 of 20 [Best Val Loss: 33.49059629, Val Loss: 34.25500298]
100%|██████████| 30/30 [00:07<00:00, 4.08it/s]
100%|██████████| 8/8 [00:00<00:00, 14.64it/s]
Epoch 221/1000 - Train loss: 30.58950119 - Val loss: 34.25318170
INFO: Early stopping counter 2 of 20 [Best Val Loss: 33.49059629, Val Loss: 34.25318170]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 14.74it/s]
Epoch 222/1000 - Train loss: 30.82144953 - Val loss: 33.45077729
INFO: Val_Loss improved (33.49059629 --> 33.45077729). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 13.59it/s]
Epoch 223/1000 - Train loss: 30.96196607 - Val loss: 33.86279774
INFO: Early stopping counter 1 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.86279774]
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 14.68it/s]
Epoch 224/1000 - Train loss: 29.15051905 - Val loss: 35.09655619
INFO: Early stopping counter 2 of 20 [Best Val Loss: 33.45077729, Val Loss: 35.09655619]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 225/1000 - Train loss: 32.76077836 - Val loss: 33.51816082
INFO: Early stopping counter 3 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.51816082]
100%|██████████| 30/30 [00:07<00:00, 3.88it/s]
100%|██████████| 8/8 [00:00<00:00, 15.26it/s]
Epoch 226/1000 - Train loss: 30.84669577 - Val loss: 33.66254926
INFO: Early stopping counter 4 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.66254926]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.29it/s]
Epoch 227/1000 - Train loss: 30.29721724 - Val loss: 33.67138219
INFO: Early stopping counter 5 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.67138219]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 15.58it/s]
Epoch 228/1000 - Train loss: 30.57647864 - Val loss: 33.46742749
INFO: Early stopping counter 6 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.46742749]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.03it/s]
Epoch 229/1000 - Train loss: 29.75591106 - Val loss: 33.64926910
INFO: Early stopping counter 7 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.64926910]
100%|██████████| 30/30 [00:07<00:00, 3.95it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 230/1000 - Train loss: 29.97538643 - Val loss: 33.92829490
INFO: Early stopping counter 8 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.92829490]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.01it/s]
Epoch 231/1000 - Train loss: 31.07698701 - Val loss: 33.83837986
INFO: Early stopping counter 9 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.83837986]
100%|██████████| 30/30 [00:07<00:00, 4.05it/s]
100%|██████████| 8/8 [00:00<00:00, 11.96it/s]
Epoch 232/1000 - Train loss: 31.61250674 - Val loss: 33.63209629
INFO: Early stopping counter 10 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.63209629]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 15.36it/s]
Epoch 233/1000 - Train loss: 33.61406199 - Val loss: 33.55203414
INFO: Early stopping counter 11 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.55203414]
Epoch 00233: reducing learning rate of group 0 to 1.0000e-07.
100%|██████████| 30/30 [00:07<00:00, 4.04it/s]
100%|██████████| 8/8 [00:00<00:00, 11.53it/s]
Epoch 234/1000 - Train loss: 31.06579107 - Val loss: 33.54258704
INFO: Early stopping counter 12 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.54258704]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 15.63it/s]
Epoch 235/1000 - Train loss: 29.48905837 - Val loss: 33.72266030
INFO: Early stopping counter 13 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.72266030]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 14.98it/s]
Epoch 236/1000 - Train loss: 30.70894318 - Val loss: 33.61566901
INFO: Early stopping counter 14 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.61566901]
100%|██████████| 30/30 [00:07<00:00, 4.00it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 237/1000 - Train loss: 29.87960555 - Val loss: 33.53357887
INFO: Early stopping counter 15 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.53357887]
100%|██████████| 30/30 [00:07<00:00, 4.07it/s]
100%|██████████| 8/8 [00:00<00:00, 14.97it/s]
Epoch 238/1000 - Train loss: 30.74196650 - Val loss: 33.52477837
INFO: Early stopping counter 16 of 20 [Best Val Loss: 33.45077729, Val Loss: 33.52477837]
100%|██████████| 30/30 [00:07<00:00, 4.11it/s]
100%|██████████| 8/8 [00:00<00:00, 12.89it/s]
Epoch 239/1000 - Train loss: 29.69309336 - Val loss: 33.44511509
INFO: Val_Loss improved (33.45077729 --> 33.44511509). Saving model ...
100%|██████████| 30/30 [00:07<00:00, 4.06it/s]
100%|██████████| 8/8 [00:00<00:00, 15.31it/s]
Epoch 240/1000 - Train loss: 31.17034079 - Val loss: 33.52552247
INFO: Early stopping counter 1 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.52552247]
100%|██████████| 30/30 [00:07<00:00, 3.97it/s]
100%|██████████| 8/8 [00:00<00:00, 11.89it/s]
Epoch 241/1000 - Train loss: 30.37048874 - Val loss: 33.67828321
INFO: Early stopping counter 2 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.67828321]
100%|██████████| 30/30 [00:07<00:00, 4.11it/s]
100%|██████████| 8/8 [00:00<00:00, 15.55it/s]
Epoch 242/1000 - Train loss: 31.06290627 - Val loss: 33.63413858
INFO: Early stopping counter 3 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.63413858]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.28it/s]
Epoch 243/1000 - Train loss: 29.24258073 - Val loss: 33.60249090
INFO: Early stopping counter 4 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.60249090]
100%|██████████| 30/30 [00:07<00:00, 3.90it/s]
100%|██████████| 8/8 [00:00<00:00, 15.23it/s]
Epoch 244/1000 - Train loss: 28.31119436 - Val loss: 33.59322262
INFO: Early stopping counter 5 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.59322262]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 14.97it/s]
Epoch 245/1000 - Train loss: 29.73287519 - Val loss: 33.58637333
INFO: Early stopping counter 6 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.58637333]
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.04it/s]
Epoch 246/1000 - Train loss: 31.60354048 - Val loss: 33.67522287
INFO: Early stopping counter 7 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.67522287]
100%|██████████| 30/30 [00:07<00:00, 4.08it/s]
100%|██████████| 8/8 [00:00<00:00, 15.59it/s]
Epoch 247/1000 - Train loss: 33.53346100 - Val loss: 33.64921856
INFO: Early stopping counter 8 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.64921856]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 11.88it/s]
Epoch 248/1000 - Train loss: 31.65353432 - Val loss: 33.59572816
INFO: Early stopping counter 9 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.59572816]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.45it/s]
Epoch 249/1000 - Train loss: 29.99492232 - Val loss: 33.70040941
INFO: Early stopping counter 10 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.70040941]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 12.05it/s]
Epoch 250/1000 - Train loss: 30.16097647 - Val loss: 33.66899109
INFO: Early stopping counter 11 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.66899109]
Epoch 00250: reducing learning rate of group 0 to 1.0000e-08.
100%|██████████| 30/30 [00:07<00:00, 3.98it/s]
100%|██████████| 8/8 [00:00<00:00, 15.45it/s]
Epoch 251/1000 - Train loss: 31.14798743 - Val loss: 33.68243337
INFO: Early stopping counter 12 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.68243337]
100%|██████████| 30/30 [00:07<00:00, 4.01it/s]
100%|██████████| 8/8 [00:00<00:00, 11.78it/s]
Epoch 252/1000 - Train loss: 29.06325385 - Val loss: 33.68182397
INFO: Early stopping counter 13 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.68182397]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 15.11it/s]
Epoch 253/1000 - Train loss: 27.38106060 - Val loss: 33.68635321
INFO: Early stopping counter 14 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.68635321]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.30it/s]
Epoch 254/1000 - Train loss: 29.72787266 - Val loss: 33.69096208
INFO: Early stopping counter 15 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.69096208]
100%|██████████| 30/30 [00:07<00:00, 4.03it/s]
100%|██████████| 8/8 [00:00<00:00, 15.63it/s]
Epoch 255/1000 - Train loss: 32.04155248 - Val loss: 33.67702913
INFO: Early stopping counter 16 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.67702913]
100%|██████████| 30/30 [00:07<00:00, 3.99it/s]
100%|██████████| 8/8 [00:00<00:00, 15.33it/s]
Epoch 256/1000 - Train loss: 29.89670734 - Val loss: 33.66881680
INFO: Early stopping counter 17 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.66881680]
100%|██████████| 30/30 [00:07<00:00, 3.92it/s]
100%|██████████| 8/8 [00:00<00:00, 15.33it/s]
Epoch 257/1000 - Train loss: 28.80437730 - Val loss: 33.67543626
INFO: Early stopping counter 18 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.67543626]
100%|██████████| 30/30 [00:07<00:00, 4.11it/s]
100%|██████████| 8/8 [00:00<00:00, 15.53it/s]
Epoch 258/1000 - Train loss: 29.39330750 - Val loss: 33.67518425
INFO: Early stopping counter 19 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.67518425]
100%|██████████| 30/30 [00:07<00:00, 4.02it/s]
100%|██████████| 8/8 [00:00<00:00, 13.12it/s]
Epoch 259/1000 - Train loss: 30.12864227 - Val loss: 33.67101192
INFO: Early stopping counter 20 of 20 [Best Val Loss: 33.44511509, Val Loss: 33.67101192]
INFO: Early stopping DONE!
INFO: Best Val Loss: 33.44511509
Loading best model...
results_uncertainty_unaware = indirectUncertaintyUnawareHeatmapRegressionPipeline.evaluate(image_paths_test1, landmarks_test1, pixel_spacings_test1)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:03<00:00, 47.73it/s]
100%|██████████| 150/150 [00:02<00:00, 52.91it/s]
Detection report:
1# Point-to-point error (PE) statistics:
======================================================================
Class Mean Median Std Min Max
----------------------------------------------------------------------
Landmark 0 123.46 122.84 10.00 102.38 146.31
Landmark 1 112.95 113.62 8.43 82.32 139.87
Landmark 2 89.53 88.91 7.64 71.85 108.62
Landmark 3 120.90 119.42 10.27 98.76 144.31
Landmark 4 56.40 56.65 5.07 35.55 68.80
Landmark 5 26.53 26.62 3.33 5.92 34.57
Landmark 6 14.11 14.03 2.70 6.22 20.56
Landmark 7 14.95 14.94 3.24 2.88 23.62
Landmark 8 12.69 12.70 2.75 4.58 21.13
Landmark 9 79.18 79.23 6.88 52.60 98.88
Landmark 10 39.95 39.92 4.03 18.74 51.52
Landmark 11 39.01 38.84 4.46 20.20 49.50
Landmark 12 47.56 48.15 6.54 8.84 60.84
Landmark 13 29.86 29.36 4.50 11.30 46.40
Landmark 14 59.56 59.26 5.68 34.53 78.03
Landmark 15 6.82 6.78 2.49 1.79 15.79
Landmark 16 82.30 81.26 7.71 65.93 101.29
Landmark 17 65.33 65.29 6.23 39.11 85.99
Landmark 18 108.08 106.84 10.07 85.59 131.33
======================================================================
2# Success detection rate (SDR):
================================================================================
Class SDR (PE≤2mm) SDR (PE≤2.5mm) SDR (PE≤3mm) SDR (PE≤4mm)
--------------------------------------------------------------------------------
Landmark 0 0.00 0.00 0.00 0.00
Landmark 1 0.00 0.00 0.00 0.00
Landmark 2 0.00 0.00 0.00 0.00
Landmark 3 0.00 0.00 0.00 0.00
Landmark 4 0.00 0.00 0.00 0.00
Landmark 5 0.00 0.00 0.00 0.00
Landmark 6 0.00 0.00 0.00 0.00
Landmark 7 0.00 0.00 0.67 0.67
Landmark 8 0.00 0.00 0.00 0.00
Landmark 9 0.00 0.00 0.00 0.00
Landmark 10 0.00 0.00 0.00 0.00
Landmark 11 0.00 0.00 0.00 0.00
Landmark 12 0.00 0.00 0.00 0.00
Landmark 13 0.00 0.00 0.00 0.00
Landmark 14 0.00 0.00 0.00 0.00
Landmark 15 1.33 2.67 4.67 13.33
Landmark 16 0.00 0.00 0.00 0.00
Landmark 17 0.00 0.00 0.00 0.00
Landmark 18 0.00 0.00 0.00 0.00
================================================================================
Coordinate Regression#
from landmarker.pipeline import CoordinateRegressionPipeline
from monai.networks.nets import DenseNet121
train_config = {
"criterion": "l1",
"optimizer": "adam",
"lr": lr,
"batch_size": batch_size,
"epochs": epochs,
"lr_scheduler": True,
}
coordinateRegressionPipeline = CoordinateRegressionPipeline(
model=DenseNet121(spatial_dims=2, in_channels=1, out_channels=19*2),
in_channels_img=1, nb_landmarks=19, train_config=train_config, verbose=True, device=device, dim_img=dim_img)
coordinateRegressionPipeline.fit(image_paths_train, landmarks_train,
transform=composed_transformd, cache_ds=cache_ds)
Reading, normalizing and resizing 150 images and landmarks...
100%|██████████| 150/150 [00:03<00:00, 48.42it/s]
100%|██████████| 30/30 [00:05<00:00, 5.82it/s]
100%|██████████| 8/8 [00:00<00:00, 23.18it/s]
Epoch 1/1000 - Train loss: 324.49358622 - Val loss: 323.91516495
100%|██████████| 30/30 [00:05<00:00, 5.72it/s]
100%|██████████| 8/8 [00:00<00:00, 32.02it/s]
Epoch 2/1000 - Train loss: 324.07745972 - Val loss: 322.07958603
INFO: Val_Loss improved (323.91516495 --> 322.07958603). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.81it/s]
100%|██████████| 8/8 [00:00<00:00, 32.16it/s]
Epoch 3/1000 - Train loss: 321.31649475 - Val loss: 320.57791138
INFO: Val_Loss improved (322.07958603 --> 320.57791138). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.83it/s]
100%|██████████| 8/8 [00:00<00:00, 31.74it/s]
Epoch 4/1000 - Train loss: 320.53109233 - Val loss: 319.04413986
INFO: Val_Loss improved (320.57791138 --> 319.04413986). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.84it/s]
100%|██████████| 8/8 [00:00<00:00, 31.22it/s]
Epoch 5/1000 - Train loss: 318.33055827 - Val loss: 317.20576477
INFO: Val_Loss improved (319.04413986 --> 317.20576477). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 31.39it/s]
Epoch 6/1000 - Train loss: 316.57220968 - Val loss: 315.44232559
INFO: Val_Loss improved (317.20576477 --> 315.44232559). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.89it/s]
100%|██████████| 8/8 [00:00<00:00, 21.74it/s]
Epoch 7/1000 - Train loss: 313.60923767 - Val loss: 313.39818573
INFO: Val_Loss improved (315.44232559 --> 313.39818573). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.03it/s]
100%|██████████| 8/8 [00:00<00:00, 31.96it/s]
Epoch 8/1000 - Train loss: 311.76450195 - Val loss: 311.54541397
INFO: Val_Loss improved (313.39818573 --> 311.54541397). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 32.06it/s]
Epoch 9/1000 - Train loss: 310.97837524 - Val loss: 309.60527039
INFO: Val_Loss improved (311.54541397 --> 309.60527039). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 32.13it/s]
Epoch 10/1000 - Train loss: 307.46528524 - Val loss: 307.65011597
INFO: Val_Loss improved (309.60527039 --> 307.65011597). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 18.39it/s]
Epoch 11/1000 - Train loss: 307.67766317 - Val loss: 305.64563751
INFO: Val_Loss improved (307.65011597 --> 305.64563751). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.00it/s]
100%|██████████| 8/8 [00:00<00:00, 32.28it/s]
Epoch 12/1000 - Train loss: 305.41239014 - Val loss: 303.70878220
INFO: Val_Loss improved (305.64563751 --> 303.70878220). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 31.27it/s]
Epoch 13/1000 - Train loss: 301.28639221 - Val loss: 301.56871414
INFO: Val_Loss improved (303.70878220 --> 301.56871414). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 31.97it/s]
Epoch 14/1000 - Train loss: 299.98548482 - Val loss: 299.44775772
INFO: Val_Loss improved (301.56871414 --> 299.44775772). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 19.59it/s]
Epoch 15/1000 - Train loss: 299.53161519 - Val loss: 297.21219254
INFO: Val_Loss improved (299.44775772 --> 297.21219254). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.82it/s]
100%|██████████| 8/8 [00:00<00:00, 32.24it/s]
Epoch 16/1000 - Train loss: 295.80570679 - Val loss: 295.26533890
INFO: Val_Loss improved (297.21219254 --> 295.26533890). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.89it/s]
100%|██████████| 8/8 [00:00<00:00, 30.91it/s]
Epoch 17/1000 - Train loss: 294.85815837 - Val loss: 292.86143494
INFO: Val_Loss improved (295.26533890 --> 292.86143494). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.64it/s]
100%|██████████| 8/8 [00:00<00:00, 31.82it/s]
Epoch 18/1000 - Train loss: 292.41820577 - Val loss: 290.76452255
INFO: Val_Loss improved (292.86143494 --> 290.76452255). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 19.63it/s]
Epoch 19/1000 - Train loss: 288.93051758 - Val loss: 288.57660294
INFO: Val_Loss improved (290.76452255 --> 288.57660294). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 31.92it/s]
Epoch 20/1000 - Train loss: 286.94795837 - Val loss: 286.25555801
INFO: Val_Loss improved (288.57660294 --> 286.25555801). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 32.10it/s]
Epoch 21/1000 - Train loss: 285.20654195 - Val loss: 283.62291336
INFO: Val_Loss improved (286.25555801 --> 283.62291336). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 32.06it/s]
Epoch 22/1000 - Train loss: 282.69975891 - Val loss: 281.33041000
INFO: Val_Loss improved (283.62291336 --> 281.33041000). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.96it/s]
100%|██████████| 8/8 [00:00<00:00, 32.50it/s]
Epoch 23/1000 - Train loss: 280.47198181 - Val loss: 279.17103577
INFO: Val_Loss improved (281.33041000 --> 279.17103577). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.84it/s]
100%|██████████| 8/8 [00:00<00:00, 32.28it/s]
Epoch 24/1000 - Train loss: 277.67555745 - Val loss: 276.74807358
INFO: Val_Loss improved (279.17103577 --> 276.74807358). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.95it/s]
100%|██████████| 8/8 [00:00<00:00, 31.15it/s]
Epoch 25/1000 - Train loss: 275.32124634 - Val loss: 274.26492310
INFO: Val_Loss improved (276.74807358 --> 274.26492310). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 32.18it/s]
Epoch 26/1000 - Train loss: 273.04855245 - Val loss: 272.01168823
INFO: Val_Loss improved (274.26492310 --> 272.01168823). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.95it/s]
100%|██████████| 8/8 [00:00<00:00, 32.03it/s]
Epoch 27/1000 - Train loss: 271.03771261 - Val loss: 268.95243073
INFO: Val_Loss improved (272.01168823 --> 268.95243073). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.90it/s]
100%|██████████| 8/8 [00:00<00:00, 30.74it/s]
Epoch 28/1000 - Train loss: 268.43554382 - Val loss: 266.45378494
INFO: Val_Loss improved (268.95243073 --> 266.45378494). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.81it/s]
100%|██████████| 8/8 [00:00<00:00, 32.10it/s]
Epoch 29/1000 - Train loss: 265.61177470 - Val loss: 263.88727570
INFO: Val_Loss improved (266.45378494 --> 263.88727570). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.95it/s]
100%|██████████| 8/8 [00:00<00:00, 18.23it/s]
Epoch 30/1000 - Train loss: 262.51192067 - Val loss: 261.24892998
INFO: Val_Loss improved (263.88727570 --> 261.24892998). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 32.56it/s]
Epoch 31/1000 - Train loss: 259.83848521 - Val loss: 258.63055229
INFO: Val_Loss improved (261.24892998 --> 258.63055229). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.87it/s]
100%|██████████| 8/8 [00:00<00:00, 31.92it/s]
Epoch 32/1000 - Train loss: 257.04504852 - Val loss: 255.29321671
INFO: Val_Loss improved (258.63055229 --> 255.29321671). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.64it/s]
100%|██████████| 8/8 [00:00<00:00, 32.31it/s]
Epoch 33/1000 - Train loss: 254.69827372 - Val loss: 253.20640564
INFO: Val_Loss improved (255.29321671 --> 253.20640564). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.96it/s]
100%|██████████| 8/8 [00:00<00:00, 31.50it/s]
Epoch 34/1000 - Train loss: 250.89113770 - Val loss: 250.18988800
INFO: Val_Loss improved (253.20640564 --> 250.18988800). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.00it/s]
100%|██████████| 8/8 [00:00<00:00, 32.27it/s]
Epoch 35/1000 - Train loss: 249.36909587 - Val loss: 247.70144081
INFO: Val_Loss improved (250.18988800 --> 247.70144081). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 31.87it/s]
Epoch 36/1000 - Train loss: 245.84058685 - Val loss: 244.76080322
INFO: Val_Loss improved (247.70144081 --> 244.76080322). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.88it/s]
100%|██████████| 8/8 [00:00<00:00, 29.57it/s]
Epoch 37/1000 - Train loss: 242.37921855 - Val loss: 242.15181923
INFO: Val_Loss improved (244.76080322 --> 242.15181923). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.90it/s]
100%|██████████| 8/8 [00:00<00:00, 32.76it/s]
Epoch 38/1000 - Train loss: 240.86630096 - Val loss: 238.82681084
INFO: Val_Loss improved (242.15181923 --> 238.82681084). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 31.70it/s]
Epoch 39/1000 - Train loss: 237.55419871 - Val loss: 235.35786819
INFO: Val_Loss improved (238.82681084 --> 235.35786819). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.92it/s]
100%|██████████| 8/8 [00:00<00:00, 29.88it/s]
Epoch 40/1000 - Train loss: 235.43022817 - Val loss: 233.82107925
INFO: Val_Loss improved (235.35786819 --> 233.82107925). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.95it/s]
100%|██████████| 8/8 [00:00<00:00, 19.08it/s]
Epoch 41/1000 - Train loss: 231.79271291 - Val loss: 229.93412018
INFO: Val_Loss improved (233.82107925 --> 229.93412018). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.97it/s]
100%|██████████| 8/8 [00:00<00:00, 32.32it/s]
Epoch 42/1000 - Train loss: 228.58647766 - Val loss: 227.22703171
INFO: Val_Loss improved (229.93412018 --> 227.22703171). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.74it/s]
100%|██████████| 8/8 [00:00<00:00, 32.55it/s]
Epoch 43/1000 - Train loss: 226.49010010 - Val loss: 223.91348839
INFO: Val_Loss improved (227.22703171 --> 223.91348839). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.85it/s]
100%|██████████| 8/8 [00:00<00:00, 32.50it/s]
Epoch 44/1000 - Train loss: 222.12014669 - Val loss: 221.21554756
INFO: Val_Loss improved (223.91348839 --> 221.21554756). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.83it/s]
100%|██████████| 8/8 [00:00<00:00, 31.97it/s]
Epoch 45/1000 - Train loss: 219.28648834 - Val loss: 217.59727097
INFO: Val_Loss improved (221.21554756 --> 217.59727097). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.01it/s]
100%|██████████| 8/8 [00:00<00:00, 31.21it/s]
Epoch 46/1000 - Train loss: 215.61562602 - Val loss: 214.50060654
INFO: Val_Loss improved (217.59727097 --> 214.50060654). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.86it/s]
100%|██████████| 8/8 [00:00<00:00, 32.64it/s]
Epoch 47/1000 - Train loss: 212.56311747 - Val loss: 211.56612587
INFO: Val_Loss improved (214.50060654 --> 211.56612587). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.00it/s]
100%|██████████| 8/8 [00:00<00:00, 19.20it/s]
Epoch 48/1000 - Train loss: 209.12319234 - Val loss: 208.63616562
INFO: Val_Loss improved (211.56612587 --> 208.63616562). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.97it/s]
100%|██████████| 8/8 [00:00<00:00, 32.08it/s]
Epoch 49/1000 - Train loss: 205.59744670 - Val loss: 204.91158485
INFO: Val_Loss improved (208.63616562 --> 204.91158485). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.70it/s]
100%|██████████| 8/8 [00:00<00:00, 32.22it/s]
Epoch 50/1000 - Train loss: 203.56117910 - Val loss: 201.36930084
INFO: Val_Loss improved (204.91158485 --> 201.36930084). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.89it/s]
100%|██████████| 8/8 [00:00<00:00, 30.75it/s]
Epoch 51/1000 - Train loss: 200.57910817 - Val loss: 198.76955795
INFO: Val_Loss improved (201.36930084 --> 198.76955795). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 20.61it/s]
Epoch 52/1000 - Train loss: 196.60879924 - Val loss: 194.91108131
INFO: Val_Loss improved (198.76955795 --> 194.91108131). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 31.82it/s]
Epoch 53/1000 - Train loss: 193.08856710 - Val loss: 191.35575485
INFO: Val_Loss improved (194.91108131 --> 191.35575485). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.90it/s]
100%|██████████| 8/8 [00:00<00:00, 32.48it/s]
Epoch 54/1000 - Train loss: 188.76516724 - Val loss: 188.03713417
INFO: Val_Loss improved (191.35575485 --> 188.03713417). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 31.41it/s]
Epoch 55/1000 - Train loss: 186.28011627 - Val loss: 183.74325943
INFO: Val_Loss improved (188.03713417 --> 183.74325943). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.01it/s]
100%|██████████| 8/8 [00:00<00:00, 31.90it/s]
Epoch 56/1000 - Train loss: 182.78583171 - Val loss: 180.18282700
INFO: Val_Loss improved (183.74325943 --> 180.18282700). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.92it/s]
100%|██████████| 8/8 [00:00<00:00, 32.21it/s]
Epoch 57/1000 - Train loss: 179.84035594 - Val loss: 176.88437080
INFO: Val_Loss improved (180.18282700 --> 176.88437080). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.62it/s]
100%|██████████| 8/8 [00:00<00:00, 31.78it/s]
Epoch 58/1000 - Train loss: 176.30698497 - Val loss: 174.24897385
INFO: Val_Loss improved (176.88437080 --> 174.24897385). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.82it/s]
100%|██████████| 8/8 [00:00<00:00, 32.08it/s]
Epoch 59/1000 - Train loss: 172.26444906 - Val loss: 170.75498009
INFO: Val_Loss improved (174.24897385 --> 170.75498009). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.97it/s]
100%|██████████| 8/8 [00:00<00:00, 32.60it/s]
Epoch 60/1000 - Train loss: 167.63980713 - Val loss: 166.24442101
INFO: Val_Loss improved (170.75498009 --> 166.24442101). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 27.99it/s]
Epoch 61/1000 - Train loss: 165.39171600 - Val loss: 163.46861458
INFO: Val_Loss improved (166.24442101 --> 163.46861458). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.92it/s]
100%|██████████| 8/8 [00:00<00:00, 32.33it/s]
Epoch 62/1000 - Train loss: 162.52741241 - Val loss: 158.89403725
INFO: Val_Loss improved (163.46861458 --> 158.89403725). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.03it/s]
100%|██████████| 8/8 [00:00<00:00, 18.91it/s]
Epoch 63/1000 - Train loss: 157.34274902 - Val loss: 155.85675430
INFO: Val_Loss improved (158.89403725 --> 155.85675430). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.99it/s]
100%|██████████| 8/8 [00:00<00:00, 32.24it/s]
Epoch 64/1000 - Train loss: 154.14562124 - Val loss: 152.42170906
INFO: Val_Loss improved (155.85675430 --> 152.42170906). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.82it/s]
100%|██████████| 8/8 [00:00<00:00, 32.25it/s]
Epoch 65/1000 - Train loss: 150.27391154 - Val loss: 148.58396339
INFO: Val_Loss improved (152.42170906 --> 148.58396339). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.83it/s]
100%|██████████| 8/8 [00:00<00:00, 32.36it/s]
Epoch 66/1000 - Train loss: 146.54411926 - Val loss: 144.47958183
INFO: Val_Loss improved (148.58396339 --> 144.47958183). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.01it/s]
100%|██████████| 8/8 [00:00<00:00, 32.16it/s]
Epoch 67/1000 - Train loss: 145.36394450 - Val loss: 140.07986069
INFO: Val_Loss improved (144.47958183 --> 140.07986069). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.84it/s]
100%|██████████| 8/8 [00:00<00:00, 30.80it/s]
Epoch 68/1000 - Train loss: 140.47779719 - Val loss: 137.96055412
INFO: Val_Loss improved (140.07986069 --> 137.96055412). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.76it/s]
100%|██████████| 8/8 [00:00<00:00, 29.35it/s]
Epoch 69/1000 - Train loss: 135.90234375 - Val loss: 132.62303543
INFO: Val_Loss improved (137.96055412 --> 132.62303543). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 32.58it/s]
Epoch 70/1000 - Train loss: 131.86049220 - Val loss: 129.09889317
INFO: Val_Loss improved (132.62303543 --> 129.09889317). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.75it/s]
100%|██████████| 8/8 [00:00<00:00, 32.10it/s]
Epoch 71/1000 - Train loss: 129.10680008 - Val loss: 125.08409023
INFO: Val_Loss improved (129.09889317 --> 125.08409023). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.86it/s]
100%|██████████| 8/8 [00:00<00:00, 32.49it/s]
Epoch 72/1000 - Train loss: 125.55691833 - Val loss: 122.97891617
INFO: Val_Loss improved (125.08409023 --> 122.97891617). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 31.05it/s]
Epoch 73/1000 - Train loss: 121.94600856 - Val loss: 119.56179237
INFO: Val_Loss improved (122.97891617 --> 119.56179237). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.88it/s]
100%|██████████| 8/8 [00:00<00:00, 32.17it/s]
Epoch 74/1000 - Train loss: 118.38614502 - Val loss: 116.64122391
INFO: Val_Loss improved (119.56179237 --> 116.64122391). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.98it/s]
100%|██████████| 8/8 [00:00<00:00, 32.48it/s]
Epoch 75/1000 - Train loss: 115.36249924 - Val loss: 112.06975842
INFO: Val_Loss improved (116.64122391 --> 112.06975842). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.76it/s]
100%|██████████| 8/8 [00:00<00:00, 32.08it/s]
Epoch 76/1000 - Train loss: 112.20607630 - Val loss: 110.26883507
INFO: Val_Loss improved (112.06975842 --> 110.26883507). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.82it/s]
100%|██████████| 8/8 [00:00<00:00, 31.24it/s]
Epoch 77/1000 - Train loss: 107.60957692 - Val loss: 105.54768753
INFO: Val_Loss improved (110.26883507 --> 105.54768753). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.96it/s]
100%|██████████| 8/8 [00:00<00:00, 19.54it/s]
Epoch 78/1000 - Train loss: 105.00187556 - Val loss: 101.88102245
INFO: Val_Loss improved (105.54768753 --> 101.88102245). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.95it/s]
100%|██████████| 8/8 [00:00<00:00, 32.52it/s]
Epoch 79/1000 - Train loss: 101.86699041 - Val loss: 98.19607353
INFO: Val_Loss improved (101.88102245 --> 98.19607353). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.72it/s]
100%|██████████| 8/8 [00:00<00:00, 32.50it/s]
Epoch 80/1000 - Train loss: 99.38721135 - Val loss: 95.92329311
INFO: Val_Loss improved (98.19607353 --> 95.92329311). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.90it/s]
100%|██████████| 8/8 [00:00<00:00, 31.99it/s]
Epoch 81/1000 - Train loss: 94.53010534 - Val loss: 92.38360977
INFO: Val_Loss improved (95.92329311 --> 92.38360977). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.73it/s]
100%|██████████| 8/8 [00:00<00:00, 17.67it/s]
Epoch 82/1000 - Train loss: 90.46144969 - Val loss: 87.62308407
INFO: Val_Loss improved (92.38360977 --> 87.62308407). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.98it/s]
100%|██████████| 8/8 [00:00<00:00, 31.96it/s]
Epoch 83/1000 - Train loss: 87.49473089 - Val loss: 83.76825714
INFO: Val_Loss improved (87.62308407 --> 83.76825714). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.84it/s]
100%|██████████| 8/8 [00:00<00:00, 32.54it/s]
Epoch 84/1000 - Train loss: 85.14325994 - Val loss: 80.24900532
INFO: Val_Loss improved (83.76825714 --> 80.24900532). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.67it/s]
100%|██████████| 8/8 [00:00<00:00, 32.28it/s]
Epoch 85/1000 - Train loss: 82.10529404 - Val loss: 77.43313789
INFO: Val_Loss improved (80.24900532 --> 77.43313789). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.96it/s]
100%|██████████| 8/8 [00:00<00:00, 32.29it/s]
Epoch 86/1000 - Train loss: 77.68520101 - Val loss: 72.15098572
INFO: Val_Loss improved (77.43313789 --> 72.15098572). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.99it/s]
100%|██████████| 8/8 [00:00<00:00, 32.37it/s]
Epoch 87/1000 - Train loss: 75.38254573 - Val loss: 68.57639933
INFO: Val_Loss improved (72.15098572 --> 68.57639933). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 30.26it/s]
Epoch 88/1000 - Train loss: 71.95964864 - Val loss: 69.60506296
INFO: Early stopping counter 1 of 20 [Best Val Loss: 68.57639933, Val Loss: 69.60506296]
100%|██████████| 30/30 [00:04<00:00, 6.09it/s]
100%|██████████| 8/8 [00:00<00:00, 18.14it/s]
Epoch 89/1000 - Train loss: 69.96128527 - Val loss: 65.90269756
INFO: Val_Loss improved (68.57639933 --> 65.90269756). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.85it/s]
100%|██████████| 8/8 [00:00<00:00, 32.55it/s]
Epoch 90/1000 - Train loss: 65.82599093 - Val loss: 60.41327572
INFO: Val_Loss improved (65.90269756 --> 60.41327572). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 32.59it/s]
Epoch 91/1000 - Train loss: 62.15390053 - Val loss: 60.32777882
INFO: Val_Loss improved (60.41327572 --> 60.32777882). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.81it/s]
100%|██████████| 8/8 [00:00<00:00, 31.91it/s]
Epoch 92/1000 - Train loss: 59.52071597 - Val loss: 56.33614874
INFO: Val_Loss improved (60.32777882 --> 56.33614874). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 30.01it/s]
Epoch 93/1000 - Train loss: 57.07577337 - Val loss: 49.44332170
INFO: Val_Loss improved (56.33614874 --> 49.44332170). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.88it/s]
100%|██████████| 8/8 [00:00<00:00, 32.36it/s]
Epoch 94/1000 - Train loss: 53.88066483 - Val loss: 52.60208893
INFO: Early stopping counter 1 of 20 [Best Val Loss: 49.44332170, Val Loss: 52.60208893]
100%|██████████| 30/30 [00:05<00:00, 5.76it/s]
100%|██████████| 8/8 [00:00<00:00, 32.20it/s]
Epoch 95/1000 - Train loss: 51.65313517 - Val loss: 43.56783295
INFO: Val_Loss improved (49.44332170 --> 43.56783295). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.02it/s]
100%|██████████| 8/8 [00:00<00:00, 19.80it/s]
Epoch 96/1000 - Train loss: 48.17310168 - Val loss: 50.46462440
INFO: Early stopping counter 1 of 20 [Best Val Loss: 43.56783295, Val Loss: 50.46462440]
100%|██████████| 30/30 [00:04<00:00, 6.01it/s]
100%|██████████| 8/8 [00:00<00:00, 32.54it/s]
Epoch 97/1000 - Train loss: 46.94108785 - Val loss: 43.21062946
INFO: Val_Loss improved (43.56783295 --> 43.21062946). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.89it/s]
100%|██████████| 8/8 [00:00<00:00, 29.46it/s]
Epoch 98/1000 - Train loss: 43.68964272 - Val loss: 37.26890635
INFO: Val_Loss improved (43.21062946 --> 37.26890635). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.00it/s]
100%|██████████| 8/8 [00:00<00:00, 22.91it/s]
Epoch 99/1000 - Train loss: 42.42243398 - Val loss: 36.43628573
INFO: Val_Loss improved (37.26890635 --> 36.43628573). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.97it/s]
100%|██████████| 8/8 [00:00<00:00, 31.94it/s]
Epoch 100/1000 - Train loss: 39.00224253 - Val loss: 32.68550730
INFO: Val_Loss improved (36.43628573 --> 32.68550730). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 32.18it/s]
Epoch 101/1000 - Train loss: 37.56638889 - Val loss: 39.47828817
INFO: Early stopping counter 1 of 20 [Best Val Loss: 32.68550730, Val Loss: 39.47828817]
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 30.49it/s]
Epoch 102/1000 - Train loss: 35.55153364 - Val loss: 27.67134261
INFO: Val_Loss improved (32.68550730 --> 27.67134261). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.89it/s]
100%|██████████| 8/8 [00:00<00:00, 32.54it/s]
Epoch 103/1000 - Train loss: 33.12477868 - Val loss: 25.26964688
INFO: Val_Loss improved (27.67134261 --> 25.26964688). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 31.92it/s]
Epoch 104/1000 - Train loss: 32.60506884 - Val loss: 26.32351637
INFO: Early stopping counter 1 of 20 [Best Val Loss: 25.26964688, Val Loss: 26.32351637]
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 31.31it/s]
Epoch 105/1000 - Train loss: 30.37842197 - Val loss: 26.56780839
INFO: Early stopping counter 2 of 20 [Best Val Loss: 25.26964688, Val Loss: 26.56780839]
100%|██████████| 30/30 [00:05<00:00, 5.87it/s]
100%|██████████| 8/8 [00:00<00:00, 31.16it/s]
Epoch 106/1000 - Train loss: 29.16577689 - Val loss: 22.31558633
INFO: Val_Loss improved (25.26964688 --> 22.31558633). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.96it/s]
100%|██████████| 8/8 [00:00<00:00, 31.93it/s]
Epoch 107/1000 - Train loss: 28.06209666 - Val loss: 21.51533818
INFO: Val_Loss improved (22.31558633 --> 21.51533818). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.92it/s]
100%|██████████| 8/8 [00:00<00:00, 31.96it/s]
Epoch 108/1000 - Train loss: 26.33553486 - Val loss: 18.55366063
INFO: Val_Loss improved (21.51533818 --> 18.55366063). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.99it/s]
100%|██████████| 8/8 [00:00<00:00, 20.13it/s]
Epoch 109/1000 - Train loss: 24.96661803 - Val loss: 21.73521042
INFO: Early stopping counter 1 of 20 [Best Val Loss: 18.55366063, Val Loss: 21.73521042]
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 31.96it/s]
Epoch 110/1000 - Train loss: 24.77166341 - Val loss: 17.40944767
INFO: Val_Loss improved (18.55366063 --> 17.40944767). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.83it/s]
100%|██████████| 8/8 [00:00<00:00, 32.06it/s]
Epoch 111/1000 - Train loss: 23.02669595 - Val loss: 17.49527371
INFO: Early stopping counter 1 of 20 [Best Val Loss: 17.40944767, Val Loss: 17.49527371]
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 20.37it/s]
Epoch 112/1000 - Train loss: 22.66434469 - Val loss: 15.68894887
INFO: Val_Loss improved (17.40944767 --> 15.68894887). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.04it/s]
100%|██████████| 8/8 [00:00<00:00, 31.72it/s]
Epoch 113/1000 - Train loss: 22.63106330 - Val loss: 15.69560063
INFO: Early stopping counter 1 of 20 [Best Val Loss: 15.68894887, Val Loss: 15.69560063]
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 32.51it/s]
Epoch 114/1000 - Train loss: 21.76268838 - Val loss: 16.40685701
INFO: Early stopping counter 2 of 20 [Best Val Loss: 15.68894887, Val Loss: 16.40685701]
100%|██████████| 30/30 [00:05<00:00, 6.00it/s]
100%|██████████| 8/8 [00:00<00:00, 19.75it/s]
Epoch 115/1000 - Train loss: 19.75447788 - Val loss: 14.02927089
INFO: Val_Loss improved (15.68894887 --> 14.02927089). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.88it/s]
100%|██████████| 8/8 [00:00<00:00, 32.16it/s]
Epoch 116/1000 - Train loss: 20.10812442 - Val loss: 15.20696378
INFO: Early stopping counter 1 of 20 [Best Val Loss: 14.02927089, Val Loss: 15.20696378]
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 30.56it/s]
Epoch 117/1000 - Train loss: 19.65195293 - Val loss: 16.31894505
INFO: Early stopping counter 2 of 20 [Best Val Loss: 14.02927089, Val Loss: 16.31894505]
100%|██████████| 30/30 [00:05<00:00, 5.90it/s]
100%|██████████| 8/8 [00:00<00:00, 30.68it/s]
Epoch 118/1000 - Train loss: 19.63163563 - Val loss: 12.74156284
INFO: Val_Loss improved (14.02927089 --> 12.74156284). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.98it/s]
100%|██████████| 8/8 [00:00<00:00, 30.85it/s]
Epoch 119/1000 - Train loss: 20.44105886 - Val loss: 12.63579750
INFO: Val_Loss improved (12.74156284 --> 12.63579750). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 32.09it/s]
Epoch 120/1000 - Train loss: 19.02729858 - Val loss: 11.77134919
INFO: Val_Loss improved (12.63579750 --> 11.77134919). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 32.02it/s]
Epoch 121/1000 - Train loss: 19.29289748 - Val loss: 12.47423220
INFO: Early stopping counter 1 of 20 [Best Val Loss: 11.77134919, Val Loss: 12.47423220]
100%|██████████| 30/30 [00:05<00:00, 5.99it/s]
100%|██████████| 8/8 [00:00<00:00, 31.83it/s]
Epoch 122/1000 - Train loss: 18.76131751 - Val loss: 12.08792639
INFO: Early stopping counter 2 of 20 [Best Val Loss: 11.77134919, Val Loss: 12.08792639]
100%|██████████| 30/30 [00:05<00:00, 5.76it/s]
100%|██████████| 8/8 [00:00<00:00, 32.07it/s]
Epoch 123/1000 - Train loss: 17.39456431 - Val loss: 11.54369533
INFO: Val_Loss improved (11.77134919 --> 11.54369533). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.94it/s]
100%|██████████| 8/8 [00:00<00:00, 31.91it/s]
Epoch 124/1000 - Train loss: 18.05216783 - Val loss: 18.96053529
INFO: Early stopping counter 1 of 20 [Best Val Loss: 11.54369533, Val Loss: 18.96053529]
100%|██████████| 30/30 [00:05<00:00, 5.86it/s]
100%|██████████| 8/8 [00:00<00:00, 32.18it/s]
Epoch 125/1000 - Train loss: 18.55572424 - Val loss: 15.93144655
INFO: Early stopping counter 2 of 20 [Best Val Loss: 11.54369533, Val Loss: 15.93144655]
100%|██████████| 30/30 [00:05<00:00, 5.84it/s]
100%|██████████| 8/8 [00:00<00:00, 32.24it/s]
Epoch 126/1000 - Train loss: 18.13236777 - Val loss: 10.03004378
INFO: Val_Loss improved (11.54369533 --> 10.03004378). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 31.64it/s]
Epoch 127/1000 - Train loss: 17.17478523 - Val loss: 10.16187167
INFO: Early stopping counter 1 of 20 [Best Val Loss: 10.03004378, Val Loss: 10.16187167]
100%|██████████| 30/30 [00:05<00:00, 5.81it/s]
100%|██████████| 8/8 [00:00<00:00, 32.39it/s]
Epoch 128/1000 - Train loss: 16.37463732 - Val loss: 10.36763310
INFO: Early stopping counter 2 of 20 [Best Val Loss: 10.03004378, Val Loss: 10.36763310]
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 31.40it/s]
Epoch 129/1000 - Train loss: 17.92197237 - Val loss: 11.44554365
INFO: Early stopping counter 3 of 20 [Best Val Loss: 10.03004378, Val Loss: 11.44554365]
100%|██████████| 30/30 [00:05<00:00, 5.81it/s]
100%|██████████| 8/8 [00:00<00:00, 32.51it/s]
Epoch 130/1000 - Train loss: 16.47363803 - Val loss: 9.36801767
INFO: Val_Loss improved (10.03004378 --> 9.36801767). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.06it/s]
100%|██████████| 8/8 [00:00<00:00, 32.40it/s]
Epoch 131/1000 - Train loss: 17.42978767 - Val loss: 9.89415377
INFO: Early stopping counter 1 of 20 [Best Val Loss: 9.36801767, Val Loss: 9.89415377]
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 32.05it/s]
Epoch 132/1000 - Train loss: 16.63763870 - Val loss: 9.25365096
INFO: Val_Loss improved (9.36801767 --> 9.25365096). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 30.41it/s]
Epoch 133/1000 - Train loss: 16.24448760 - Val loss: 10.00427121
INFO: Early stopping counter 1 of 20 [Best Val Loss: 9.25365096, Val Loss: 10.00427121]
100%|██████████| 30/30 [00:05<00:00, 5.90it/s]
100%|██████████| 8/8 [00:00<00:00, 32.51it/s]
Epoch 134/1000 - Train loss: 17.09589853 - Val loss: 9.38489401
INFO: Early stopping counter 2 of 20 [Best Val Loss: 9.25365096, Val Loss: 9.38489401]
100%|██████████| 30/30 [00:05<00:00, 5.91it/s]
100%|██████████| 8/8 [00:00<00:00, 30.93it/s]
Epoch 135/1000 - Train loss: 16.56568530 - Val loss: 9.11441940
INFO: Val_Loss improved (9.25365096 --> 9.11441940). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 32.52it/s]
Epoch 136/1000 - Train loss: 16.59870710 - Val loss: 9.31099236
INFO: Early stopping counter 1 of 20 [Best Val Loss: 9.11441940, Val Loss: 9.31099236]
100%|██████████| 30/30 [00:05<00:00, 5.94it/s]
100%|██████████| 8/8 [00:00<00:00, 31.36it/s]
Epoch 137/1000 - Train loss: 16.82937164 - Val loss: 9.55657333
INFO: Early stopping counter 2 of 20 [Best Val Loss: 9.11441940, Val Loss: 9.55657333]
100%|██████████| 30/30 [00:05<00:00, 5.88it/s]
100%|██████████| 8/8 [00:00<00:00, 31.97it/s]
Epoch 138/1000 - Train loss: 17.23116624 - Val loss: 14.00735557
INFO: Early stopping counter 3 of 20 [Best Val Loss: 9.11441940, Val Loss: 14.00735557]
100%|██████████| 30/30 [00:05<00:00, 5.84it/s]
100%|██████████| 8/8 [00:00<00:00, 30.54it/s]
Epoch 139/1000 - Train loss: 17.38641338 - Val loss: 9.08644658
INFO: Val_Loss improved (9.11441940 --> 9.08644658). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 32.45it/s]
Epoch 140/1000 - Train loss: 16.78854987 - Val loss: 8.88170397
INFO: Val_Loss improved (9.08644658 --> 8.88170397). Saving model ...
100%|██████████| 30/30 [00:04<00:00, 6.13it/s]
100%|██████████| 8/8 [00:00<00:00, 31.96it/s]
Epoch 141/1000 - Train loss: 16.71876779 - Val loss: 12.03038323
INFO: Early stopping counter 1 of 20 [Best Val Loss: 8.88170397, Val Loss: 12.03038323]
100%|██████████| 30/30 [00:05<00:00, 5.81it/s]
100%|██████████| 8/8 [00:00<00:00, 28.44it/s]
Epoch 142/1000 - Train loss: 17.21769164 - Val loss: 9.53168583
INFO: Early stopping counter 2 of 20 [Best Val Loss: 8.88170397, Val Loss: 9.53168583]
100%|██████████| 30/30 [00:04<00:00, 6.07it/s]
100%|██████████| 8/8 [00:00<00:00, 32.08it/s]
Epoch 143/1000 - Train loss: 17.42705564 - Val loss: 8.85243350
INFO: Val_Loss improved (8.88170397 --> 8.85243350). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 30.73it/s]
Epoch 144/1000 - Train loss: 15.04922034 - Val loss: 9.44384068
INFO: Early stopping counter 1 of 20 [Best Val Loss: 8.85243350, Val Loss: 9.44384068]
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 31.80it/s]
Epoch 145/1000 - Train loss: 16.72757832 - Val loss: 10.30191946
INFO: Early stopping counter 2 of 20 [Best Val Loss: 8.85243350, Val Loss: 10.30191946]
100%|██████████| 30/30 [00:05<00:00, 5.97it/s]
100%|██████████| 8/8 [00:00<00:00, 32.20it/s]
Epoch 146/1000 - Train loss: 17.42754758 - Val loss: 8.69546705
INFO: Val_Loss improved (8.85243350 --> 8.69546705). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.79it/s]
100%|██████████| 8/8 [00:00<00:00, 31.46it/s]
Epoch 147/1000 - Train loss: 16.52345877 - Val loss: 9.00765157
INFO: Early stopping counter 1 of 20 [Best Val Loss: 8.69546705, Val Loss: 9.00765157]
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 32.24it/s]
Epoch 148/1000 - Train loss: 16.29480960 - Val loss: 9.03373230
INFO: Early stopping counter 2 of 20 [Best Val Loss: 8.69546705, Val Loss: 9.03373230]
100%|██████████| 30/30 [00:04<00:00, 6.04it/s]
100%|██████████| 8/8 [00:00<00:00, 32.28it/s]
Epoch 149/1000 - Train loss: 16.12279644 - Val loss: 9.83441895
INFO: Early stopping counter 3 of 20 [Best Val Loss: 8.69546705, Val Loss: 9.83441895]
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 32.03it/s]
Epoch 150/1000 - Train loss: 16.56296597 - Val loss: 8.81636685
INFO: Early stopping counter 4 of 20 [Best Val Loss: 8.69546705, Val Loss: 8.81636685]
100%|██████████| 30/30 [00:05<00:00, 5.91it/s]
100%|██████████| 8/8 [00:00<00:00, 23.49it/s]
Epoch 151/1000 - Train loss: 17.61890141 - Val loss: 10.16962945
INFO: Early stopping counter 5 of 20 [Best Val Loss: 8.69546705, Val Loss: 10.16962945]
100%|██████████| 30/30 [00:05<00:00, 5.98it/s]
100%|██████████| 8/8 [00:00<00:00, 32.22it/s]
Epoch 152/1000 - Train loss: 16.16431955 - Val loss: 8.65596259
INFO: Val_Loss improved (8.69546705 --> 8.65596259). Saving model ...
100%|██████████| 30/30 [00:05<00:00, 5.83it/s]
100%|██████████| 8/8 [00:00<00:00, 31.91it/s]
Epoch 153/1000 - Train loss: 16.95704845 - Val loss: 8.75781107
INFO: Early stopping counter 1 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.75781107]
100%|██████████| 30/30 [00:04<00:00, 6.05it/s]
100%|██████████| 8/8 [00:00<00:00, 17.84it/s]
Epoch 154/1000 - Train loss: 16.89016250 - Val loss: 10.80084980
INFO: Early stopping counter 2 of 20 [Best Val Loss: 8.65596259, Val Loss: 10.80084980]
100%|██████████| 30/30 [00:04<00:00, 6.00it/s]
100%|██████████| 8/8 [00:00<00:00, 32.23it/s]
Epoch 155/1000 - Train loss: 16.44982106 - Val loss: 8.99804312
INFO: Early stopping counter 3 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.99804312]
100%|██████████| 30/30 [00:05<00:00, 5.90it/s]
100%|██████████| 8/8 [00:00<00:00, 32.44it/s]
Epoch 156/1000 - Train loss: 17.03195524 - Val loss: 8.72102225
INFO: Early stopping counter 4 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.72102225]
100%|██████████| 30/30 [00:05<00:00, 5.97it/s]
100%|██████████| 8/8 [00:00<00:00, 32.46it/s]
Epoch 157/1000 - Train loss: 15.36453902 - Val loss: 8.89173955
INFO: Early stopping counter 5 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.89173955]
100%|██████████| 30/30 [00:05<00:00, 5.82it/s]
100%|██████████| 8/8 [00:00<00:00, 32.20it/s]
Epoch 158/1000 - Train loss: 17.04786266 - Val loss: 9.96935630
INFO: Early stopping counter 6 of 20 [Best Val Loss: 8.65596259, Val Loss: 9.96935630]
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 30.89it/s]
Epoch 159/1000 - Train loss: 16.89169496 - Val loss: 9.57703024
INFO: Early stopping counter 7 of 20 [Best Val Loss: 8.65596259, Val Loss: 9.57703024]
100%|██████████| 30/30 [00:05<00:00, 5.96it/s]
100%|██████████| 8/8 [00:00<00:00, 31.21it/s]
Epoch 160/1000 - Train loss: 17.58814090 - Val loss: 9.13229108
INFO: Early stopping counter 8 of 20 [Best Val Loss: 8.65596259, Val Loss: 9.13229108]
100%|██████████| 30/30 [00:05<00:00, 5.93it/s]
100%|██████████| 8/8 [00:00<00:00, 31.70it/s]
Epoch 161/1000 - Train loss: 14.82893731 - Val loss: 9.12412882
INFO: Early stopping counter 9 of 20 [Best Val Loss: 8.65596259, Val Loss: 9.12412882]
100%|██████████| 30/30 [00:04<00:00, 6.02it/s]
100%|██████████| 8/8 [00:00<00:00, 18.57it/s]
Epoch 162/1000 - Train loss: 16.90921043 - Val loss: 9.12528139
INFO: Early stopping counter 10 of 20 [Best Val Loss: 8.65596259, Val Loss: 9.12528139]
100%|██████████| 30/30 [00:05<00:00, 5.98it/s]
100%|██████████| 8/8 [00:00<00:00, 32.13it/s]
Epoch 163/1000 - Train loss: 17.16013110 - Val loss: 9.07301247
INFO: Early stopping counter 11 of 20 [Best Val Loss: 8.65596259, Val Loss: 9.07301247]
Epoch 00163: reducing learning rate of group 0 to 1.0000e-05.
100%|██████████| 30/30 [00:05<00:00, 5.78it/s]
100%|██████████| 8/8 [00:00<00:00, 31.88it/s]
Epoch 164/1000 - Train loss: 16.81863829 - Val loss: 8.96239024
INFO: Early stopping counter 12 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.96239024]
100%|██████████| 30/30 [00:05<00:00, 5.94it/s]
100%|██████████| 8/8 [00:00<00:00, 31.64it/s]
Epoch 165/1000 - Train loss: 15.74133835 - Val loss: 8.89574474
INFO: Early stopping counter 13 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.89574474]
100%|██████████| 30/30 [00:05<00:00, 5.80it/s]
100%|██████████| 8/8 [00:00<00:00, 32.15it/s]
Epoch 166/1000 - Train loss: 15.76310212 - Val loss: 8.92132795
INFO: Early stopping counter 14 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.92132795]
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 31.68it/s]
Epoch 167/1000 - Train loss: 16.44784368 - Val loss: 8.91570193
INFO: Early stopping counter 15 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.91570193]
100%|██████████| 30/30 [00:05<00:00, 5.88it/s]
100%|██████████| 8/8 [00:00<00:00, 32.04it/s]
Epoch 168/1000 - Train loss: 16.14062608 - Val loss: 8.91657645
INFO: Early stopping counter 16 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.91657645]
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 32.20it/s]
Epoch 169/1000 - Train loss: 14.62881838 - Val loss: 8.92999268
INFO: Early stopping counter 17 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.92999268]
100%|██████████| 30/30 [00:05<00:00, 5.71it/s]
100%|██████████| 8/8 [00:00<00:00, 31.79it/s]
Epoch 170/1000 - Train loss: 16.02510535 - Val loss: 8.84706682
INFO: Early stopping counter 18 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.84706682]
100%|██████████| 30/30 [00:04<00:00, 6.00it/s]
100%|██████████| 8/8 [00:00<00:00, 32.29it/s]
Epoch 171/1000 - Train loss: 14.85677077 - Val loss: 8.83096230
INFO: Early stopping counter 19 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.83096230]
100%|██████████| 30/30 [00:05<00:00, 5.77it/s]
100%|██████████| 8/8 [00:00<00:00, 31.56it/s]
Epoch 172/1000 - Train loss: 15.99702911 - Val loss: 8.85800487
INFO: Early stopping counter 20 of 20 [Best Val Loss: 8.65596259, Val Loss: 8.85800487]
INFO: Early stopping DONE!
INFO: Best Val Loss: 8.65596259
Loading best model...
results_coordinate = coordinateRegressionPipeline.evaluate(image_paths_test1, landmarks_test1, pixel_spacings_test1)
Reading and normalizing 150 images...
100%|██████████| 150/150 [00:02<00:00, 61.70it/s]
100%|██████████| 150/150 [00:17<00:00, 8.60it/s]
Detection report:
1# Point-to-point error (PE) statistics:
======================================================================
Class Mean Median Std Min Max
----------------------------------------------------------------------
Landmark 0 112.22 112.28 4.21 101.73 123.62
Landmark 1 136.47 136.53 5.06 123.94 150.03
Landmark 2 144.14 144.21 5.37 130.80 158.59
Landmark 3 116.74 116.80 4.37 105.85 128.54
Landmark 4 167.86 167.95 6.24 152.39 184.59
Landmark 5 187.13 187.23 6.96 169.86 205.77
Landmark 6 196.39 196.49 7.33 178.20 216.06
Landmark 7 198.34 198.45 7.40 179.99 218.15
Landmark 8 198.40 198.51 7.39 180.07 218.19
Landmark 9 159.48 159.56 5.98 144.60 175.53
Landmark 10 179.67 179.76 6.69 163.06 197.61
Landmark 11 181.07 181.16 6.73 164.37 199.12
Landmark 12 181.24 181.33 6.75 164.48 199.34
Landmark 13 191.99 192.08 7.16 174.21 211.18
Landmark 14 170.61 170.69 6.35 154.87 187.63
Landmark 15 204.16 204.27 7.61 185.27 224.55
Landmark 16 144.44 144.51 5.39 131.04 158.93
Landmark 17 163.40 163.47 6.08 148.33 179.70
Landmark 18 126.18 126.24 4.73 114.40 138.92
======================================================================
2# Success detection rate (SDR):
================================================================================
Class SDR (PE≤2mm) SDR (PE≤2.5mm) SDR (PE≤3mm) SDR (PE≤4mm)
--------------------------------------------------------------------------------
Landmark 0 0.00 0.00 0.00 0.00
Landmark 1 0.00 0.00 0.00 0.00
Landmark 2 0.00 0.00 0.00 0.00
Landmark 3 0.00 0.00 0.00 0.00
Landmark 4 0.00 0.00 0.00 0.00
Landmark 5 0.00 0.00 0.00 0.00
Landmark 6 0.00 0.00 0.00 0.00
Landmark 7 0.00 0.00 0.00 0.00
Landmark 8 0.00 0.00 0.00 0.00
Landmark 9 0.00 0.00 0.00 0.00
Landmark 10 0.00 0.00 0.00 0.00
Landmark 11 0.00 0.00 0.00 0.00
Landmark 12 0.00 0.00 0.00 0.00
Landmark 13 0.00 0.00 0.00 0.00
Landmark 14 0.00 0.00 0.00 0.00
Landmark 15 0.00 0.00 0.00 0.00
Landmark 16 0.00 0.00 0.00 0.00
Landmark 17 0.00 0.00 0.00 0.00
Landmark 18 0.00 0.00 0.00 0.00
================================================================================