2021 ML Dev-Matching | 미술 작품 분류하기 : 출제자가 추천한 우수 코드 A

 

 

목차

• Dataset
• Import Lib
• ConfigManager
• SEED고정
• Load Data
• Model
• Utils
• CosineAnnealingWarmupRestarts
• CutMix
• 지표 계산을 위한 average meter
• Training
• Inference
• model ensemble을 위해서 npy 파일 형태로 저장
• Ensemble

---

Dataset¶

• EDA(Exploratory Data Analysis, 탐색적 데이터 분석)를 진행하면서 동시에 csv를 만들었습니다.
• image와 label을 반환합니다.
• albumentations augmentation을 염두해두고 설계했기 때문에 이미지를 opencv로 읽었습니다.

In [1]:

import os
import cv2
import numpy as np

from torch.utils.data import Dataset

In [2]:

class ArtPaintDataset(Dataset):
    def __init__(self, df, transform=None):
        super().__init__()
        self.df = df.reset_index()
        self.image_id = self.df.image_id
        self.labels = self.df.label
        self.transform = transform        
    
    def __len__(self):
        return len(self.df)

    def set_transform(self, transform):        
        self.transform = transform

    def __getitem__(self, idx):
        image_path = self.image_id[idx]
        image = cv2.imread(image_path)
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

        label = self.labels[idx]
        
        if self.transform:
            image = self.transform(image=np.array(image))['image']

        return {'image' : image, 'label' : label}

 

Import Lib¶

In [3]:

import os
import math
import timm
import yaml
import torch
import random
import numpy as np
import pandas as pd
import torch.nn as nn
import albumentations
import albumentations.pytorch

from tqdm.auto import tqdm
from adamp import AdamP
from easydict import EasyDict
from prettyprinter import cpprint
from torchsummary import summary as summary_
from torch.utils.data import Dataset, DataLoader
from torch.optim.lr_scheduler import _LRScheduler
from torch.optim.lr_scheduler import CosineAnnealingLR
from sklearn.model_selection import StratifiedKFold, train_test_split

 

ConfigManager¶

• .yaml 파일을 사용하여 config를 관리했습니다.
• config를 바꿔가며 실험했습니다.
• 최종적으로 사용한 config는 다음과 같습니다.

---

base:
  seed: 77
  model_arc: 'nfnet_l0'
  num_classes: 7
  input_dir: './train/train.csv'
  output_dir: './results/'
  train_only: False
  image_size: 227
  cutmix_args:
    use_cutmix: False
    beta: 1.0
    cutmix_prob: 0.5
  train_args:
    num_epochs: 6
    train_batch_size: 32
    val_batch_size: 32
    max_lr: 0.0001
    min_lr: 0.00001
    cycle: 3
    gamma: 0.5
    weight_decay: 0.000001
    log_intervals: 10
    eval_metric: 'accuracy'    
    n_splits: 5

---

swin:
  seed: 777
  model_arc: 'swin_base_patch4_window7_224'
  num_classes: 7
  input_dir: './train/train.csv'
  output_dir: './results/'
  train_only: False
  image_size: 224
  cutmix_args:
    use_cutmix: True
    beta: 1.0
    cutmix_prob: 0.5
  train_args:
    num_epochs: 10
    train_batch_size: 16
    val_batch_size: 16
    max_lr: 0.0001
    min_lr: 0.00001
    cycle: 3
    gamma: 0.5
    weight_decay: 0.000001
    log_intervals: 10
    eval_metric: 'accuracy'    
    n_splits: 5

In [4]:

# Set Config
class YamlConfigManager:
    def __init__(self, config_file_path='./config.yml', config_name='swin'):
        super().__init__()
        self.values = EasyDict()        
        if config_file_path:
            self.config_file_path = config_file_path
            self.config_name = config_name
            self.reload()
    
    def reload(self):
        self.clear()
        if self.config_file_path:
            with open(self.config_file_path, 'r') as f:
                self.values.update(yaml.safe_load(f)[self.config_name])

    def clear(self):
        self.values.clear()
    
    def update(self, yml_dict):
        for (k1, v1) in yml_dict.items():
            if isinstance(v1, dict):
                for (k2, v2) in v1.items():
                    if isinstance(v2, dict):
                        for (k3, v3) in v2.items():
                            self.values[k1][k2][k3] = v3
                    else:
                        self.values[k1][k2] = v2
            else:
                self.values[k1] = v1

    def export(self, save_file_path):
        if save_file_path:
            with open(save_file_path, 'w') as f:
                yaml.dump(dict(self.values), f)

In [5]:

cfg = YamlConfigManager()

SEED = cfg.values.seed
INPUT_DIR = cfg.values.input_dir
TRAIN_ONLY = cfg.values.train_only
IMAGE_SIZE = cfg.values.image_size
TRAIN_BATCH_SIZE = cfg.values.train_args.train_batch_size
VAL_BATCH_SIZE = cfg.values.train_args.val_batch_size

SEED 고정¶

In [6]:

# Fix random seed
def seed_everything(seed: int = 42):
    random.seed(seed)
    np.random.seed(seed)
    os.environ["PYTHONHASHSEED"] = str(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)  # type: ignore
    torch.backends.cudnn.deterministic = True  # type: ignore
    torch.backends.cudnn.benchmark = True  # type: ignore

In [7]:

seed_everything(SEED)

Load Data¶

• 만들어둔 csv 파일을 불러왔습니다.

Augmentation¶

• HorizontalFlip() : TTA(Test Time Augmentation)를 염두해두고 선택하였습니다.
• ToGray() : 그림 이미지라서 색조가 너무 다양할 것이라고 생각하여 gray scale에서도 feature를 잡을 수 있었으면 했습니다.
• Blur() : 그림 이미지 특성상 blurring된 효과가 많이 있을 것이라 생각하여 채택했습니다.
• Normalize RGB mean, std 값은 eda.ipynb를 통해 직접 계산했습니다.

In [20]:

# # Caculate mean and std

# # pixel count
# count = len(train_df) * 227 * 227

# # mean and std
# total_mean = psum / count
# total_var  = (psum_sq / count) - (total_mean ** 2)
# total_std  = torch.sqrt(total_var)

# # output
# print('mean: '  + str(total_mean))
# print('std:  '  + str(total_std))

 

mean: tensor([0.4569, 0.5074, 0.5557]) std: tensor([0.2888, 0.2743, 0.2829])

In [8]:

whole_df = pd.read_csv(INPUT_DIR)
whole_label = whole_df['label'].values

train_transform = albumentations.Compose([
    albumentations.OneOf([
        albumentations.HorizontalFlip(),
        albumentations.ToGray(),
        albumentations.Blur()
    ]), 
    albumentations.Resize(IMAGE_SIZE, IMAGE_SIZE),
    albumentations.Normalize(mean=(0.4569, 0.5074, 0.5557), std=(0.2888, 0.2743, 0.2829)),
    albumentations.pytorch.transforms.ToTensorV2()])

val_transform = albumentations.Compose([
    albumentations.Resize(IMAGE_SIZE, IMAGE_SIZE),
    albumentations.Normalize(mean=(0.4569, 0.5074, 0.5557), std=(0.2888, 0.2743, 0.2829)),
    albumentations.pytorch.transforms.ToTensorV2()])

 

Model¶

• timm library를 사용했습니다.
• NFNet, EfficientNet
  • kaggle ailen signal search 대회에 참가 중인데 nfnet과 efficientnet이 대체로 성능이 좋았습니다.
  • 첫 제출 당시에 NFNet을 사용했는데 1.0이 나와서 놀랐습니다.
  • 문제가 쉬운 대회인 만큼 shake up이 심할거라 생각해서 다른 모델들도 테스트했지만 val score가 좋지 않았습니다.
  • LB(LeaderBoard) score
    • efficientnet_b0 : 90.0
    • nfnet_l0 : 100.0
• Swin-transformer
  • naver boostcamp에서 competition 진행했었는데 detection 대회에서 swin-transformer를 사용해서 2등을 했습니다.
  • 당시에 성능이 좋았습니다.
  • 구조가 다른 모델끼리 앙상블할 경우에 generalized performance가 올라갈 것이라고 생각했습니다.
  • LB score
    • swin-base-224-22k : 97.14

In [9]:

class PretrainedModel(nn.Module):
    def __init__(self, model_arc='resnet18d', num_classes=7):
        super().__init__()
        self.net = timm.create_model(model_arc, pretrained=True, num_classes=num_classes)
    
    def forward(self, x):
        x = self.net(x)

        return x

Utils¶

• 여러가지 유틸 함수들을 정의했습니다.

In [10]:

def get_dataloader(df, transform, batch_size, shuffle):
    dataset = ArtPaintDataset(df=df, transform=transform)

    loader = DataLoader(
        dataset,
        batch_size=batch_size,
        shuffle=shuffle
    )

    return loader

CosineAnnealingWarmupRestarts¶

• gamma 비율로 감소하는 cosine annealing warmup restart scheduler가 lr 분석하기 제일 편했습니다.
• 경험에 의해서 선택했으며, wandb와 함께 사용할 경우 optimizer 분석에 매우 용이했습니다.

In [11]:

class CosineAnnealingWarmupRestarts(_LRScheduler):
    """
        optimizer (Optimizer): Wrapped optimizer.
        first_cycle_steps (int): First cycle step size.
        cycle_mult(float): Cycle steps magnification. Default: -1.
        max_lr(float): First cycle's max learning rate. Default: 0.1.
        min_lr(float): Min learning rate. Default: 0.001.
        warmup_steps(int): Linear warmup step size. Default: 0.
        gamma(float): Decrease rate of max learning rate by cycle. Default: 1.
        last_epoch (int): The index of last epoch. Default: -1.
    """
    
    def __init__(self,
                 optimizer : torch.optim.Optimizer,
                 first_cycle_steps : int,
                 cycle_mult : float = 1.,
                 max_lr : float = 0.1,
                 min_lr : float = 0.001,
                 warmup_steps : int = 0,
                 gamma : float = 1.,
                 last_epoch : int = -1
        ):
        assert warmup_steps < first_cycle_steps
        
        self.first_cycle_steps = first_cycle_steps # first cycle step size
        self.cycle_mult = cycle_mult # cycle steps magnification
        self.base_max_lr = max_lr # first max learning rate
        self.max_lr = max_lr # max learning rate in the current cycle
        self.min_lr = min_lr # min learning rate
        self.warmup_steps = warmup_steps # warmup step size
        self.gamma = gamma # decrease rate of max learning rate by cycle
        
        self.cur_cycle_steps = first_cycle_steps # first cycle step size
        self.cycle = 0 # cycle count
        self.step_in_cycle = last_epoch # step size of the current cycle
        
        super(CosineAnnealingWarmupRestarts, self).__init__(optimizer, last_epoch)
        
        # set learning rate min_lr
        self.init_lr()
    
    def init_lr(self):
        self.base_lrs = []
        for param_group in self.optimizer.param_groups:
            param_group['lr'] = self.min_lr
            self.base_lrs.append(self.min_lr)
    
    def get_lr(self):
        if self.step_in_cycle == -1:
            return self.base_lrs
        elif self.step_in_cycle < self.warmup_steps:
            return [(self.max_lr - base_lr)*self.step_in_cycle / self.warmup_steps + base_lr for base_lr in self.base_lrs]
        else:
            return [base_lr + (self.max_lr - base_lr) \
                    * (1 + math.cos(math.pi * (self.step_in_cycle-self.warmup_steps) \
                                    / (self.cur_cycle_steps - self.warmup_steps))) / 2
                    for base_lr in self.base_lrs]

    def step(self, epoch=None):
        if epoch is None:
            epoch = self.last_epoch + 1
            self.step_in_cycle = self.step_in_cycle + 1
            if self.step_in_cycle >= self.cur_cycle_steps:
                self.cycle += 1
                self.step_in_cycle = self.step_in_cycle - self.cur_cycle_steps
                self.cur_cycle_steps = int((self.cur_cycle_steps - self.warmup_steps) * self.cycle_mult) + self.warmup_steps
        else:
            if epoch >= self.first_cycle_steps:
                if self.cycle_mult == 1.:
                    self.step_in_cycle = epoch % self.first_cycle_steps
                    self.cycle = epoch // self.first_cycle_steps
                else:
                    n = int(math.log((epoch / self.first_cycle_steps * (self.cycle_mult - 1) + 1), self.cycle_mult))
                    self.cycle = n
                    self.step_in_cycle = epoch - int(self.first_cycle_steps * (self.cycle_mult ** n - 1) / (self.cycle_mult - 1))
                    self.cur_cycle_steps = self.first_cycle_steps * self.cycle_mult ** (n)
            else:
                self.cur_cycle_steps = self.first_cycle_steps
                self.step_in_cycle = epoch
                
        self.max_lr = self.base_max_lr * (self.gamma**self.cycle)
        self.last_epoch = math.floor(epoch)
        for param_group, lr in zip(self.optimizer.param_groups, self.get_lr()):
            param_group['lr'] = lr

CutMix¶

• Robust한 모델을 만들기 위해 선택한 방법입니다.
• 쉬운 문제에서 사용할 경우 성능이 많이 좋아졌었던 경험이 있습니다.
• 학습을 더욱 복잡하게 만들어서 generalized performance의 향상을 노렸습니다.
• NFNet에서는 사용하지 않았고, Swin Transformer에만 사용해서 앙상블했습니다.

In [12]:

def rand_bbox(size, lam):
    width = size[2]
    height = size[3]
    cut_ratio = np.sqrt(1. - lam)
    cut_width = np.int(width * cut_ratio)
    cut_height = np.int(height * cut_ratio)

    # uniform
    cx = np.random.randint(width)
    cy = np.random.randint(height)

    bbx1 = np.clip(cx - cut_width // 2, 0, width)
    bby1 = np.clip(cy - cut_height // 2, 0, height)
    bbx2 = np.clip(cx + cut_width // 2, 0, width)
    bby2 = np.clip(cy + cut_height // 2, 0, height)

    return bbx1, bby1, bbx2, bby2

In [13]:

class CutMix(object):
    def __init__(self, beta, cutmix_prob) -> None:
        super().__init__()
        self.beta = beta
        self.cutmix_prob = cutmix_prob 

    def forward(self, images, labels):
        # generate mixed sample
        lam = np.random.beta(self.beta, self.beta)
        rand_index = torch.randperm(images.size()[0]).cuda()
        label_1 = labels
        label_2 = labels[rand_index]
        bbx1, bby1, bbx2, bby2 = rand_bbox(images.size(), lam)
        images[:, :, bbx1:bbx2, bby1:bby2] = images[rand_index, :, bbx1:bbx2, bby1:bby2]

        # adjust lambda to exactly match pixel ratio
        lam = 1 - ((bbx2 - bbx1) * (bby2 - bby1) / (images.size()[-1] * images.size()[-2]))

        return {'lam' : lam, 'image' : images, 'label_1' : label_1, 'label_2' : label_2}

In [14]:

class ComputeMetric(object):
    def __init__(self, metric) -> None:
        super().__init__() 
        self.metric = metric    

    def cutmix_accuracy(self, logits, labels, topk=(1, 5)):
        max_k = max(topk)
        batch_size = labels.size(0)

        _, pred = logits.topk(max_k, 1, True, True)
        pred = pred.t()
        matches = pred.eq(labels.view(1, -1).expand_as(pred))

        res = []
        for k in topk:
            matches_k = matches[:k].reshape(-1).float().sum(0, keepdim=True)
            wrong_k = batch_size - matches_k
            res.append(matches_k.mul_(100.0 / batch_size))

        return res

    def compute(self, logits, labels, topk=(1, 5)):
        if self.metric == 'accuracy':
            out = self.cutmix_accuracy(logits=logits, labels=labels, topk=topk)

        return out

지표 계산을 위한 average meter¶

In [15]:

class AverageMeter(object):
    def __init__(self) -> None:
        super().__init__()
        self.reset()
    
    def reset(self):
        self.val = 0
        self.avg = 0
        self.sum = 0
        self.count = 0

    def update(self, val, n=1):
        self.val = val
        self.sum += val * n
        self.count += n
        self.avg = self.sum / self.count

 

Training¶

• 5 Fold cross validation 사용
• 모델 별로 lr, epoch 조절해가면서 다르게 학습했습니다.
• EDA(eda.ipynb 파일 만들어서 진행) 해봤을 때 guitar와 person같이 class imbalance가 있어 stratified하게 train과 validation 분할 했습니다.

In [33]:

# import matplotlib as pyplot
# import seaborn as sns

# ax = sns.countplot(train_df['label'])

# for p, label in zip(ax.patches, label_list):
#     ax.annotate(label, (p.get_x(), p.get_height() + 0.15))

 

In [16]:

def train(cfg, k, train_loader, val_loader):
    # Set Config
    MODEL_ARC = cfg.values.model_arc
    OUTPUT_DIR = cfg.values.output_dir
    NUM_CLASSES = cfg.values.num_classes
    TRAIN_ONLY = cfg.values.train_only

    os.makedirs(OUTPUT_DIR, exist_ok=True)

    # Set train arguments
    num_epochs = cfg.values.train_args.num_epochs
    train_batch_size = cfg.values.train_args.train_batch_size
    log_intervals = cfg.values.train_args.log_intervals
    max_lr = cfg.values.train_args.max_lr
    min_lr = cfg.values.train_args.min_lr
    cycle = cfg.values.train_args.cycle
    gamma = cfg.values.train_args.gamma

    # Set CutMix arguments
    USE_CUTMIX = cfg.values.cutmix_args.use_cutmix    
    beta = cfg.values.cutmix_args.beta
    cutmix_prob = cfg.values.cutmix_args.cutmix_prob    
    cutmix = CutMix(beta=beta, cutmix_prob=cutmix_prob)
    
    device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

    model = PretrainedModel(model_arc=MODEL_ARC, num_classes=NUM_CLASSES)
    model.to(device)
    if k < 2:
        summary_(model, (3, IMAGE_SIZE, IMAGE_SIZE), batch_size=train_batch_size)

    optimizer = AdamP(model.parameters(), lr=max_lr, weight_decay=cfg.values.train_args.weight_decay)
    first_cycle_steps = len(train_loader) * num_epochs // cycle
    scheduler = CosineAnnealingWarmupRestarts(
        optimizer, 
        first_cycle_steps=first_cycle_steps, 
        cycle_mult=1.0,
        max_lr=max_lr, 
        min_lr=min_lr, 
        warmup_steps=int(first_cycle_steps * 0.2), 
        gamma=gamma
    )
    criterion = nn.CrossEntropyLoss()

    eval_metric = ComputeMetric(cfg.values.train_args.eval_metric)
    best_acc = 0.
    os.makedirs(os.path.join(OUTPUT_DIR, MODEL_ARC), exist_ok=True)

    for epoch in range(num_epochs):
        model.train()

        loss_values = AverageMeter()
        top1 = AverageMeter()
        top5 = AverageMeter()

        for i, train_batch in enumerate(tqdm(train_loader, desc=f'Training')):
            sample = train_batch
            images = sample['image'].float().to(device)
            labels = sample['label'].long().to(device)

            ratio = np.random.rand(1)

            if USE_CUTMIX:
                if beta > 0 and ratio < cutmix_prob:
                    # generate mixed sample
                    sample = cutmix.forward(images, labels)

                    logits = model(sample['image'])                    
                    loss = criterion(logits, sample['label_1']) * sample['lam'] + criterion(logits, sample['label_2']) * (1. - sample['lam'])
                else:
                    logits = model(images)
                    loss = criterion(logits, labels)
            else:
                logits = model(images)
                loss = criterion(logits, labels)

            # measure evaluation metric and record loss
            top1_err, top5_err = eval_metric.compute(logits.data, labels, topk=(1, 5))

            loss_values.update(loss.item(), images.size(0))
            top1.update(top1_err.item(), images.size(0))
            top5.update(top5_err.item(), images.size(0))

            # compute gradient and do optimizer step
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            scheduler.step()

            if i % log_intervals == 0:
                current_lr = scheduler.get_lr()[0]
                tqdm.write(f'Epoch : [{epoch + 1}/{num_epochs}][{i}/{len(train_loader)}] || '
                           f'LR : {current_lr:.5f} || '
                           f'Train Loss : {loss_values.val:.4f} ({loss_values.avg:.4f}) || '                        
                           f'Train Top 1-acc : {top1.val:.3f}% ({top1.avg:.3f})% || '
                           f'Train Top 5-acc : {top5.val:.3f}% ({top5.avg:.3f})%')

        loss_values = AverageMeter()
        top1 = AverageMeter()
        top5 = AverageMeter()
        
        if not TRAIN_ONLY:
            with torch.no_grad():
                model.eval()

                loss_values = AverageMeter()
                top1 = AverageMeter()
                top5 = AverageMeter()

                for i, val_batch in enumerate(tqdm(val_loader, desc=f'Validation')):
                    sample = val_batch
                    images = sample['image'].float().to(device)
                    labels = sample['label'].long().to(device)

                    logits = model(images)
                    loss = criterion(logits, labels)
                    preds = torch.argmax(logits, -1)

                    top1_err, top5_err = eval_metric.compute(logits.data, labels, topk=(1, 5))
                    loss_values.update(loss.item(), images.size(0))
                    top1.update(top1_err.item(), images.size(0))
                    top5.update(top5_err.item(), images.size(0))

            tqdm.write(f'Epoch : [{epoch + 1}/{num_epochs}] || '
                       f'Val Loss : {loss_values.avg:.4f} || '                        
                       f'Val Top 1-acc : {top1.avg:.3f}% || '
                       f'Val Top 5-acc : {top5.avg:.3f}%')

            is_best = top1.avg >= best_acc
            best_acc = max(top1.avg, best_acc)

            if is_best:
                if k > 0:
                    os.makedirs(os.path.join(OUTPUT_DIR, MODEL_ARC, f'{k}_fold'), exist_ok=True)
                    torch.save(model.state_dict(), os.path.join(OUTPUT_DIR, MODEL_ARC, f'{k}_fold', f'{epoch + 1}_epoch_{best_acc:.2f}%_with_val.pth'))
                else:                    
                    torch.save(model.state_dict(), os.path.join(OUTPUT_DIR, MODEL_ARC, f'{epoch + 1}_epoch_{best_acc:.2f}%_with_val.pth'))
        
        else:
            torch.save(model.state_dict(), os.path.join(OUTPUT_DIR, MODEL_ARC, f'_{epoch + 1}_epoch_{top1.avg:.2f}%_only_train.pth'))

In [17]:

kfold = StratifiedKFold(n_splits=cfg.values.train_args.n_splits)
k = 1
for train_idx, val_idx in kfold.split(whole_df, whole_label):
    print('\n')
    cpprint('=' * 15 + f'{k}-Fold Cross Validation' + '=' * 15)
    train_df = whole_df.iloc[train_idx]
    val_df = whole_df.iloc[val_idx]

    train_loader = get_dataloader(df=train_df, transform=train_transform, batch_size=TRAIN_BATCH_SIZE, shuffle=True)
    val_loader = get_dataloader(df=val_df, transform=val_transform, batch_size=VAL_BATCH_SIZE, shuffle=False)

    train(cfg, k, train_loader, val_loader)

    k += 1

 

'===============1-Fold Cross Validation==============='
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1          [16, 128, 56, 56]           6,272
         LayerNorm-2            [16, 3136, 128]             256
        PatchEmbed-3            [16, 3136, 128]               0
           Dropout-4            [16, 3136, 128]               0
         LayerNorm-5            [16, 3136, 128]             256
            Linear-6              [16, 49, 384]          49,536
           Softmax-7            [16, 4, 49, 49]               0
           Dropout-8            [16, 4, 49, 49]               0
            Linear-9              [16, 49, 128]          16,512
          Dropout-10              [16, 49, 128]               0
  WindowAttention-11              [16, 49, 128]               0
         Identity-12            [16, 3136, 128]               0
        LayerNorm-13            [16, 3136, 128]             256
           Linear-14            [16, 3136, 512]          66,048
             GELU-15            [16, 3136, 512]               0
          Dropout-16            [16, 3136, 512]               0
           Linear-17            [16, 3136, 128]          65,664
          Dropout-18            [16, 3136, 128]               0
              Mlp-19            [16, 3136, 128]               0
         Identity-20            [16, 3136, 128]               0
SwinTransformerBlock-21            [16, 3136, 128]               0
        LayerNorm-22            [16, 3136, 128]             256
           Linear-23              [16, 49, 384]          49,536
          Softmax-24            [16, 4, 49, 49]               0
          Dropout-25            [16, 4, 49, 49]               0
           Linear-26              [16, 49, 128]          16,512
          Dropout-27              [16, 49, 128]               0
  WindowAttention-28              [16, 49, 128]               0
         DropPath-29            [16, 3136, 128]               0
        LayerNorm-30            [16, 3136, 128]             256
           Linear-31            [16, 3136, 512]          66,048
             GELU-32            [16, 3136, 512]               0
          Dropout-33            [16, 3136, 512]               0
           Linear-34            [16, 3136, 128]          65,664
          Dropout-35            [16, 3136, 128]               0
              Mlp-36            [16, 3136, 128]               0
         DropPath-37            [16, 3136, 128]               0
SwinTransformerBlock-38            [16, 3136, 128]               0
        LayerNorm-39             [16, 784, 512]           1,024
           Linear-40             [16, 784, 256]         131,072
     PatchMerging-41             [16, 784, 256]               0
       BasicLayer-42             [16, 784, 256]               0
        LayerNorm-43             [16, 784, 256]             512
           Linear-44              [16, 49, 768]         197,376
          Softmax-45            [16, 8, 49, 49]               0
          Dropout-46            [16, 8, 49, 49]               0
           Linear-47              [16, 49, 256]          65,792
          Dropout-48              [16, 49, 256]               0
  WindowAttention-49              [16, 49, 256]               0
         DropPath-50             [16, 784, 256]               0
        LayerNorm-51             [16, 784, 256]             512
           Linear-52            [16, 784, 1024]         263,168
             GELU-53            [16, 784, 1024]               0
          Dropout-54            [16, 784, 1024]               0
           Linear-55             [16, 784, 256]         262,400
          Dropout-56             [16, 784, 256]               0
              Mlp-57             [16, 784, 256]               0
         DropPath-58             [16, 784, 256]               0
SwinTransformerBlock-59             [16, 784, 256]               0
        LayerNorm-60             [16, 784, 256]             512
           Linear-61              [16, 49, 768]         197,376
          Softmax-62            [16, 8, 49, 49]               0
          Dropout-63            [16, 8, 49, 49]               0
           Linear-64              [16, 49, 256]          65,792
          Dropout-65              [16, 49, 256]               0
  WindowAttention-66              [16, 49, 256]               0
         DropPath-67             [16, 784, 256]               0
        LayerNorm-68             [16, 784, 256]             512
           Linear-69            [16, 784, 1024]         263,168
             GELU-70            [16, 784, 1024]               0
          Dropout-71            [16, 784, 1024]               0
           Linear-72             [16, 784, 256]         262,400
          Dropout-73             [16, 784, 256]               0
              Mlp-74             [16, 784, 256]               0
         DropPath-75             [16, 784, 256]               0
SwinTransformerBlock-76             [16, 784, 256]               0
        LayerNorm-77            [16, 196, 1024]           2,048
           Linear-78             [16, 196, 512]         524,288
     PatchMerging-79             [16, 196, 512]               0
       BasicLayer-80             [16, 196, 512]               0
        LayerNorm-81             [16, 196, 512]           1,024
           Linear-82             [16, 49, 1536]         787,968
          Softmax-83           [16, 16, 49, 49]               0
          Dropout-84           [16, 16, 49, 49]               0
           Linear-85              [16, 49, 512]         262,656
          Dropout-86              [16, 49, 512]               0
  WindowAttention-87              [16, 49, 512]               0
         DropPath-88             [16, 196, 512]               0
        LayerNorm-89             [16, 196, 512]           1,024
           Linear-90            [16, 196, 2048]       1,050,624
             GELU-91            [16, 196, 2048]               0
          Dropout-92            [16, 196, 2048]               0
           Linear-93             [16, 196, 512]       1,049,088
          Dropout-94             [16, 196, 512]               0
              Mlp-95             [16, 196, 512]               0
         DropPath-96             [16, 196, 512]               0
SwinTransformerBlock-97             [16, 196, 512]               0
        LayerNorm-98             [16, 196, 512]           1,024
           Linear-99             [16, 49, 1536]         787,968
         Softmax-100           [16, 16, 49, 49]               0
         Dropout-101           [16, 16, 49, 49]               0
          Linear-102              [16, 49, 512]         262,656
         Dropout-103              [16, 49, 512]               0
 WindowAttention-104              [16, 49, 512]               0
        DropPath-105             [16, 196, 512]               0
       LayerNorm-106             [16, 196, 512]           1,024
          Linear-107            [16, 196, 2048]       1,050,624
            GELU-108            [16, 196, 2048]               0
         Dropout-109            [16, 196, 2048]               0
          Linear-110             [16, 196, 512]       1,049,088
         Dropout-111             [16, 196, 512]               0
             Mlp-112             [16, 196, 512]               0
        DropPath-113             [16, 196, 512]               0
SwinTransformerBlock-114             [16, 196, 512]               0
       LayerNorm-115             [16, 196, 512]           1,024
          Linear-116             [16, 49, 1536]         787,968
         Softmax-117           [16, 16, 49, 49]               0
         Dropout-118           [16, 16, 49, 49]               0
          Linear-119              [16, 49, 512]         262,656
         Dropout-120              [16, 49, 512]               0
 WindowAttention-121              [16, 49, 512]               0
        DropPath-122             [16, 196, 512]               0
       LayerNorm-123             [16, 196, 512]           1,024
          Linear-124            [16, 196, 2048]       1,050,624
            GELU-125            [16, 196, 2048]               0
         Dropout-126            [16, 196, 2048]               0
          Linear-127             [16, 196, 512]       1,049,088
         Dropout-128             [16, 196, 512]               0
             Mlp-129             [16, 196, 512]               0
        DropPath-130             [16, 196, 512]               0
SwinTransformerBlock-131             [16, 196, 512]               0
       LayerNorm-132             [16, 196, 512]           1,024
          Linear-133             [16, 49, 1536]         787,968
         Softmax-134           [16, 16, 49, 49]               0
         Dropout-135           [16, 16, 49, 49]               0
          Linear-136              [16, 49, 512]         262,656
         Dropout-137              [16, 49, 512]               0
 WindowAttention-138              [16, 49, 512]               0
        DropPath-139             [16, 196, 512]               0
       LayerNorm-140             [16, 196, 512]           1,024
          Linear-141            [16, 196, 2048]       1,050,624
            GELU-142            [16, 196, 2048]               0
         Dropout-143            [16, 196, 2048]               0
          Linear-144             [16, 196, 512]       1,049,088
         Dropout-145             [16, 196, 512]               0
             Mlp-146             [16, 196, 512]               0
        DropPath-147             [16, 196, 512]               0
SwinTransformerBlock-148             [16, 196, 512]               0
       LayerNorm-149             [16, 196, 512]           1,024
          Linear-150             [16, 49, 1536]         787,968
         Softmax-151           [16, 16, 49, 49]               0
         Dropout-152           [16, 16, 49, 49]               0
          Linear-153              [16, 49, 512]         262,656
         Dropout-154              [16, 49, 512]               0
 WindowAttention-155              [16, 49, 512]               0
        DropPath-156             [16, 196, 512]               0
       LayerNorm-157             [16, 196, 512]           1,024
          Linear-158            [16, 196, 2048]       1,050,624
            GELU-159            [16, 196, 2048]               0
         Dropout-160            [16, 196, 2048]               0
          Linear-161             [16, 196, 512]       1,049,088
         Dropout-162             [16, 196, 512]               0
             Mlp-163             [16, 196, 512]               0
        DropPath-164             [16, 196, 512]               0
SwinTransformerBlock-165             [16, 196, 512]               0
       LayerNorm-166             [16, 196, 512]           1,024
          Linear-167             [16, 49, 1536]         787,968
         Softmax-168           [16, 16, 49, 49]               0
         Dropout-169           [16, 16, 49, 49]               0
          Linear-170              [16, 49, 512]         262,656
         Dropout-171              [16, 49, 512]               0
 WindowAttention-172              [16, 49, 512]               0
        DropPath-173             [16, 196, 512]               0
       LayerNorm-174             [16, 196, 512]           1,024
          Linear-175            [16, 196, 2048]       1,050,624
            GELU-176            [16, 196, 2048]               0
         Dropout-177            [16, 196, 2048]               0
          Linear-178             [16, 196, 512]       1,049,088
         Dropout-179             [16, 196, 512]               0
             Mlp-180             [16, 196, 512]               0
        DropPath-181             [16, 196, 512]               0
SwinTransformerBlock-182             [16, 196, 512]               0
       LayerNorm-183             [16, 196, 512]           1,024
          Linear-184             [16, 49, 1536]         787,968
         Softmax-185           [16, 16, 49, 49]               0
         Dropout-186           [16, 16, 49, 49]               0
          Linear-187              [16, 49, 512]         262,656
         Dropout-188              [16, 49, 512]               0
 WindowAttention-189              [16, 49, 512]               0
        DropPath-190             [16, 196, 512]               0
       LayerNorm-191             [16, 196, 512]           1,024
          Linear-192            [16, 196, 2048]       1,050,624
            GELU-193            [16, 196, 2048]               0
         Dropout-194            [16, 196, 2048]               0
          Linear-195             [16, 196, 512]       1,049,088
         Dropout-196             [16, 196, 512]               0
             Mlp-197             [16, 196, 512]               0
        DropPath-198             [16, 196, 512]               0
SwinTransformerBlock-199             [16, 196, 512]               0
       LayerNorm-200             [16, 196, 512]           1,024
          Linear-201             [16, 49, 1536]         787,968
         Softmax-202           [16, 16, 49, 49]               0
         Dropout-203           [16, 16, 49, 49]               0
          Linear-204              [16, 49, 512]         262,656
         Dropout-205              [16, 49, 512]               0
 WindowAttention-206              [16, 49, 512]               0
        DropPath-207             [16, 196, 512]               0
       LayerNorm-208             [16, 196, 512]           1,024
          Linear-209            [16, 196, 2048]       1,050,624
            GELU-210            [16, 196, 2048]               0
         Dropout-211            [16, 196, 2048]               0
          Linear-212             [16, 196, 512]       1,049,088
         Dropout-213             [16, 196, 512]               0
             Mlp-214             [16, 196, 512]               0
        DropPath-215             [16, 196, 512]               0
SwinTransformerBlock-216             [16, 196, 512]               0
       LayerNorm-217             [16, 196, 512]           1,024
          Linear-218             [16, 49, 1536]         787,968
         Softmax-219           [16, 16, 49, 49]               0
         Dropout-220           [16, 16, 49, 49]               0
          Linear-221              [16, 49, 512]         262,656
         Dropout-222              [16, 49, 512]               0
 WindowAttention-223              [16, 49, 512]               0
        DropPath-224             [16, 196, 512]               0
       LayerNorm-225             [16, 196, 512]           1,024
          Linear-226            [16, 196, 2048]       1,050,624
            GELU-227            [16, 196, 2048]               0
         Dropout-228            [16, 196, 2048]               0
          Linear-229             [16, 196, 512]       1,049,088
         Dropout-230             [16, 196, 512]               0
             Mlp-231             [16, 196, 512]               0
        DropPath-232             [16, 196, 512]               0
SwinTransformerBlock-233             [16, 196, 512]               0
       LayerNorm-234             [16, 196, 512]           1,024
          Linear-235             [16, 49, 1536]         787,968
         Softmax-236           [16, 16, 49, 49]               0
         Dropout-237           [16, 16, 49, 49]               0
          Linear-238              [16, 49, 512]         262,656
         Dropout-239              [16, 49, 512]               0
 WindowAttention-240              [16, 49, 512]               0
        DropPath-241             [16, 196, 512]               0
       LayerNorm-242             [16, 196, 512]           1,024
          Linear-243            [16, 196, 2048]       1,050,624
            GELU-244            [16, 196, 2048]               0
         Dropout-245            [16, 196, 2048]               0
          Linear-246             [16, 196, 512]       1,049,088
         Dropout-247             [16, 196, 512]               0
             Mlp-248             [16, 196, 512]               0
        DropPath-249             [16, 196, 512]               0
SwinTransformerBlock-250             [16, 196, 512]               0
       LayerNorm-251             [16, 196, 512]           1,024
          Linear-252             [16, 49, 1536]         787,968
         Softmax-253           [16, 16, 49, 49]               0
         Dropout-254           [16, 16, 49, 49]               0
          Linear-255              [16, 49, 512]         262,656
         Dropout-256              [16, 49, 512]               0
 WindowAttention-257              [16, 49, 512]               0
        DropPath-258             [16, 196, 512]               0
       LayerNorm-259             [16, 196, 512]           1,024
          Linear-260            [16, 196, 2048]       1,050,624
            GELU-261            [16, 196, 2048]               0
         Dropout-262            [16, 196, 2048]               0
          Linear-263             [16, 196, 512]       1,049,088
         Dropout-264             [16, 196, 512]               0
             Mlp-265             [16, 196, 512]               0
        DropPath-266             [16, 196, 512]               0
SwinTransformerBlock-267             [16, 196, 512]               0
       LayerNorm-268             [16, 196, 512]           1,024
          Linear-269             [16, 49, 1536]         787,968
         Softmax-270           [16, 16, 49, 49]               0
         Dropout-271           [16, 16, 49, 49]               0
          Linear-272              [16, 49, 512]         262,656
         Dropout-273              [16, 49, 512]               0
 WindowAttention-274              [16, 49, 512]               0
        DropPath-275             [16, 196, 512]               0
       LayerNorm-276             [16, 196, 512]           1,024
          Linear-277            [16, 196, 2048]       1,050,624
            GELU-278            [16, 196, 2048]               0
         Dropout-279            [16, 196, 2048]               0
          Linear-280             [16, 196, 512]       1,049,088
         Dropout-281             [16, 196, 512]               0
             Mlp-282             [16, 196, 512]               0
        DropPath-283             [16, 196, 512]               0
SwinTransformerBlock-284             [16, 196, 512]               0
       LayerNorm-285             [16, 196, 512]           1,024
          Linear-286             [16, 49, 1536]         787,968
         Softmax-287           [16, 16, 49, 49]               0
         Dropout-288           [16, 16, 49, 49]               0
          Linear-289              [16, 49, 512]         262,656
         Dropout-290              [16, 49, 512]               0
 WindowAttention-291              [16, 49, 512]               0
        DropPath-292             [16, 196, 512]               0
       LayerNorm-293             [16, 196, 512]           1,024
          Linear-294            [16, 196, 2048]       1,050,624
            GELU-295            [16, 196, 2048]               0
         Dropout-296            [16, 196, 2048]               0
          Linear-297             [16, 196, 512]       1,049,088
         Dropout-298             [16, 196, 512]               0
             Mlp-299             [16, 196, 512]               0
        DropPath-300             [16, 196, 512]               0
SwinTransformerBlock-301             [16, 196, 512]               0
       LayerNorm-302             [16, 196, 512]           1,024
          Linear-303             [16, 49, 1536]         787,968
         Softmax-304           [16, 16, 49, 49]               0
         Dropout-305           [16, 16, 49, 49]               0
          Linear-306              [16, 49, 512]         262,656
         Dropout-307              [16, 49, 512]               0
 WindowAttention-308              [16, 49, 512]               0
        DropPath-309             [16, 196, 512]               0
       LayerNorm-310             [16, 196, 512]           1,024
          Linear-311            [16, 196, 2048]       1,050,624
            GELU-312            [16, 196, 2048]               0
         Dropout-313            [16, 196, 2048]               0
          Linear-314             [16, 196, 512]       1,049,088
         Dropout-315             [16, 196, 512]               0
             Mlp-316             [16, 196, 512]               0
        DropPath-317             [16, 196, 512]               0
SwinTransformerBlock-318             [16, 196, 512]               0
       LayerNorm-319             [16, 196, 512]           1,024
          Linear-320             [16, 49, 1536]         787,968
         Softmax-321           [16, 16, 49, 49]               0
         Dropout-322           [16, 16, 49, 49]               0
          Linear-323              [16, 49, 512]         262,656
         Dropout-324              [16, 49, 512]               0
 WindowAttention-325              [16, 49, 512]               0
        DropPath-326             [16, 196, 512]               0
       LayerNorm-327             [16, 196, 512]           1,024
          Linear-328            [16, 196, 2048]       1,050,624
            GELU-329            [16, 196, 2048]               0
         Dropout-330            [16, 196, 2048]               0
          Linear-331             [16, 196, 512]       1,049,088
         Dropout-332             [16, 196, 512]               0
             Mlp-333             [16, 196, 512]               0
        DropPath-334             [16, 196, 512]               0
SwinTransformerBlock-335             [16, 196, 512]               0
       LayerNorm-336             [16, 196, 512]           1,024
          Linear-337             [16, 49, 1536]         787,968
         Softmax-338           [16, 16, 49, 49]               0
         Dropout-339           [16, 16, 49, 49]               0
          Linear-340              [16, 49, 512]         262,656
         Dropout-341              [16, 49, 512]               0
 WindowAttention-342              [16, 49, 512]               0
        DropPath-343             [16, 196, 512]               0
       LayerNorm-344             [16, 196, 512]           1,024
          Linear-345            [16, 196, 2048]       1,050,624
            GELU-346            [16, 196, 2048]               0
         Dropout-347            [16, 196, 2048]               0
          Linear-348             [16, 196, 512]       1,049,088
         Dropout-349             [16, 196, 512]               0
             Mlp-350             [16, 196, 512]               0
        DropPath-351             [16, 196, 512]               0
SwinTransformerBlock-352             [16, 196, 512]               0
       LayerNorm-353             [16, 196, 512]           1,024
          Linear-354             [16, 49, 1536]         787,968
         Softmax-355           [16, 16, 49, 49]               0
         Dropout-356           [16, 16, 49, 49]               0
          Linear-357              [16, 49, 512]         262,656
         Dropout-358              [16, 49, 512]               0
 WindowAttention-359              [16, 49, 512]               0
        DropPath-360             [16, 196, 512]               0
       LayerNorm-361             [16, 196, 512]           1,024
          Linear-362            [16, 196, 2048]       1,050,624
            GELU-363            [16, 196, 2048]               0
         Dropout-364            [16, 196, 2048]               0
          Linear-365             [16, 196, 512]       1,049,088
         Dropout-366             [16, 196, 512]               0
             Mlp-367             [16, 196, 512]               0
        DropPath-368             [16, 196, 512]               0
SwinTransformerBlock-369             [16, 196, 512]               0
       LayerNorm-370             [16, 196, 512]           1,024
          Linear-371             [16, 49, 1536]         787,968
         Softmax-372           [16, 16, 49, 49]               0
         Dropout-373           [16, 16, 49, 49]               0
          Linear-374              [16, 49, 512]         262,656
         Dropout-375              [16, 49, 512]               0
 WindowAttention-376              [16, 49, 512]               0
        DropPath-377             [16, 196, 512]               0
       LayerNorm-378             [16, 196, 512]           1,024
          Linear-379            [16, 196, 2048]       1,050,624
            GELU-380            [16, 196, 2048]               0
         Dropout-381            [16, 196, 2048]               0
          Linear-382             [16, 196, 512]       1,049,088
         Dropout-383             [16, 196, 512]               0
             Mlp-384             [16, 196, 512]               0
        DropPath-385             [16, 196, 512]               0
SwinTransformerBlock-386             [16, 196, 512]               0
       LayerNorm-387             [16, 49, 2048]           4,096
          Linear-388             [16, 49, 1024]       2,097,152
    PatchMerging-389             [16, 49, 1024]               0
      BasicLayer-390             [16, 49, 1024]               0
       LayerNorm-391             [16, 49, 1024]           2,048
          Linear-392             [16, 49, 3072]       3,148,800
         Softmax-393           [16, 32, 49, 49]               0
         Dropout-394           [16, 32, 49, 49]               0
          Linear-395             [16, 49, 1024]       1,049,600
         Dropout-396             [16, 49, 1024]               0
 WindowAttention-397             [16, 49, 1024]               0
        DropPath-398             [16, 49, 1024]               0
       LayerNorm-399             [16, 49, 1024]           2,048
          Linear-400             [16, 49, 4096]       4,198,400
            GELU-401             [16, 49, 4096]               0
         Dropout-402             [16, 49, 4096]               0
          Linear-403             [16, 49, 1024]       4,195,328
         Dropout-404             [16, 49, 1024]               0
             Mlp-405             [16, 49, 1024]               0
        DropPath-406             [16, 49, 1024]               0
SwinTransformerBlock-407             [16, 49, 1024]               0
       LayerNorm-408             [16, 49, 1024]           2,048
          Linear-409             [16, 49, 3072]       3,148,800
         Softmax-410           [16, 32, 49, 49]               0
         Dropout-411           [16, 32, 49, 49]               0
          Linear-412             [16, 49, 1024]       1,049,600
         Dropout-413             [16, 49, 1024]               0
 WindowAttention-414             [16, 49, 1024]               0
        DropPath-415             [16, 49, 1024]               0
       LayerNorm-416             [16, 49, 1024]           2,048
          Linear-417             [16, 49, 4096]       4,198,400
            GELU-418             [16, 49, 4096]               0
         Dropout-419             [16, 49, 4096]               0
          Linear-420             [16, 49, 1024]       4,195,328
         Dropout-421             [16, 49, 1024]               0
             Mlp-422             [16, 49, 1024]               0
        DropPath-423             [16, 49, 1024]               0
SwinTransformerBlock-424             [16, 49, 1024]               0
      BasicLayer-425             [16, 49, 1024]               0
       LayerNorm-426             [16, 49, 1024]           2,048
AdaptiveAvgPool1d-427              [16, 1024, 1]               0
          Linear-428                    [16, 7]           7,175
 SwinTransformer-429                    [16, 7]               0
================================================================
Total params: 86,686,855
Trainable params: 86,686,855
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 9.19
Forward/backward pass size (MB): 8669.97
Params size (MB): 330.68
Estimated Total Size (MB): 9009.84
----------------------------------------------------------------

Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.1090 (2.1090) || Train Top 1-acc : 12.500% (12.500)% || Train Top 5-acc : 62.500% (62.500)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.8202 (1.9161) || Train Top 1-acc : 25.000% (19.318)% || Train Top 5-acc : 87.500% (80.682)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.1825 (1.7320) || Train Top 1-acc : 68.750% (34.226)% || Train Top 5-acc : 93.750% (86.012)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.3470 (1.5649) || Train Top 1-acc : 62.500% (44.556)% || Train Top 5-acc : 100.000% (89.113)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.2557 (1.3340) || Train Top 1-acc : 93.750% (53.659)% || Train Top 5-acc : 100.000% (91.159)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.1729 (1.1790) || Train Top 1-acc : 93.750% (59.191)% || Train Top 5-acc : 100.000% (92.525)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 1.3103 (1.1023) || Train Top 1-acc : 75.000% (61.988)% || Train Top 5-acc : 100.000% (93.443)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 0.4908 (1.0439) || Train Top 1-acc : 87.500% (64.261)% || Train Top 5-acc : 100.000% (94.014)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.8526 (0.9797) || Train Top 1-acc : 18.750% (66.975)% || Train Top 5-acc : 100.000% (94.753)%

Epoch : [1/10] || Val Loss : 0.1027 || Val Top 1-acc : 97.059% || Val Top 5-acc : 100.000%

Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 1.0165 (1.0165) || Train Top 1-acc : 81.250% (81.250)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.1882 (0.4326) || Train Top 1-acc : 93.750% (92.614)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.0179 (0.4095) || Train Top 1-acc : 100.000% (88.393)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.1978 (0.4609) || Train Top 1-acc : 93.750% (87.702)% || Train Top 5-acc : 100.000% (99.194)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.3267 (0.4471) || Train Top 1-acc : 87.500% (88.110)% || Train Top 5-acc : 100.000% (99.238)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.8365 (0.4656) || Train Top 1-acc : 68.750% (88.235)% || Train Top 5-acc : 100.000% (99.387)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.0348 (0.4488) || Train Top 1-acc : 100.000% (88.525)% || Train Top 5-acc : 100.000% (99.385)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.2624 (0.4268) || Train Top 1-acc : 93.750% (88.732)% || Train Top 5-acc : 100.000% (99.472)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.0438 (0.4027) || Train Top 1-acc : 100.000% (88.657)% || Train Top 5-acc : 100.000% (99.537)%

Epoch : [2/10] || Val Loss : 0.0685 || Val Top 1-acc : 97.941% || Val Top 5-acc : 100.000%

Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.6699 (0.6699) || Train Top 1-acc : 37.500% (37.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.0201 (0.1761) || Train Top 1-acc : 100.000% (92.614)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0125 (0.2776) || Train Top 1-acc : 100.000% (94.345)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.6409 (0.3282) || Train Top 1-acc : 100.000% (92.137)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.0147 (0.3249) || Train Top 1-acc : 100.000% (92.835)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0040 (0.2935) || Train Top 1-acc : 100.000% (93.873)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0191 (0.2901) || Train Top 1-acc : 100.000% (94.570)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0112 (0.2966) || Train Top 1-acc : 100.000% (93.662)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.0798 (0.2730) || Train Top 1-acc : 100.000% (94.444)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [3/10] || Val Loss : 0.0472 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%

Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.0122 (0.0122) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.8667 (0.3944) || Train Top 1-acc : 87.500% (86.932)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.0085 (0.3728) || Train Top 1-acc : 100.000% (90.774)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.0063 (0.3491) || Train Top 1-acc : 100.000% (90.323)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.0090 (0.3131) || Train Top 1-acc : 100.000% (89.482)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.2690 (0.2854) || Train Top 1-acc : 100.000% (90.564)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.6340 (0.2916) || Train Top 1-acc : 93.750% (90.266)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.9933 (0.3147) || Train Top 1-acc : 12.500% (89.525)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0077 (0.2940) || Train Top 1-acc : 100.000% (89.969)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [4/10] || Val Loss : 0.0554 || Val Top 1-acc : 98.529% || Val Top 5-acc : 100.000%

Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.7766 (0.7766) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.6723 (0.4551) || Train Top 1-acc : 100.000% (89.205)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.4227 (0.4078) || Train Top 1-acc : 100.000% (87.798)% || Train Top 5-acc : 100.000% (99.702)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.6683 (0.4091) || Train Top 1-acc : 87.500% (87.298)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.0084 (0.4049) || Train Top 1-acc : 100.000% (87.957)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0036 (0.3768) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.0031 (0.3554) || Train Top 1-acc : 100.000% (88.012)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.5924 (0.3393) || Train Top 1-acc : 93.750% (89.261)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.0027 (0.3399) || Train Top 1-acc : 100.000% (89.275)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [5/10] || Val Loss : 0.0485 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%

Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.0060 (0.0060) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.6237 (0.1738) || Train Top 1-acc : 93.750% (95.455)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.8690 (0.2624) || Train Top 1-acc : 81.250% (94.345)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.7609 (0.3277) || Train Top 1-acc : 87.500% (92.944)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.3789 (0.3237) || Train Top 1-acc : 100.000% (92.530)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.0037 (0.3117) || Train Top 1-acc : 100.000% (92.402)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.4470 (0.3068) || Train Top 1-acc : 100.000% (93.648)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.7470 (0.3078) || Train Top 1-acc : 100.000% (93.398)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.0084 (0.3012) || Train Top 1-acc : 100.000% (93.673)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [6/10] || Val Loss : 0.0286 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%

Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.0056 (0.0056) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.2165 (0.2785) || Train Top 1-acc : 100.000% (92.045)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.4463 (0.3574) || Train Top 1-acc : 100.000% (87.202)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.5680 (0.4040) || Train Top 1-acc : 87.500% (86.089)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.6585 (0.4326) || Train Top 1-acc : 50.000% (81.402)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.0033 (0.4182) || Train Top 1-acc : 100.000% (82.353)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.4754 (0.4248) || Train Top 1-acc : 100.000% (82.480)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.0022 (0.3955) || Train Top 1-acc : 100.000% (83.275)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.1527 (0.3748) || Train Top 1-acc : 100.000% (84.799)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [7/10] || Val Loss : 0.0241 || Val Top 1-acc : 98.529% || Val Top 5-acc : 100.000%

Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.7053 (0.7053) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.0123 (0.4296) || Train Top 1-acc : 100.000% (88.068)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.5860 (0.3488) || Train Top 1-acc : 56.250% (89.583)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0035 (0.2884) || Train Top 1-acc : 100.000% (92.742)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.0017 (0.2458) || Train Top 1-acc : 100.000% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.6352 (0.2529) || Train Top 1-acc : 25.000% (92.157)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.4637 (0.2591) || Train Top 1-acc : 93.750% (91.291)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.0913 (0.2458) || Train Top 1-acc : 100.000% (92.430)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.0029 (0.2675) || Train Top 1-acc : 100.000% (91.898)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [8/10] || Val Loss : 0.0548 || Val Top 1-acc : 98.529% || Val Top 5-acc : 100.000%

Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.0085 (0.0085) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.0032 (0.2898) || Train Top 1-acc : 100.000% (93.182)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.0025 (0.2206) || Train Top 1-acc : 100.000% (96.131)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.0021 (0.2624) || Train Top 1-acc : 100.000% (92.540)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0024 (0.2308) || Train Top 1-acc : 100.000% (93.902)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0037 (0.2124) || Train Top 1-acc : 100.000% (94.240)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.6163 (0.2243) || Train Top 1-acc : 81.250% (94.570)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.5209 (0.2377) || Train Top 1-acc : 100.000% (94.190)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.0028 (0.2328) || Train Top 1-acc : 100.000% (93.981)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [9/10] || Val Loss : 0.0499 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%

Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.0012 (0.0012) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.4145 (0.2699) || Train Top 1-acc : 100.000% (97.727)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.0012 (0.2629) || Train Top 1-acc : 100.000% (97.619)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.5747 (0.2955) || Train Top 1-acc : 100.000% (91.532)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.0017 (0.2580) || Train Top 1-acc : 100.000% (91.311)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.7088 (0.2563) || Train Top 1-acc : 43.750% (89.583)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.6859 (0.2471) || Train Top 1-acc : 62.500% (90.369)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.6162 (0.2428) || Train Top 1-acc : 87.500% (90.933)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.0065 (0.2444) || Train Top 1-acc : 100.000% (91.127)% || Train Top 5-acc : 100.000% (99.846)%

Epoch : [10/10] || Val Loss : 0.0379 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%


'===============2-Fold Cross Validation==============='
Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 1.9786 (1.9786) || Train Top 1-acc : 18.750% (18.750)% || Train Top 5-acc : 75.000% (75.000)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.9056 (1.9197) || Train Top 1-acc : 12.500% (18.750)% || Train Top 5-acc : 68.750% (81.818)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.3824 (1.7464) || Train Top 1-acc : 50.000% (31.845)% || Train Top 5-acc : 93.750% (87.500)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 0.7627 (1.5567) || Train Top 1-acc : 81.250% (41.935)% || Train Top 5-acc : 100.000% (90.121)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.7342 (1.3341) || Train Top 1-acc : 100.000% (52.744)% || Train Top 5-acc : 100.000% (92.378)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.7819 (1.2193) || Train Top 1-acc : 81.250% (58.211)% || Train Top 5-acc : 100.000% (93.505)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 0.9304 (1.1261) || Train Top 1-acc : 87.500% (63.115)% || Train Top 5-acc : 100.000% (94.570)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 1.1189 (1.0639) || Train Top 1-acc : 68.750% (65.493)% || Train Top 5-acc : 100.000% (95.335)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.5482 (1.0153) || Train Top 1-acc : 87.500% (67.515)% || Train Top 5-acc : 100.000% (95.756)%

Epoch : [1/10] || Val Loss : 0.1106 || Val Top 1-acc : 97.647% || Val Top 5-acc : 99.412%

Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 1.0929 (1.0929) || Train Top 1-acc : 75.000% (75.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.9855 (0.5546) || Train Top 1-acc : 100.000% (86.364)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.0425 (0.4892) || Train Top 1-acc : 100.000% (86.607)% || Train Top 5-acc : 100.000% (98.810)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.4465 (0.4609) || Train Top 1-acc : 100.000% (87.903)% || Train Top 5-acc : 100.000% (99.194)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.1961 (0.4454) || Train Top 1-acc : 93.750% (85.366)% || Train Top 5-acc : 100.000% (99.238)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.3380 (0.4155) || Train Top 1-acc : 87.500% (84.926)% || Train Top 5-acc : 100.000% (99.265)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.0853 (0.4314) || Train Top 1-acc : 100.000% (84.324)% || Train Top 5-acc : 100.000% (99.385)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.8239 (0.4433) || Train Top 1-acc : 87.500% (84.683)% || Train Top 5-acc : 100.000% (99.384)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.7969 (0.4773) || Train Top 1-acc : 87.500% (84.491)% || Train Top 5-acc : 100.000% (99.383)%

Epoch : [2/10] || Val Loss : 0.0584 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%

Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.0847 (0.0847) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.0401 (0.4712) || Train Top 1-acc : 100.000% (78.409)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0320 (0.3945) || Train Top 1-acc : 100.000% (79.762)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.0384 (0.4485) || Train Top 1-acc : 100.000% (76.613)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.2413 (0.4236) || Train Top 1-acc : 100.000% (80.793)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0809 (0.3980) || Train Top 1-acc : 93.750% (83.088)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0139 (0.3604) || Train Top 1-acc : 100.000% (85.143)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0348 (0.3448) || Train Top 1-acc : 100.000% (86.180)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.0095 (0.3409) || Train Top 1-acc : 100.000% (86.343)% || Train Top 5-acc : 100.000% (99.691)%

Epoch : [3/10] || Val Loss : 0.0292 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%

Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.0142 (0.0142) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.7210 (0.1531) || Train Top 1-acc : 68.750% (92.045)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.0117 (0.1085) || Train Top 1-acc : 100.000% (94.940)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.6184 (0.1680) || Train Top 1-acc : 43.750% (93.548)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.6395 (0.2423) || Train Top 1-acc : 93.750% (93.598)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.0177 (0.2327) || Train Top 1-acc : 100.000% (94.363)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.0082 (0.2169) || Train Top 1-acc : 100.000% (94.365)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.0111 (0.2166) || Train Top 1-acc : 100.000% (94.190)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.4796 (0.2176) || Train Top 1-acc : 93.750% (93.827)% || Train Top 5-acc : 100.000% (99.846)%

Epoch : [4/10] || Val Loss : 0.0124 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%

Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.4371 (0.4371) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.0152 (0.5655) || Train Top 1-acc : 100.000% (78.409)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.0176 (0.3995) || Train Top 1-acc : 100.000% (87.202)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.0067 (0.3112) || Train Top 1-acc : 100.000% (90.524)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.8374 (0.2774) || Train Top 1-acc : 93.750% (91.921)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.2836 (0.2649) || Train Top 1-acc : 100.000% (92.525)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.0054 (0.2688) || Train Top 1-acc : 100.000% (90.881)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.0297 (0.2616) || Train Top 1-acc : 100.000% (91.813)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.6366 (0.2890) || Train Top 1-acc : 87.500% (90.818)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [5/10] || Val Loss : 0.0240 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%

Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.0070 (0.0070) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.0140 (0.2751) || Train Top 1-acc : 100.000% (89.205)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.8425 (0.3024) || Train Top 1-acc : 18.750% (87.202)% || Train Top 5-acc : 93.750% (99.702)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.0099 (0.2850) || Train Top 1-acc : 100.000% (86.290)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.7025 (0.2905) || Train Top 1-acc : 75.000% (85.213)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.8478 (0.2985) || Train Top 1-acc : 81.250% (87.255)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.3189 (0.3177) || Train Top 1-acc : 100.000% (86.988)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.0085 (0.2965) || Train Top 1-acc : 100.000% (88.028)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.0053 (0.2851) || Train Top 1-acc : 100.000% (89.352)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [6/10] || Val Loss : 0.0096 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%

Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.8451 (0.8451) || Train Top 1-acc : 68.750% (68.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.3970 (0.1916) || Train Top 1-acc : 93.750% (96.591)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.4457 (0.2410) || Train Top 1-acc : 93.750% (91.964)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.6965 (0.2587) || Train Top 1-acc : 93.750% (94.153)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.7570 (0.3079) || Train Top 1-acc : 93.750% (90.244)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.0028 (0.3068) || Train Top 1-acc : 100.000% (88.725)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0070 (0.2952) || Train Top 1-acc : 100.000% (90.471)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.6754 (0.3054) || Train Top 1-acc : 100.000% (89.789)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0094 (0.2912) || Train Top 1-acc : 100.000% (90.664)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [7/10] || Val Loss : 0.0050 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%

Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.7817 (0.7817) || Train Top 1-acc : 81.250% (81.250)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.7176 (0.4617) || Train Top 1-acc : 62.500% (84.659)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.8492 (0.4741) || Train Top 1-acc : 37.500% (80.655)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.5736 (0.4226) || Train Top 1-acc : 100.000% (83.871)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.4147 (0.4164) || Train Top 1-acc : 100.000% (84.909)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.0020 (0.3762) || Train Top 1-acc : 100.000% (86.642)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.6746 (0.3975) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.5722 (0.3860) || Train Top 1-acc : 50.000% (87.676)% || Train Top 5-acc : 93.750% (99.648)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.7933 (0.3881) || Train Top 1-acc : 87.500% (87.114)% || Train Top 5-acc : 100.000% (99.691)%

Epoch : [8/10] || Val Loss : 0.0110 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%

Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.0027 (0.0027) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.0083 (0.0644) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.0031 (0.1934) || Train Top 1-acc : 100.000% (91.964)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.6657 (0.2242) || Train Top 1-acc : 37.500% (90.726)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0052 (0.2519) || Train Top 1-acc : 100.000% (89.787)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0018 (0.2451) || Train Top 1-acc : 100.000% (89.216)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.0038 (0.2543) || Train Top 1-acc : 100.000% (89.754)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.0036 (0.2378) || Train Top 1-acc : 100.000% (90.581)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.0073 (0.2398) || Train Top 1-acc : 100.000% (91.204)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [9/10] || Val Loss : 0.0074 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%

Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.8033 (0.8033) || Train Top 1-acc : 18.750% (18.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.0049 (0.3229) || Train Top 1-acc : 100.000% (83.523)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.4185 (0.2718) || Train Top 1-acc : 87.500% (84.821)% || Train Top 5-acc : 100.000% (99.702)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0085 (0.2215) || Train Top 1-acc : 100.000% (89.113)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.0023 (0.2758) || Train Top 1-acc : 100.000% (89.177)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.4751 (0.2923) || Train Top 1-acc : 100.000% (88.603)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.0058 (0.2990) || Train Top 1-acc : 100.000% (87.295)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0025 (0.2934) || Train Top 1-acc : 100.000% (88.556)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.0030 (0.2960) || Train Top 1-acc : 100.000% (89.043)% || Train Top 5-acc : 100.000% (99.846)%

Epoch : [10/10] || Val Loss : 0.0043 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%


'===============3-Fold Cross Validation==============='

Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.0720 (2.0720) || Train Top 1-acc : 12.500% (12.500)% || Train Top 5-acc : 62.500% (62.500)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.7776 (1.9316) || Train Top 1-acc : 43.750% (23.864)% || Train Top 5-acc : 75.000% (76.136)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.4177 (1.8146) || Train Top 1-acc : 50.000% (29.762)% || Train Top 5-acc : 100.000% (82.738)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.5192 (1.6751) || Train Top 1-acc : 37.500% (36.895)% || Train Top 5-acc : 100.000% (86.694)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.7066 (1.4569) || Train Top 1-acc : 87.500% (48.018)% || Train Top 5-acc : 100.000% (89.634)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.8877 (1.3557) || Train Top 1-acc : 75.000% (52.819)% || Train Top 5-acc : 93.750% (91.299)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 1.1762 (1.2324) || Train Top 1-acc : 75.000% (57.582)% || Train Top 5-acc : 100.000% (92.623)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 0.2511 (1.1340) || Train Top 1-acc : 93.750% (61.180)% || Train Top 5-acc : 100.000% (93.134)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.0398 (1.0670) || Train Top 1-acc : 100.000% (64.275)% || Train Top 5-acc : 100.000% (93.981)%

Epoch : [1/10] || Val Loss : 0.1052 || Val Top 1-acc : 97.353% || Val Top 5-acc : 99.706%

Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 0.1522 (0.1522) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.7374 (0.5063) || Train Top 1-acc : 100.000% (88.636)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.0710 (0.4124) || Train Top 1-acc : 100.000% (88.393)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.0451 (0.4122) || Train Top 1-acc : 100.000% (86.290)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.9314 (0.4282) || Train Top 1-acc : 62.500% (85.366)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.0468 (0.4101) || Train Top 1-acc : 100.000% (85.784)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.7474 (0.4185) || Train Top 1-acc : 93.750% (86.475)% || Train Top 5-acc : 100.000% (99.590)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.0288 (0.4013) || Train Top 1-acc : 100.000% (87.764)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.0481 (0.4107) || Train Top 1-acc : 100.000% (87.346)% || Train Top 5-acc : 100.000% (99.614)%

Epoch : [2/10] || Val Loss : 0.0466 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%

Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.0975 (0.0975) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.6538 (0.3331) || Train Top 1-acc : 93.750% (97.159)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0456 (0.3037) || Train Top 1-acc : 100.000% (97.917)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.8460 (0.3477) || Train Top 1-acc : 68.750% (94.556)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.2994 (0.3534) || Train Top 1-acc : 93.750% (92.073)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0195 (0.3358) || Train Top 1-acc : 100.000% (92.034)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0105 (0.3068) || Train Top 1-acc : 100.000% (92.623)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0128 (0.3065) || Train Top 1-acc : 100.000% (92.694)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.3932 (0.3081) || Train Top 1-acc : 93.750% (92.901)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [3/10] || Val Loss : 0.0331 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%

Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.5570 (0.5570) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.3258 (0.3006) || Train Top 1-acc : 100.000% (94.318)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.5734 (0.3007) || Train Top 1-acc : 100.000% (90.774)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.0303 (0.3164) || Train Top 1-acc : 100.000% (89.718)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.0232 (0.3054) || Train Top 1-acc : 100.000% (91.768)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.0116 (0.3024) || Train Top 1-acc : 100.000% (92.157)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.8345 (0.3167) || Train Top 1-acc : 87.500% (91.701)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.0108 (0.3137) || Train Top 1-acc : 100.000% (90.845)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0077 (0.3195) || Train Top 1-acc : 100.000% (89.429)% || Train Top 5-acc : 100.000% (99.846)%

Epoch : [4/10] || Val Loss : 0.0223 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%

Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.0156 (0.0156) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.0072 (0.2611) || Train Top 1-acc : 100.000% (84.659)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.0092 (0.1958) || Train Top 1-acc : 100.000% (89.286)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.0100 (0.1986) || Train Top 1-acc : 100.000% (92.742)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.6878 (0.2660) || Train Top 1-acc : 100.000% (92.226)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0070 (0.2976) || Train Top 1-acc : 100.000% (90.441)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.9078 (0.3341) || Train Top 1-acc : 81.250% (89.139)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.6196 (0.3450) || Train Top 1-acc : 81.250% (88.556)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.0594 (0.3470) || Train Top 1-acc : 100.000% (88.580)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [5/10] || Val Loss : 0.0287 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%

Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.5246 (0.5246) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.6923 (0.3866) || Train Top 1-acc : 87.500% (89.205)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.5975 (0.4660) || Train Top 1-acc : 62.500% (83.631)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.7972 (0.4186) || Train Top 1-acc : 18.750% (83.065)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.0336 (0.4082) || Train Top 1-acc : 100.000% (83.994)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.0021 (0.3904) || Train Top 1-acc : 100.000% (86.887)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.0051 (0.3658) || Train Top 1-acc : 100.000% (88.320)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.5820 (0.3447) || Train Top 1-acc : 100.000% (89.525)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.0028 (0.3448) || Train Top 1-acc : 100.000% (89.275)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [6/10] || Val Loss : 0.0236 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%

Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.7110 (0.7110) || Train Top 1-acc : 87.500% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.9598 (0.4299) || Train Top 1-acc : 25.000% (80.682)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.5731 (0.3490) || Train Top 1-acc : 100.000% (89.286)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.0206 (0.3777) || Train Top 1-acc : 100.000% (87.702)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.0045 (0.3647) || Train Top 1-acc : 100.000% (89.177)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.5658 (0.3217) || Train Top 1-acc : 62.500% (90.074)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0022 (0.3149) || Train Top 1-acc : 100.000% (90.164)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.5986 (0.3160) || Train Top 1-acc : 37.500% (88.556)% || Train Top 5-acc : 93.750% (99.736)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0064 (0.3103) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [7/10] || Val Loss : 0.0229 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%

Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.0126 (0.0126) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.0046 (0.3209) || Train Top 1-acc : 100.000% (84.659)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.0030 (0.2900) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0098 (0.2692) || Train Top 1-acc : 100.000% (91.331)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.0039 (0.2477) || Train Top 1-acc : 100.000% (92.530)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.8142 (0.2741) || Train Top 1-acc : 62.500% (91.667)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.6055 (0.2918) || Train Top 1-acc : 93.750% (89.447)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.6436 (0.3183) || Train Top 1-acc : 100.000% (88.644)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.0013 (0.2972) || Train Top 1-acc : 100.000% (89.043)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [8/10] || Val Loss : 0.0184 || Val Top 1-acc : 99.412% || Val Top 5-acc : 100.000%

Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.0027 (0.0027) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.5898 (0.2487) || Train Top 1-acc : 75.000% (89.205)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.0021 (0.1797) || Train Top 1-acc : 100.000% (94.345)% || Train Top 5-acc : 100.000% (99.107)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.0068 (0.2011) || Train Top 1-acc : 100.000% (94.758)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.3292 (0.2237) || Train Top 1-acc : 100.000% (95.122)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0079 (0.2265) || Train Top 1-acc : 100.000% (93.995)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.0014 (0.2211) || Train Top 1-acc : 100.000% (93.852)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.7926 (0.2363) || Train Top 1-acc : 75.000% (93.046)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.6087 (0.2463) || Train Top 1-acc : 31.250% (92.901)% || Train Top 5-acc : 87.500% (99.537)%

Epoch : [9/10] || Val Loss : 0.0131 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%

Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.4328 (0.4328) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.6455 (0.4515) || Train Top 1-acc : 100.000% (88.068)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.2457 (0.3538) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0033 (0.2960) || Train Top 1-acc : 100.000% (90.726)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.5825 (0.3055) || Train Top 1-acc : 100.000% (90.244)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.6785 (0.3167) || Train Top 1-acc : 100.000% (91.544)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.5924 (0.3040) || Train Top 1-acc : 100.000% (92.316)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0020 (0.2927) || Train Top 1-acc : 100.000% (92.870)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.5519 (0.2729) || Train Top 1-acc : 100.000% (92.824)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [10/10] || Val Loss : 0.0160 || Val Top 1-acc : 99.412% || Val Top 5-acc : 100.000%


'===============4-Fold Cross Validation==============='

Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.1108 (2.1108) || Train Top 1-acc : 0.000% (0.000)% || Train Top 5-acc : 62.500% (62.500)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.8617 (1.8976) || Train Top 1-acc : 12.500% (15.909)% || Train Top 5-acc : 100.000% (79.545)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.1728 (1.7297) || Train Top 1-acc : 62.500% (29.762)% || Train Top 5-acc : 100.000% (87.202)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.0114 (1.5880) || Train Top 1-acc : 43.750% (37.702)% || Train Top 5-acc : 100.000% (89.516)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 1.1918 (1.3777) || Train Top 1-acc : 87.500% (48.018)% || Train Top 5-acc : 93.750% (91.463)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.3317 (1.1855) || Train Top 1-acc : 93.750% (55.147)% || Train Top 5-acc : 100.000% (92.770)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 0.7014 (1.1024) || Train Top 1-acc : 93.750% (60.041)% || Train Top 5-acc : 100.000% (93.852)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 1.0357 (1.0330) || Train Top 1-acc : 56.250% (62.500)% || Train Top 5-acc : 100.000% (94.630)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.1170 (0.9727) || Train Top 1-acc : 93.750% (65.355)% || Train Top 5-acc : 100.000% (95.139)%

Epoch : [1/10] || Val Loss : 0.1316 || Val Top 1-acc : 96.755% || Val Top 5-acc : 100.000%

Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 0.9091 (0.9091) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.0604 (0.5182) || Train Top 1-acc : 100.000% (82.386)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.4621 (0.4736) || Train Top 1-acc : 87.500% (85.714)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.0275 (0.4028) || Train Top 1-acc : 100.000% (89.113)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.1133 (0.3619) || Train Top 1-acc : 100.000% (90.549)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.0364 (0.3784) || Train Top 1-acc : 100.000% (88.971)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.0336 (0.3608) || Train Top 1-acc : 100.000% (90.164)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.8431 (0.3535) || Train Top 1-acc : 81.250% (90.141)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 1.1812 (0.3608) || Train Top 1-acc : 37.500% (89.429)% || Train Top 5-acc : 93.750% (99.691)%

Epoch : [2/10] || Val Loss : 0.1468 || Val Top 1-acc : 96.165% || Val Top 5-acc : 100.000%

Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.2246 (0.2246) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.9671 (0.2779) || Train Top 1-acc : 68.750% (94.318)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0288 (0.2994) || Train Top 1-acc : 100.000% (91.964)% || Train Top 5-acc : 100.000% (99.107)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.7963 (0.3208) || Train Top 1-acc : 75.000% (92.339)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.0200 (0.3825) || Train Top 1-acc : 100.000% (88.415)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0376 (0.3615) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0171 (0.3721) || Train Top 1-acc : 100.000% (89.242)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0067 (0.3339) || Train Top 1-acc : 100.000% (90.317)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.0069 (0.3072) || Train Top 1-acc : 100.000% (90.818)% || Train Top 5-acc : 100.000% (99.691)%

Epoch : [3/10] || Val Loss : 0.0580 || Val Top 1-acc : 97.345% || Val Top 5-acc : 100.000%
 
Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.5756 (0.5756) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.0418 (0.3361) || Train Top 1-acc : 100.000% (82.955)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.6049 (0.3751) || Train Top 1-acc : 100.000% (84.524)% || Train Top 5-acc : 100.000% (99.107)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.1730 (0.3705) || Train Top 1-acc : 93.750% (84.879)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.7159 (0.3748) || Train Top 1-acc : 100.000% (87.805)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.3958 (0.3576) || Train Top 1-acc : 100.000% (89.216)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.0115 (0.3394) || Train Top 1-acc : 100.000% (89.857)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.2760 (0.3272) || Train Top 1-acc : 100.000% (90.845)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0097 (0.3161) || Train Top 1-acc : 100.000% (90.664)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [4/10] || Val Loss : 0.0969 || Val Top 1-acc : 97.050% || Val Top 5-acc : 100.000%
 
Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.2456 (0.2456) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.0148 (0.3081) || Train Top 1-acc : 100.000% (91.477)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.6745 (0.3977) || Train Top 1-acc : 93.750% (90.774)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.0173 (0.3700) || Train Top 1-acc : 100.000% (90.323)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.0148 (0.3435) || Train Top 1-acc : 100.000% (91.768)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0073 (0.3391) || Train Top 1-acc : 100.000% (91.422)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.4713 (0.3340) || Train Top 1-acc : 100.000% (92.008)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.0109 (0.3454) || Train Top 1-acc : 100.000% (92.430)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.8212 (0.3591) || Train Top 1-acc : 87.500% (91.358)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [5/10] || Val Loss : 0.0336 || Val Top 1-acc : 98.820% || Val Top 5-acc : 100.000%

Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.3899 (0.3899) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.0154 (0.0537) || Train Top 1-acc : 100.000% (99.432)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.7629 (0.1997) || Train Top 1-acc : 93.750% (89.286)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.5958 (0.2350) || Train Top 1-acc : 87.500% (91.129)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.0045 (0.2186) || Train Top 1-acc : 100.000% (92.073)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.5276 (0.2654) || Train Top 1-acc : 93.750% (90.686)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.0084 (0.2493) || Train Top 1-acc : 100.000% (91.189)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.0057 (0.2545) || Train Top 1-acc : 100.000% (89.613)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.6041 (0.2553) || Train Top 1-acc : 62.500% (89.352)% || Train Top 5-acc : 100.000% (99.691)%

Epoch : [6/10] || Val Loss : 0.0320 || Val Top 1-acc : 98.230% || Val Top 5-acc : 100.000%

Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.5307 (0.5307) || Train Top 1-acc : 87.500% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.0054 (0.3147) || Train Top 1-acc : 100.000% (91.477)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.0055 (0.2666) || Train Top 1-acc : 100.000% (93.452)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.0080 (0.2908) || Train Top 1-acc : 100.000% (89.315)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.0085 (0.2561) || Train Top 1-acc : 100.000% (90.244)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.6985 (0.2960) || Train Top 1-acc : 81.250% (87.132)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0053 (0.3152) || Train Top 1-acc : 100.000% (85.246)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.2425 (0.3452) || Train Top 1-acc : 100.000% (84.419)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0069 (0.3274) || Train Top 1-acc : 100.000% (84.491)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [7/10] || Val Loss : 0.0270 || Val Top 1-acc : 99.410% || Val Top 5-acc : 100.000%

Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.4553 (0.4553) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.5086 (0.1545) || Train Top 1-acc : 93.750% (99.432)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.0089 (0.2241) || Train Top 1-acc : 100.000% (95.238)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0119 (0.2344) || Train Top 1-acc : 100.000% (91.935)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.0201 (0.2248) || Train Top 1-acc : 100.000% (90.549)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.0034 (0.2730) || Train Top 1-acc : 100.000% (88.971)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.0028 (0.2727) || Train Top 1-acc : 100.000% (89.754)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.0142 (0.2914) || Train Top 1-acc : 100.000% (90.141)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.0028 (0.2728) || Train Top 1-acc : 100.000% (91.204)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [8/10] || Val Loss : 0.0294 || Val Top 1-acc : 98.525% || Val Top 5-acc : 100.000%

Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.7818 (0.7818) || Train Top 1-acc : 75.000% (75.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.7282 (0.3394) || Train Top 1-acc : 68.750% (85.227)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.2990 (0.3756) || Train Top 1-acc : 100.000% (82.143)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.0026 (0.3406) || Train Top 1-acc : 100.000% (84.677)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0021 (0.2912) || Train Top 1-acc : 100.000% (88.262)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0030 (0.2494) || Train Top 1-acc : 100.000% (90.319)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.7077 (0.2870) || Train Top 1-acc : 18.750% (90.164)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.0023 (0.2583) || Train Top 1-acc : 100.000% (91.373)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.6054 (0.2762) || Train Top 1-acc : 100.000% (90.664)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [9/10] || Val Loss : 0.0227 || Val Top 1-acc : 99.410% || Val Top 5-acc : 100.000%

Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.0028 (0.0028) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.2013 (0.3793) || Train Top 1-acc : 100.000% (98.295)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.6386 (0.4370) || Train Top 1-acc : 100.000% (91.071)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0031 (0.3684) || Train Top 1-acc : 100.000% (93.347)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.5983 (0.3663) || Train Top 1-acc : 100.000% (91.768)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.8164 (0.3471) || Train Top 1-acc : 81.250% (91.544)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.2539 (0.3466) || Train Top 1-acc : 100.000% (91.496)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0026 (0.3320) || Train Top 1-acc : 100.000% (91.637)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.3697 (0.3306) || Train Top 1-acc : 100.000% (91.821)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [10/10] || Val Loss : 0.0249 || Val Top 1-acc : 99.115% || Val Top 5-acc : 100.000%


'===============5-Fold Cross Validation==============='

Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.0386 (2.0386) || Train Top 1-acc : 6.250% (6.250)% || Train Top 5-acc : 75.000% (75.000)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 2.0713 (1.9109) || Train Top 1-acc : 31.250% (20.455)% || Train Top 5-acc : 56.250% (79.545)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.3254 (1.7304) || Train Top 1-acc : 75.000% (34.524)% || Train Top 5-acc : 100.000% (86.310)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.0685 (1.5021) || Train Top 1-acc : 81.250% (46.169)% || Train Top 5-acc : 100.000% (90.524)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.2387 (1.2839) || Train Top 1-acc : 93.750% (56.402)% || Train Top 5-acc : 100.000% (92.835)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 1.2306 (1.1856) || Train Top 1-acc : 81.250% (61.520)% || Train Top 5-acc : 93.750% (93.995)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 0.4093 (1.0717) || Train Top 1-acc : 93.750% (65.881)% || Train Top 5-acc : 100.000% (94.877)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 0.9882 (1.0018) || Train Top 1-acc : 43.750% (68.134)% || Train Top 5-acc : 93.750% (95.511)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.4428 (0.9365) || Train Top 1-acc : 93.750% (70.679)% || Train Top 5-acc : 100.000% (96.065)%

Epoch : [1/10] || Val Loss : 0.2146 || Val Top 1-acc : 94.690% || Val Top 5-acc : 100.000%

Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 0.0805 (0.0805) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 1.1480 (0.6476) || Train Top 1-acc : 50.000% (82.386)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.1377 (0.5387) || Train Top 1-acc : 93.750% (86.310)% || Train Top 5-acc : 100.000% (98.810)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.0670 (0.5262) || Train Top 1-acc : 100.000% (84.677)% || Train Top 5-acc : 100.000% (98.992)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.2268 (0.5196) || Train Top 1-acc : 93.750% (84.756)% || Train Top 5-acc : 100.000% (99.238)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.8585 (0.5349) || Train Top 1-acc : 81.250% (84.436)% || Train Top 5-acc : 100.000% (99.142)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.5709 (0.5292) || Train Top 1-acc : 100.000% (85.143)% || Train Top 5-acc : 100.000% (99.078)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.5153 (0.5029) || Train Top 1-acc : 93.750% (86.092)% || Train Top 5-acc : 100.000% (99.208)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.9393 (0.4761) || Train Top 1-acc : 93.750% (87.423)% || Train Top 5-acc : 100.000% (99.306)%

Epoch : [2/10] || Val Loss : 0.1453 || Val Top 1-acc : 95.575% || Val Top 5-acc : 99.410%

Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.0124 (0.0124) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.0187 (0.2988) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0271 (0.2670) || Train Top 1-acc : 100.000% (91.964)% || Train Top 5-acc : 100.000% (99.702)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.0087 (0.2915) || Train Top 1-acc : 100.000% (89.718)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.0161 (0.2893) || Train Top 1-acc : 100.000% (90.549)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0083 (0.3079) || Train Top 1-acc : 100.000% (91.176)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0309 (0.3210) || Train Top 1-acc : 100.000% (92.008)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0124 (0.3300) || Train Top 1-acc : 100.000% (91.461)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.8605 (0.3338) || Train Top 1-acc : 75.000% (90.664)% || Train Top 5-acc : 100.000% (99.614)%

Epoch : [3/10] || Val Loss : 0.1214 || Val Top 1-acc : 97.345% || Val Top 5-acc : 99.705%

Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.0080 (0.0080) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.0059 (0.2478) || Train Top 1-acc : 100.000% (92.614)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.9131 (0.2578) || Train Top 1-acc : 68.750% (91.964)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.7515 (0.2385) || Train Top 1-acc : 100.000% (94.153)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.4925 (0.2670) || Train Top 1-acc : 100.000% (93.750)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.0057 (0.2657) || Train Top 1-acc : 100.000% (94.240)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.3083 (0.2825) || Train Top 1-acc : 100.000% (93.135)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.0053 (0.2897) || Train Top 1-acc : 100.000% (91.285)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0127 (0.2777) || Train Top 1-acc : 100.000% (91.435)% || Train Top 5-acc : 100.000% (99.846)%

Epoch : [4/10] || Val Loss : 0.1148 || Val Top 1-acc : 96.755% || Val Top 5-acc : 100.000%

Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.6259 (0.6259) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.7918 (0.2159) || Train Top 1-acc : 0.000% (89.773)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.7312 (0.3097) || Train Top 1-acc : 75.000% (86.607)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.2967 (0.2696) || Train Top 1-acc : 100.000% (89.113)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.6948 (0.3178) || Train Top 1-acc : 62.500% (85.671)% || Train Top 5-acc : 100.000% (99.390)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0035 (0.3003) || Train Top 1-acc : 100.000% (87.868)% || Train Top 5-acc : 100.000% (99.510)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.4196 (0.2807) || Train Top 1-acc : 93.750% (89.242)% || Train Top 5-acc : 100.000% (99.590)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.6039 (0.2869) || Train Top 1-acc : 100.000% (89.613)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.9320 (0.2970) || Train Top 1-acc : 81.250% (89.506)% || Train Top 5-acc : 100.000% (99.691)%

Epoch : [5/10] || Val Loss : 0.1448 || Val Top 1-acc : 95.870% || Val Top 5-acc : 99.705%

Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.0042 (0.0042) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.3565 (0.2846) || Train Top 1-acc : 100.000% (95.455)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.0052 (0.2608) || Train Top 1-acc : 100.000% (95.238)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.6910 (0.2762) || Train Top 1-acc : 87.500% (91.129)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.6822 (0.2907) || Train Top 1-acc : 100.000% (92.988)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.0330 (0.3036) || Train Top 1-acc : 100.000% (91.667)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.0039 (0.3174) || Train Top 1-acc : 100.000% (91.393)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.0062 (0.2918) || Train Top 1-acc : 100.000% (91.725)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.7268 (0.2846) || Train Top 1-acc : 43.750% (92.052)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [6/10] || Val Loss : 0.1204 || Val Top 1-acc : 97.050% || Val Top 5-acc : 100.000%

Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.0055 (0.0055) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.4340 (0.3952) || Train Top 1-acc : 50.000% (78.977)% || Train Top 5-acc : 100.000% (98.864)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.0042 (0.3191) || Train Top 1-acc : 100.000% (82.440)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.0018 (0.2937) || Train Top 1-acc : 100.000% (85.081)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.7229 (0.2791) || Train Top 1-acc : 87.500% (87.500)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.6283 (0.3096) || Train Top 1-acc : 62.500% (87.868)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0016 (0.3001) || Train Top 1-acc : 100.000% (89.139)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.2362 (0.2912) || Train Top 1-acc : 100.000% (88.556)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0154 (0.2831) || Train Top 1-acc : 100.000% (88.812)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [7/10] || Val Loss : 0.1291 || Val Top 1-acc : 97.050% || Val Top 5-acc : 100.000%

Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.5337 (0.5337) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.0025 (0.1729) || Train Top 1-acc : 100.000% (96.591)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.0025 (0.1783) || Train Top 1-acc : 100.000% (94.643)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0010 (0.1866) || Train Top 1-acc : 100.000% (95.363)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.5238 (0.2408) || Train Top 1-acc : 100.000% (94.360)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.5768 (0.2467) || Train Top 1-acc : 100.000% (93.505)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.0017 (0.2528) || Train Top 1-acc : 100.000% (94.262)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.0026 (0.2893) || Train Top 1-acc : 100.000% (92.958)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.5362 (0.3126) || Train Top 1-acc : 100.000% (91.049)% || Train Top 5-acc : 100.000% (99.923)%

Epoch : [8/10] || Val Loss : 0.1266 || Val Top 1-acc : 96.755% || Val Top 5-acc : 99.705%

Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.5063 (0.5063) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.2515 (0.3973) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.7745 (0.3355) || Train Top 1-acc : 100.000% (93.452)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.5575 (0.3380) || Train Top 1-acc : 93.750% (90.927)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0040 (0.3345) || Train Top 1-acc : 100.000% (89.177)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0038 (0.3008) || Train Top 1-acc : 100.000% (91.299)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.5112 (0.2993) || Train Top 1-acc : 87.500% (92.316)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.5287 (0.2962) || Train Top 1-acc : 100.000% (92.606)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.6574 (0.3093) || Train Top 1-acc : 56.250% (92.130)% || Train Top 5-acc : 100.000% (99.769)%

Epoch : [9/10] || Val Loss : 0.1025 || Val Top 1-acc : 97.935% || Val Top 5-acc : 100.000%

Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.7395 (0.7395) || Train Top 1-acc : 6.250% (6.250)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.0021 (0.3403) || Train Top 1-acc : 100.000% (86.932)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.0024 (0.2719) || Train Top 1-acc : 100.000% (88.393)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0027 (0.2210) || Train Top 1-acc : 100.000% (91.734)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.0007 (0.2749) || Train Top 1-acc : 100.000% (88.872)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.0010 (0.2503) || Train Top 1-acc : 100.000% (90.074)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.4998 (0.2308) || Train Top 1-acc : 100.000% (90.471)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0066 (0.2262) || Train Top 1-acc : 100.000% (91.813)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.5944 (0.2119) || Train Top 1-acc : 87.500% (92.593)% || Train Top 5-acc : 100.000% (100.000)%

Epoch : [10/10] || Val Loss : 0.1011 || Val Top 1-acc : 97.935% || Val Top 5-acc : 100.000%

Inference¶

• Test Time Augmentation 사용 (only hflip)

In [2]:

# class TestDataset(Dataset):
#     def __init__(self, data_path='./test/0/', transform=None):
#         self.data_path = data_path
#         self.data = os.listdir(data_path)
#         self.transform = transform
        
#     def __len__(self):
#         return len(self.data)

#     def __getitem__(self, idx):        
#         image_path = os.path.join(self.data_path, self.data[idx])
#         image = cv2.imread(image_path)
#         image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
#         if self.transform:
#             image = self.transform(image=np.array(image))['image']
            
#         return image

In [3]:

# SEED = 77  
# BATCH_SIZE = 32    
# IMAGE_SIZE = 227
# MODEL_ARC = 'nfnet_l0'
# NUM_CLASSES = 7
# MODEL_DIR = './results'
# NUM_FOLD = 5

In [4]:

# device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

In [5]:

# # Fix random seed
# def seed_everything(seed: int = 42):
#     random.seed(seed)
#     np.random.seed(seed)
#     os.environ["PYTHONHASHSEED"] = str(seed)
#     torch.manual_seed(seed)
#     torch.cuda.manual_seed(seed)  # type: ignore
#     torch.backends.cudnn.deterministic = True  # type: ignore
#     torch.backends.cudnn.benchmark = True  # type: ignore

In [6]:

# seed_everything(SEED)

In [7]:

# test_transform = albumentations.Compose([               
#         albumentations.Resize(IMAGE_SIZE, IMAGE_SIZE),
#         albumentations.Normalize(mean=(0.4569, 0.5074, 0.5557), std=(0.2888, 0.2743, 0.2829)),
#         albumentations.pytorch.transforms.ToTensorV2()])

In [8]:

# test_dataset = TestDataset(transform=test_transform)
# test_loader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False)

In [9]:

# class PretrainedModel(nn.Module):
#     def __init__(self, model_arc='swin_tiny_patch4_window7_224', num_classes=7):
#         super().__init__()
#         self.net = timm.create_model(model_arc, pretrained=False, num_classes=num_classes)
    
#     def forward(self, x):
#         x = self.net(x)

#         return x

In [10]:

# model = PretrainedModel(model_arc=MODEL_ARC, num_classes=NUM_CLASSES)
# model.to(device)

Out[10]:

PretrainedModel(
  (net): NormFreeNet(
    (stem): Sequential(
      (conv1): ScaledStdConv2d(3, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (act2): SiLU(inplace=True)
      (conv2): ScaledStdConv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (act3): SiLU(inplace=True)
      (conv3): ScaledStdConv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (act4): SiLU(inplace=True)
      (conv4): ScaledStdConv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
    )
    (stages): Sequential(
      (0): Sequential(
        (0): NormFreeBlock(
          (downsample): DownsampleAvg(
            (pool): Identity()
            (conv): ScaledStdConv2d(128, 256, kernel_size=(1, 1), stride=(1, 1))
          )
          (act1): SiLU()
          (conv1): ScaledStdConv2d(128, 64, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (act3): SiLU()
          (conv3): ScaledStdConv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
      )
      (1): Sequential(
        (0): NormFreeBlock(
          (downsample): DownsampleAvg(
            (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
            (conv): ScaledStdConv2d(256, 512, kernel_size=(1, 1), stride=(1, 1))
          )
          (act1): SiLU()
          (conv1): ScaledStdConv2d(256, 128, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=2)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=2)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (1): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=2)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=2)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
      )
      (2): Sequential(
        (0): NormFreeBlock(
          (downsample): DownsampleAvg(
            (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
            (conv): ScaledStdConv2d(512, 1536, kernel_size=(1, 1), stride=(1, 1))
          )
          (act1): SiLU()
          (conv1): ScaledStdConv2d(512, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (1): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (2): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (3): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (4): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (5): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
      )
      (3): Sequential(
        (0): NormFreeBlock(
          (downsample): DownsampleAvg(
            (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
            (conv): ScaledStdConv2d(1536, 1536, kernel_size=(1, 1), stride=(1, 1))
          )
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (1): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
        (2): NormFreeBlock(
          (act1): SiLU()
          (conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
          (act2): SiLU(inplace=True)
          (conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act2b): SiLU(inplace=True)
          (conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
          (act3): SiLU()
          (conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
          (attn_last): SEModule(
            (fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
            (act): ReLU(inplace=True)
            (fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
            (gate): Sigmoid()
          )
          (drop_path): Identity()
        )
      )
    )
    (final_conv): ScaledStdConv2d(1536, 2304, kernel_size=(1, 1), stride=(1, 1))
    (final_act): SiLU(inplace=True)
    (head): ClassifierHead(
      (global_pool): SelectAdaptivePool2d (pool_type=avg, flatten=True)
      (fc): Linear(in_features=2304, out_features=7, bias=True)
    )
  )
)

In [11]:

# states = [torch.load(glob(MODEL_DIR + f'/{MODEL_ARC}/{k}_fold/*.pth')[-1]) for k in range(1, NUM_FOLD + 1)]

In [12]:

# transforms = tta.Compose(
#     [
#         tta.HorizontalFlip(),
#         # tta.VerticalFlip(),
#         # tta.Multiply(factors=[0.9, 1, 1.1])
#     ]
# )

model ensemble을 위해서 npy 파일 형태로 저장¶

In [13]:

# probs = []
# save_ = []
# for i, images in enumerate(tqdm(test_loader)):
#     images = images.to(device)
#     avg_preds = []
#     for state in states:
#         model.load_state_dict(state)
#         model.eval()
#         tta_model = tta.ClassificationTTAWrapper(model, transforms)
#         tta_model.to(device)
#         tta_model.eval()
#         with torch.no_grad():
#             logits = tta_model(images)
#         avg_preds.append(logits.to('cpu').numpy())
#     avg_preds = np.mean(avg_preds, axis=0)
#     save_.append(avg_preds)
#     probs.append(avg_preds.argmax(-1))
# save_ = np.concatenate(save_)
# probs = np.concatenate(probs)

 

In [14]:

# df = pd.read_csv('./test_answer_sample_.csv')

In [15]:

# len(probs)

Out[15]:

350

In [16]:

# save_.shape

Out[16]:

(350, 7)

In [17]:

# np.save(f'./{MODEL_ARC}.npy', save_)

In [18]:

# df['answer value'] = probs

In [ ]:

# df.to_csv(f'submission_{MODEL_ARC}.csv', index=False)

Ensemble¶

• nfnet_l0와 swin transformer 사용
• stacking ensemble

In [10]:

# import numpy as np
# import pandas as pd

In [3]:

# npy_list = [
#     'nfnet_l0.npy',
#     'swin_base_patch4_window7_224.npy'
# ]

# predictions = []
# for f in npy_list:
#     predictions.append(np.load(f))

In [5]:

# np.array(predictions).shape

Out[5]:

(2, 350, 7)

In [15]:

# final = np.array(predictions).mean(axis=0).argmax(-1)
# final.shape

Out[15]:

(350,)

In [16]:

# df = pd.read_csv('./test_answer_sample_.csv')

In [17]:

# df['answer value'] = final

In [18]:

# df.to_csv(f'submission_ensemble.csv', index=False)