취업 이야기/데브매칭 문제 해설
2021 ML Dev-Matching | 미술 작품 분류하기 : 출제자가 추천한 우수 코드 A
grepp
2021. 7. 6. 09:29
목차
- Dataset
- Import Lib
- ConfigManager
- SEED고정
- Load Data
- Model
- Utils
- CosineAnnealingWarmupRestarts
- CutMix
- 지표 계산을 위한 average meter
- Training
- Inference
- model ensemble을 위해서 npy 파일 형태로 저장
- Ensemble
Dataset¶
- EDA(Exploratory Data Analysis, 탐색적 데이터 분석)를 진행하면서 동시에 csv를 만들었습니다.
- image와 label을 반환합니다.
- albumentations augmentation을 염두해두고 설계했기 때문에 이미지를 opencv로 읽었습니다.
In [1]:
import os
import cv2
import numpy as np
from torch.utils.data import Dataset
In [2]:
class ArtPaintDataset(Dataset):
def __init__(self, df, transform=None):
super().__init__()
self.df = df.reset_index()
self.image_id = self.df.image_id
self.labels = self.df.label
self.transform = transform
def __len__(self):
return len(self.df)
def set_transform(self, transform):
self.transform = transform
def __getitem__(self, idx):
image_path = self.image_id[idx]
image = cv2.imread(image_path)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
label = self.labels[idx]
if self.transform:
image = self.transform(image=np.array(image))['image']
return {'image' : image, 'label' : label}
Import Lib¶
In [3]:
import os
import math
import timm
import yaml
import torch
import random
import numpy as np
import pandas as pd
import torch.nn as nn
import albumentations
import albumentations.pytorch
from tqdm.auto import tqdm
from adamp import AdamP
from easydict import EasyDict
from prettyprinter import cpprint
from torchsummary import summary as summary_
from torch.utils.data import Dataset, DataLoader
from torch.optim.lr_scheduler import _LRScheduler
from torch.optim.lr_scheduler import CosineAnnealingLR
from sklearn.model_selection import StratifiedKFold, train_test_split
ConfigManager¶
- .yaml 파일을 사용하여 config를 관리했습니다.
- config를 바꿔가며 실험했습니다.
- 최종적으로 사용한 config는 다음과 같습니다.
base:
seed: 77
model_arc: 'nfnet_l0'
num_classes: 7
input_dir: './train/train.csv'
output_dir: './results/'
train_only: False
image_size: 227
cutmix_args:
use_cutmix: False
beta: 1.0
cutmix_prob: 0.5
train_args:
num_epochs: 6
train_batch_size: 32
val_batch_size: 32
max_lr: 0.0001
min_lr: 0.00001
cycle: 3
gamma: 0.5
weight_decay: 0.000001
log_intervals: 10
eval_metric: 'accuracy'
n_splits: 5swin:
seed: 777
model_arc: 'swin_base_patch4_window7_224'
num_classes: 7
input_dir: './train/train.csv'
output_dir: './results/'
train_only: False
image_size: 224
cutmix_args:
use_cutmix: True
beta: 1.0
cutmix_prob: 0.5
train_args:
num_epochs: 10
train_batch_size: 16
val_batch_size: 16
max_lr: 0.0001
min_lr: 0.00001
cycle: 3
gamma: 0.5
weight_decay: 0.000001
log_intervals: 10
eval_metric: 'accuracy'
n_splits: 5In [4]:
# Set Config
class YamlConfigManager:
def __init__(self, config_file_path='./config.yml', config_name='swin'):
super().__init__()
self.values = EasyDict()
if config_file_path:
self.config_file_path = config_file_path
self.config_name = config_name
self.reload()
def reload(self):
self.clear()
if self.config_file_path:
with open(self.config_file_path, 'r') as f:
self.values.update(yaml.safe_load(f)[self.config_name])
def clear(self):
self.values.clear()
def update(self, yml_dict):
for (k1, v1) in yml_dict.items():
if isinstance(v1, dict):
for (k2, v2) in v1.items():
if isinstance(v2, dict):
for (k3, v3) in v2.items():
self.values[k1][k2][k3] = v3
else:
self.values[k1][k2] = v2
else:
self.values[k1] = v1
def export(self, save_file_path):
if save_file_path:
with open(save_file_path, 'w') as f:
yaml.dump(dict(self.values), f)
In [5]:
cfg = YamlConfigManager()
SEED = cfg.values.seed
INPUT_DIR = cfg.values.input_dir
TRAIN_ONLY = cfg.values.train_only
IMAGE_SIZE = cfg.values.image_size
TRAIN_BATCH_SIZE = cfg.values.train_args.train_batch_size
VAL_BATCH_SIZE = cfg.values.train_args.val_batch_size
SEED 고정¶
In [6]:
# Fix random seed
def seed_everything(seed: int = 42):
random.seed(seed)
np.random.seed(seed)
os.environ["PYTHONHASHSEED"] = str(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed(seed) # type: ignore
torch.backends.cudnn.deterministic = True # type: ignore
torch.backends.cudnn.benchmark = True # type: ignore
In [7]:
seed_everything(SEED)
In [20]:
# # Caculate mean and std
# # pixel count
# count = len(train_df) * 227 * 227
# # mean and std
# total_mean = psum / count
# total_var = (psum_sq / count) - (total_mean ** 2)
# total_std = torch.sqrt(total_var)
# # output
# print('mean: ' + str(total_mean))
# print('std: ' + str(total_std))
mean: tensor([0.4569, 0.5074, 0.5557]) std: tensor([0.2888, 0.2743, 0.2829])
In [8]:
whole_df = pd.read_csv(INPUT_DIR)
whole_label = whole_df['label'].values
train_transform = albumentations.Compose([
albumentations.OneOf([
albumentations.HorizontalFlip(),
albumentations.ToGray(),
albumentations.Blur()
]),
albumentations.Resize(IMAGE_SIZE, IMAGE_SIZE),
albumentations.Normalize(mean=(0.4569, 0.5074, 0.5557), std=(0.2888, 0.2743, 0.2829)),
albumentations.pytorch.transforms.ToTensorV2()])
val_transform = albumentations.Compose([
albumentations.Resize(IMAGE_SIZE, IMAGE_SIZE),
albumentations.Normalize(mean=(0.4569, 0.5074, 0.5557), std=(0.2888, 0.2743, 0.2829)),
albumentations.pytorch.transforms.ToTensorV2()])
Model¶
- timm library를 사용했습니다.
- NFNet, EfficientNet
- kaggle ailen signal search 대회에 참가 중인데 nfnet과 efficientnet이 대체로 성능이 좋았습니다.
- 첫 제출 당시에 NFNet을 사용했는데 1.0이 나와서 놀랐습니다.
- 문제가 쉬운 대회인 만큼 shake up이 심할거라 생각해서 다른 모델들도 테스트했지만 val score가 좋지 않았습니다.
- LB(LeaderBoard) score
- efficientnet_b0 :
90.0 - nfnet_l0 :
100.0
- efficientnet_b0 :
- Swin-transformer
- naver boostcamp에서 competition 진행했었는데 detection 대회에서 swin-transformer를 사용해서 2등을 했습니다.
- 당시에 성능이 좋았습니다.
- 구조가 다른 모델끼리 앙상블할 경우에 generalized performance가 올라갈 것이라고 생각했습니다.
- LB score
- swin-base-224-22k :
97.14
- swin-base-224-22k :
In [9]:
class PretrainedModel(nn.Module):
def __init__(self, model_arc='resnet18d', num_classes=7):
super().__init__()
self.net = timm.create_model(model_arc, pretrained=True, num_classes=num_classes)
def forward(self, x):
x = self.net(x)
return x
Utils¶
- 여러가지 유틸 함수들을 정의했습니다.
In [10]:
def get_dataloader(df, transform, batch_size, shuffle):
dataset = ArtPaintDataset(df=df, transform=transform)
loader = DataLoader(
dataset,
batch_size=batch_size,
shuffle=shuffle
)
return loader
CosineAnnealingWarmupRestarts¶
- gamma 비율로 감소하는 cosine annealing warmup restart scheduler가 lr 분석하기 제일 편했습니다.
- 경험에 의해서 선택했으며, wandb와 함께 사용할 경우 optimizer 분석에 매우 용이했습니다.
In [11]:
class CosineAnnealingWarmupRestarts(_LRScheduler):
"""
optimizer (Optimizer): Wrapped optimizer.
first_cycle_steps (int): First cycle step size.
cycle_mult(float): Cycle steps magnification. Default: -1.
max_lr(float): First cycle's max learning rate. Default: 0.1.
min_lr(float): Min learning rate. Default: 0.001.
warmup_steps(int): Linear warmup step size. Default: 0.
gamma(float): Decrease rate of max learning rate by cycle. Default: 1.
last_epoch (int): The index of last epoch. Default: -1.
"""
def __init__(self,
optimizer : torch.optim.Optimizer,
first_cycle_steps : int,
cycle_mult : float = 1.,
max_lr : float = 0.1,
min_lr : float = 0.001,
warmup_steps : int = 0,
gamma : float = 1.,
last_epoch : int = -1
):
assert warmup_steps < first_cycle_steps
self.first_cycle_steps = first_cycle_steps # first cycle step size
self.cycle_mult = cycle_mult # cycle steps magnification
self.base_max_lr = max_lr # first max learning rate
self.max_lr = max_lr # max learning rate in the current cycle
self.min_lr = min_lr # min learning rate
self.warmup_steps = warmup_steps # warmup step size
self.gamma = gamma # decrease rate of max learning rate by cycle
self.cur_cycle_steps = first_cycle_steps # first cycle step size
self.cycle = 0 # cycle count
self.step_in_cycle = last_epoch # step size of the current cycle
super(CosineAnnealingWarmupRestarts, self).__init__(optimizer, last_epoch)
# set learning rate min_lr
self.init_lr()
def init_lr(self):
self.base_lrs = []
for param_group in self.optimizer.param_groups:
param_group['lr'] = self.min_lr
self.base_lrs.append(self.min_lr)
def get_lr(self):
if self.step_in_cycle == -1:
return self.base_lrs
elif self.step_in_cycle < self.warmup_steps:
return [(self.max_lr - base_lr)*self.step_in_cycle / self.warmup_steps + base_lr for base_lr in self.base_lrs]
else:
return [base_lr + (self.max_lr - base_lr) \
* (1 + math.cos(math.pi * (self.step_in_cycle-self.warmup_steps) \
/ (self.cur_cycle_steps - self.warmup_steps))) / 2
for base_lr in self.base_lrs]
def step(self, epoch=None):
if epoch is None:
epoch = self.last_epoch + 1
self.step_in_cycle = self.step_in_cycle + 1
if self.step_in_cycle >= self.cur_cycle_steps:
self.cycle += 1
self.step_in_cycle = self.step_in_cycle - self.cur_cycle_steps
self.cur_cycle_steps = int((self.cur_cycle_steps - self.warmup_steps) * self.cycle_mult) + self.warmup_steps
else:
if epoch >= self.first_cycle_steps:
if self.cycle_mult == 1.:
self.step_in_cycle = epoch % self.first_cycle_steps
self.cycle = epoch // self.first_cycle_steps
else:
n = int(math.log((epoch / self.first_cycle_steps * (self.cycle_mult - 1) + 1), self.cycle_mult))
self.cycle = n
self.step_in_cycle = epoch - int(self.first_cycle_steps * (self.cycle_mult ** n - 1) / (self.cycle_mult - 1))
self.cur_cycle_steps = self.first_cycle_steps * self.cycle_mult ** (n)
else:
self.cur_cycle_steps = self.first_cycle_steps
self.step_in_cycle = epoch
self.max_lr = self.base_max_lr * (self.gamma**self.cycle)
self.last_epoch = math.floor(epoch)
for param_group, lr in zip(self.optimizer.param_groups, self.get_lr()):
param_group['lr'] = lr
CutMix¶
- Robust한 모델을 만들기 위해 선택한 방법입니다.
- 쉬운 문제에서 사용할 경우 성능이 많이 좋아졌었던 경험이 있습니다.
- 학습을 더욱 복잡하게 만들어서 generalized performance의 향상을 노렸습니다.
- NFNet에서는 사용하지 않았고, Swin Transformer에만 사용해서 앙상블했습니다.
In [12]:
def rand_bbox(size, lam):
width = size[2]
height = size[3]
cut_ratio = np.sqrt(1. - lam)
cut_width = np.int(width * cut_ratio)
cut_height = np.int(height * cut_ratio)
# uniform
cx = np.random.randint(width)
cy = np.random.randint(height)
bbx1 = np.clip(cx - cut_width // 2, 0, width)
bby1 = np.clip(cy - cut_height // 2, 0, height)
bbx2 = np.clip(cx + cut_width // 2, 0, width)
bby2 = np.clip(cy + cut_height // 2, 0, height)
return bbx1, bby1, bbx2, bby2
In [13]:
class CutMix(object):
def __init__(self, beta, cutmix_prob) -> None:
super().__init__()
self.beta = beta
self.cutmix_prob = cutmix_prob
def forward(self, images, labels):
# generate mixed sample
lam = np.random.beta(self.beta, self.beta)
rand_index = torch.randperm(images.size()[0]).cuda()
label_1 = labels
label_2 = labels[rand_index]
bbx1, bby1, bbx2, bby2 = rand_bbox(images.size(), lam)
images[:, :, bbx1:bbx2, bby1:bby2] = images[rand_index, :, bbx1:bbx2, bby1:bby2]
# adjust lambda to exactly match pixel ratio
lam = 1 - ((bbx2 - bbx1) * (bby2 - bby1) / (images.size()[-1] * images.size()[-2]))
return {'lam' : lam, 'image' : images, 'label_1' : label_1, 'label_2' : label_2}
In [14]:
class ComputeMetric(object):
def __init__(self, metric) -> None:
super().__init__()
self.metric = metric
def cutmix_accuracy(self, logits, labels, topk=(1, 5)):
max_k = max(topk)
batch_size = labels.size(0)
_, pred = logits.topk(max_k, 1, True, True)
pred = pred.t()
matches = pred.eq(labels.view(1, -1).expand_as(pred))
res = []
for k in topk:
matches_k = matches[:k].reshape(-1).float().sum(0, keepdim=True)
wrong_k = batch_size - matches_k
res.append(matches_k.mul_(100.0 / batch_size))
return res
def compute(self, logits, labels, topk=(1, 5)):
if self.metric == 'accuracy':
out = self.cutmix_accuracy(logits=logits, labels=labels, topk=topk)
return out
지표 계산을 위한 average meter¶
In [15]:
class AverageMeter(object):
def __init__(self) -> None:
super().__init__()
self.reset()
def reset(self):
self.val = 0
self.avg = 0
self.sum = 0
self.count = 0
def update(self, val, n=1):
self.val = val
self.sum += val * n
self.count += n
self.avg = self.sum / self.count
Training¶
- 5 Fold cross validation 사용
- 모델 별로 lr, epoch 조절해가면서 다르게 학습했습니다.
- EDA(eda.ipynb 파일 만들어서 진행) 해봤을 때 guitar와 person같이 class imbalance가 있어 stratified하게 train과 validation 분할 했습니다.
In [33]:
# import matplotlib as pyplot
# import seaborn as sns
# ax = sns.countplot(train_df['label'])
# for p, label in zip(ax.patches, label_list):
# ax.annotate(label, (p.get_x(), p.get_height() + 0.15))
In [16]:
def train(cfg, k, train_loader, val_loader):
# Set Config
MODEL_ARC = cfg.values.model_arc
OUTPUT_DIR = cfg.values.output_dir
NUM_CLASSES = cfg.values.num_classes
TRAIN_ONLY = cfg.values.train_only
os.makedirs(OUTPUT_DIR, exist_ok=True)
# Set train arguments
num_epochs = cfg.values.train_args.num_epochs
train_batch_size = cfg.values.train_args.train_batch_size
log_intervals = cfg.values.train_args.log_intervals
max_lr = cfg.values.train_args.max_lr
min_lr = cfg.values.train_args.min_lr
cycle = cfg.values.train_args.cycle
gamma = cfg.values.train_args.gamma
# Set CutMix arguments
USE_CUTMIX = cfg.values.cutmix_args.use_cutmix
beta = cfg.values.cutmix_args.beta
cutmix_prob = cfg.values.cutmix_args.cutmix_prob
cutmix = CutMix(beta=beta, cutmix_prob=cutmix_prob)
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
model = PretrainedModel(model_arc=MODEL_ARC, num_classes=NUM_CLASSES)
model.to(device)
if k < 2:
summary_(model, (3, IMAGE_SIZE, IMAGE_SIZE), batch_size=train_batch_size)
optimizer = AdamP(model.parameters(), lr=max_lr, weight_decay=cfg.values.train_args.weight_decay)
first_cycle_steps = len(train_loader) * num_epochs // cycle
scheduler = CosineAnnealingWarmupRestarts(
optimizer,
first_cycle_steps=first_cycle_steps,
cycle_mult=1.0,
max_lr=max_lr,
min_lr=min_lr,
warmup_steps=int(first_cycle_steps * 0.2),
gamma=gamma
)
criterion = nn.CrossEntropyLoss()
eval_metric = ComputeMetric(cfg.values.train_args.eval_metric)
best_acc = 0.
os.makedirs(os.path.join(OUTPUT_DIR, MODEL_ARC), exist_ok=True)
for epoch in range(num_epochs):
model.train()
loss_values = AverageMeter()
top1 = AverageMeter()
top5 = AverageMeter()
for i, train_batch in enumerate(tqdm(train_loader, desc=f'Training')):
sample = train_batch
images = sample['image'].float().to(device)
labels = sample['label'].long().to(device)
ratio = np.random.rand(1)
if USE_CUTMIX:
if beta > 0 and ratio < cutmix_prob:
# generate mixed sample
sample = cutmix.forward(images, labels)
logits = model(sample['image'])
loss = criterion(logits, sample['label_1']) * sample['lam'] + criterion(logits, sample['label_2']) * (1. - sample['lam'])
else:
logits = model(images)
loss = criterion(logits, labels)
else:
logits = model(images)
loss = criterion(logits, labels)
# measure evaluation metric and record loss
top1_err, top5_err = eval_metric.compute(logits.data, labels, topk=(1, 5))
loss_values.update(loss.item(), images.size(0))
top1.update(top1_err.item(), images.size(0))
top5.update(top5_err.item(), images.size(0))
# compute gradient and do optimizer step
optimizer.zero_grad()
loss.backward()
optimizer.step()
scheduler.step()
if i % log_intervals == 0:
current_lr = scheduler.get_lr()[0]
tqdm.write(f'Epoch : [{epoch + 1}/{num_epochs}][{i}/{len(train_loader)}] || '
f'LR : {current_lr:.5f} || '
f'Train Loss : {loss_values.val:.4f} ({loss_values.avg:.4f}) || '
f'Train Top 1-acc : {top1.val:.3f}% ({top1.avg:.3f})% || '
f'Train Top 5-acc : {top5.val:.3f}% ({top5.avg:.3f})%')
loss_values = AverageMeter()
top1 = AverageMeter()
top5 = AverageMeter()
if not TRAIN_ONLY:
with torch.no_grad():
model.eval()
loss_values = AverageMeter()
top1 = AverageMeter()
top5 = AverageMeter()
for i, val_batch in enumerate(tqdm(val_loader, desc=f'Validation')):
sample = val_batch
images = sample['image'].float().to(device)
labels = sample['label'].long().to(device)
logits = model(images)
loss = criterion(logits, labels)
preds = torch.argmax(logits, -1)
top1_err, top5_err = eval_metric.compute(logits.data, labels, topk=(1, 5))
loss_values.update(loss.item(), images.size(0))
top1.update(top1_err.item(), images.size(0))
top5.update(top5_err.item(), images.size(0))
tqdm.write(f'Epoch : [{epoch + 1}/{num_epochs}] || '
f'Val Loss : {loss_values.avg:.4f} || '
f'Val Top 1-acc : {top1.avg:.3f}% || '
f'Val Top 5-acc : {top5.avg:.3f}%')
is_best = top1.avg >= best_acc
best_acc = max(top1.avg, best_acc)
if is_best:
if k > 0:
os.makedirs(os.path.join(OUTPUT_DIR, MODEL_ARC, f'{k}_fold'), exist_ok=True)
torch.save(model.state_dict(), os.path.join(OUTPUT_DIR, MODEL_ARC, f'{k}_fold', f'{epoch + 1}_epoch_{best_acc:.2f}%_with_val.pth'))
else:
torch.save(model.state_dict(), os.path.join(OUTPUT_DIR, MODEL_ARC, f'{epoch + 1}_epoch_{best_acc:.2f}%_with_val.pth'))
else:
torch.save(model.state_dict(), os.path.join(OUTPUT_DIR, MODEL_ARC, f'_{epoch + 1}_epoch_{top1.avg:.2f}%_only_train.pth'))
In [17]:
kfold = StratifiedKFold(n_splits=cfg.values.train_args.n_splits)
k = 1
for train_idx, val_idx in kfold.split(whole_df, whole_label):
print('\n')
cpprint('=' * 15 + f'{k}-Fold Cross Validation' + '=' * 15)
train_df = whole_df.iloc[train_idx]
val_df = whole_df.iloc[val_idx]
train_loader = get_dataloader(df=train_df, transform=train_transform, batch_size=TRAIN_BATCH_SIZE, shuffle=True)
val_loader = get_dataloader(df=val_df, transform=val_transform, batch_size=VAL_BATCH_SIZE, shuffle=False)
train(cfg, k, train_loader, val_loader)
k += 1
'===============1-Fold Cross Validation==============='
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Conv2d-1 [16, 128, 56, 56] 6,272
LayerNorm-2 [16, 3136, 128] 256
PatchEmbed-3 [16, 3136, 128] 0
Dropout-4 [16, 3136, 128] 0
LayerNorm-5 [16, 3136, 128] 256
Linear-6 [16, 49, 384] 49,536
Softmax-7 [16, 4, 49, 49] 0
Dropout-8 [16, 4, 49, 49] 0
Linear-9 [16, 49, 128] 16,512
Dropout-10 [16, 49, 128] 0
WindowAttention-11 [16, 49, 128] 0
Identity-12 [16, 3136, 128] 0
LayerNorm-13 [16, 3136, 128] 256
Linear-14 [16, 3136, 512] 66,048
GELU-15 [16, 3136, 512] 0
Dropout-16 [16, 3136, 512] 0
Linear-17 [16, 3136, 128] 65,664
Dropout-18 [16, 3136, 128] 0
Mlp-19 [16, 3136, 128] 0
Identity-20 [16, 3136, 128] 0
SwinTransformerBlock-21 [16, 3136, 128] 0
LayerNorm-22 [16, 3136, 128] 256
Linear-23 [16, 49, 384] 49,536
Softmax-24 [16, 4, 49, 49] 0
Dropout-25 [16, 4, 49, 49] 0
Linear-26 [16, 49, 128] 16,512
Dropout-27 [16, 49, 128] 0
WindowAttention-28 [16, 49, 128] 0
DropPath-29 [16, 3136, 128] 0
LayerNorm-30 [16, 3136, 128] 256
Linear-31 [16, 3136, 512] 66,048
GELU-32 [16, 3136, 512] 0
Dropout-33 [16, 3136, 512] 0
Linear-34 [16, 3136, 128] 65,664
Dropout-35 [16, 3136, 128] 0
Mlp-36 [16, 3136, 128] 0
DropPath-37 [16, 3136, 128] 0
SwinTransformerBlock-38 [16, 3136, 128] 0
LayerNorm-39 [16, 784, 512] 1,024
Linear-40 [16, 784, 256] 131,072
PatchMerging-41 [16, 784, 256] 0
BasicLayer-42 [16, 784, 256] 0
LayerNorm-43 [16, 784, 256] 512
Linear-44 [16, 49, 768] 197,376
Softmax-45 [16, 8, 49, 49] 0
Dropout-46 [16, 8, 49, 49] 0
Linear-47 [16, 49, 256] 65,792
Dropout-48 [16, 49, 256] 0
WindowAttention-49 [16, 49, 256] 0
DropPath-50 [16, 784, 256] 0
LayerNorm-51 [16, 784, 256] 512
Linear-52 [16, 784, 1024] 263,168
GELU-53 [16, 784, 1024] 0
Dropout-54 [16, 784, 1024] 0
Linear-55 [16, 784, 256] 262,400
Dropout-56 [16, 784, 256] 0
Mlp-57 [16, 784, 256] 0
DropPath-58 [16, 784, 256] 0
SwinTransformerBlock-59 [16, 784, 256] 0
LayerNorm-60 [16, 784, 256] 512
Linear-61 [16, 49, 768] 197,376
Softmax-62 [16, 8, 49, 49] 0
Dropout-63 [16, 8, 49, 49] 0
Linear-64 [16, 49, 256] 65,792
Dropout-65 [16, 49, 256] 0
WindowAttention-66 [16, 49, 256] 0
DropPath-67 [16, 784, 256] 0
LayerNorm-68 [16, 784, 256] 512
Linear-69 [16, 784, 1024] 263,168
GELU-70 [16, 784, 1024] 0
Dropout-71 [16, 784, 1024] 0
Linear-72 [16, 784, 256] 262,400
Dropout-73 [16, 784, 256] 0
Mlp-74 [16, 784, 256] 0
DropPath-75 [16, 784, 256] 0
SwinTransformerBlock-76 [16, 784, 256] 0
LayerNorm-77 [16, 196, 1024] 2,048
Linear-78 [16, 196, 512] 524,288
PatchMerging-79 [16, 196, 512] 0
BasicLayer-80 [16, 196, 512] 0
LayerNorm-81 [16, 196, 512] 1,024
Linear-82 [16, 49, 1536] 787,968
Softmax-83 [16, 16, 49, 49] 0
Dropout-84 [16, 16, 49, 49] 0
Linear-85 [16, 49, 512] 262,656
Dropout-86 [16, 49, 512] 0
WindowAttention-87 [16, 49, 512] 0
DropPath-88 [16, 196, 512] 0
LayerNorm-89 [16, 196, 512] 1,024
Linear-90 [16, 196, 2048] 1,050,624
GELU-91 [16, 196, 2048] 0
Dropout-92 [16, 196, 2048] 0
Linear-93 [16, 196, 512] 1,049,088
Dropout-94 [16, 196, 512] 0
Mlp-95 [16, 196, 512] 0
DropPath-96 [16, 196, 512] 0
SwinTransformerBlock-97 [16, 196, 512] 0
LayerNorm-98 [16, 196, 512] 1,024
Linear-99 [16, 49, 1536] 787,968
Softmax-100 [16, 16, 49, 49] 0
Dropout-101 [16, 16, 49, 49] 0
Linear-102 [16, 49, 512] 262,656
Dropout-103 [16, 49, 512] 0
WindowAttention-104 [16, 49, 512] 0
DropPath-105 [16, 196, 512] 0
LayerNorm-106 [16, 196, 512] 1,024
Linear-107 [16, 196, 2048] 1,050,624
GELU-108 [16, 196, 2048] 0
Dropout-109 [16, 196, 2048] 0
Linear-110 [16, 196, 512] 1,049,088
Dropout-111 [16, 196, 512] 0
Mlp-112 [16, 196, 512] 0
DropPath-113 [16, 196, 512] 0
SwinTransformerBlock-114 [16, 196, 512] 0
LayerNorm-115 [16, 196, 512] 1,024
Linear-116 [16, 49, 1536] 787,968
Softmax-117 [16, 16, 49, 49] 0
Dropout-118 [16, 16, 49, 49] 0
Linear-119 [16, 49, 512] 262,656
Dropout-120 [16, 49, 512] 0
WindowAttention-121 [16, 49, 512] 0
DropPath-122 [16, 196, 512] 0
LayerNorm-123 [16, 196, 512] 1,024
Linear-124 [16, 196, 2048] 1,050,624
GELU-125 [16, 196, 2048] 0
Dropout-126 [16, 196, 2048] 0
Linear-127 [16, 196, 512] 1,049,088
Dropout-128 [16, 196, 512] 0
Mlp-129 [16, 196, 512] 0
DropPath-130 [16, 196, 512] 0
SwinTransformerBlock-131 [16, 196, 512] 0
LayerNorm-132 [16, 196, 512] 1,024
Linear-133 [16, 49, 1536] 787,968
Softmax-134 [16, 16, 49, 49] 0
Dropout-135 [16, 16, 49, 49] 0
Linear-136 [16, 49, 512] 262,656
Dropout-137 [16, 49, 512] 0
WindowAttention-138 [16, 49, 512] 0
DropPath-139 [16, 196, 512] 0
LayerNorm-140 [16, 196, 512] 1,024
Linear-141 [16, 196, 2048] 1,050,624
GELU-142 [16, 196, 2048] 0
Dropout-143 [16, 196, 2048] 0
Linear-144 [16, 196, 512] 1,049,088
Dropout-145 [16, 196, 512] 0
Mlp-146 [16, 196, 512] 0
DropPath-147 [16, 196, 512] 0
SwinTransformerBlock-148 [16, 196, 512] 0
LayerNorm-149 [16, 196, 512] 1,024
Linear-150 [16, 49, 1536] 787,968
Softmax-151 [16, 16, 49, 49] 0
Dropout-152 [16, 16, 49, 49] 0
Linear-153 [16, 49, 512] 262,656
Dropout-154 [16, 49, 512] 0
WindowAttention-155 [16, 49, 512] 0
DropPath-156 [16, 196, 512] 0
LayerNorm-157 [16, 196, 512] 1,024
Linear-158 [16, 196, 2048] 1,050,624
GELU-159 [16, 196, 2048] 0
Dropout-160 [16, 196, 2048] 0
Linear-161 [16, 196, 512] 1,049,088
Dropout-162 [16, 196, 512] 0
Mlp-163 [16, 196, 512] 0
DropPath-164 [16, 196, 512] 0
SwinTransformerBlock-165 [16, 196, 512] 0
LayerNorm-166 [16, 196, 512] 1,024
Linear-167 [16, 49, 1536] 787,968
Softmax-168 [16, 16, 49, 49] 0
Dropout-169 [16, 16, 49, 49] 0
Linear-170 [16, 49, 512] 262,656
Dropout-171 [16, 49, 512] 0
WindowAttention-172 [16, 49, 512] 0
DropPath-173 [16, 196, 512] 0
LayerNorm-174 [16, 196, 512] 1,024
Linear-175 [16, 196, 2048] 1,050,624
GELU-176 [16, 196, 2048] 0
Dropout-177 [16, 196, 2048] 0
Linear-178 [16, 196, 512] 1,049,088
Dropout-179 [16, 196, 512] 0
Mlp-180 [16, 196, 512] 0
DropPath-181 [16, 196, 512] 0
SwinTransformerBlock-182 [16, 196, 512] 0
LayerNorm-183 [16, 196, 512] 1,024
Linear-184 [16, 49, 1536] 787,968
Softmax-185 [16, 16, 49, 49] 0
Dropout-186 [16, 16, 49, 49] 0
Linear-187 [16, 49, 512] 262,656
Dropout-188 [16, 49, 512] 0
WindowAttention-189 [16, 49, 512] 0
DropPath-190 [16, 196, 512] 0
LayerNorm-191 [16, 196, 512] 1,024
Linear-192 [16, 196, 2048] 1,050,624
GELU-193 [16, 196, 2048] 0
Dropout-194 [16, 196, 2048] 0
Linear-195 [16, 196, 512] 1,049,088
Dropout-196 [16, 196, 512] 0
Mlp-197 [16, 196, 512] 0
DropPath-198 [16, 196, 512] 0
SwinTransformerBlock-199 [16, 196, 512] 0
LayerNorm-200 [16, 196, 512] 1,024
Linear-201 [16, 49, 1536] 787,968
Softmax-202 [16, 16, 49, 49] 0
Dropout-203 [16, 16, 49, 49] 0
Linear-204 [16, 49, 512] 262,656
Dropout-205 [16, 49, 512] 0
WindowAttention-206 [16, 49, 512] 0
DropPath-207 [16, 196, 512] 0
LayerNorm-208 [16, 196, 512] 1,024
Linear-209 [16, 196, 2048] 1,050,624
GELU-210 [16, 196, 2048] 0
Dropout-211 [16, 196, 2048] 0
Linear-212 [16, 196, 512] 1,049,088
Dropout-213 [16, 196, 512] 0
Mlp-214 [16, 196, 512] 0
DropPath-215 [16, 196, 512] 0
SwinTransformerBlock-216 [16, 196, 512] 0
LayerNorm-217 [16, 196, 512] 1,024
Linear-218 [16, 49, 1536] 787,968
Softmax-219 [16, 16, 49, 49] 0
Dropout-220 [16, 16, 49, 49] 0
Linear-221 [16, 49, 512] 262,656
Dropout-222 [16, 49, 512] 0
WindowAttention-223 [16, 49, 512] 0
DropPath-224 [16, 196, 512] 0
LayerNorm-225 [16, 196, 512] 1,024
Linear-226 [16, 196, 2048] 1,050,624
GELU-227 [16, 196, 2048] 0
Dropout-228 [16, 196, 2048] 0
Linear-229 [16, 196, 512] 1,049,088
Dropout-230 [16, 196, 512] 0
Mlp-231 [16, 196, 512] 0
DropPath-232 [16, 196, 512] 0
SwinTransformerBlock-233 [16, 196, 512] 0
LayerNorm-234 [16, 196, 512] 1,024
Linear-235 [16, 49, 1536] 787,968
Softmax-236 [16, 16, 49, 49] 0
Dropout-237 [16, 16, 49, 49] 0
Linear-238 [16, 49, 512] 262,656
Dropout-239 [16, 49, 512] 0
WindowAttention-240 [16, 49, 512] 0
DropPath-241 [16, 196, 512] 0
LayerNorm-242 [16, 196, 512] 1,024
Linear-243 [16, 196, 2048] 1,050,624
GELU-244 [16, 196, 2048] 0
Dropout-245 [16, 196, 2048] 0
Linear-246 [16, 196, 512] 1,049,088
Dropout-247 [16, 196, 512] 0
Mlp-248 [16, 196, 512] 0
DropPath-249 [16, 196, 512] 0
SwinTransformerBlock-250 [16, 196, 512] 0
LayerNorm-251 [16, 196, 512] 1,024
Linear-252 [16, 49, 1536] 787,968
Softmax-253 [16, 16, 49, 49] 0
Dropout-254 [16, 16, 49, 49] 0
Linear-255 [16, 49, 512] 262,656
Dropout-256 [16, 49, 512] 0
WindowAttention-257 [16, 49, 512] 0
DropPath-258 [16, 196, 512] 0
LayerNorm-259 [16, 196, 512] 1,024
Linear-260 [16, 196, 2048] 1,050,624
GELU-261 [16, 196, 2048] 0
Dropout-262 [16, 196, 2048] 0
Linear-263 [16, 196, 512] 1,049,088
Dropout-264 [16, 196, 512] 0
Mlp-265 [16, 196, 512] 0
DropPath-266 [16, 196, 512] 0
SwinTransformerBlock-267 [16, 196, 512] 0
LayerNorm-268 [16, 196, 512] 1,024
Linear-269 [16, 49, 1536] 787,968
Softmax-270 [16, 16, 49, 49] 0
Dropout-271 [16, 16, 49, 49] 0
Linear-272 [16, 49, 512] 262,656
Dropout-273 [16, 49, 512] 0
WindowAttention-274 [16, 49, 512] 0
DropPath-275 [16, 196, 512] 0
LayerNorm-276 [16, 196, 512] 1,024
Linear-277 [16, 196, 2048] 1,050,624
GELU-278 [16, 196, 2048] 0
Dropout-279 [16, 196, 2048] 0
Linear-280 [16, 196, 512] 1,049,088
Dropout-281 [16, 196, 512] 0
Mlp-282 [16, 196, 512] 0
DropPath-283 [16, 196, 512] 0
SwinTransformerBlock-284 [16, 196, 512] 0
LayerNorm-285 [16, 196, 512] 1,024
Linear-286 [16, 49, 1536] 787,968
Softmax-287 [16, 16, 49, 49] 0
Dropout-288 [16, 16, 49, 49] 0
Linear-289 [16, 49, 512] 262,656
Dropout-290 [16, 49, 512] 0
WindowAttention-291 [16, 49, 512] 0
DropPath-292 [16, 196, 512] 0
LayerNorm-293 [16, 196, 512] 1,024
Linear-294 [16, 196, 2048] 1,050,624
GELU-295 [16, 196, 2048] 0
Dropout-296 [16, 196, 2048] 0
Linear-297 [16, 196, 512] 1,049,088
Dropout-298 [16, 196, 512] 0
Mlp-299 [16, 196, 512] 0
DropPath-300 [16, 196, 512] 0
SwinTransformerBlock-301 [16, 196, 512] 0
LayerNorm-302 [16, 196, 512] 1,024
Linear-303 [16, 49, 1536] 787,968
Softmax-304 [16, 16, 49, 49] 0
Dropout-305 [16, 16, 49, 49] 0
Linear-306 [16, 49, 512] 262,656
Dropout-307 [16, 49, 512] 0
WindowAttention-308 [16, 49, 512] 0
DropPath-309 [16, 196, 512] 0
LayerNorm-310 [16, 196, 512] 1,024
Linear-311 [16, 196, 2048] 1,050,624
GELU-312 [16, 196, 2048] 0
Dropout-313 [16, 196, 2048] 0
Linear-314 [16, 196, 512] 1,049,088
Dropout-315 [16, 196, 512] 0
Mlp-316 [16, 196, 512] 0
DropPath-317 [16, 196, 512] 0
SwinTransformerBlock-318 [16, 196, 512] 0
LayerNorm-319 [16, 196, 512] 1,024
Linear-320 [16, 49, 1536] 787,968
Softmax-321 [16, 16, 49, 49] 0
Dropout-322 [16, 16, 49, 49] 0
Linear-323 [16, 49, 512] 262,656
Dropout-324 [16, 49, 512] 0
WindowAttention-325 [16, 49, 512] 0
DropPath-326 [16, 196, 512] 0
LayerNorm-327 [16, 196, 512] 1,024
Linear-328 [16, 196, 2048] 1,050,624
GELU-329 [16, 196, 2048] 0
Dropout-330 [16, 196, 2048] 0
Linear-331 [16, 196, 512] 1,049,088
Dropout-332 [16, 196, 512] 0
Mlp-333 [16, 196, 512] 0
DropPath-334 [16, 196, 512] 0
SwinTransformerBlock-335 [16, 196, 512] 0
LayerNorm-336 [16, 196, 512] 1,024
Linear-337 [16, 49, 1536] 787,968
Softmax-338 [16, 16, 49, 49] 0
Dropout-339 [16, 16, 49, 49] 0
Linear-340 [16, 49, 512] 262,656
Dropout-341 [16, 49, 512] 0
WindowAttention-342 [16, 49, 512] 0
DropPath-343 [16, 196, 512] 0
LayerNorm-344 [16, 196, 512] 1,024
Linear-345 [16, 196, 2048] 1,050,624
GELU-346 [16, 196, 2048] 0
Dropout-347 [16, 196, 2048] 0
Linear-348 [16, 196, 512] 1,049,088
Dropout-349 [16, 196, 512] 0
Mlp-350 [16, 196, 512] 0
DropPath-351 [16, 196, 512] 0
SwinTransformerBlock-352 [16, 196, 512] 0
LayerNorm-353 [16, 196, 512] 1,024
Linear-354 [16, 49, 1536] 787,968
Softmax-355 [16, 16, 49, 49] 0
Dropout-356 [16, 16, 49, 49] 0
Linear-357 [16, 49, 512] 262,656
Dropout-358 [16, 49, 512] 0
WindowAttention-359 [16, 49, 512] 0
DropPath-360 [16, 196, 512] 0
LayerNorm-361 [16, 196, 512] 1,024
Linear-362 [16, 196, 2048] 1,050,624
GELU-363 [16, 196, 2048] 0
Dropout-364 [16, 196, 2048] 0
Linear-365 [16, 196, 512] 1,049,088
Dropout-366 [16, 196, 512] 0
Mlp-367 [16, 196, 512] 0
DropPath-368 [16, 196, 512] 0
SwinTransformerBlock-369 [16, 196, 512] 0
LayerNorm-370 [16, 196, 512] 1,024
Linear-371 [16, 49, 1536] 787,968
Softmax-372 [16, 16, 49, 49] 0
Dropout-373 [16, 16, 49, 49] 0
Linear-374 [16, 49, 512] 262,656
Dropout-375 [16, 49, 512] 0
WindowAttention-376 [16, 49, 512] 0
DropPath-377 [16, 196, 512] 0
LayerNorm-378 [16, 196, 512] 1,024
Linear-379 [16, 196, 2048] 1,050,624
GELU-380 [16, 196, 2048] 0
Dropout-381 [16, 196, 2048] 0
Linear-382 [16, 196, 512] 1,049,088
Dropout-383 [16, 196, 512] 0
Mlp-384 [16, 196, 512] 0
DropPath-385 [16, 196, 512] 0
SwinTransformerBlock-386 [16, 196, 512] 0
LayerNorm-387 [16, 49, 2048] 4,096
Linear-388 [16, 49, 1024] 2,097,152
PatchMerging-389 [16, 49, 1024] 0
BasicLayer-390 [16, 49, 1024] 0
LayerNorm-391 [16, 49, 1024] 2,048
Linear-392 [16, 49, 3072] 3,148,800
Softmax-393 [16, 32, 49, 49] 0
Dropout-394 [16, 32, 49, 49] 0
Linear-395 [16, 49, 1024] 1,049,600
Dropout-396 [16, 49, 1024] 0
WindowAttention-397 [16, 49, 1024] 0
DropPath-398 [16, 49, 1024] 0
LayerNorm-399 [16, 49, 1024] 2,048
Linear-400 [16, 49, 4096] 4,198,400
GELU-401 [16, 49, 4096] 0
Dropout-402 [16, 49, 4096] 0
Linear-403 [16, 49, 1024] 4,195,328
Dropout-404 [16, 49, 1024] 0
Mlp-405 [16, 49, 1024] 0
DropPath-406 [16, 49, 1024] 0
SwinTransformerBlock-407 [16, 49, 1024] 0
LayerNorm-408 [16, 49, 1024] 2,048
Linear-409 [16, 49, 3072] 3,148,800
Softmax-410 [16, 32, 49, 49] 0
Dropout-411 [16, 32, 49, 49] 0
Linear-412 [16, 49, 1024] 1,049,600
Dropout-413 [16, 49, 1024] 0
WindowAttention-414 [16, 49, 1024] 0
DropPath-415 [16, 49, 1024] 0
LayerNorm-416 [16, 49, 1024] 2,048
Linear-417 [16, 49, 4096] 4,198,400
GELU-418 [16, 49, 4096] 0
Dropout-419 [16, 49, 4096] 0
Linear-420 [16, 49, 1024] 4,195,328
Dropout-421 [16, 49, 1024] 0
Mlp-422 [16, 49, 1024] 0
DropPath-423 [16, 49, 1024] 0
SwinTransformerBlock-424 [16, 49, 1024] 0
BasicLayer-425 [16, 49, 1024] 0
LayerNorm-426 [16, 49, 1024] 2,048
AdaptiveAvgPool1d-427 [16, 1024, 1] 0
Linear-428 [16, 7] 7,175
SwinTransformer-429 [16, 7] 0
================================================================
Total params: 86,686,855
Trainable params: 86,686,855
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 9.19
Forward/backward pass size (MB): 8669.97
Params size (MB): 330.68
Estimated Total Size (MB): 9009.84
----------------------------------------------------------------
Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.1090 (2.1090) || Train Top 1-acc : 12.500% (12.500)% || Train Top 5-acc : 62.500% (62.500)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.8202 (1.9161) || Train Top 1-acc : 25.000% (19.318)% || Train Top 5-acc : 87.500% (80.682)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.1825 (1.7320) || Train Top 1-acc : 68.750% (34.226)% || Train Top 5-acc : 93.750% (86.012)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.3470 (1.5649) || Train Top 1-acc : 62.500% (44.556)% || Train Top 5-acc : 100.000% (89.113)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.2557 (1.3340) || Train Top 1-acc : 93.750% (53.659)% || Train Top 5-acc : 100.000% (91.159)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.1729 (1.1790) || Train Top 1-acc : 93.750% (59.191)% || Train Top 5-acc : 100.000% (92.525)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 1.3103 (1.1023) || Train Top 1-acc : 75.000% (61.988)% || Train Top 5-acc : 100.000% (93.443)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 0.4908 (1.0439) || Train Top 1-acc : 87.500% (64.261)% || Train Top 5-acc : 100.000% (94.014)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.8526 (0.9797) || Train Top 1-acc : 18.750% (66.975)% || Train Top 5-acc : 100.000% (94.753)%
Epoch : [1/10] || Val Loss : 0.1027 || Val Top 1-acc : 97.059% || Val Top 5-acc : 100.000%
Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 1.0165 (1.0165) || Train Top 1-acc : 81.250% (81.250)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.1882 (0.4326) || Train Top 1-acc : 93.750% (92.614)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.0179 (0.4095) || Train Top 1-acc : 100.000% (88.393)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.1978 (0.4609) || Train Top 1-acc : 93.750% (87.702)% || Train Top 5-acc : 100.000% (99.194)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.3267 (0.4471) || Train Top 1-acc : 87.500% (88.110)% || Train Top 5-acc : 100.000% (99.238)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.8365 (0.4656) || Train Top 1-acc : 68.750% (88.235)% || Train Top 5-acc : 100.000% (99.387)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.0348 (0.4488) || Train Top 1-acc : 100.000% (88.525)% || Train Top 5-acc : 100.000% (99.385)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.2624 (0.4268) || Train Top 1-acc : 93.750% (88.732)% || Train Top 5-acc : 100.000% (99.472)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.0438 (0.4027) || Train Top 1-acc : 100.000% (88.657)% || Train Top 5-acc : 100.000% (99.537)%
Epoch : [2/10] || Val Loss : 0.0685 || Val Top 1-acc : 97.941% || Val Top 5-acc : 100.000%
Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.6699 (0.6699) || Train Top 1-acc : 37.500% (37.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.0201 (0.1761) || Train Top 1-acc : 100.000% (92.614)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0125 (0.2776) || Train Top 1-acc : 100.000% (94.345)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.6409 (0.3282) || Train Top 1-acc : 100.000% (92.137)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.0147 (0.3249) || Train Top 1-acc : 100.000% (92.835)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0040 (0.2935) || Train Top 1-acc : 100.000% (93.873)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0191 (0.2901) || Train Top 1-acc : 100.000% (94.570)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0112 (0.2966) || Train Top 1-acc : 100.000% (93.662)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.0798 (0.2730) || Train Top 1-acc : 100.000% (94.444)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [3/10] || Val Loss : 0.0472 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%
Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.0122 (0.0122) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.8667 (0.3944) || Train Top 1-acc : 87.500% (86.932)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.0085 (0.3728) || Train Top 1-acc : 100.000% (90.774)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.0063 (0.3491) || Train Top 1-acc : 100.000% (90.323)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.0090 (0.3131) || Train Top 1-acc : 100.000% (89.482)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.2690 (0.2854) || Train Top 1-acc : 100.000% (90.564)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.6340 (0.2916) || Train Top 1-acc : 93.750% (90.266)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.9933 (0.3147) || Train Top 1-acc : 12.500% (89.525)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0077 (0.2940) || Train Top 1-acc : 100.000% (89.969)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [4/10] || Val Loss : 0.0554 || Val Top 1-acc : 98.529% || Val Top 5-acc : 100.000%
Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.7766 (0.7766) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.6723 (0.4551) || Train Top 1-acc : 100.000% (89.205)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.4227 (0.4078) || Train Top 1-acc : 100.000% (87.798)% || Train Top 5-acc : 100.000% (99.702)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.6683 (0.4091) || Train Top 1-acc : 87.500% (87.298)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.0084 (0.4049) || Train Top 1-acc : 100.000% (87.957)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0036 (0.3768) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.0031 (0.3554) || Train Top 1-acc : 100.000% (88.012)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.5924 (0.3393) || Train Top 1-acc : 93.750% (89.261)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.0027 (0.3399) || Train Top 1-acc : 100.000% (89.275)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [5/10] || Val Loss : 0.0485 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%
Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.0060 (0.0060) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.6237 (0.1738) || Train Top 1-acc : 93.750% (95.455)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.8690 (0.2624) || Train Top 1-acc : 81.250% (94.345)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.7609 (0.3277) || Train Top 1-acc : 87.500% (92.944)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.3789 (0.3237) || Train Top 1-acc : 100.000% (92.530)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.0037 (0.3117) || Train Top 1-acc : 100.000% (92.402)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.4470 (0.3068) || Train Top 1-acc : 100.000% (93.648)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.7470 (0.3078) || Train Top 1-acc : 100.000% (93.398)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.0084 (0.3012) || Train Top 1-acc : 100.000% (93.673)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [6/10] || Val Loss : 0.0286 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%
Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.0056 (0.0056) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.2165 (0.2785) || Train Top 1-acc : 100.000% (92.045)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.4463 (0.3574) || Train Top 1-acc : 100.000% (87.202)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.5680 (0.4040) || Train Top 1-acc : 87.500% (86.089)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.6585 (0.4326) || Train Top 1-acc : 50.000% (81.402)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.0033 (0.4182) || Train Top 1-acc : 100.000% (82.353)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.4754 (0.4248) || Train Top 1-acc : 100.000% (82.480)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.0022 (0.3955) || Train Top 1-acc : 100.000% (83.275)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.1527 (0.3748) || Train Top 1-acc : 100.000% (84.799)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10] || Val Loss : 0.0241 || Val Top 1-acc : 98.529% || Val Top 5-acc : 100.000%
Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.7053 (0.7053) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.0123 (0.4296) || Train Top 1-acc : 100.000% (88.068)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.5860 (0.3488) || Train Top 1-acc : 56.250% (89.583)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0035 (0.2884) || Train Top 1-acc : 100.000% (92.742)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.0017 (0.2458) || Train Top 1-acc : 100.000% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.6352 (0.2529) || Train Top 1-acc : 25.000% (92.157)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.4637 (0.2591) || Train Top 1-acc : 93.750% (91.291)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.0913 (0.2458) || Train Top 1-acc : 100.000% (92.430)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.0029 (0.2675) || Train Top 1-acc : 100.000% (91.898)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10] || Val Loss : 0.0548 || Val Top 1-acc : 98.529% || Val Top 5-acc : 100.000%
Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.0085 (0.0085) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.0032 (0.2898) || Train Top 1-acc : 100.000% (93.182)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.0025 (0.2206) || Train Top 1-acc : 100.000% (96.131)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.0021 (0.2624) || Train Top 1-acc : 100.000% (92.540)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0024 (0.2308) || Train Top 1-acc : 100.000% (93.902)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0037 (0.2124) || Train Top 1-acc : 100.000% (94.240)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.6163 (0.2243) || Train Top 1-acc : 81.250% (94.570)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.5209 (0.2377) || Train Top 1-acc : 100.000% (94.190)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.0028 (0.2328) || Train Top 1-acc : 100.000% (93.981)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10] || Val Loss : 0.0499 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%
Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.0012 (0.0012) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.4145 (0.2699) || Train Top 1-acc : 100.000% (97.727)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.0012 (0.2629) || Train Top 1-acc : 100.000% (97.619)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.5747 (0.2955) || Train Top 1-acc : 100.000% (91.532)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.0017 (0.2580) || Train Top 1-acc : 100.000% (91.311)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.7088 (0.2563) || Train Top 1-acc : 43.750% (89.583)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.6859 (0.2471) || Train Top 1-acc : 62.500% (90.369)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.6162 (0.2428) || Train Top 1-acc : 87.500% (90.933)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.0065 (0.2444) || Train Top 1-acc : 100.000% (91.127)% || Train Top 5-acc : 100.000% (99.846)%
Epoch : [10/10] || Val Loss : 0.0379 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%
'===============2-Fold Cross Validation==============='
Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 1.9786 (1.9786) || Train Top 1-acc : 18.750% (18.750)% || Train Top 5-acc : 75.000% (75.000)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.9056 (1.9197) || Train Top 1-acc : 12.500% (18.750)% || Train Top 5-acc : 68.750% (81.818)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.3824 (1.7464) || Train Top 1-acc : 50.000% (31.845)% || Train Top 5-acc : 93.750% (87.500)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 0.7627 (1.5567) || Train Top 1-acc : 81.250% (41.935)% || Train Top 5-acc : 100.000% (90.121)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.7342 (1.3341) || Train Top 1-acc : 100.000% (52.744)% || Train Top 5-acc : 100.000% (92.378)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.7819 (1.2193) || Train Top 1-acc : 81.250% (58.211)% || Train Top 5-acc : 100.000% (93.505)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 0.9304 (1.1261) || Train Top 1-acc : 87.500% (63.115)% || Train Top 5-acc : 100.000% (94.570)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 1.1189 (1.0639) || Train Top 1-acc : 68.750% (65.493)% || Train Top 5-acc : 100.000% (95.335)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.5482 (1.0153) || Train Top 1-acc : 87.500% (67.515)% || Train Top 5-acc : 100.000% (95.756)%
Epoch : [1/10] || Val Loss : 0.1106 || Val Top 1-acc : 97.647% || Val Top 5-acc : 99.412%
Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 1.0929 (1.0929) || Train Top 1-acc : 75.000% (75.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.9855 (0.5546) || Train Top 1-acc : 100.000% (86.364)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.0425 (0.4892) || Train Top 1-acc : 100.000% (86.607)% || Train Top 5-acc : 100.000% (98.810)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.4465 (0.4609) || Train Top 1-acc : 100.000% (87.903)% || Train Top 5-acc : 100.000% (99.194)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.1961 (0.4454) || Train Top 1-acc : 93.750% (85.366)% || Train Top 5-acc : 100.000% (99.238)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.3380 (0.4155) || Train Top 1-acc : 87.500% (84.926)% || Train Top 5-acc : 100.000% (99.265)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.0853 (0.4314) || Train Top 1-acc : 100.000% (84.324)% || Train Top 5-acc : 100.000% (99.385)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.8239 (0.4433) || Train Top 1-acc : 87.500% (84.683)% || Train Top 5-acc : 100.000% (99.384)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.7969 (0.4773) || Train Top 1-acc : 87.500% (84.491)% || Train Top 5-acc : 100.000% (99.383)%
Epoch : [2/10] || Val Loss : 0.0584 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%
Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.0847 (0.0847) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.0401 (0.4712) || Train Top 1-acc : 100.000% (78.409)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0320 (0.3945) || Train Top 1-acc : 100.000% (79.762)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.0384 (0.4485) || Train Top 1-acc : 100.000% (76.613)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.2413 (0.4236) || Train Top 1-acc : 100.000% (80.793)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0809 (0.3980) || Train Top 1-acc : 93.750% (83.088)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0139 (0.3604) || Train Top 1-acc : 100.000% (85.143)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0348 (0.3448) || Train Top 1-acc : 100.000% (86.180)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.0095 (0.3409) || Train Top 1-acc : 100.000% (86.343)% || Train Top 5-acc : 100.000% (99.691)%
Epoch : [3/10] || Val Loss : 0.0292 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%
Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.0142 (0.0142) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.7210 (0.1531) || Train Top 1-acc : 68.750% (92.045)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.0117 (0.1085) || Train Top 1-acc : 100.000% (94.940)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.6184 (0.1680) || Train Top 1-acc : 43.750% (93.548)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.6395 (0.2423) || Train Top 1-acc : 93.750% (93.598)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.0177 (0.2327) || Train Top 1-acc : 100.000% (94.363)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.0082 (0.2169) || Train Top 1-acc : 100.000% (94.365)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.0111 (0.2166) || Train Top 1-acc : 100.000% (94.190)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.4796 (0.2176) || Train Top 1-acc : 93.750% (93.827)% || Train Top 5-acc : 100.000% (99.846)%
Epoch : [4/10] || Val Loss : 0.0124 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%
Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.4371 (0.4371) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.0152 (0.5655) || Train Top 1-acc : 100.000% (78.409)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.0176 (0.3995) || Train Top 1-acc : 100.000% (87.202)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.0067 (0.3112) || Train Top 1-acc : 100.000% (90.524)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.8374 (0.2774) || Train Top 1-acc : 93.750% (91.921)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.2836 (0.2649) || Train Top 1-acc : 100.000% (92.525)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.0054 (0.2688) || Train Top 1-acc : 100.000% (90.881)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.0297 (0.2616) || Train Top 1-acc : 100.000% (91.813)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.6366 (0.2890) || Train Top 1-acc : 87.500% (90.818)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10] || Val Loss : 0.0240 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%
Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.0070 (0.0070) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.0140 (0.2751) || Train Top 1-acc : 100.000% (89.205)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.8425 (0.3024) || Train Top 1-acc : 18.750% (87.202)% || Train Top 5-acc : 93.750% (99.702)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.0099 (0.2850) || Train Top 1-acc : 100.000% (86.290)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.7025 (0.2905) || Train Top 1-acc : 75.000% (85.213)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.8478 (0.2985) || Train Top 1-acc : 81.250% (87.255)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.3189 (0.3177) || Train Top 1-acc : 100.000% (86.988)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.0085 (0.2965) || Train Top 1-acc : 100.000% (88.028)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.0053 (0.2851) || Train Top 1-acc : 100.000% (89.352)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [6/10] || Val Loss : 0.0096 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%
Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.8451 (0.8451) || Train Top 1-acc : 68.750% (68.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.3970 (0.1916) || Train Top 1-acc : 93.750% (96.591)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.4457 (0.2410) || Train Top 1-acc : 93.750% (91.964)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.6965 (0.2587) || Train Top 1-acc : 93.750% (94.153)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.7570 (0.3079) || Train Top 1-acc : 93.750% (90.244)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.0028 (0.3068) || Train Top 1-acc : 100.000% (88.725)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0070 (0.2952) || Train Top 1-acc : 100.000% (90.471)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.6754 (0.3054) || Train Top 1-acc : 100.000% (89.789)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0094 (0.2912) || Train Top 1-acc : 100.000% (90.664)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [7/10] || Val Loss : 0.0050 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%
Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.7817 (0.7817) || Train Top 1-acc : 81.250% (81.250)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.7176 (0.4617) || Train Top 1-acc : 62.500% (84.659)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.8492 (0.4741) || Train Top 1-acc : 37.500% (80.655)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.5736 (0.4226) || Train Top 1-acc : 100.000% (83.871)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.4147 (0.4164) || Train Top 1-acc : 100.000% (84.909)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.0020 (0.3762) || Train Top 1-acc : 100.000% (86.642)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.6746 (0.3975) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.5722 (0.3860) || Train Top 1-acc : 50.000% (87.676)% || Train Top 5-acc : 93.750% (99.648)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.7933 (0.3881) || Train Top 1-acc : 87.500% (87.114)% || Train Top 5-acc : 100.000% (99.691)%
Epoch : [8/10] || Val Loss : 0.0110 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%
Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.0027 (0.0027) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.0083 (0.0644) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.0031 (0.1934) || Train Top 1-acc : 100.000% (91.964)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.6657 (0.2242) || Train Top 1-acc : 37.500% (90.726)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0052 (0.2519) || Train Top 1-acc : 100.000% (89.787)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0018 (0.2451) || Train Top 1-acc : 100.000% (89.216)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.0038 (0.2543) || Train Top 1-acc : 100.000% (89.754)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.0036 (0.2378) || Train Top 1-acc : 100.000% (90.581)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.0073 (0.2398) || Train Top 1-acc : 100.000% (91.204)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [9/10] || Val Loss : 0.0074 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%
Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.8033 (0.8033) || Train Top 1-acc : 18.750% (18.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.0049 (0.3229) || Train Top 1-acc : 100.000% (83.523)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.4185 (0.2718) || Train Top 1-acc : 87.500% (84.821)% || Train Top 5-acc : 100.000% (99.702)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0085 (0.2215) || Train Top 1-acc : 100.000% (89.113)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.0023 (0.2758) || Train Top 1-acc : 100.000% (89.177)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.4751 (0.2923) || Train Top 1-acc : 100.000% (88.603)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.0058 (0.2990) || Train Top 1-acc : 100.000% (87.295)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0025 (0.2934) || Train Top 1-acc : 100.000% (88.556)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.0030 (0.2960) || Train Top 1-acc : 100.000% (89.043)% || Train Top 5-acc : 100.000% (99.846)%
Epoch : [10/10] || Val Loss : 0.0043 || Val Top 1-acc : 100.000% || Val Top 5-acc : 100.000%
'===============3-Fold Cross Validation==============='
Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.0720 (2.0720) || Train Top 1-acc : 12.500% (12.500)% || Train Top 5-acc : 62.500% (62.500)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.7776 (1.9316) || Train Top 1-acc : 43.750% (23.864)% || Train Top 5-acc : 75.000% (76.136)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.4177 (1.8146) || Train Top 1-acc : 50.000% (29.762)% || Train Top 5-acc : 100.000% (82.738)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.5192 (1.6751) || Train Top 1-acc : 37.500% (36.895)% || Train Top 5-acc : 100.000% (86.694)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.7066 (1.4569) || Train Top 1-acc : 87.500% (48.018)% || Train Top 5-acc : 100.000% (89.634)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.8877 (1.3557) || Train Top 1-acc : 75.000% (52.819)% || Train Top 5-acc : 93.750% (91.299)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 1.1762 (1.2324) || Train Top 1-acc : 75.000% (57.582)% || Train Top 5-acc : 100.000% (92.623)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 0.2511 (1.1340) || Train Top 1-acc : 93.750% (61.180)% || Train Top 5-acc : 100.000% (93.134)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.0398 (1.0670) || Train Top 1-acc : 100.000% (64.275)% || Train Top 5-acc : 100.000% (93.981)%
Epoch : [1/10] || Val Loss : 0.1052 || Val Top 1-acc : 97.353% || Val Top 5-acc : 99.706%
Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 0.1522 (0.1522) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.7374 (0.5063) || Train Top 1-acc : 100.000% (88.636)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.0710 (0.4124) || Train Top 1-acc : 100.000% (88.393)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.0451 (0.4122) || Train Top 1-acc : 100.000% (86.290)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.9314 (0.4282) || Train Top 1-acc : 62.500% (85.366)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.0468 (0.4101) || Train Top 1-acc : 100.000% (85.784)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.7474 (0.4185) || Train Top 1-acc : 93.750% (86.475)% || Train Top 5-acc : 100.000% (99.590)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.0288 (0.4013) || Train Top 1-acc : 100.000% (87.764)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.0481 (0.4107) || Train Top 1-acc : 100.000% (87.346)% || Train Top 5-acc : 100.000% (99.614)%
Epoch : [2/10] || Val Loss : 0.0466 || Val Top 1-acc : 98.824% || Val Top 5-acc : 100.000%
Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.0975 (0.0975) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.6538 (0.3331) || Train Top 1-acc : 93.750% (97.159)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0456 (0.3037) || Train Top 1-acc : 100.000% (97.917)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.8460 (0.3477) || Train Top 1-acc : 68.750% (94.556)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.2994 (0.3534) || Train Top 1-acc : 93.750% (92.073)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0195 (0.3358) || Train Top 1-acc : 100.000% (92.034)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0105 (0.3068) || Train Top 1-acc : 100.000% (92.623)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0128 (0.3065) || Train Top 1-acc : 100.000% (92.694)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.3932 (0.3081) || Train Top 1-acc : 93.750% (92.901)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [3/10] || Val Loss : 0.0331 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%
Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.5570 (0.5570) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.3258 (0.3006) || Train Top 1-acc : 100.000% (94.318)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.5734 (0.3007) || Train Top 1-acc : 100.000% (90.774)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.0303 (0.3164) || Train Top 1-acc : 100.000% (89.718)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.0232 (0.3054) || Train Top 1-acc : 100.000% (91.768)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.0116 (0.3024) || Train Top 1-acc : 100.000% (92.157)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.8345 (0.3167) || Train Top 1-acc : 87.500% (91.701)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.0108 (0.3137) || Train Top 1-acc : 100.000% (90.845)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0077 (0.3195) || Train Top 1-acc : 100.000% (89.429)% || Train Top 5-acc : 100.000% (99.846)%
Epoch : [4/10] || Val Loss : 0.0223 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%
Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.0156 (0.0156) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.0072 (0.2611) || Train Top 1-acc : 100.000% (84.659)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.0092 (0.1958) || Train Top 1-acc : 100.000% (89.286)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.0100 (0.1986) || Train Top 1-acc : 100.000% (92.742)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.6878 (0.2660) || Train Top 1-acc : 100.000% (92.226)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0070 (0.2976) || Train Top 1-acc : 100.000% (90.441)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.9078 (0.3341) || Train Top 1-acc : 81.250% (89.139)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.6196 (0.3450) || Train Top 1-acc : 81.250% (88.556)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.0594 (0.3470) || Train Top 1-acc : 100.000% (88.580)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [5/10] || Val Loss : 0.0287 || Val Top 1-acc : 99.118% || Val Top 5-acc : 100.000%
Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.5246 (0.5246) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.6923 (0.3866) || Train Top 1-acc : 87.500% (89.205)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.5975 (0.4660) || Train Top 1-acc : 62.500% (83.631)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.7972 (0.4186) || Train Top 1-acc : 18.750% (83.065)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.0336 (0.4082) || Train Top 1-acc : 100.000% (83.994)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.0021 (0.3904) || Train Top 1-acc : 100.000% (86.887)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.0051 (0.3658) || Train Top 1-acc : 100.000% (88.320)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.5820 (0.3447) || Train Top 1-acc : 100.000% (89.525)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.0028 (0.3448) || Train Top 1-acc : 100.000% (89.275)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [6/10] || Val Loss : 0.0236 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%
Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.7110 (0.7110) || Train Top 1-acc : 87.500% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.9598 (0.4299) || Train Top 1-acc : 25.000% (80.682)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.5731 (0.3490) || Train Top 1-acc : 100.000% (89.286)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.0206 (0.3777) || Train Top 1-acc : 100.000% (87.702)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.0045 (0.3647) || Train Top 1-acc : 100.000% (89.177)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.5658 (0.3217) || Train Top 1-acc : 62.500% (90.074)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0022 (0.3149) || Train Top 1-acc : 100.000% (90.164)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.5986 (0.3160) || Train Top 1-acc : 37.500% (88.556)% || Train Top 5-acc : 93.750% (99.736)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0064 (0.3103) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [7/10] || Val Loss : 0.0229 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%
Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.0126 (0.0126) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.0046 (0.3209) || Train Top 1-acc : 100.000% (84.659)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.0030 (0.2900) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0098 (0.2692) || Train Top 1-acc : 100.000% (91.331)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.0039 (0.2477) || Train Top 1-acc : 100.000% (92.530)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.8142 (0.2741) || Train Top 1-acc : 62.500% (91.667)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.6055 (0.2918) || Train Top 1-acc : 93.750% (89.447)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.6436 (0.3183) || Train Top 1-acc : 100.000% (88.644)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.0013 (0.2972) || Train Top 1-acc : 100.000% (89.043)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10] || Val Loss : 0.0184 || Val Top 1-acc : 99.412% || Val Top 5-acc : 100.000%
Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.0027 (0.0027) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.5898 (0.2487) || Train Top 1-acc : 75.000% (89.205)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.0021 (0.1797) || Train Top 1-acc : 100.000% (94.345)% || Train Top 5-acc : 100.000% (99.107)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.0068 (0.2011) || Train Top 1-acc : 100.000% (94.758)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.3292 (0.2237) || Train Top 1-acc : 100.000% (95.122)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0079 (0.2265) || Train Top 1-acc : 100.000% (93.995)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.0014 (0.2211) || Train Top 1-acc : 100.000% (93.852)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.7926 (0.2363) || Train Top 1-acc : 75.000% (93.046)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.6087 (0.2463) || Train Top 1-acc : 31.250% (92.901)% || Train Top 5-acc : 87.500% (99.537)%
Epoch : [9/10] || Val Loss : 0.0131 || Val Top 1-acc : 99.706% || Val Top 5-acc : 100.000%
Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.4328 (0.4328) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.6455 (0.4515) || Train Top 1-acc : 100.000% (88.068)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.2457 (0.3538) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0033 (0.2960) || Train Top 1-acc : 100.000% (90.726)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.5825 (0.3055) || Train Top 1-acc : 100.000% (90.244)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.6785 (0.3167) || Train Top 1-acc : 100.000% (91.544)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.5924 (0.3040) || Train Top 1-acc : 100.000% (92.316)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0020 (0.2927) || Train Top 1-acc : 100.000% (92.870)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.5519 (0.2729) || Train Top 1-acc : 100.000% (92.824)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10] || Val Loss : 0.0160 || Val Top 1-acc : 99.412% || Val Top 5-acc : 100.000%
'===============4-Fold Cross Validation==============='
Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.1108 (2.1108) || Train Top 1-acc : 0.000% (0.000)% || Train Top 5-acc : 62.500% (62.500)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 1.8617 (1.8976) || Train Top 1-acc : 12.500% (15.909)% || Train Top 5-acc : 100.000% (79.545)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.1728 (1.7297) || Train Top 1-acc : 62.500% (29.762)% || Train Top 5-acc : 100.000% (87.202)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.0114 (1.5880) || Train Top 1-acc : 43.750% (37.702)% || Train Top 5-acc : 100.000% (89.516)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 1.1918 (1.3777) || Train Top 1-acc : 87.500% (48.018)% || Train Top 5-acc : 93.750% (91.463)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 0.3317 (1.1855) || Train Top 1-acc : 93.750% (55.147)% || Train Top 5-acc : 100.000% (92.770)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 0.7014 (1.1024) || Train Top 1-acc : 93.750% (60.041)% || Train Top 5-acc : 100.000% (93.852)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 1.0357 (1.0330) || Train Top 1-acc : 56.250% (62.500)% || Train Top 5-acc : 100.000% (94.630)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.1170 (0.9727) || Train Top 1-acc : 93.750% (65.355)% || Train Top 5-acc : 100.000% (95.139)%
Epoch : [1/10] || Val Loss : 0.1316 || Val Top 1-acc : 96.755% || Val Top 5-acc : 100.000%
Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 0.9091 (0.9091) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 0.0604 (0.5182) || Train Top 1-acc : 100.000% (82.386)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.4621 (0.4736) || Train Top 1-acc : 87.500% (85.714)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.0275 (0.4028) || Train Top 1-acc : 100.000% (89.113)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.1133 (0.3619) || Train Top 1-acc : 100.000% (90.549)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.0364 (0.3784) || Train Top 1-acc : 100.000% (88.971)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.0336 (0.3608) || Train Top 1-acc : 100.000% (90.164)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.8431 (0.3535) || Train Top 1-acc : 81.250% (90.141)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 1.1812 (0.3608) || Train Top 1-acc : 37.500% (89.429)% || Train Top 5-acc : 93.750% (99.691)%
Epoch : [2/10] || Val Loss : 0.1468 || Val Top 1-acc : 96.165% || Val Top 5-acc : 100.000%
Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.2246 (0.2246) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.9671 (0.2779) || Train Top 1-acc : 68.750% (94.318)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0288 (0.2994) || Train Top 1-acc : 100.000% (91.964)% || Train Top 5-acc : 100.000% (99.107)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.7963 (0.3208) || Train Top 1-acc : 75.000% (92.339)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.0200 (0.3825) || Train Top 1-acc : 100.000% (88.415)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0376 (0.3615) || Train Top 1-acc : 100.000% (89.583)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0171 (0.3721) || Train Top 1-acc : 100.000% (89.242)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0067 (0.3339) || Train Top 1-acc : 100.000% (90.317)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.0069 (0.3072) || Train Top 1-acc : 100.000% (90.818)% || Train Top 5-acc : 100.000% (99.691)%
Epoch : [3/10] || Val Loss : 0.0580 || Val Top 1-acc : 97.345% || Val Top 5-acc : 100.000%
Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.5756 (0.5756) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.0418 (0.3361) || Train Top 1-acc : 100.000% (82.955)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.6049 (0.3751) || Train Top 1-acc : 100.000% (84.524)% || Train Top 5-acc : 100.000% (99.107)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.1730 (0.3705) || Train Top 1-acc : 93.750% (84.879)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.7159 (0.3748) || Train Top 1-acc : 100.000% (87.805)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.3958 (0.3576) || Train Top 1-acc : 100.000% (89.216)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.0115 (0.3394) || Train Top 1-acc : 100.000% (89.857)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.2760 (0.3272) || Train Top 1-acc : 100.000% (90.845)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0097 (0.3161) || Train Top 1-acc : 100.000% (90.664)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [4/10] || Val Loss : 0.0969 || Val Top 1-acc : 97.050% || Val Top 5-acc : 100.000%
Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.2456 (0.2456) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.0148 (0.3081) || Train Top 1-acc : 100.000% (91.477)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.6745 (0.3977) || Train Top 1-acc : 93.750% (90.774)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.0173 (0.3700) || Train Top 1-acc : 100.000% (90.323)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.0148 (0.3435) || Train Top 1-acc : 100.000% (91.768)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0073 (0.3391) || Train Top 1-acc : 100.000% (91.422)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.4713 (0.3340) || Train Top 1-acc : 100.000% (92.008)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.0109 (0.3454) || Train Top 1-acc : 100.000% (92.430)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.8212 (0.3591) || Train Top 1-acc : 87.500% (91.358)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [5/10] || Val Loss : 0.0336 || Val Top 1-acc : 98.820% || Val Top 5-acc : 100.000%
Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.3899 (0.3899) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.0154 (0.0537) || Train Top 1-acc : 100.000% (99.432)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.7629 (0.1997) || Train Top 1-acc : 93.750% (89.286)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.5958 (0.2350) || Train Top 1-acc : 87.500% (91.129)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.0045 (0.2186) || Train Top 1-acc : 100.000% (92.073)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.5276 (0.2654) || Train Top 1-acc : 93.750% (90.686)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.0084 (0.2493) || Train Top 1-acc : 100.000% (91.189)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.0057 (0.2545) || Train Top 1-acc : 100.000% (89.613)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.6041 (0.2553) || Train Top 1-acc : 62.500% (89.352)% || Train Top 5-acc : 100.000% (99.691)%
Epoch : [6/10] || Val Loss : 0.0320 || Val Top 1-acc : 98.230% || Val Top 5-acc : 100.000%
Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.5307 (0.5307) || Train Top 1-acc : 87.500% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.0054 (0.3147) || Train Top 1-acc : 100.000% (91.477)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.0055 (0.2666) || Train Top 1-acc : 100.000% (93.452)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.0080 (0.2908) || Train Top 1-acc : 100.000% (89.315)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.0085 (0.2561) || Train Top 1-acc : 100.000% (90.244)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.6985 (0.2960) || Train Top 1-acc : 81.250% (87.132)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0053 (0.3152) || Train Top 1-acc : 100.000% (85.246)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.2425 (0.3452) || Train Top 1-acc : 100.000% (84.419)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0069 (0.3274) || Train Top 1-acc : 100.000% (84.491)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [7/10] || Val Loss : 0.0270 || Val Top 1-acc : 99.410% || Val Top 5-acc : 100.000%
Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.4553 (0.4553) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.5086 (0.1545) || Train Top 1-acc : 93.750% (99.432)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.0089 (0.2241) || Train Top 1-acc : 100.000% (95.238)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0119 (0.2344) || Train Top 1-acc : 100.000% (91.935)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.0201 (0.2248) || Train Top 1-acc : 100.000% (90.549)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.0034 (0.2730) || Train Top 1-acc : 100.000% (88.971)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.0028 (0.2727) || Train Top 1-acc : 100.000% (89.754)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.0142 (0.2914) || Train Top 1-acc : 100.000% (90.141)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.0028 (0.2728) || Train Top 1-acc : 100.000% (91.204)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10] || Val Loss : 0.0294 || Val Top 1-acc : 98.525% || Val Top 5-acc : 100.000%
Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.7818 (0.7818) || Train Top 1-acc : 75.000% (75.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.7282 (0.3394) || Train Top 1-acc : 68.750% (85.227)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.2990 (0.3756) || Train Top 1-acc : 100.000% (82.143)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.0026 (0.3406) || Train Top 1-acc : 100.000% (84.677)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0021 (0.2912) || Train Top 1-acc : 100.000% (88.262)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0030 (0.2494) || Train Top 1-acc : 100.000% (90.319)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.7077 (0.2870) || Train Top 1-acc : 18.750% (90.164)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.0023 (0.2583) || Train Top 1-acc : 100.000% (91.373)% || Train Top 5-acc : 100.000% (99.824)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.6054 (0.2762) || Train Top 1-acc : 100.000% (90.664)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [9/10] || Val Loss : 0.0227 || Val Top 1-acc : 99.410% || Val Top 5-acc : 100.000%
Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.0028 (0.0028) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.2013 (0.3793) || Train Top 1-acc : 100.000% (98.295)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.6386 (0.4370) || Train Top 1-acc : 100.000% (91.071)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0031 (0.3684) || Train Top 1-acc : 100.000% (93.347)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.5983 (0.3663) || Train Top 1-acc : 100.000% (91.768)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.8164 (0.3471) || Train Top 1-acc : 81.250% (91.544)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.2539 (0.3466) || Train Top 1-acc : 100.000% (91.496)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0026 (0.3320) || Train Top 1-acc : 100.000% (91.637)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.3697 (0.3306) || Train Top 1-acc : 100.000% (91.821)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [10/10] || Val Loss : 0.0249 || Val Top 1-acc : 99.115% || Val Top 5-acc : 100.000%
'===============5-Fold Cross Validation==============='
Epoch : [1/10][0/85] || LR : 0.00001 || Train Loss : 2.0386 (2.0386) || Train Top 1-acc : 6.250% (6.250)% || Train Top 5-acc : 75.000% (75.000)%
Epoch : [1/10][10/85] || LR : 0.00003 || Train Loss : 2.0713 (1.9109) || Train Top 1-acc : 31.250% (20.455)% || Train Top 5-acc : 56.250% (79.545)%
Epoch : [1/10][20/85] || LR : 0.00004 || Train Loss : 1.3254 (1.7304) || Train Top 1-acc : 75.000% (34.524)% || Train Top 5-acc : 100.000% (86.310)%
Epoch : [1/10][30/85] || LR : 0.00006 || Train Loss : 1.0685 (1.5021) || Train Top 1-acc : 81.250% (46.169)% || Train Top 5-acc : 100.000% (90.524)%
Epoch : [1/10][40/85] || LR : 0.00008 || Train Loss : 0.2387 (1.2839) || Train Top 1-acc : 93.750% (56.402)% || Train Top 5-acc : 100.000% (92.835)%
Epoch : [1/10][50/85] || LR : 0.00009 || Train Loss : 1.2306 (1.1856) || Train Top 1-acc : 81.250% (61.520)% || Train Top 5-acc : 93.750% (93.995)%
Epoch : [1/10][60/85] || LR : 0.00010 || Train Loss : 0.4093 (1.0717) || Train Top 1-acc : 93.750% (65.881)% || Train Top 5-acc : 100.000% (94.877)%
Epoch : [1/10][70/85] || LR : 0.00010 || Train Loss : 0.9882 (1.0018) || Train Top 1-acc : 43.750% (68.134)% || Train Top 5-acc : 93.750% (95.511)%
Epoch : [1/10][80/85] || LR : 0.00010 || Train Loss : 0.4428 (0.9365) || Train Top 1-acc : 93.750% (70.679)% || Train Top 5-acc : 100.000% (96.065)%
Epoch : [1/10] || Val Loss : 0.2146 || Val Top 1-acc : 94.690% || Val Top 5-acc : 100.000%
Epoch : [2/10][0/85] || LR : 0.00010 || Train Loss : 0.0805 (0.0805) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [2/10][10/85] || LR : 0.00009 || Train Loss : 1.1480 (0.6476) || Train Top 1-acc : 50.000% (82.386)% || Train Top 5-acc : 100.000% (98.295)%
Epoch : [2/10][20/85] || LR : 0.00009 || Train Loss : 0.1377 (0.5387) || Train Top 1-acc : 93.750% (86.310)% || Train Top 5-acc : 100.000% (98.810)%
Epoch : [2/10][30/85] || LR : 0.00009 || Train Loss : 0.0670 (0.5262) || Train Top 1-acc : 100.000% (84.677)% || Train Top 5-acc : 100.000% (98.992)%
Epoch : [2/10][40/85] || LR : 0.00008 || Train Loss : 0.2268 (0.5196) || Train Top 1-acc : 93.750% (84.756)% || Train Top 5-acc : 100.000% (99.238)%
Epoch : [2/10][50/85] || LR : 0.00008 || Train Loss : 0.8585 (0.5349) || Train Top 1-acc : 81.250% (84.436)% || Train Top 5-acc : 100.000% (99.142)%
Epoch : [2/10][60/85] || LR : 0.00007 || Train Loss : 0.5709 (0.5292) || Train Top 1-acc : 100.000% (85.143)% || Train Top 5-acc : 100.000% (99.078)%
Epoch : [2/10][70/85] || LR : 0.00006 || Train Loss : 0.5153 (0.5029) || Train Top 1-acc : 93.750% (86.092)% || Train Top 5-acc : 100.000% (99.208)%
Epoch : [2/10][80/85] || LR : 0.00006 || Train Loss : 0.9393 (0.4761) || Train Top 1-acc : 93.750% (87.423)% || Train Top 5-acc : 100.000% (99.306)%
Epoch : [2/10] || Val Loss : 0.1453 || Val Top 1-acc : 95.575% || Val Top 5-acc : 99.410%
Epoch : [3/10][0/85] || LR : 0.00005 || Train Loss : 0.0124 (0.0124) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [3/10][10/85] || LR : 0.00005 || Train Loss : 0.0187 (0.2988) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (99.432)%
Epoch : [3/10][20/85] || LR : 0.00004 || Train Loss : 0.0271 (0.2670) || Train Top 1-acc : 100.000% (91.964)% || Train Top 5-acc : 100.000% (99.702)%
Epoch : [3/10][30/85] || LR : 0.00004 || Train Loss : 0.0087 (0.2915) || Train Top 1-acc : 100.000% (89.718)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [3/10][40/85] || LR : 0.00003 || Train Loss : 0.0161 (0.2893) || Train Top 1-acc : 100.000% (90.549)% || Train Top 5-acc : 100.000% (99.695)%
Epoch : [3/10][50/85] || LR : 0.00003 || Train Loss : 0.0083 (0.3079) || Train Top 1-acc : 100.000% (91.176)% || Train Top 5-acc : 100.000% (99.755)%
Epoch : [3/10][60/85] || LR : 0.00002 || Train Loss : 0.0309 (0.3210) || Train Top 1-acc : 100.000% (92.008)% || Train Top 5-acc : 100.000% (99.795)%
Epoch : [3/10][70/85] || LR : 0.00002 || Train Loss : 0.0124 (0.3300) || Train Top 1-acc : 100.000% (91.461)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [3/10][80/85] || LR : 0.00001 || Train Loss : 0.8605 (0.3338) || Train Top 1-acc : 75.000% (90.664)% || Train Top 5-acc : 100.000% (99.614)%
Epoch : [3/10] || Val Loss : 0.1214 || Val Top 1-acc : 97.345% || Val Top 5-acc : 99.705%
Epoch : [4/10][0/85] || LR : 0.00001 || Train Loss : 0.0080 (0.0080) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][10/85] || LR : 0.00001 || Train Loss : 0.0059 (0.2478) || Train Top 1-acc : 100.000% (92.614)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][20/85] || LR : 0.00001 || Train Loss : 0.9131 (0.2578) || Train Top 1-acc : 68.750% (91.964)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][30/85] || LR : 0.00001 || Train Loss : 0.7515 (0.2385) || Train Top 1-acc : 100.000% (94.153)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [4/10][40/85] || LR : 0.00002 || Train Loss : 0.4925 (0.2670) || Train Top 1-acc : 100.000% (93.750)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [4/10][50/85] || LR : 0.00003 || Train Loss : 0.0057 (0.2657) || Train Top 1-acc : 100.000% (94.240)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [4/10][60/85] || LR : 0.00003 || Train Loss : 0.3083 (0.2825) || Train Top 1-acc : 100.000% (93.135)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [4/10][70/85] || LR : 0.00004 || Train Loss : 0.0053 (0.2897) || Train Top 1-acc : 100.000% (91.285)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [4/10][80/85] || LR : 0.00005 || Train Loss : 0.0127 (0.2777) || Train Top 1-acc : 100.000% (91.435)% || Train Top 5-acc : 100.000% (99.846)%
Epoch : [4/10] || Val Loss : 0.1148 || Val Top 1-acc : 96.755% || Val Top 5-acc : 100.000%
Epoch : [5/10][0/85] || LR : 0.00005 || Train Loss : 0.6259 (0.6259) || Train Top 1-acc : 93.750% (93.750)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][10/85] || LR : 0.00005 || Train Loss : 0.7918 (0.2159) || Train Top 1-acc : 0.000% (89.773)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [5/10][20/85] || LR : 0.00005 || Train Loss : 0.7312 (0.3097) || Train Top 1-acc : 75.000% (86.607)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [5/10][30/85] || LR : 0.00005 || Train Loss : 0.2967 (0.2696) || Train Top 1-acc : 100.000% (89.113)% || Train Top 5-acc : 100.000% (99.597)%
Epoch : [5/10][40/85] || LR : 0.00005 || Train Loss : 0.6948 (0.3178) || Train Top 1-acc : 62.500% (85.671)% || Train Top 5-acc : 100.000% (99.390)%
Epoch : [5/10][50/85] || LR : 0.00005 || Train Loss : 0.0035 (0.3003) || Train Top 1-acc : 100.000% (87.868)% || Train Top 5-acc : 100.000% (99.510)%
Epoch : [5/10][60/85] || LR : 0.00004 || Train Loss : 0.4196 (0.2807) || Train Top 1-acc : 93.750% (89.242)% || Train Top 5-acc : 100.000% (99.590)%
Epoch : [5/10][70/85] || LR : 0.00004 || Train Loss : 0.6039 (0.2869) || Train Top 1-acc : 100.000% (89.613)% || Train Top 5-acc : 100.000% (99.648)%
Epoch : [5/10][80/85] || LR : 0.00004 || Train Loss : 0.9320 (0.2970) || Train Top 1-acc : 81.250% (89.506)% || Train Top 5-acc : 100.000% (99.691)%
Epoch : [5/10] || Val Loss : 0.1448 || Val Top 1-acc : 95.870% || Val Top 5-acc : 99.705%
Epoch : [6/10][0/85] || LR : 0.00004 || Train Loss : 0.0042 (0.0042) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][10/85] || LR : 0.00003 || Train Loss : 0.3565 (0.2846) || Train Top 1-acc : 100.000% (95.455)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][20/85] || LR : 0.00003 || Train Loss : 0.0052 (0.2608) || Train Top 1-acc : 100.000% (95.238)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [6/10][30/85] || LR : 0.00003 || Train Loss : 0.6910 (0.2762) || Train Top 1-acc : 87.500% (91.129)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [6/10][40/85] || LR : 0.00003 || Train Loss : 0.6822 (0.2907) || Train Top 1-acc : 100.000% (92.988)% || Train Top 5-acc : 100.000% (99.848)%
Epoch : [6/10][50/85] || LR : 0.00002 || Train Loss : 0.0330 (0.3036) || Train Top 1-acc : 100.000% (91.667)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [6/10][60/85] || LR : 0.00002 || Train Loss : 0.0039 (0.3174) || Train Top 1-acc : 100.000% (91.393)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [6/10][70/85] || LR : 0.00002 || Train Loss : 0.0062 (0.2918) || Train Top 1-acc : 100.000% (91.725)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [6/10][80/85] || LR : 0.00002 || Train Loss : 0.7268 (0.2846) || Train Top 1-acc : 43.750% (92.052)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [6/10] || Val Loss : 0.1204 || Val Top 1-acc : 97.050% || Val Top 5-acc : 100.000%
Epoch : [7/10][0/85] || LR : 0.00002 || Train Loss : 0.0055 (0.0055) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [7/10][10/85] || LR : 0.00001 || Train Loss : 0.4340 (0.3952) || Train Top 1-acc : 50.000% (78.977)% || Train Top 5-acc : 100.000% (98.864)%
Epoch : [7/10][20/85] || LR : 0.00001 || Train Loss : 0.0042 (0.3191) || Train Top 1-acc : 100.000% (82.440)% || Train Top 5-acc : 100.000% (99.405)%
Epoch : [7/10][30/85] || LR : 0.00001 || Train Loss : 0.0018 (0.2937) || Train Top 1-acc : 100.000% (85.081)% || Train Top 5-acc : 100.000% (99.395)%
Epoch : [7/10][40/85] || LR : 0.00001 || Train Loss : 0.7229 (0.2791) || Train Top 1-acc : 87.500% (87.500)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [7/10][50/85] || LR : 0.00001 || Train Loss : 0.6283 (0.3096) || Train Top 1-acc : 62.500% (87.868)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [7/10][60/85] || LR : 0.00001 || Train Loss : 0.0016 (0.3001) || Train Top 1-acc : 100.000% (89.139)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [7/10][70/85] || LR : 0.00001 || Train Loss : 0.2362 (0.2912) || Train Top 1-acc : 100.000% (88.556)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [7/10][80/85] || LR : 0.00002 || Train Loss : 0.0154 (0.2831) || Train Top 1-acc : 100.000% (88.812)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [7/10] || Val Loss : 0.1291 || Val Top 1-acc : 97.050% || Val Top 5-acc : 100.000%
Epoch : [8/10][0/85] || LR : 0.00002 || Train Loss : 0.5337 (0.5337) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][10/85] || LR : 0.00002 || Train Loss : 0.0025 (0.1729) || Train Top 1-acc : 100.000% (96.591)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][20/85] || LR : 0.00002 || Train Loss : 0.0025 (0.1783) || Train Top 1-acc : 100.000% (94.643)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][30/85] || LR : 0.00002 || Train Loss : 0.0010 (0.1866) || Train Top 1-acc : 100.000% (95.363)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][40/85] || LR : 0.00002 || Train Loss : 0.5238 (0.2408) || Train Top 1-acc : 100.000% (94.360)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [8/10][50/85] || LR : 0.00002 || Train Loss : 0.5768 (0.2467) || Train Top 1-acc : 100.000% (93.505)% || Train Top 5-acc : 100.000% (99.877)%
Epoch : [8/10][60/85] || LR : 0.00002 || Train Loss : 0.0017 (0.2528) || Train Top 1-acc : 100.000% (94.262)% || Train Top 5-acc : 100.000% (99.898)%
Epoch : [8/10][70/85] || LR : 0.00002 || Train Loss : 0.0026 (0.2893) || Train Top 1-acc : 100.000% (92.958)% || Train Top 5-acc : 100.000% (99.912)%
Epoch : [8/10][80/85] || LR : 0.00002 || Train Loss : 0.5362 (0.3126) || Train Top 1-acc : 100.000% (91.049)% || Train Top 5-acc : 100.000% (99.923)%
Epoch : [8/10] || Val Loss : 0.1266 || Val Top 1-acc : 96.755% || Val Top 5-acc : 99.705%
Epoch : [9/10][0/85] || LR : 0.00002 || Train Loss : 0.5063 (0.5063) || Train Top 1-acc : 100.000% (100.000)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][10/85] || LR : 0.00002 || Train Loss : 0.2515 (0.3973) || Train Top 1-acc : 100.000% (87.500)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][20/85] || LR : 0.00002 || Train Loss : 0.7745 (0.3355) || Train Top 1-acc : 100.000% (93.452)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [9/10][30/85] || LR : 0.00002 || Train Loss : 0.5575 (0.3380) || Train Top 1-acc : 93.750% (90.927)% || Train Top 5-acc : 100.000% (99.798)%
Epoch : [9/10][40/85] || LR : 0.00002 || Train Loss : 0.0040 (0.3345) || Train Top 1-acc : 100.000% (89.177)% || Train Top 5-acc : 100.000% (99.543)%
Epoch : [9/10][50/85] || LR : 0.00002 || Train Loss : 0.0038 (0.3008) || Train Top 1-acc : 100.000% (91.299)% || Train Top 5-acc : 100.000% (99.632)%
Epoch : [9/10][60/85] || LR : 0.00002 || Train Loss : 0.5112 (0.2993) || Train Top 1-acc : 87.500% (92.316)% || Train Top 5-acc : 100.000% (99.693)%
Epoch : [9/10][70/85] || LR : 0.00002 || Train Loss : 0.5287 (0.2962) || Train Top 1-acc : 100.000% (92.606)% || Train Top 5-acc : 100.000% (99.736)%
Epoch : [9/10][80/85] || LR : 0.00001 || Train Loss : 0.6574 (0.3093) || Train Top 1-acc : 56.250% (92.130)% || Train Top 5-acc : 100.000% (99.769)%
Epoch : [9/10] || Val Loss : 0.1025 || Val Top 1-acc : 97.935% || Val Top 5-acc : 100.000%
Epoch : [10/10][0/85] || LR : 0.00001 || Train Loss : 0.7395 (0.7395) || Train Top 1-acc : 6.250% (6.250)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][10/85] || LR : 0.00001 || Train Loss : 0.0021 (0.3403) || Train Top 1-acc : 100.000% (86.932)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][20/85] || LR : 0.00001 || Train Loss : 0.0024 (0.2719) || Train Top 1-acc : 100.000% (88.393)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][30/85] || LR : 0.00001 || Train Loss : 0.0027 (0.2210) || Train Top 1-acc : 100.000% (91.734)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][40/85] || LR : 0.00001 || Train Loss : 0.0007 (0.2749) || Train Top 1-acc : 100.000% (88.872)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][50/85] || LR : 0.00001 || Train Loss : 0.0010 (0.2503) || Train Top 1-acc : 100.000% (90.074)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][60/85] || LR : 0.00001 || Train Loss : 0.4998 (0.2308) || Train Top 1-acc : 100.000% (90.471)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][70/85] || LR : 0.00001 || Train Loss : 0.0066 (0.2262) || Train Top 1-acc : 100.000% (91.813)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10][80/85] || LR : 0.00001 || Train Loss : 0.5944 (0.2119) || Train Top 1-acc : 87.500% (92.593)% || Train Top 5-acc : 100.000% (100.000)%
Epoch : [10/10] || Val Loss : 0.1011 || Val Top 1-acc : 97.935% || Val Top 5-acc : 100.000%
Inference¶
- Test Time Augmentation 사용 (only hflip)
In [2]:
# class TestDataset(Dataset):
# def __init__(self, data_path='./test/0/', transform=None):
# self.data_path = data_path
# self.data = os.listdir(data_path)
# self.transform = transform
# def __len__(self):
# return len(self.data)
# def __getitem__(self, idx):
# image_path = os.path.join(self.data_path, self.data[idx])
# image = cv2.imread(image_path)
# image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# if self.transform:
# image = self.transform(image=np.array(image))['image']
# return image
In [3]:
# SEED = 77
# BATCH_SIZE = 32
# IMAGE_SIZE = 227
# MODEL_ARC = 'nfnet_l0'
# NUM_CLASSES = 7
# MODEL_DIR = './results'
# NUM_FOLD = 5
In [4]:
# device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
In [5]:
# # Fix random seed
# def seed_everything(seed: int = 42):
# random.seed(seed)
# np.random.seed(seed)
# os.environ["PYTHONHASHSEED"] = str(seed)
# torch.manual_seed(seed)
# torch.cuda.manual_seed(seed) # type: ignore
# torch.backends.cudnn.deterministic = True # type: ignore
# torch.backends.cudnn.benchmark = True # type: ignore
In [6]:
# seed_everything(SEED)
In [7]:
# test_transform = albumentations.Compose([
# albumentations.Resize(IMAGE_SIZE, IMAGE_SIZE),
# albumentations.Normalize(mean=(0.4569, 0.5074, 0.5557), std=(0.2888, 0.2743, 0.2829)),
# albumentations.pytorch.transforms.ToTensorV2()])
In [8]:
# test_dataset = TestDataset(transform=test_transform)
# test_loader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False)
In [9]:
# class PretrainedModel(nn.Module):
# def __init__(self, model_arc='swin_tiny_patch4_window7_224', num_classes=7):
# super().__init__()
# self.net = timm.create_model(model_arc, pretrained=False, num_classes=num_classes)
# def forward(self, x):
# x = self.net(x)
# return x
In [10]:
# model = PretrainedModel(model_arc=MODEL_ARC, num_classes=NUM_CLASSES)
# model.to(device)
Out[10]:
PretrainedModel(
(net): NormFreeNet(
(stem): Sequential(
(conv1): ScaledStdConv2d(3, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(act3): SiLU(inplace=True)
(conv3): ScaledStdConv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(act4): SiLU(inplace=True)
(conv4): ScaledStdConv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
)
(stages): Sequential(
(0): Sequential(
(0): NormFreeBlock(
(downsample): DownsampleAvg(
(pool): Identity()
(conv): ScaledStdConv2d(128, 256, kernel_size=(1, 1), stride=(1, 1))
)
(act1): SiLU()
(conv1): ScaledStdConv2d(128, 64, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(act3): SiLU()
(conv3): ScaledStdConv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
)
(1): Sequential(
(0): NormFreeBlock(
(downsample): DownsampleAvg(
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
(conv): ScaledStdConv2d(256, 512, kernel_size=(1, 1), stride=(1, 1))
)
(act1): SiLU()
(conv1): ScaledStdConv2d(256, 128, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=2)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=2)
(act3): SiLU()
(conv3): ScaledStdConv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(1): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=2)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=2)
(act3): SiLU()
(conv3): ScaledStdConv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
)
(2): Sequential(
(0): NormFreeBlock(
(downsample): DownsampleAvg(
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
(conv): ScaledStdConv2d(512, 1536, kernel_size=(1, 1), stride=(1, 1))
)
(act1): SiLU()
(conv1): ScaledStdConv2d(512, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(1): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(2): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(3): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(4): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(5): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
)
(3): Sequential(
(0): NormFreeBlock(
(downsample): DownsampleAvg(
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
(conv): ScaledStdConv2d(1536, 1536, kernel_size=(1, 1), stride=(1, 1))
)
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(1): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
(2): NormFreeBlock(
(act1): SiLU()
(conv1): ScaledStdConv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act2): SiLU(inplace=True)
(conv2): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act2b): SiLU(inplace=True)
(conv2b): ScaledStdConv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6)
(act3): SiLU()
(conv3): ScaledStdConv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(attn_last): SEModule(
(fc1): Conv2d(1536, 384, kernel_size=(1, 1), stride=(1, 1))
(act): ReLU(inplace=True)
(fc2): Conv2d(384, 1536, kernel_size=(1, 1), stride=(1, 1))
(gate): Sigmoid()
)
(drop_path): Identity()
)
)
)
(final_conv): ScaledStdConv2d(1536, 2304, kernel_size=(1, 1), stride=(1, 1))
(final_act): SiLU(inplace=True)
(head): ClassifierHead(
(global_pool): SelectAdaptivePool2d (pool_type=avg, flatten=True)
(fc): Linear(in_features=2304, out_features=7, bias=True)
)
)
)In [11]:
# states = [torch.load(glob(MODEL_DIR + f'/{MODEL_ARC}/{k}_fold/*.pth')[-1]) for k in range(1, NUM_FOLD + 1)]
In [12]:
# transforms = tta.Compose(
# [
# tta.HorizontalFlip(),
# # tta.VerticalFlip(),
# # tta.Multiply(factors=[0.9, 1, 1.1])
# ]
# )
model ensemble을 위해서 npy 파일 형태로 저장¶
In [13]:
# probs = []
# save_ = []
# for i, images in enumerate(tqdm(test_loader)):
# images = images.to(device)
# avg_preds = []
# for state in states:
# model.load_state_dict(state)
# model.eval()
# tta_model = tta.ClassificationTTAWrapper(model, transforms)
# tta_model.to(device)
# tta_model.eval()
# with torch.no_grad():
# logits = tta_model(images)
# avg_preds.append(logits.to('cpu').numpy())
# avg_preds = np.mean(avg_preds, axis=0)
# save_.append(avg_preds)
# probs.append(avg_preds.argmax(-1))
# save_ = np.concatenate(save_)
# probs = np.concatenate(probs)
In [14]:
# df = pd.read_csv('./test_answer_sample_.csv')
In [15]:
# len(probs)
Out[15]:
350In [16]:
# save_.shape
Out[16]:
(350, 7)In [17]:
# np.save(f'./{MODEL_ARC}.npy', save_)
In [18]:
# df['answer value'] = probs
In [ ]:
# df.to_csv(f'submission_{MODEL_ARC}.csv', index=False)
Ensemble¶
- nfnet_l0와 swin transformer 사용
- stacking ensemble
In [10]:
# import numpy as np
# import pandas as pd
In [3]:
# npy_list = [
# 'nfnet_l0.npy',
# 'swin_base_patch4_window7_224.npy'
# ]
# predictions = []
# for f in npy_list:
# predictions.append(np.load(f))
In [5]:
# np.array(predictions).shape
Out[5]:
(2, 350, 7)In [15]:
# final = np.array(predictions).mean(axis=0).argmax(-1)
# final.shape
Out[15]:
(350,)In [16]:
# df = pd.read_csv('./test_answer_sample_.csv')
In [17]:
# df['answer value'] = final
In [18]:
# df.to_csv(f'submission_ensemble.csv', index=False)