Ubuntu 20.04的物理系统下，我的一个NTFS分区可能由于在Windows更新时进行了强制关机出现了错误，因此在Ubuntu下无法正常写入，出现了read-only状态。

问题解决

查找了一些修复方法，有针对ext2/ext3/ext4和NTFS两种文件系统的修复方法。

我在这里找到了针对NTFS的修复指令：

• 假设需要修复的分区挂载在/dev/sdb1
• 首先在文件管理器或使用命令解除分区挂载

sudo umount /dev/sdb1

• 然后使用ntfsfix命令尝试修复NTFS分区

sudo ntfsfix /dev/sdb1

• 成功后输出结果，分区可以正常写入

Mounting volume... The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
FAILED
Attempting to correct errors... 
Processing $MFT and $MFTMirr...
Reading $MFT... OK
Reading $MFTMirr... OK
Comparing $MFTMirr to $MFT... OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition... OK
Going to empty the journal ($LogFile)... OK
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/sdb1 was processed successfully.

如果你的受损分区是EXT4的，还可以参照这里进行修复：

• 假设需要修复的分区挂载在/dev/sdb1
• 首先在文件管理器或使用命令解除分区挂载

sudo umount /dev/sdb1

• 然后使用fsck命令尝试修复EXT4分区

sudo fsck.ext4 -y /dev/sdb1

修复时请选择对应的文件系统的修复指令。

Python的使用频率和范围越来越大，在一些开发工作中由于需要可视化的图形界面，常常需要进行图形用户界面（Graphic User Interface, GUI）开发。例如，目前最火热的大模型应用，常常是以一个网页界面进行操作和展示，从而免去了控制台或接口操作的复杂性。因此本文总结记录了我接触了解过的GUI开发相关工具和依赖库。

本文将Python开发中的GUI分为了原生GUI和Web GUI两大类，前者指的是以Windows或Android系统程序窗口的形式展示的界面，其界面风格受具体系统影响；后者是基于Web技术展示的网页界面，具有跨平台一致性的好处。此外为了原生GUI能够方便地移植到其他用户的系统进行使用，本文还简单介绍了可执行程序打包工具。

目录

原生GUI

Tkinter

Python中自带的免费开源GUI框架，无需额外安装即可使用。上手容易，使用简单，教程文档众多。具备可视化界面设计器 ^[1]。有较多的主题和美化资源。

wxPython ^[2]

免费开源GUI框架，需额外安装使用。具备可视化界面设计器^{[3] [4]}。上手难度适中。

^[5]

PyQT^[6]

个人与商业双重许可的跨平台GUI框架，需额外安装使用。具备官方可视化界面设计器^[7]。上手难度较高，功能也相对强大。

PySimpleGUI ^[8]

Kivy ^[9]

支持Windows, Linux, MacOS, Android和iOS的跨平台免费开源GUI框架，需额外安装使用。

Web UI

Streamlit ^[10]

非常热门的Python Web UI框架，有非常多组件，具备对Dataframe的可视化支持，针对机器学习应用做了很多优化，社区活跃，界面简约美观，支持云部署。

Gradio ^[11]

与Streamlit相似的非Python Web UI框架，同样非常适合机器学习应用。界面是简约的Hugging Face Style。

NiceGUI ^[12]

相对Streamlit等更简单容易上手，界面非常简约。

可执行程序打包

Pyinstaller ^[13]

免费开源的支持Windows, Linux和MacOS系统的应用程序打包工具，操作简单，应用广泛。

Nuitka ^[14]

个人免费商用付费的跨平台应用程序打包工具，应用程序编译为C文件，代码效率与安全性有提升。

Reference

通过USB Type-C数据线将Redmi K40手机连接至台式机Ubuntu 20.04后，手机切换至访问存储模式，Ubuntu上可以访问手机存储，并正常进行文件传输。标题所述问题的诱发原因可能是将Ubuntu上的文件传输到Redmi上的过程中，数据线松动导致传输意外中断，从而在Ubuntu上访问手机存储时出现如下错误消息并无法正常访问手机存储：

Unhandled error message: The name:1.153 was not provided by any .service files

解决方案

可行方案在askoverflow找到：在Ubuntu 20.04的终端中输入nautilus -q命令，并重新连接手机即可。
该命令使得GNOME桌面自带的文件管理器所打开的多个目录全部退出，相当于重启Ubuntu文件管理器。

此外，尝试重启手机再重新连接的方法无法解决。

1.1 硬件配置

• 处理器：2*AMD EPYC 7773X 64-Core
• 内存：1.5TB
• 显卡：8*NVIDIA GeForce RTX 3090 24GB

1.2 工具环境

• Python：3.10.12
• Anaconda：23.7.4
• 系统：Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-91-generic x86_64)
• IDE：VS Code 1.85.1
• gym：0.26.2
• Pytorch：2.1.2

2. 实现

2.1 Breakout for Atari 2600

Breakout是一款经典的雅达利游戏，也就是我们所熟知的“打砖块”。玩家需要左右移动在屏幕下方的短平板子将一颗不断弹跳的小球反弹回屏幕上方，使其将一块块矩形砖块组成的六行砖块墙面打碎，并防止小球从屏幕底部掉落。在Atari 2600版本的Breakout中，玩家共有5次小球掉落机会，一旦用完就标志游戏结束，每打掉一块砖块得1分，全部打掉则游戏胜利结束。

2.2.2 Double DQN agent

在普通的DQN agent中，只有一个Q-Network用于估计动作价值函数时，存在过估计问题，会导致学习到的策略不稳定。Hasselt等人2015年提出的Double Q-Learning很好缓解了过估计问题[ Deep Reinforcement Learning with Double Q-learning]。其中agent使用主网络和目标网络，主网络用于计算当前状态下每个动作的估计值，而目标网络则用于计算下一个状态的最大动作价值的估计值。
动作选择方面，采用的是典型的epsilon-greedy策略。每步在当前状态下以的概率选择随机动作，以$(1-\epsilon)$的概率通过贪心策略选择主网络给出的最大动作价值对应的动作。
遵循DQN agent的经典设置，本节也采用了经验回放缓冲区 (Experience Replay Buffer)方法，用于在训练时将采样的$(s,a,r,s')$轨迹缓存，并提供随机批量采样供给网络进行小批量学习，可以有效提高训练稳定性和效率[ Playing Atari with Deep Reinforcement Learning]。本节Double DQN方法中的经验缓冲区在缓存满前并不开始训练，具有冷启动特征。
对于主网络的学习，使用SmoothL1Loss，使用目标网络的价值估计结果作为监督，与主网络的价值估计结果计算loss，并对主网络进行梯度反向传播更新参数。

$L_i(\theta _i) = \mathbb{E}_{(s,a,r,s') \sim D} [SmoothL1Loss(y_i,Q(s,a;\theta_i))]$

$y_i=\mathbb{E}_{(s,a,r,s') \sim D} [r+\gamma \max_{a'}Q(s',a';\theta_{c* \lfloor i/c \rfloor })]$

$SmoothL1Loss(x,y)=\frac{1}{n} \sum^n_{j=1} \left\{ \begin{aligned} & 0.5(x_j-y_j)^2 & if |x_j-y_j|<\delta\\ & \delta * |x_j-y_j|-0.5*\delta^2 &otherwise \end{aligned} \right.$

其中$Q(s,a;\theta_i)$是主网络在第$i$次迭代时，在状态$s$下采取动作$a$所估计的动作价值，$y_i$是由奖励$r$和目标网络计算组合得到的目标值。特别地，由于目标网络每隔$c$次迭代与主网络参数同步一次，因此除了初始时相同，其余时候的目标网络参数为上次同步，即第$c* \lfloor i/c \rfloor$次迭代的参数。

2.2.3 模型训练技巧

平衡探索与利用：这是强化学习方法中几乎无法避开的主题，探索和利用的平衡牵涉到模型性能和训练效率，因此需要设置合适地训练策略来使得agent平衡探索和利用。在本节的实现中，探索与利用的平衡主要通过训练时选择动作所使用的epsilon-greedy策略来实现，更具体而言是由参数epsilon控制。本节方法首先在前期一定步数内只进行完全随机动作选择来进行探索。此外还采用了指数衰减式的动态epsilon，使得训练前期可以进行最大化的探索，而在后期则最大化利用agent学习的策略。如下式，在第$t$步时的定义为：

$\epsilon_t = \epsilon _{END} + (\epsilon _{START}-\epsilon _{END})*e^{-t/\epsilon _{DECAY}}$

其中$\epsilon _{START}$为epsilon在起步时的最大值，\epsilon _{END}为衰减结束时的最小值，$\epsilon _{DECAY}$为衰减过程控制系数，$\epsilon _{DECAY}$越大，衰减过程越长。当$t$不断增大，$\epsilon_t $的值将不断逼近。
Gym环境配置：Gym提供了wrapper方式对环境进行修改配置，结合训练效率方面的考虑，需要对环境的可观测状态进行和每幕结束时机进行修改。
在Breakout-v5环境中，可观测状态主要是一幅三通道彩色图像，形状为(210, 160, 3)，值为[0,255]的8bit无符号整数。由于彩色在此游戏中并没有特殊含义，去除后不影响游戏进行，因此首先将将图像的三通道彩色通过OpenCV库处理为单通道灰度图，形状为(210, 160)。进一步地，为了节省存储空间，加快训练速度，将该灰度图压缩变形为(84, 84)的方形图像。此外，在Breakout游戏中，玩家每次有5次机会，当小球第5次掉落时游戏结束，只能重置。为了使得agent学习到不让小球掉落的重要性，训练时将每次小球掉落作为一幕的结束，而本文若无特殊说明，则一幕仍指以第5次小球掉落为结束的一局游戏。

3. 分析评估

本节将分析介绍2.2和2.3小节实现的Double DQN方法在2.1小结介绍的Breakout游戏上的训练和评估表现。
设置超参数如表3-1所示，本节所有针对Double DQN的实验除特殊说明外所有超参数均相同。

首先设置实验定量定性地探究Double DQN的Replay Buffer和fixed target对于模型性能的影响。因此利用单一变量原则，只设置MEMORY_SIZE，分别为2e4、1e5和5e5各进行一次训练；此外只设置TARGET_UPDATE，分别为2e2、1e5和5e3各进行一次训练。上述训练均以100幕（一次生命为一幕）为一个Epoch，得到如下表3-2的6组实验结果（TARGET_UPDATE=1e3的已经在MEMORY_SIZE=1e5实验训练过，因此两个实验的结果相同）。

将上述训练过程的每个Epoch的平均幕奖励绘制为折线图，如图3-1所示。结合图表，可见TARGET_UPDATE越大，模型表现越好，且收敛时间与之并非正相关。当TARGET_UPDATE为5e3时模型收敛最好，收敛速度适中。而MEMORY_SIZE参数同样也是越大模型效果越好，但是其训练耗时同样存在很大的增长，相差5倍的MEMORY_SIZE在收敛时间上相差近1000分钟。综合模型收敛效果和速度，在合适的MEMORY_SIZE下适当增加TARGET_UPDATE可以获得最优的收敛性能。

对Double DQN方法的不同超参数训练分支，均使用历史最优模型进行评估测试，即每个模型进行100幕游戏，并统计每个模型的最高、最低和平均单幕得分，同时绘制每幕得分的箱线图。

*DDQN指Double DQN; TU指TARGET_UPDATE; MS指MEMORY_SIZE;

图3-3描绘了Double DQN在不同超参数下训练的模型在100幕Breakout游戏上评估所得单幕回报的箱线图。从中可以发现Double DQN方法在目标网络更新频率为5e3时的评估效果总体最好，得分上限最高，但相对不稳定；而Double DQN在目标网络更新频率为2e2时的评估效果虽然稳定但总体最差。这一结论与根据图3-1得出的结论基本一致。

4.结论与展望

通过实验过程和结果的分析，我们可以得出以下结论：
Double DQN的超参数选择影响模型性能：在实验中，我们发现Double DQN方法在目标网络更新频率为5e3时表现最好，具有最高的平均幕回报。同时，实验结果还表明MEMORY_SIZE参数的增加可以提升模型性能，但同时也导致训练时间的显著增加。在选择超参数时，需要平衡模型性能和训练效率。
Double DQN在不同超参数下的评估结果有较大差异： 尽管在目标网络更新频率为5e3时Double DQN表现最好，但其评估结果相对不稳定，具有较大的方差。这提示我们在选择超参数时不仅要考虑性能指标的提升，还要关注模型的稳定性。
平衡探索与利用的重要性：在强化学习中，平衡探索和利用是一个重要的主题。在实验中，我们使用了epsilon-greedy策略在DQN中来平衡探索和利用。通过调整epsilon的衰减方式，我们可以在训练的不同阶段进行不同程度的探索和利用，从而提高模型的学习效率。

基于上述的结论和对本次整个实验过程的分析总结，在此对未来的工作和本文不足之处进行以下总结展望。
进一步优化超参数：未来的工作可以通过更系统地调整超参数，尤其是对于Double DQN方法中的其他超参数，来寻找更优的组合，以提高模型性能和训练效率。
尝试其他强化学习算法：除了DQN，还有许多其他强化学习算法可以尝试，例如PPO、DDPG等。对比不同算法在相同环境下的表现，有助于更全面地了解它们的优劣势。
探索更复杂的游戏环境：在本实验中，我们使用了Atari 2600版本的Breakout游戏作为测试环境。未来的工作可以尝试在更复杂的游戏环境中验证模型的性能，以更好地适应现实世界的复杂任务。
深入研究模型稳定性：对于Double DQN在不同超参数下评估结果不稳定的问题，未来的研究可以更深入地探讨如何提高模型的稳定性，以确保在不同训练阶段都能取得良好的性能。

5. 代码

import time
import random
import numpy as np
import ale_py
# import gymnasium as gym
import gym
import torch
import torch.nn as nn
from torch import optim
from torch.autograd import Variable
import torch.nn.functional as F
from collections import deque,namedtuple
from tqdm import tqdm
import matplotlib.pyplot as plt
import cv2
from itertools import count
import random, pickle, os.path, math, glob

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)

Transition = namedtuple('Transion', ('state', 'action', 'next_state', 'reward'))

## 超参数
# epsilon = 0.9
BATCH_SIZE = 32
GAMMA = 0.99
EPS_START = 1
EPS_END = 0.02
EPS_DECAY = 1000000
EPS_RANDOM_COUNT = 50000 # 前50000步纯随机用于探索
TARGET_UPDATE = 1000 # steps
RENDER = False
lr = 1e-4
INITIAL_MEMORY = 10000
MEMORY_SIZE = 10 * INITIAL_MEMORY
n_episode = 100000#10000000


MODEL_STORE_PATH = './models'#+'DQN_pytorch_pong'
modelname = 'DQN_Breakout'
madel_path = MODEL_STORE_PATH + 'DQN_Breakout_episode60.pt'#+ '/' + 'model/'


## 超参数
# epsilon = 0.9
BATCH_SIZE = 32
GAMMA = 0.99
EPS_START = 1
EPS_END = 0.02
EPS_DECAY = 1000000
EPS_RANDOM_COUNT = 50000 # 前50000步纯随机用于探索
TARGET_UPDATE = 1000 # steps
RENDER = False
lr = 1e-4
INITIAL_MEMORY = 10000
MEMORY_SIZE = 10 * INITIAL_MEMORY
n_episode = 100000#10000000


MODEL_STORE_PATH = './models'#+'DQN_pytorch_pong'


class ReplayMemory(object):
    def __init__(self, capacity):
        self.capacity = capacity
        self.memory = []
        self.position = 0

    def push(self, *args):
        if len(self.memory) < self.capacity:
            self.memory.append(None)
        self.memory[self.position] = Transition(*args)
        self.position = (self.position + 1) % self.capacity #移动指针，经验池满了之后从最开始的位置开始将最近的经验存进经验池

    def sample(self, batch_size):
        return random.sample(self.memory, batch_size)# 从经验池中随机采样

    def __len__(self):
        return len(self.memory)
        
class DQN(nn.Module):
    def __init__(self, in_channels=4, n_actions=14):
        """
        Initialize Deep Q Network
        Args:
            in_channels (int): number of input channels
            n_actions (int): number of outputs
        """
        super(DQN, self).__init__()
        self.conv1 = nn.Conv2d(in_channels, 32, kernel_size=8, stride=4)
        # self.bn1 = nn.BatchNorm2d(32)
        self.conv2 = nn.Conv2d(32, 64, kernel_size=4, stride=2)
        # self.bn2 = nn.BatchNorm2d(64)
        self.conv3 = nn.Conv2d(64, 64, kernel_size=3, stride=1)
        # self.bn3 = nn.BatchNorm2d(64)
        self.fc4 = nn.Linear(7*7*64, 512)
        self.head = nn.Linear(512, n_actions)

    def forward(self, x):
        # print(x)
        # print(x.shape)
        # cv2.imwrite("test_x.png",x.cpu().numpy()[0][0])
        x = x.float() / 255
        x = F.relu(self.conv1(x))
        x = F.relu(self.conv2(x))
        x = F.relu(self.conv3(x))
        x = x.view(x.size(0), -1)  # 将卷积层的输出展平
        x = F.relu(self.fc4(x)) #.view(x.size(0), -1)
        out = self.head(x)
        # print(out)
        return out

class DQN_agent():
    def __init__(self,in_channels=4, action_space=[], learning_rate=1e-4, memory_size=10000, epsilon=1, trained_model_path=''):

        self.in_channels = in_channels
        self.action_space = action_space
        self.action_dim = self.action_space.n

        self.memory_buffer = ReplayMemory(memory_size)
        self.stepdone = 0
        self.DQN = DQN(self.in_channels, self.action_dim).to(device)
        self.target_DQN = DQN(self.in_channels, self.action_dim).to(device)
        # 加载之前训练好的模型，没有预训练好的模型时可以注释
        if(trained_model_path != ''):
            self.DQN.load_state_dict(torch.load(trained_model_path))

        self.target_DQN.load_state_dict(self.DQN.state_dict())
        self.optimizer = optim.RMSprop(self.DQN.parameters(),lr=learning_rate, eps=0.001, alpha=0.95)



    def select_action(self, state):

        self.stepdone += 1
        state = state.to(device)
        epsilon = EPS_END + (EPS_START - EPS_END)* \
            math.exp(-1. * self.stepdone / EPS_DECAY)            # 随机选择动作系数epsilon 衰减，也可以使用固定的epsilon
        # epsilon-greedy策略选择动作
        if self.stepdone<EPS_RANDOM_COUNT or random.random()<epsilon:
            action = torch.tensor([[random.randrange(self.action_dim)]], device=device, dtype=torch.long)
        else:
            action = self.DQN(state).detach().max(1)[1].view(1,1)  # 选择Q值最大的动作并view

        return action


    def learn(self):
        # 经验池小于BATCH_SIZE则直接返回
        if self.memory_buffer.__len__()<BATCH_SIZE:
            return
        # 从经验池中采样

        transitions = self.memory_buffer.sample(BATCH_SIZE)
        '''
        batch.state - tuple of all the states (each state is a tensor)                  （BATCH_SIZE * channel * h * w）
        batch.next_state - tuple of all the next states (each state is a tensor) （BATCH_SIZE * channel * h * w）
        batch.reward - tuple of all the rewards (each reward is a float)          （BATCH_SIZE * 1）
        batch.action - tuple of all the actions (each action is an int)               （BATCH_SIZE * 1）
        '''
        
        batch = Transition(*zip(*transitions))

        actions = tuple((map(lambda a: torch.tensor([[a]], device=device), batch.action)))
        rewards = tuple((map(lambda r: torch.tensor([r], device=device), batch.reward)))


        # 判断是不是在最后一个状态，最后一个状态的next设置为None
        non_final_mask = torch.tensor(
            tuple(map(lambda s: s is not None, batch.next_state)),
            device=device, dtype=torch.uint8).bool()

        non_final_next_states = torch.cat([s for s in batch.next_state
                                           if s is not None]).to(device)


        state_batch = torch.cat(batch.state).to(device)
        action_batch = torch.cat(actions)
        reward_batch = torch.cat(rewards)
        # 计算当前状态的Q值
        state_action_values = self.DQN(state_batch).gather(1, action_batch)

        next_state_values = torch.zeros(BATCH_SIZE, device=device)
        next_state_values[non_final_mask] = self.target_DQN(non_final_next_states).max(1)[0].detach()
        expected_state_action_values = (next_state_values * GAMMA) + reward_batch

        loss = F.smooth_l1_loss(state_action_values, expected_state_action_values.unsqueeze(1))
        self.optimizer.zero_grad()
        loss.backward()
        for param in self.DQN.parameters():
            param.grad.data.clamp_(-1, 1)

        self.optimizer.step()

class Trainer():
    def __init__(self, env, agent, n_episode):
        self.env = env
        self.n_episode = n_episode
        self.agent = agent
        # self.losslist = []
        self.rewardlist = []
        self.avg_rewardlist = []

    # 获取当前状态，将env返回的状态通过transpose调换轴后作为状态
    def get_state(self,obs):
        # print(obs.shape)
        state = np.array(obs)
        # state = state.transpose((1, 2, 0)) #将2轴放在0轴之前
        state = torch.from_numpy(state)
        return state.unsqueeze(0)    # 转化为四维的数据结构


 # 训练智能体
    def train(self):
        for episode in range(self.n_episode):

            obs = self.env.reset()

            # print('============obs = self.env.reset()============')
            # state = self.img_process(obs)
            state = np.stack((obs[0], obs[1], obs[2], obs[3]))
            # print(state.shape)
            state = self.get_state(state)
            # print(state.shape)
            episode_reward = 0.0

            # print('episode:',episode)
            for t in count():
                # print(state.shape)
                action = self.agent.select_action(state)
                if RENDER:
                    self.env.render()


                obs,reward,done,_,_ = self.env.step(action)
                episode_reward += reward

                if not done:
                    # next_state = self.get_state(obs)

                    # next_state = self.img_process(obs)
                    next_state = np.stack((obs[0], obs[1], obs[2], obs[3]))
                    next_state = self.get_state(next_state)
                else:
                    next_state = None
                # print(next_state.shape)
                reward = torch.tensor([reward], device=device)

                # 将四元组存到memory中
                '''
                state: batch_size channel h w    size: batch_size * 4
                action: size: batch_size * 1
                next_state: batch_size channel h w    size: batch_size * 4
                reward: size: batch_size * 1
                '''
                self.agent.memory_buffer.push(state, action.to('cpu'), next_state, reward.to('cpu')) # 里面的数据都是Tensor
                # print('========memory_buffer.push=========')
                # print(state.shape)
                # print(action.shape)
                # print(next_state.shape)
                # print(reward)
                # cv2.imwrite("test_1.png",state[0].numpy()[0])
                # cv2.imwrite("test_2.png",state[0].numpy()[1])
                # cv2.imwrite("test_3.png",state[0].numpy()[2])
                # cv2.imwrite("test_4.png",state[0].numpy()[3])
                # time.sleep(0.1)
                
                state = next_state
                # 经验池满了之后开始学习
                if self.agent.stepdone > INITIAL_MEMORY:
                    self.agent.learn()
                    if self.agent.stepdone % TARGET_UPDATE == 0:
                        print('======== target DQN updated =========')
                        self.agent.target_DQN.load_state_dict(self.agent.DQN.state_dict())

                if done:
                    break
            agent_epsilon = EPS_END + (EPS_START - EPS_END)* math.exp(-1. * self.agent.stepdone / EPS_DECAY) 
                 
            print('Total steps: {} \t Episode/steps: {}/{} \t Total reward: {} \t Avg reward: {} \t epsilon: {}'.format(
                self.agent.stepdone, episode, t, episode_reward, episode_reward/t, agent_epsilon))
            
            
            if episode % 20 == 0:
                torch.save(self.agent.DQN.state_dict(), MODEL_STORE_PATH + '/' + "{}_episode{}.pt".format(modelname, episode))
                # print('Total steps: {} \t Episode: {}/{} \t Total reward: {}'.format(self.agent.stepdone, episode, t, episode_reward))

            self.rewardlist.append(episode_reward)
            self.avg_rewardlist.append(episode_reward/t)
            


            self.env.close()
        return

    #绘制单幕总奖励曲线
    def plot_total_reward(self):
        plt.plot(self.rewardlist)
        plt.xlabel("Training epochs")
        plt.ylabel("Total reward per episode")
        plt.title('Total reward curve of DQN on Skiing')
        plt.savefig('DQN_train_total_reward.png')
        plt.show()

    #绘制单幕平均奖励曲线
    def plot_avg_reward(self):
        plt.plot(self.avg_rewardlist)
        plt.xlabel("Training epochs")
        plt.ylabel("Average reward per episode")
        plt.title('Average reward curve of DQN on Skiing')
        plt.savefig('DQN_train_avg_reward.png')
        plt.show()
# reward clip
class ClipRewardEnv(gym.RewardWrapper):
    def __init__(self, env):
        gym.RewardWrapper.__init__(self, env)

    def reward(self, reward):
        """Bin reward to {+1, 0, -1} by its sign."""
        return np.sign(reward)
    
# image frame process
class WarpFrame(gym.ObservationWrapper):
    def __init__(self, env, width=84, height=84, grayscale=True):
        """Warp frames to 84x84 as done in the Nature paper and later work."""
        gym.ObservationWrapper.__init__(self, env)
        self.width = width
        self.height = height
        self.grayscale = grayscale
        if self.grayscale:
            self.observation_space = gym.spaces.Box(low=0, high=255,
                shape=(self.height, self.width, 1), dtype=np.uint8)
        else:
            self.observation_space = gym.spaces.Box(low=0, high=255,
                shape=(self.height, self.width, 3), dtype=np.uint8)

    def observation(self, frame):
        if self.grayscale:
            frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
        frame = cv2.resize(frame, (self.width, self.height), interpolation=cv2.INTER_AREA)
        if self.grayscale:
            frame = np.expand_dims(frame, -1)
        return frame
    
class ScaledFloatFrame(gym.ObservationWrapper):
    def __init__(self, env):
        gym.ObservationWrapper.__init__(self, env)
        self.observation_space = gym.spaces.Box(low=0, high=1, shape=env.observation_space.shape, dtype=np.float32)

    def observation(self, observation):
        # careful! This undoes the memory optimization, use
        # with smaller replay buffers only.
        return np.array(observation).astype(np.float32) / 255.0
    
# Frame Stacking
class FrameStack(gym.Wrapper):
    def __init__(self, env, k):
        """
        Stack k last frames.
        Returns lazy array, which is much more memory efficient.
        See Also
        --------
        baselines.common.atari_wrappers.LazyFrames
        """
        gym.Wrapper.__init__(self, env)
        self.k = k
        self.frames = deque([], maxlen=k)
        shp = env.observation_space.shape
        self.observation_space = gym.spaces.Box(low=0, high=255, shape=(shp[:-1] + (shp[-1] * k,)), dtype=env.observation_space.dtype)

    def reset(self):
        ob = self.env.reset()
        if isinstance(ob, tuple): # obs is tuple in newer version of gym
            ob = ob[0]
        for _ in range(self.k):
            self.frames.append(ob)
        return self._get_ob()

    def step(self, action):
        ob, reward, done, truncated, info = self.env.step(action) 
        if isinstance(ob, tuple):
            ob = ob[0]
        self.frames.append(ob)
        return self._get_ob(), reward, done, truncated, info

    def _get_ob(self):
        assert len(self.frames) == self.k
        return LazyFrames(list(self.frames))
    
class EpisodicLifeEnv(gym.Wrapper):
    def __init__(self, env):
        """Make end-of-life == end-of-episode, but only reset on true game over.
        Done by DeepMind for the DQN and co. since it helps value estimation.
        """
        gym.Wrapper.__init__(self, env)
        self.lives = 0
        self.was_real_done  = True

    def step(self, action):
        obs, reward, done, truncated, info = self.env.step(action)
        self.was_real_done = done
        # check current lives, make loss of life terminal,
        # then update lives to handle bonus lives
        lives = self.env.unwrapped.ale.lives()

        if lives < self.lives and lives > 0:
            # for Qbert sometimes we stay in lives == 0 condition for a few frames
            # so it's important to keep lives > 0, so that we only reset once
            # the environment advertises done.
            done = True
        self.lives = lives
        return obs, reward, done, truncated, info

    def reset(self, **kwargs):
        """Reset only when lives are exhausted.
        This way all states are still reachable even though lives are episodic,
        and the learner need not know about any of this behind-the-scenes.
        """
        if self.was_real_done:
            obs, info = self.env.reset(**kwargs)
        else:
            # no-op step to advance from terminal/lost life state
            obs, _, _, _, info = self.env.step(0)
        self.lives = self.env.unwrapped.ale.lives()
        return obs, info
    
class LazyFrames(object):
    def __init__(self, frames):
        """
        This object ensures that common frames between the observations are only stored once.
        It exists purely to optimize memory usage which can be huge for DQN's 1M frames replay
        buffers.
        This object should only be converted to numpy array before being passed to the model.
        You'd not believe how complex the previous solution was.
        """
        self._frames = frames
        self._out = None

    def _force(self):
        if self._out is None:
            self._out = np.concatenate(self._frames, axis=-1)
            self._frames = None
        return self._out

    def __array__(self, dtype=None):
        out = self._force()
        if dtype is not None:
            out = out.astype(dtype)
        return out

    def __len__(self):
        return len(self._force())

    def __getitem__(self, i):
        return self._force()[..., i]
    
def env_wrap_deepmind(env, episode_life=True, clip_rewards=True, frame_stack=True, scale=True):
    """
    Configure environment for DeepMind-style Atari.
    """
    if episode_life:
        env = EpisodicLifeEnv(env)
    env = WarpFrame(env)
    if scale:
        env = ScaledFloatFrame(env) # scale the frame image
    if clip_rewards:
        env = ClipRewardEnv(env) # clip the reward into [-1,1]
    if frame_stack:
        env = FrameStack(env, 4) # stack 4 frame to replace the RGB 3 chanels image
    return env
    
# create environment and warp it into DeepMind style
env = env_wrap_deepmind(gym.make("ALE/Breakout-v5"), episode_life=True, clip_rewards=False, frame_stack=True, scale=False)
action_space = env.action_space

# use 4 stacked chanel
agent = DQN_agent(in_channels = 4, action_space = action_space, learning_rate = lr, memory_size=MEMORY_SIZE)

trainer = Trainer(env, agent, n_episode) # 这里应该用超参数里的n_episode
trainer.train()
trainer.plot_total_reward()
# trainer.plot_avg_reward()

# save total reward list
np.save('total_reward_list_breakout_1e5.npy',np.array(trainer.rewardlist))
np.save('avg_reward_list_breakout_1e5.npy',np.array(trainer.avg_rewardlist))

print('The training costs {} episodes'.format(len(trainer.rewardlist)))

print('The max episode reward is {}, at episode {}'.format(
    max(trainer.rewardlist),
    trainer.rewardlist.index(max(trainer.rewardlist))
    ))

# 合并5 episodes为1episode
assert(len(trainer.rewardlist)%5==0)
reshaped_reward_array = np.array(trainer.rewardlist).reshape((int(len(trainer.rewardlist)/5), 5))
# 沿着第二个维度求和
summed_rewawrd_array = reshaped_reward_array.sum(axis=1)


print('Now takes 5 episodes as 1, the training cost {} complete episodes'.format(len(summed_rewawrd_array)))

print('The max episode return is {}, at episode {}'.format(
    max(summed_rewawrd_array),
    np.where(summed_rewawrd_array == max(summed_rewawrd_array))
    ))

# 合并200 episodes为1episode
assert(len(summed_rewawrd_array)%200==0)
reshaped_reward_array_200 = summed_rewawrd_array.reshape((int(len(summed_rewawrd_array)/200), 200))
# 沿着第二个维度求和
summed_rewawrd_array_200 = reshaped_reward_array_200.sum(axis=1)
avg_rewawrd_array_200 = summed_rewawrd_array_200/200.0

np.save('avg_rewawrd_array_200_breakout_1e5.npy',avg_rewawrd_array_200)

print('The following graph takes 1000 games as 1 epoch where 5 games equals to 1 episode as stated before')

max_idx = np.argmax(avg_rewawrd_array_200)
max_y = max(avg_rewawrd_array_200)
print('The best average return per epoch is {}, at epoch {}'.format(max_idx,max_y))

plt.figure(figsize=(10,6))
plt.plot(avg_rewawrd_array_200,marker='o',markersize=4)
plt.xlabel("Training Epochs",fontsize=12)
plt.ylabel("Average Reward per Episode",fontsize=12)

plt.scatter(max_idx, max_y, color='red', s=60)
plt.annotate(f'max avg return: ({max_idx}, {max_y:.2f})', xy=(max_idx, max_y), xytext=(max_idx-40, max_y-1),
             arrowprops=dict(facecolor='red', shrink=0.05))
# plt.title('Average Reward of DQN on Breakout')
plt.savefig('DQN_train_total_reward_Breakout.svg')
plt.show()

Reference:
https://blog.csdn.net/libenfan/article/details/117395547
https://zhuanlan.zhihu.com/p/80185628

本文旨在深入探讨强化学习在生成式预训练语言模型中的应用，特别是在对齐优化、提示词优化和经验记忆增强提示词等方面的具体实践。通过对现有研究的综述，我们将揭示强化学习在提高生成式语言模型性能和人类对话交互的关键作用。虽然这些应用展示了巨大的潜力，但也将讨论现有方法的挑战和可能的未来发展方向。

在当今人工智能领域，生成式预训练语言模型的崛起成为自然语言处理和文本生成的一项重大突破。这一技术通过在大规模文本数据上进行预训练，使得模型能够学到语言的深层次结构和模式，从而具备出色的生成能力。生成式预训练模型的出现引领了自然语言处理的新潮流，但也伴随着一系列挑战，如模型的价值对齐、生成结果不可控、难以实现交互式学习与经验利用等问题。
强化学习，作为一种通过智能体与环境的交互来学习最优行为的方法，近年来在生成式预训练语言模型中得到了广泛关注。将强化学习引入生成式模型的训练过程，不仅可以提高模型生成结果的质量，还能够使模型更好地适应特定任务和领域。
本文的研究目的在于深入探讨强化学习在生成式预训练语言模型中的应用，着眼于理解其在不同阶段的作用机制和效果。通过对该结合应用的系统研究，我们旨在揭示强化学习如何优化模型性能、对齐人类价值观、以及优化和增强提示词等方面发挥的关键作用。
本文将围绕生成式预训练语言模型与强化学习的结合展开，结构安排如下：第二章将简要介绍生成式人工智能与预训练微调范式的基本概念，为读者提供理论基础和背景知识。第三章将详细阐述强化学习在生成式预训练语言模型中的应用，包括对齐优化、提示词优化、经验记忆增强等方面的研究与实践。

2. 生成式预训练语言模型介绍

生成式预训练语言模型作为人工智能领域的热点之一，其在自然语言生成和理解方面的表现引起了广泛关注。这些模型的背后通常是庞大的神经网络结构，其中使用了生成式人工智能和迁移学习的思想，为其在不同任务上的灵活性和性能提供了基础。

2.1 生成式人工智能

生成式人工智能是一种强调模型能够自主创造新内容和信息的人工智能范式。与传统的判别式人工智能不同，生成式人工智能不仅能够理解输入数据的特征，还可以生成具有相似特征的全新数据。这使得生成式人工智能在语言生成、图像创作、音乐合成等领域表现出色。其核心思想是通过学习数据的分布和模式，使模型能够生成与训练数据类似但又不完全相同的新样本，从而展现出一定的创造性和想象力。生成式人工智能的发展在许多应用中取得了显著成就，为人工智能的创新和进步提供了新的可能性。

2.2 迁移学习

大模型中常说的“预训练-微调”，其实是一种迁移学习下的范式，这一思想的核心在于将从一个领域中获得的知识应用到另一个相关领域，从而提升目标领域的学习性能。这种方法尤为重要，特别是在目标领域的数据相对稀缺或难以获取的情况下。通过利用先前在一个领域上获取的知识，模型能够更有效地适应新的任务或领域，为整体学习性能的改善提供了有力支持。这种迁移学习的范式为解决数据稀缺和难以获得的问题提供了一种实用而有效的方法。

图2-1 预训练范式示意图

3.强化学习在生成式预训练语言模型中的应用

强化学习作为一种强调在特定环境中通过试错学习来最大化奖励的学习范式，在生成式预训练语言模型中展现出了强大的潜力。本节将深入研究强化学习在生成式预训练语言模型中的应用，从预训练、微调到推理等不同阶段，揭示强化学习在优化模型性能、对齐人类价值观以及优化提示词等方面的关键作用。通过对相关方法和技术的介绍，我们将从多个方面了解强化学习如何推动生成式预训练语言模型的发展，为生成式人工智能领域的未来带来更多可能性。

3.1对齐优化

我们知道大语言模型 (Large Language Model, LLM)在经历预训练（Pre-Training）和有监督微调（Supervised Fine-Tuning, SFT）后，由于自监督预训练任务通常只是简单的词预测任务，因此仍然普遍存在忠实性、伦理道德、数据安全等多方面的缺陷，好似一个口无遮拦的模型。上述这些问题缺陷恰恰较难以用严格的数学形式进行描述定义，因为其通常是隐含在人类的价值观中的一种主观偏好。因此一个用于与人类交互的生成式预训练语言模型需要进行对齐 (Alignment)，通俗地说，是将上游基础模型和人类意图价值这两块长短不一的板子给对齐了，得到一个更符合人类价值观和意图的大语言模型。
对齐的方法可以分为生成器优化对齐（generator improvement）和推理时附加对齐（inference-time add-on）^[1]，主要区别在与前者是需要进行参数更新的，而后者作用于推理阶段故不用进行参数更新。

3.1.1 生成器优化对齐

生成器优化对齐中的生成器，指的是用于生成自然语言文本序列的模型，多数情况下都是预训练Transformer模型。而生成器优化对齐，指在训练阶段对模型进行参数微调对齐以达到优化生成结果、与人类对齐的目的。举例来说其实最常见的生成器优化方法就是有监督微调（Supervised Fine-Tuning, SFT）和人类反馈强化学习（Reinforcement Learning with Human Feedback, RLHF）。本小节将主要介绍基于人类反馈强化学习的对齐微调。
有监督微调依赖于有标签文本数据，数量和质量上存在限制，难以使模型高效学习到人类偏好。而ChatGPT发布之初能如此惊艳的一大功臣，就是基于人类反馈强化学习的微调对齐方法。
人类反馈强化学习的起源可以追溯到收录于NIPS 2017的Christiano等人所著的“Deep reinforcement learning from human preferences”^[2], 其中RLHF被用于利用人类反馈优化训练一个模拟环境中的火柴棒小人做后空翻。这篇文章很好地介绍了如何通过建模人类偏好让强化学习agent学习到如何做一个人类认为好的后空翻，为后来RLHF被用于大语言模型奠定了基础。
OpenAI在接连发布三代GPT后，发表了InstructGPT的论文^[3]，公开了RLHF这项OpenAI的独门秘籍，下面就简单介绍一下这篇文章的RLHF方法。
文章在GPT3的基础模型上进行微调，微调分三步走，如下图：

图3-1 InstructGPT中的人类反馈强化学习方法的三个主要步骤

其中第二步的奖励模型（Reward Model, RM）是一个从6B参数的SFT后的GPT模型开始进行梯度下降训练的，其最后的unembedding层被移除了。之所以不用175B参数的是因为不稳定，文章附件有介绍这一原因。
RM的训练数据收集很有巧思，因为他们并不是只给两个输出结果要求人类标签员去选一个好的，而是针对一个prompt生成K个结果，并要求标签员对这些结果从好到坏排序，那么这样一次排序任取其中两个结果排列组合可以产生$C_2^K$个comparison pair，形如$(x,y_w,y_l)$，其中是输入prompt，$y_w$是比$y_l$更好的一个输出结果（我猜下标w代表win，l代表lose）。
RM训练时的loss函数为：

$loss(\theta)=\frac{1}{C_2^K} E_{(x,y_w,y_l) \sim D} [\log{(\sigma(r_\theta(x,y_l)))}]$

其中$r_\theta(x,y)$是奖励模型的标量输出，$D$是整个人类comparison pair数据集，E_{(x,y_w,y_l) \sim D表示从数据分布$D$中采样一个提示词输入$x$及其对应的一好一坏两个生成结果$y_w$和$y_l$，并计算期望。
简而言之，奖励模型的训练采用了对比学习的思想，希望RM扩大正样本和负样本间的得分差异，且希望让正样本得分高于负样本。
第三步的强化学习阶段非常关键，其混合了PPO梯度和预训练梯度，并将这种混合梯度训练的模型称为PPO-ptx, 具体来说RL训练时的混合目标函数为：

$objective(\phi)=E_{(x,y)\sim D_{\pi_{\phi}^{RL}}} [r_\theta(x,y)-\beta \log (\pi_{\phi}^{RL} (y | x) / \pi^{SFT} (y | x))] + \gamma E_{x \sim D_{pretrain}} [log (\pi_{\phi}^{RL} (x))]$

其中$\pi_{\phi}^{RL}$是要学习的RL策略，$\pi^{SFT}$是有监督微调过的模型，$D_{pretrain}$是预训练数据集分布，$\beta$是KL散度奖励系数，$\gamma$是预训练损失系数。
简而言之，该目标函数希望RL模型可以最大化来自RM的奖励，最小化RL策略和SFT模型的KL散度（即希望RL策略不偏离SFT模型，从而提高稳定性和鲁棒性），并且最后还将预训练梯度也纳入考虑，希望提高稳定性和训练效率。

3.1.2 推理时附加对齐

了解完大家最熟悉的基于RLHF的微调对齐，这里介绍一个作用于推理阶段，不用更新模型参数的对齐方式：语言模型受控解码，由Google Research的Mudgal^[1:1]等人发表。
这篇论文提出了一种名为受控解码（Controlled Decoding，简称CD）的新型off-policy强化学习方法，用于控制语言模型的自回归生成过程，使其朝向高奖励结果的推理路径进行推理。CD通过一个名为前缀评分器（prefix scorer）的价值函数来解决离策略强化学习问题，该前缀评分器在推理阶段用于引导生成过程朝向更高奖励结果。文章中强化学习的应用概括如下：
问题建模：作者将控制语言模型生成过程的问题建模为一个离策略强化学习问题。在这个问题中，目标是学习一个解码策略（decoding policy），使得在给定上下文（prompt）的情况下，生成的文本序列能够获得更高的奖励（reward）。
价值函数设计：作者提出了一个名为前缀评分器（prefix scorer）的价值函数，用于预测从当前部分解码的响应继续解码时的预期奖励。这个前缀评分器可以在离策略数据上进行训练，从而避免了在线策略学习中的样本效率问题。
推理策略：在推理阶段，作者提出了两种使用前缀评分器的策略。一种是逐个标记（token-wise）采样，另一种是分块（block-wise）采样和重排。这两种策略都可以在不改变训练时的模型结构的情况下，实现对生成过程的有效控制。
多目标优化：作者展示了如何通过调整前缀评分器的权重，实现在多个奖励目标之间的权衡。这使得CD方法可以解决多目标强化学习问题，而无需增加额外的复杂性。

3.2 提示词优化

提示词 (prompt)往往是一段自然语言文本序列，在研究中其还有连续形式，即一个多维向量。提示词用于输入到生成式预训练语言模型并引导其生成结果。经验表明，经验和研究表明，不同提示词输入到生成式预训练语言模型中会导致显著的输出结果差异。下面介绍利用强化学习对提示词进行最优搜索和增强的相关研究。

3.2.1 提示词优化搜索

文本形式的提示词由于其离散性质，其优化非常困难。针对提示词优化搜索的研究中，相关研究可根据提示词的连续或离散而分别划分为软提示 (Soft Prompt, Continuous Prompt)和硬提示 (Hard Prompt, or Discrete Prompt)。其中软提示需要访问语言模型的梯度，而算梯度需要很高的计算成本 (有时梯度甚至并不可用)，且软提示的优化结果不具有普适性，即一个模型上优化的提示词无法在别的模型上适用。此外由于软提示本身是多维向量的数学形式，天然难以被人类阅读和理解。针对上述软提示缺点，硬提示作为离散文本形式，以无需访问模型梯度、易于人类理解、普适性高等优点被人们关注和研究。
然而，硬提示由于其离散性质，其优化相比连续性的软提示具有更大的困难。有研究为了解决这一困难，将离散文本提示优化问题建模为强化学习问题^[4]。其目标是在不需要访问预训练语言模型梯度的情况下优化提示词。代理通过策略来逐步选择提示的每个词，并最大化根据输出结果计算的奖励。该研究使用了soft Q-Learning (SQL)方法的on-policy组件。其目标是最大化奖励，即

$\max_{\theta}R(\bold{y}_{LM}(\bold{\hat{z},\bold{x}})), \bold{z} \sim \prod_{t=1}^T \pi_{\bold{\theta}}(z_t | \bold{z}_{<t})$

其中$\bold{y}_{LM}(\bold{\hat{z},\bold{x}})$是预语言模型以$\bold{x}$为输入，以$\bold{\hat{z}}$为提示词时，预语言模型的输出结果。而$R(\bold{y})$是奖励函数，文章中针对不同的下游自然语言处理任务有不同的$R(\bold{y})$。例如对于文本分类 (text classification)任务，文章中的奖励函数为：

$R(\bold{x},c)=\lambda_1^{1-Correct}\lambda_2^{Correct} GAP_{\bold{z}}(c)$

对于无监督文本风格迁移，其奖励函数为：

$R(\bold{x},\bold{y},s)=Content(\bold{x},\bold{y})+Style(\bold{y},s)$

此外，为了训练效率和稳定性，该研究还对不同的下游任务特定的奖励函数进行了统一处理，即提出z-score的奖励函数后处理：

$z-score(\bold{z},\bold{x})=\frac{R_{\bold{x}}(\bold{z})-mean_{z'\in Z(\bold{x})}R_{\bold{x}}(\bold{Z'})}{stdev_{z'\in Z(\bold{x})}R_{\bold{x}}(\bold{Z'})}$

其中$R_{\bold{x}}(\bold{z})$是$R(\bold{y}_{LM})$的缩写，stdev表示样本标准差。
对于带参数$\bold{\theta}$的策略网络，作者通过将一个简单的下游任务特定的MLP层插入到冻结参数的用于生成提示词的预训练语言模型中进行实现，更具体地，是插入到LLM的输出头前。这使得该方法具有很好的普适性和的易用性，无需额外构建大型的策略网络。
上述这项研究中的提示词优化范式还可以应用到其他领域，显示出了提示词优化范式的应用广泛性。有研究^[5]将基于策略梯度强化学习的离散提示词优化范式应用到了分子信息学领域，基于生成式预训练Transformer (GPT)模型生成具有预期性质的药物分子的分子式SMILES字符串。不同于前文的离散提示词优化范式，此研究的提示词优化其实是需要访问梯度的。其奖励函数的设计特定于数据集类型，目标函数和策略梯度基于经典策略梯度方法，并无改动。
该研究考虑以一种名为SMILES的字符串作为药物分子的分子式表达，将具有空间结构等复杂形态的药物分子以计算机可以处理的字符串形式表示，进而可以利用语言建模任务在大量药物分子的SMILES字符串上进行预训练，解决了药物分子的分子式生成问题。

图3-2环丙沙星的 SMILES表示过程(最下方为SMILES字符串)^[5:1]

而为了生成更符合特定性质，例如抗癌的药物性质的分子式，该研究利用离散提示词优化来控制药物分子式预训练模型的输入提示词，使得生成的分子式更贴合预期性质，如图3-3。
针对生成结果，该研究利用了药物分子的领域相关指标进行评估，包括Validity, Novelty, Diversity, QED, SAS. 分别评估生成分子式的有效性 (要符合基本的理化规则)、新颖性 (和已有的药物至少不能重复)、多样性、类药性 (在一定理化参数空间的化合物成为类药物 ，即drug-like)、分子易合成性 (根据分子结构复杂性计算是否难以在现实合成)。

图3-3 基于策略梯度强化学习的药物分子GPT的提示优化与分子式生成过程

3.2.2 经验记忆增强提示词

人类可以利用过往的经验和记忆中进行学习，现有LLMs在部署后，受限于参数更新带来的计算量，从而较难从交互中有效通过参数更新来学习新的经验。为了实现有效地交互式学习，常见的方法是用RL对模型进行微调，但微调需要大量计算，难以部署和长期实现交互式学习。也有方法无需进行微调，而是利用LLMs上下文学习能力直接将历史经验嵌入到提示词。这种方法需要微调才能利用经验，且受LLMs输入长度限制。
基于上述背景，有研究^[6]考虑利用强化学习方法进行提示词记忆增强。该研究提出了强化学习与经验记忆(RLEM)的方法。该方法通过强化学习的过程更新外部持久化的经验记忆，而不是调整LLM的参数。在LLM交互时，利用观测到的状态去检索存储在经验记忆中的若干经验，即一组观察值Ox、动作Ax和对应的Q值估计Qx . LLM再根据本次交互的观测 、上次交互得到的反馈以及检索到的经验决定接下来在环境中的动作，并与环境交互后得到相应奖励反馈。如此，本次交互产生一个新的元组并存储到经验记忆中，有些类似Replay Buffer。

4. 参考文献

0.1 目录

[TOC]

0.2 论文、代码、数据集与阅读笔记

• paper: https://openreview.net/forum?id=LzQQ89U1qm_

• codes: https://github.com/thuml/Anomaly-Transformer

• datasets(不完整): https://drive.google.com/drive/folders/1gisthCoE-RrKJ0j3KPV7xiibhHWT9qRm?usp=sharing

• datasets()

• my review:

1. Startup

初始条件介绍和必要准备工作，代码来自https://github.com/thuml/Anomaly-Transformer，论文数据来自作者提供的Google Cloud

初始环境信息

显卡：耕升GTX 1660 6GB

CPU：Intel i7-10700 2.90GHz

内存：16GB DDR4

系统：Ubuntu 20.04.1 内核5.15.0-89-generic (非虚拟机)

CUDA：release 11.5

显卡驱动信息：

Thu Nov 30 16:24:15 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 1660        Off | 00000000:01:00.0  On |                  N/A |
| 77%   78C    P0              96W / 120W |   5636MiB /  6144MiB |     99%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

安装Pytorch 1.8.0

在已经安装conda 22.9.0，并用conda创建了python 3.6虚拟环境（环境命名为Anomaly-Transformer）的前提下，尝试使用conda安装pytorch（失败，网络原因导致较大文件下载失败，conda参照了这个链接换清华源仍然无法解决）

conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge

因此尝试pip安装（成功，可以正常使用import torch）

pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html

一开始比较困扰的就是CUDA版本对应的问题，但后面看似乎cudatoolkit版本和机器安装的CUDA版本不用完全对应也能安装并使用上pytorch.

2. 论文实验复现

将论文提出方法应用到SDM、PSM、MSL、SMAP、SWaT共计五个数据集，复现文章评估数据。

2.1 SMD

作为第一个登场的脚本，很明显是要报一堆大大小小的错的。还好问题都不大，搞定了后面就畅通无阻了。

首次运行SMD.sh

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ bash ./scripts/SMD.sh
./scripts/SMD.sh: line 2: $'\r': command not found
Traceback (most recent call last):
  File "main.py", line 7, in <module>
    from solver import Solver
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 9, in <module>
    from data_factory.data_loader import get_loader_segment
  File "/media/username/folder/Dev/Anomaly-Transformer/data_factory/data_loader.py", line 11, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'
Traceback (most recent call last):
  File "main.py", line 7, in <module>
    from solver import Solver
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 9, in <module>
    from data_factory.data_loader import get_loader_segment
  File "/media/username/folder/Dev/Anomaly-Transformer/data_factory/data_loader.py", line 11, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'

问题定位与解决：可见问题主要都是package缺失，缺失package和安装命令如下：

• sklearn: 命令行输入pip install scikit-learn
• pandas : 命令行输入pip install pandas

再次运行SMD.sh

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ bash ./scripts/SMD.sh
./scripts/SMD.sh: line 2: $'\r': command not found
------------ Options -------------
anormly_ratio: 0.5
batch_size: 256
data_path: dataset/SMD
dataset: SMD
input_c: 38
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 10
output_c: 38
pretrained_model: None
win_size: 100
-------------- End ----------------
Traceback (most recent call last):
  File "main.py", line 52, in <module>
    main(config)
  File "main.py", line 18, in main
    solver = Solver(vars(config))
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 74, in __init__
    dataset=self.dataset)
  File "/media/username/folder/Dev/Anomaly-Transformer/data_factory/data_loader.py", line 204, in get_loader_segment
    dataset = SMDSegLoader(data_path, win_size, step, mode)
  File "/media/username/folder/Dev/Anomaly-Transformer/data_factory/data_loader.py", line 166, in __init__
    data = np.load(data_path + "/SMD_train.npy")
  File "/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/numpy/lib/npyio.py", line 416, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'dataset/SMD/SMD_train.npy'
------------ Options -------------
anormly_ratio: 0.5
batch_size: 256
data_path: dataset/SMD
dataset: SMD
input_c: 38
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 38
pretrained_model: 20
win_size: 100
-------------- End ----------------
Traceback (most recent call last):
  File "main.py", line 52, in <module>
    main(config)
  File "main.py", line 18, in main
    solver = Solver(vars(config))
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 74, in __init__
    dataset=self.dataset)
  File "/media/username/folder/Dev/Anomaly-Transformer/data_factory/data_loader.py", line 204, in get_loader_segment
    dataset = SMDSegLoader(data_path, win_size, step, mode)
  File "/media/username/folder/Dev/Anomaly-Transformer/data_factory/data_loader.py", line 166, in __init__
    data = np.load(data_path + "/SMD_train.npy")
  File "/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/numpy/lib/npyio.py", line 416, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'dataset/SMD/SMD_train.npy'

问题定位与解决：问题主要为数据集文件找不到：'dataset/SMD/SMD_train.npy'，根据该提示将下载的数据集文件(Tsinghua Cloud or Google Cloud)整理后按照如下结构存放：

Anomoly_Transformer/
├── dataset/
│     ├── SMD/
│     │    ├── SMD_test.npy
│     │    ├── SMD_train.npy
│     │    └── ......
│     ├── PSM/
│     │    ├── test.csv
│     │    ├── train.csv
│     │    └── ......
│     ├── MSL/
│     │    ├── MSL_test.npy
│     │    └── ......
│     └── SMAP/
│          ├── SMAP_test.npy
│          └── ......
└── ......

第三次及之后运行SMD.sh

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ bash ./scripts/SMD.sh
./scripts/SMD.sh: line 2: $'\r': command not found
------------ Options -------------
anormly_ratio: 0.5
batch_size: 256
data_path: dataset/SMD
dataset: SMD
input_c: 38
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 10
output_c: 38
pretrained_model: None
win_size: 100
-------------- End ----------------
======================TRAIN MODE======================
Traceback (most recent call last):
  File "main.py", line 52, in <module>
    main(config)
  File "main.py", line 21, in main
    solver.train()
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 161, in train
    self.win_size)).detach())) + torch.mean(
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 13, in my_kl_loss
    res = p * (torch.log(p + 0.0001) - torch.log(q + 0.0001))
RuntimeError: CUDA out of memory. Tried to allocate 80.00 MiB (GPU 0; 5.79 GiB total capacity; 3.97 GiB already allocated; 49.75 MiB free; 4.11 GiB reserved in total by PyTorch)
------------ Options -------------
anormly_ratio: 0.5
batch_size: 256
data_path: dataset/SMD
dataset: SMD
input_c: 38
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 38
pretrained_model: 20
win_size: 100
-------------- End ----------------
Traceback (most recent call last):
  File "main.py", line 52, in <module>
    main(config)
  File "main.py", line 23, in main
    solver.test()
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 210, in test
    os.path.join(str(self.model_save_path), str(self.dataset) + '_checkpoint.pth')))
  File "/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/serialization.py", line 579, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/serialization.py", line 230, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/serialization.py", line 211, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/SMD_checkpoint.pth'

问题定位与解决：

• 问题1：CUDA out of memory: RuntimeError: CUDA out of memory. Tried to allocate 80.00 MiB (GPU 0; 5.79 GiB total capacity; 3.97 GiB already allocated; 49.75 MiB free; 4.11 GiB reserved in total by PyTorch)，初步认为是CUDA显存分配问题，模型所需显存没有得到满足。
• 问题2：模型checkpoint文件缺失：由于训练未成功进行，使得模型checkpoint文件沒有成功生成，从而在test阶段想要读取模型时无法读取。

因此应该围绕CUDA显存分配优化进行研究。

解决过程：

• 在博文中找到方案1：减小batch_size
• 尝试将启动命令中训练与测试的batch_size均从256改为128，然后重新运行./scripts/SMD.sh
• 仍然爆显存：RuntimeError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 5.79 GiB total capacity; 3.89 GiB already allocated; 82.94 MiB free; 4.05 GiB reserved in total by PyTorch)
• 尝试修改batch_size为64，然后重新运行./scripts/SMD.sh
• 问题依旧，尝试修改batch_size为32，然后重新运行./scripts/SMD.sh
• 成功开始训练，迹象为观察到如下训练过程打印的epoch信息：

======================TRAIN MODE======================
        speed: 0.1335s/iter; left time: 283.1503s
        speed: 0.1289s/iter; left time: 260.5845s
Epoch: 1 cost time: 29.139591455459595
Epoch: 1, Steps: 222 | Train Loss: -40.3103769 Vali Loss: -46.1086967 
Validation loss decreased (inf --> -46.108697).  Saving model ...
Updating learning rate to 0.0001
        speed: 0.2505s/iter; left time: 475.7060s
        speed: 0.1302s/iter; left time: 234.2822s
Epoch: 2 cost time: 28.97248649597168

在经过四个epoch后停止，进入test阶段，并输出了最终实验结果：

Threshold : 0.06388568006455485
pred:    (708400,)
gt:      (708400,)
pred:  (708400,)
gt:    (708400,)
Accuracy : 0.9926, Precision : 0.8927, Recall : 0.9329, F-score : 0.9124 

完整的训练、测试过程控制台输出如下：

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ bash ./scripts/SMD.sh
------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/SMD
dataset: SMD
input_c: 38
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 10
output_c: 38
pretrained_model: None
win_size: 100
-------------- End ----------------
======================TRAIN MODE======================
        speed: 0.1335s/iter; left time: 283.1503s
        speed: 0.1289s/iter; left time: 260.5845s
Epoch: 1 cost time: 29.139591455459595
Epoch: 1, Steps: 222 | Train Loss: -40.3103769 Vali Loss: -46.1086967 
Validation loss decreased (inf --> -46.108697).  Saving model ...
Updating learning rate to 0.0001
        speed: 0.2505s/iter; left time: 475.7060s
        speed: 0.1302s/iter; left time: 234.2822s
Epoch: 2 cost time: 28.97248649597168
Epoch: 2, Steps: 222 | Train Loss: -47.4852449 Vali Loss: -46.8629997 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 0.2555s/iter; left time: 428.5185s
        speed: 0.1307s/iter; left time: 206.1918s
Epoch: 3 cost time: 29.593196392059326
Epoch: 3, Steps: 222 | Train Loss: -47.8205990 Vali Loss: -47.0798451 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
        speed: 0.2540s/iter; left time: 369.4981s
        speed: 0.1327s/iter; left time: 179.8330s
Epoch: 4 cost time: 29.744439840316772
Epoch: 4, Steps: 222 | Train Loss: -47.9206608 Vali Loss: -47.1366013 
EarlyStopping counter: 3 out of 3
Early stopping
------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/SMD
dataset: SMD
input_c: 38
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 38
pretrained_model: 20
win_size: 100
-------------- End ----------------
======================TEST MODE======================
/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.06388568006455485
pred:    (708400,)
gt:      (708400,)
pred:  (708400,)
gt:    (708400,)
Accuracy : 0.9926, Precision : 0.8927, Recall : 0.9329, F-score : 0.9124 

2.2 PSM

首次运行PSM.sh

成功结束，测试结果摘要如下：

======================TEST MODE======================
Threshold : 0.0011754722148179996
pred:    (87800,)
gt:      (87800,)
pred:  (87800,)
gt:    (87800,)
Accuracy : 0.9882, Precision : 0.9697, Recall : 0.9883, F-score : 0.9789

完整执行过程如下：

nomaly-Transformer$ bash ./scripts/PSM.sh
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/PSM
dataset: PSM
input_c: 25
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 25
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (87841, 25)
train: (132481, 25)
test: (87841, 25)
train: (132481, 25)
test: (87841, 25)
train: (132481, 25)
test: (87841, 25)
train: (132481, 25)
======================TRAIN MODE======================
        speed: 0.1336s/iter; left time: 1644.4129s
        speed: 0.1275s/iter; left time: 1556.9812s
        speed: 0.1276s/iter; left time: 1545.0969s
        speed: 0.1277s/iter; left time: 1534.3159s
        speed: 0.1279s/iter; left time: 1524.0642s
        speed: 0.1279s/iter; left time: 1510.8930s
        speed: 0.1279s/iter; left time: 1498.2409s
        speed: 0.1279s/iter; left time: 1485.3983s
        speed: 0.1279s/iter; left time: 1472.8507s
        speed: 0.1280s/iter; left time: 1460.4547s
        speed: 0.1280s/iter; left time: 1448.4515s
        speed: 0.1282s/iter; left time: 1437.4174s
        speed: 0.1284s/iter; left time: 1427.1299s
        speed: 0.1284s/iter; left time: 1414.0394s
        speed: 0.1284s/iter; left time: 1401.1101s
        speed: 0.1285s/iter; left time: 1389.8639s
        speed: 0.1284s/iter; left time: 1375.8026s
        speed: 0.1283s/iter; left time: 1361.4072s
        speed: 0.1284s/iter; left time: 1349.9602s
        speed: 0.1284s/iter; left time: 1336.6942s
        speed: 0.1282s/iter; left time: 1322.4420s
        speed: 0.1283s/iter; left time: 1310.0931s
        speed: 0.1283s/iter; left time: 1297.5508s
        speed: 0.1282s/iter; left time: 1283.9510s
        speed: 0.1283s/iter; left time: 1271.7995s
        speed: 0.1283s/iter; left time: 1259.0883s
        speed: 0.1283s/iter; left time: 1245.8020s
        speed: 0.1283s/iter; left time: 1233.0175s
        speed: 0.1283s/iter; left time: 1220.1547s
        speed: 0.1283s/iter; left time: 1207.7776s
        speed: 0.1284s/iter; left time: 1195.7177s
        speed: 0.1282s/iter; left time: 1181.4120s
        speed: 0.1283s/iter; left time: 1168.8951s
        speed: 0.1282s/iter; left time: 1155.4520s
        speed: 0.1283s/iter; left time: 1143.4009s
        speed: 0.1284s/iter; left time: 1131.5084s
        speed: 0.1283s/iter; left time: 1117.7446s
        speed: 0.1282s/iter; left time: 1104.4219s
        speed: 0.1282s/iter; left time: 1091.2835s
        speed: 0.1283s/iter; left time: 1078.9449s
        speed: 0.1282s/iter; left time: 1065.9970s
Epoch: 1 cost time: 531.1504812240601
Epoch: 1, Steps: 4137 | Train Loss: -48.0091480 Vali Loss: -48.8543076 
Validation loss decreased (inf --> -48.854308).  Saving model ...
Updating learning rate to 0.0001
        speed: 1.2588s/iter; left time: 10290.9493s
        speed: 0.1282s/iter; left time: 1035.4373s
        speed: 0.1282s/iter; left time: 1022.6818s
        speed: 0.1283s/iter; left time: 1010.5991s
        speed: 0.1282s/iter; left time: 996.7476s
        speed: 0.1283s/iter; left time: 984.5289s
        speed: 0.1282s/iter; left time: 971.1445s
        speed: 0.1282s/iter; left time: 958.5275s
        speed: 0.1282s/iter; left time: 945.7043s
        speed: 0.1283s/iter; left time: 933.1298s
        speed: 0.1282s/iter; left time: 919.9409s
        speed: 0.1282s/iter; left time: 907.2530s
        speed: 0.1282s/iter; left time: 894.4075s
        speed: 0.1282s/iter; left time: 881.5417s
        speed: 0.1283s/iter; left time: 869.0542s
        speed: 0.1283s/iter; left time: 856.4396s
        speed: 0.1284s/iter; left time: 844.1700s
        speed: 0.1283s/iter; left time: 830.4890s
        speed: 0.1282s/iter; left time: 816.9863s
        speed: 0.1283s/iter; left time: 804.9645s
        speed: 0.1282s/iter; left time: 791.7809s
        speed: 0.1283s/iter; left time: 779.5495s
        speed: 0.1283s/iter; left time: 766.7960s
        speed: 0.1282s/iter; left time: 753.4139s
        speed: 0.1282s/iter; left time: 740.1944s
        speed: 0.1282s/iter; left time: 727.6711s
        speed: 0.1284s/iter; left time: 715.8365s
        speed: 0.1282s/iter; left time: 701.6651s
        speed: 0.1283s/iter; left time: 689.5141s
        speed: 0.1282s/iter; left time: 676.0763s
        speed: 0.1282s/iter; left time: 663.4497s
        speed: 0.1282s/iter; left time: 650.6223s
        speed: 0.1284s/iter; left time: 638.5416s
        speed: 0.1282s/iter; left time: 625.1153s
        speed: 0.1283s/iter; left time: 612.6931s
        speed: 0.1283s/iter; left time: 599.7292s
        speed: 0.1282s/iter; left time: 586.6867s
        speed: 0.1284s/iter; left time: 574.6260s
        speed: 0.1283s/iter; left time: 561.4819s
        speed: 0.1283s/iter; left time: 548.4647s
        speed: 0.1283s/iter; left time: 535.5813s
Epoch: 2 cost time: 530.542858839035
Epoch: 2, Steps: 4137 | Train Loss: -48.9527894 Vali Loss: -48.9326362 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 1.2538s/iter; left time: 5062.9567s
        speed: 0.1284s/iter; left time: 505.7279s
        speed: 0.1284s/iter; left time: 492.9298s
        speed: 0.1283s/iter; left time: 479.5802s
        speed: 0.1282s/iter; left time: 466.2639s
        speed: 0.1283s/iter; left time: 453.8794s
        speed: 0.1284s/iter; left time: 441.3263s
        speed: 0.1282s/iter; left time: 428.0605s
        speed: 0.1284s/iter; left time: 415.8170s
        speed: 0.1283s/iter; left time: 402.4540s
        speed: 0.1282s/iter; left time: 389.4098s
        speed: 0.1283s/iter; left time: 376.9801s
        speed: 0.1283s/iter; left time: 364.0838s
        speed: 0.1283s/iter; left time: 351.2112s
        speed: 0.1282s/iter; left time: 338.1965s
        speed: 0.1283s/iter; left time: 325.5066s
        speed: 0.1282s/iter; left time: 312.6431s
        speed: 0.1284s/iter; left time: 300.1481s
        speed: 0.1283s/iter; left time: 287.0474s
        speed: 0.1284s/iter; left time: 274.4572s
        speed: 0.1282s/iter; left time: 261.2532s
        speed: 0.1282s/iter; left time: 248.4272s
        speed: 0.1282s/iter; left time: 235.6939s
        speed: 0.1282s/iter; left time: 222.7621s
        speed: 0.1282s/iter; left time: 209.9875s
        speed: 0.1282s/iter; left time: 197.1853s
        speed: 0.1282s/iter; left time: 184.3661s
        speed: 0.1283s/iter; left time: 171.6811s
        speed: 0.1285s/iter; left time: 159.0394s
        speed: 0.1283s/iter; left time: 145.9588s
        speed: 0.1283s/iter; left time: 133.1463s
        speed: 0.1283s/iter; left time: 120.3105s
        speed: 0.1282s/iter; left time: 107.4170s
        speed: 0.1283s/iter; left time: 94.6591s
        speed: 0.1282s/iter; left time: 81.8221s
        speed: 0.1282s/iter; left time: 68.9838s
        speed: 0.1282s/iter; left time: 56.1650s
        speed: 0.1282s/iter; left time: 43.3404s
        speed: 0.1283s/iter; left time: 30.5237s
        speed: 0.1282s/iter; left time: 17.6921s
        speed: 0.1282s/iter; left time: 4.8703s
Epoch: 3 cost time: 530.5573189258575
Epoch: 3, Steps: 4137 | Train Loss: -48.9824078 Vali Loss: -48.9623636 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/PSM
dataset: PSM
input_c: 25
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 25
pretrained_model: 20
win_size: 100
-------------- End ----------------
test: (87841, 25)
train: (132481, 25)
test: (87841, 25)
train: (132481, 25)
test: (87841, 25)
train: (132481, 25)
test: (87841, 25)
train: (132481, 25)
======================TEST MODE======================
/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0011754722148179996
pred:    (87800,)
gt:      (87800,)
pred:  (87800,)
gt:    (87800,)
Accuracy : 0.9882, Precision : 0.9697, Recall : 0.9883, F-score : 0.9789

2.3 MSL

首次运行MSL.sh

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ bash ./scripts/MSL.sh
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/MSL
dataset: MSL
input_c: 55
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 55
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
Traceback (most recent call last):
  File "main.py", line 52, in <module>
    main(config)
  File "main.py", line 18, in main
    solver = Solver(vars(config))
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 85, in __init__
    self.build_model()
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 90, in build_model
    self.model = AnomalyTransformer(win_size=self.win_size, enc_in=self.input_c, c_out=self.output_c, e_layers=3)
  File "/media/username/folder/Dev/Anomaly-Transformer/model/AnomalyTransformer.py", line 77, in __init__
    ) for l in range(e_layers)
  File "/media/username/folder/Dev/Anomaly-Transformer/model/AnomalyTransformer.py", line 77, in <listcomp>
    ) for l in range(e_layers)
  File "/media/username/folder/Dev/Anomaly-Transformer/model/attn.py", line 29, in __init__
    self.distances = torch.zeros((window_size, window_size)).cuda()
  File "/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/cuda/__init__.py", line 170, in _lazy_init
    torch._C._cuda_init()
RuntimeError: No CUDA GPUs are available
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/MSL
dataset: MSL
input_c: 55
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 55
pretrained_model: 20
win_size: 100
-------------- End ----------------
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
Traceback (most recent call last):
  File "main.py", line 52, in <module>
    main(config)
  File "main.py", line 18, in main
    solver = Solver(vars(config))
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 85, in __init__
    self.build_model()
  File "/media/username/folder/Dev/Anomaly-Transformer/solver.py", line 90, in build_model
    self.model = AnomalyTransformer(win_size=self.win_size, enc_in=self.input_c, c_out=self.output_c, e_layers=3)
  File "/media/username/folder/Dev/Anomaly-Transformer/model/AnomalyTransformer.py", line 77, in __init__
    ) for l in range(e_layers)
  File "/media/username/folder/Dev/Anomaly-Transformer/model/AnomalyTransformer.py", line 77, in <listcomp>
    ) for l in range(e_layers)
  File "/media/username/folder/Dev/Anomaly-Transformer/model/attn.py", line 29, in __init__
    self.distances = torch.zeros((window_size, window_size)).cuda()
  File "/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/cuda/__init__.py", line 170, in _lazy_init
    torch._C._cuda_init()
RuntimeError: No CUDA GPUs are available

运行失败，原因为RuntimeError: No CUDA GPUs are available，不知道为什么GPU不可用了。尝试看看GPU是否可用。

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ python
Python 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.cuda.device_count())
1
>>> print(torch.cuda.is_available())
True

结果正常？？？再尝试运行MSL.sh仍然有问题，检查MSL.sh本身，发现第一行有问题：

export CUDA_VISIBLE_DEVICES=7

因为电脑只有一张显卡，序号不应该是7，应该为0。修改后再运行MSL.sh，正常了...(大无语，干嘛突然写个7，其他脚本的明明都是0)

再次运行MSL.sh

修改GPU序号后正常运行，测试结果摘要如下

======================TEST MODE======================
Threshold : 0.0012788161612115718
pred:    (73700,)
gt:      (73700,)
pred:  (73700,)
gt:    (73700,)
Accuracy : 0.9863, Precision : 0.9186, Recall : 0.9545, F-score : 0.9362 

完整执行过程如下：

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ bash ./scripts/MSL.sh
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/MSL
dataset: MSL
input_c: 55
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 55
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
======================TRAIN MODE======================
        speed: 0.1424s/iter; left time: 763.5606s
        speed: 0.1358s/iter; left time: 714.4375s
        speed: 0.1387s/iter; left time: 716.0470s
        speed: 0.1354s/iter; left time: 685.1058s
        speed: 0.1357s/iter; left time: 673.0052s
        speed: 0.1402s/iter; left time: 681.4136s
        speed: 0.1380s/iter; left time: 656.8551s
        speed: 0.1355s/iter; left time: 631.5301s
        speed: 0.1360s/iter; left time: 620.2718s
        speed: 0.1350s/iter; left time: 602.2681s
        speed: 0.1344s/iter; left time: 586.1590s
        speed: 0.1352s/iter; left time: 576.0738s
        speed: 0.1372s/iter; left time: 570.8950s
        speed: 0.1358s/iter; left time: 551.6753s
        speed: 0.1326s/iter; left time: 525.1665s
        speed: 0.1336s/iter; left time: 515.8068s
        speed: 0.1349s/iter; left time: 507.2338s
        speed: 0.1348s/iter; left time: 493.3304s
Epoch: 1 cost time: 247.81738114356995
Epoch: 1, Steps: 1820 | Train Loss: -47.0458832 Vali Loss: -46.7697310 
Validation loss decreased (inf --> -46.769731).  Saving model ...
Updating learning rate to 0.0001
        speed: 1.1043s/iter; left time: 3910.1910s
        speed: 0.1326s/iter; left time: 456.2046s
        speed: 0.1330s/iter; left time: 444.4217s
        speed: 0.1336s/iter; left time: 433.0193s
        speed: 0.1396s/iter; left time: 438.4048s
        speed: 0.1384s/iter; left time: 420.9290s
        speed: 0.1358s/iter; left time: 399.4554s
        speed: 0.1363s/iter; left time: 387.1776s
        speed: 0.1354s/iter; left time: 371.1777s
        speed: 0.1354s/iter; left time: 357.5956s
        speed: 0.1351s/iter; left time: 343.2376s
        speed: 0.1355s/iter; left time: 330.8024s
        speed: 0.1362s/iter; left time: 318.7658s
        speed: 0.1367s/iter; left time: 306.3689s
        speed: 0.1363s/iter; left time: 291.7184s
        speed: 0.1358s/iter; left time: 277.2149s
        speed: 0.1362s/iter; left time: 264.4203s
        speed: 0.1352s/iter; left time: 248.9006s
Epoch: 2 cost time: 246.60186314582825
Epoch: 2, Steps: 1820 | Train Loss: -48.5221037 Vali Loss: -47.3841785 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 1.1290s/iter; left time: 1942.9595s
        speed: 0.1394s/iter; left time: 225.9686s
        speed: 0.1351s/iter; left time: 205.5129s
        speed: 0.1406s/iter; left time: 199.7962s
        speed: 0.1332s/iter; left time: 175.9856s
        speed: 0.1326s/iter; left time: 161.8460s
        speed: 0.1314s/iter; left time: 147.2958s
        speed: 0.1334s/iter; left time: 136.1576s
        speed: 0.1319s/iter; left time: 121.5173s
        speed: 0.1389s/iter; left time: 114.0600s
        speed: 0.1306s/iter; left time: 94.1768s
        speed: 0.1396s/iter; left time: 86.6974s
        speed: 0.1352s/iter; left time: 70.4401s
        speed: 0.1373s/iter; left time: 57.8087s
        speed: 0.1379s/iter; left time: 44.2689s
        speed: 0.1322s/iter; left time: 29.2235s
        speed: 0.1308s/iter; left time: 15.8220s
        speed: 0.1308s/iter; left time: 2.7468s
Epoch: 3 cost time: 245.10069799423218
Epoch: 3, Steps: 1820 | Train Loss: -48.7357392 Vali Loss: -47.5481951 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/MSL
dataset: MSL
input_c: 55
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 55
pretrained_model: 20
win_size: 100
-------------- End ----------------
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
test: (73729, 55)
train: (58317, 55)
======================TEST MODE======================
/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0012788161612115718
pred:    (73700,)
gt:      (73700,)
pred:  (73700,)
gt:    (73700,)
Accuracy : 0.9863, Precision : 0.9186, Recall : 0.9545, F-score : 0.9362 

2.4 SMAP

首次运行SMAP.sh

这次留了个心眼看看脚本第一行的GPU编号是否正确，没有问题，得到测试结果摘要如下：

======================TEST MODE======================
Threshold : 0.0005670388956787038
pred:    (427600,)
gt:      (427600,)
pred:  (427600,)
gt:    (427600,)
Accuracy : 0.9906, Precision : 0.9360, Recall : 0.9943, F-score : 0.9642 

完整执行过程如下（跑得太久，机器都快烤熟了）：

(Anomaly-Transformer) username@username-ubuntu:/media/username/folder/Dev/Anomaly-Transformer$ bash ./scripts/SMAP.sh
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/SMAP
dataset: SMAP
input_c: 25
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 25
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (427617, 25)
train: (135183, 25)
test: (427617, 25)
train: (135183, 25)
test: (427617, 25)
train: (135183, 25)
test: (427617, 25)
train: (135183, 25)
======================TRAIN MODE======================
        speed: 0.1343s/iter; left time: 1687.4161s
        speed: 0.1316s/iter; left time: 1641.0434s
        speed: 0.1304s/iter; left time: 1612.4711s
        speed: 0.1304s/iter; left time: 1599.8954s
        speed: 0.1315s/iter; left time: 1600.4497s
        speed: 0.1304s/iter; left time: 1573.3904s
        speed: 0.1306s/iter; left time: 1563.0526s
        speed: 0.1313s/iter; left time: 1557.9986s
        speed: 0.1308s/iter; left time: 1539.4388s
        speed: 0.1302s/iter; left time: 1518.6118s
        speed: 0.1312s/iter; left time: 1518.1523s
        speed: 0.1305s/iter; left time: 1495.9211s
        speed: 0.1306s/iter; left time: 1484.8276s
        speed: 0.1306s/iter; left time: 1471.1267s
        speed: 0.1297s/iter; left time: 1447.8968s
        speed: 0.1304s/iter; left time: 1442.7176s
        speed: 0.1299s/iter; left time: 1424.9521s
        speed: 0.1303s/iter; left time: 1415.9097s
        speed: 0.1309s/iter; left time: 1409.4994s
        speed: 0.1311s/iter; left time: 1398.0175s
        speed: 0.1318s/iter; left time: 1392.9594s
        speed: 0.1302s/iter; left time: 1362.8479s
        speed: 0.1371s/iter; left time: 1421.0785s
        speed: 0.1292s/iter; left time: 1326.9539s
        speed: 0.1303s/iter; left time: 1324.7232s
        speed: 0.1308s/iter; left time: 1316.6065s
        speed: 0.1309s/iter; left time: 1304.8822s
        speed: 0.1306s/iter; left time: 1289.0181s
        speed: 0.1322s/iter; left time: 1291.2752s
        speed: 0.1315s/iter; left time: 1271.0320s
        speed: 0.1302s/iter; left time: 1245.4013s
        speed: 0.1310s/iter; left time: 1240.1241s
        speed: 0.1309s/iter; left time: 1225.9448s
        speed: 0.1300s/iter; left time: 1204.4843s
        speed: 0.1308s/iter; left time: 1198.7496s
        speed: 0.1329s/iter; left time: 1205.0089s
        speed: 0.1319s/iter; left time: 1183.1681s
        speed: 0.1301s/iter; left time: 1153.9812s
        speed: 0.1295s/iter; left time: 1135.2198s
        speed: 0.1307s/iter; left time: 1132.5606s
        speed: 0.1312s/iter; left time: 1124.1417s
        speed: 0.1296s/iter; left time: 1097.1383s
Epoch: 1 cost time: 553.0207221508026
Epoch: 1, Steps: 4222 | Train Loss: -47.6426614 Vali Loss: -48.1685601 
Validation loss decreased (inf --> -48.168560).  Saving model ...
Updating learning rate to 0.0001
        speed: 5.5265s/iter; left time: 46118.9601s
        speed: 0.1295s/iter; left time: 1067.8902s
        speed: 0.1297s/iter; left time: 1056.0485s
        speed: 0.1296s/iter; left time: 1042.8939s
        speed: 0.1328s/iter; left time: 1055.0172s
        speed: 0.1347s/iter; left time: 1056.7791s
        speed: 0.1300s/iter; left time: 1006.6005s
        speed: 0.1293s/iter; left time: 988.4494s
        speed: 0.1292s/iter; left time: 975.1445s
        speed: 0.1294s/iter; left time: 963.6841s
        speed: 0.1292s/iter; left time: 948.7126s
        speed: 0.1292s/iter; left time: 936.1060s
        speed: 0.1291s/iter; left time: 922.7592s
        speed: 0.1291s/iter; left time: 909.7682s
        speed: 0.1291s/iter; left time: 896.7182s
        speed: 0.1290s/iter; left time: 882.9915s
        speed: 0.1291s/iter; left time: 870.9842s
        speed: 0.1289s/iter; left time: 856.8340s
        speed: 0.1289s/iter; left time: 843.7893s
        speed: 0.1291s/iter; left time: 831.9244s
        speed: 0.1292s/iter; left time: 819.9622s
        speed: 0.1297s/iter; left time: 809.7022s
        speed: 0.1293s/iter; left time: 794.2553s
        speed: 0.1292s/iter; left time: 781.1323s
        speed: 0.1292s/iter; left time: 767.8188s
        speed: 0.1293s/iter; left time: 755.7132s
        speed: 0.1292s/iter; left time: 742.2035s
        speed: 0.1293s/iter; left time: 729.9284s
        speed: 0.1294s/iter; left time: 717.5859s
        speed: 0.1293s/iter; left time: 703.9006s
        speed: 0.1292s/iter; left time: 690.3800s
        speed: 0.1291s/iter; left time: 677.3184s
        speed: 0.1293s/iter; left time: 665.2619s
        speed: 0.1292s/iter; left time: 651.7931s
        speed: 0.1292s/iter; left time: 638.8412s
        speed: 0.1293s/iter; left time: 626.6065s
        speed: 0.1292s/iter; left time: 612.8525s
        speed: 0.1291s/iter; left time: 599.8719s
        speed: 0.1292s/iter; left time: 587.1467s
        speed: 0.1292s/iter; left time: 574.4164s
        speed: 0.1293s/iter; left time: 561.6398s
        speed: 0.1291s/iter; left time: 548.2016s
Epoch: 2 cost time: 546.9801330566406
Epoch: 2, Steps: 4222 | Train Loss: -48.5213919 Vali Loss: -48.2957534 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 5.4772s/iter; left time: 22582.6544s
        speed: 0.1305s/iter; left time: 524.9768s
        speed: 0.1301s/iter; left time: 510.4672s
        speed: 0.1292s/iter; left time: 494.0773s
        speed: 0.1291s/iter; left time: 480.7389s
        speed: 0.1293s/iter; left time: 468.5919s
        speed: 0.1292s/iter; left time: 455.1257s
        speed: 0.1292s/iter; left time: 442.3313s
        speed: 0.1293s/iter; left time: 429.8049s
        speed: 0.1293s/iter; left time: 416.6019s
        speed: 0.1291s/iter; left time: 403.3154s
        speed: 0.1292s/iter; left time: 390.4252s
        speed: 0.1291s/iter; left time: 377.3882s
        speed: 0.1292s/iter; left time: 364.6556s
        speed: 0.1293s/iter; left time: 352.1070s
        speed: 0.1291s/iter; left time: 338.6508s
        speed: 0.1292s/iter; left time: 325.8527s
        speed: 0.1291s/iter; left time: 312.7774s
        speed: 0.1292s/iter; left time: 300.1695s
        speed: 0.1291s/iter; left time: 286.9356s
        speed: 0.1291s/iter; left time: 274.0536s
        speed: 0.1299s/iter; left time: 262.8002s
        speed: 0.1324s/iter; left time: 254.5154s
        speed: 0.1298s/iter; left time: 236.6313s
        speed: 0.1328s/iter; left time: 228.8171s
        speed: 0.1327s/iter; left time: 215.4497s
        speed: 0.1304s/iter; left time: 198.6513s
        speed: 0.1295s/iter; left time: 184.3195s
        speed: 0.1299s/iter; left time: 171.8900s
        speed: 0.1292s/iter; left time: 157.9532s
        speed: 0.1290s/iter; left time: 144.8993s
        speed: 0.1292s/iter; left time: 132.2070s
        speed: 0.1293s/iter; left time: 119.3879s
        speed: 0.1291s/iter; left time: 106.2423s
        speed: 0.1291s/iter; left time: 93.3420s
        speed: 0.1291s/iter; left time: 80.4567s
        speed: 0.1292s/iter; left time: 67.5827s
        speed: 0.1292s/iter; left time: 54.6509s
        speed: 0.1293s/iter; left time: 41.7594s
        speed: 0.1292s/iter; left time: 28.8161s
        speed: 0.1292s/iter; left time: 15.8970s
        speed: 0.1291s/iter; left time: 2.9699s
Epoch: 3 cost time: 547.1292362213135
Epoch: 3, Steps: 4222 | Train Loss: -48.6120459 Vali Loss: -48.3690009 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/SMAP
dataset: SMAP
input_c: 25
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 25
pretrained_model: 20
win_size: 100
-------------- End ----------------
test: (427617, 25)
train: (135183, 25)
test: (427617, 25)
train: (135183, 25)
test: (427617, 25)
train: (135183, 25)
test: (427617, 25)
train: (135183, 25)
======================TEST MODE======================
/home/username/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0005670388956787038
pred:    (427600,)
gt:      (427600,)
pred:  (427600,)
gt:    (427600,)
Accuracy : 0.9906, Precision : 0.9360, Recall : 0.9943, F-score : 0.9642 

2.5 SWaT

此数据集较为特殊，体现在其获取和使用上。在数据集获取上，根据协议无法与他人分享，需要自行申请。因此我前往iTrust官网申请，只选择SWaT数据集即可（填表链接），并等一了天得到邮件回复。在数据集使用上，作者没有编写训练脚本，因此需要自己对数据集进行处理并使用模型进行训练和测试。

拿到数据集后，根据论文附件K中Table 13推断论文使用的是2015年版本的数据集，即SWaT共享的Google Drive中，SWAT/SWaT.A1&A2_Dec 2015/Physical/下的SWaT_dataset_Attack_v0.xlsx（作测试集）和SWaT_dataset_Normal_v1.xlsx（作训练集）。下面进行简单的处理。

数据集处理

# 1. 使用表格软件打开两者，分别删除第一行(第一行不是标题，只有P1,P2等字符，第二行的标题需要保留)后均保存为csv文件
# 2. 将两者用Python进行简单检查，转为numpy矩阵并保存为npy文件，代码如下：

import numpy as np
import pandas as pd

swat_train_pd = pd.read_csv('./dataset/SWaT/SWaT_Dataset_Normal_v1.csv')
swat_test_pd = pd.read_csv('./dataset/SWaT/SWaT_Dataset_Attack_v0.csv')

print(swat_train_pd.shape)
print(swat_test_pd.shape)
print(swat_test_pd['Normal/Attack'].unique())
print(swat_test_pd.head())
"""
(495000, 53)
(449919, 53)
['Normal' 'Attack' 'A ttack']
                 Timestamp    FIT101    LIT101  ...  P602  P603  Normal/Attack
0   28/12/2015 10:00:00 AM  2.427057  522.8467  ...     1     1         Normal
1   28/12/2015 10:00:01 AM  2.446274  522.8860  ...     1     1         Normal
2   28/12/2015 10:00:02 AM  2.489191  522.8467  ...     1     1         Normal
3   28/12/2015 10:00:03 AM  2.534350  522.9645  ...     1     1         Normal
4   28/12/2015 10:00:04 AM  2.569260  523.4748  ...     1     1         Normal

[5 rows x 53 columns]
"""

swat_test_pd = swat_test_pd.replace('Normal',0).replace('Attack',1).replace('A ttack',1)
swat_test_label_np = swat_test_pd.iloc[:,52].values
swat_test_np = swat_test_pd.drop([' Timestamp','Normal/Attack'], axis=1).values
swat_train_np = swat_train_pd.drop([' Timestamp','Normal/Attack'], axis=1).values

print(swat_train_np.shape)
print(swat_test_np.shape)
print(swat_test_label_np.shape)
"""
(495000, 51)
(449919, 51)
(449919,)
"""

np.save('./dataset/SWaT/swat_test_label.npy',swat_test_label_np)
np.save('./dataset/SWaT/swat_train.npy',swat_train_np)
np.save('./dataset/SWaT/swat_test.npy',swat_test_np)

然后新建训练测试脚本./scripts/SWaT.sh，内容是从./scripts/Start.sh复制的

export CUDA_VISIBLE_DEVICES=0

python main.py --anormly_ratio 0.5 --num_epochs 3    --batch_size 32  --mode train --dataset SWaT  --data_path dataset/SWaT --input_c 51    --output_c 51
python main.py --anormly_ratio 0.1  --num_epochs 10        --batch_size 32     --mode test    --dataset SWaT   --data_path dataset/SWaT  --input_c 51    --output_c 51  --pretrained_model 10

接着为SWaT数据集添加dataloder。编辑./data_factory/data_loader.py，添加一个SwatSegLoader类并修改原有get_loader_segment函数：

'''
Loader for SWaT dataset
'''
class SwatSegLoader(object):
    def __init__(self, data_path, win_size, step, mode="train"):
        self.mode = mode
        self.step = step
        self.win_size = win_size
        self.scaler = StandardScaler()
        data = np.load(data_path + "/swat_train.npy")
        self.scaler.fit(data)
        data = self.scaler.transform(data)
        test_data = np.load(data_path + "/swat_test.npy")
        self.test = self.scaler.transform(test_data)

        self.train = data
        self.val = self.test
        self.test_labels = np.load(data_path + "/swat_test_label.npy")
        print("test:", self.test.shape)
        print("train:", self.train.shape)

    def __len__(self):

        if self.mode == "train":
            return (self.train.shape[0] - self.win_size) // self.step + 1
        elif (self.mode == 'val'):
            return (self.val.shape[0] - self.win_size) // self.step + 1
        elif (self.mode == 'test'):
            return (self.test.shape[0] - self.win_size) // self.step + 1
        else:
            return (self.test.shape[0] - self.win_size) // self.win_size + 1

    def __getitem__(self, index):
        index = index * self.step
        if self.mode == "train":
            return np.float32(self.train[index:index + self.win_size]), np.float32(self.test_labels[0:self.win_size])
        elif (self.mode == 'val'):
            return np.float32(self.val[index:index + self.win_size]), np.float32(self.test_labels[0:self.win_size])
        elif (self.mode == 'test'):
            return np.float32(self.test[index:index + self.win_size]), np.float32(
                self.test_labels[index:index + self.win_size])
        else:
            return np.float32(self.test[
                              index // self.step * self.win_size:index // self.step * self.win_size + self.win_size]), np.float32(
                self.test_labels[index // self.step * self.win_size:index // self.step * self.win_size + self.win_size])

            
"""
Add a new line about the SWaT dataset
"""            
def get_loader_segment(data_path, batch_size, win_size=100, step=100, mode='train', dataset='KDD'):
    if (dataset == 'SMD'):
        dataset = SMDSegLoader(data_path, win_size, step, mode)
    elif (dataset == 'MSL'):
        dataset = MSLSegLoader(data_path, win_size, 1, mode)
    elif (dataset == 'SMAP'):
        dataset = SMAPSegLoader(data_path, win_size, 1, mode)
    elif (dataset == 'PSM'):
        dataset = PSMSegLoader(data_path, win_size, 1, mode)
    elif (dataset == 'SWaT'): # added this
        dataset = SwatSegLoader(data_path, win_size, 1, mode)

    shuffle = False
    if mode == 'train':
        shuffle = True

    data_loader = DataLoader(dataset=dataset,
                             batch_size=batch_size,
                             shuffle=shuffle,
                             num_workers=0)
    return data_loader

首次运行SWaT.sh

得到测试结果摘要如下：

======================TEST MODE======================
Threshold : 0.0031170047065244427
pred:    (449900,)
gt:      (449900,)
pred:  (449900,)
gt:    (449900,)
Accuracy : 0.9775, Precision : 0.8841, Recall : 0.9371, F-score : 0.9099 

完整执行过程如下（跑了大概两个小时）：

(Anomaly-Transformer) username@ranlychan-ubuntu:/media/ranlychan/3E6E20236E1FD28F/Dev/Anomaly-Transformer$ bash ./scripts/SWaT.sh
------------ Options -------------
anormly_ratio: 0.1
batch_size: 32
data_path: dataset/SWaT
dataset: SWaT
input_c: 51
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 51
pretrained_model: 10
win_size: 100
-------------- End ----------------
test: (449919, 51)
train: (496800, 51)
test: (449919, 51)
train: (496800, 51)
test: (449919, 51)
train: (496800, 51)
test: (449919, 51)
train: (496800, 51)
======================TEST MODE======================
/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0032192275498528246
pred:    (449900,)
gt:      (449900,)
pred:  (449900,)
gt:    (449900,)
Accuracy : 0.9771, Precision : 0.8965, Recall : 0.9172, F-score : 0.9067 
(Anomaly-Transformer) ranlychan@ranlychan-ubuntu:/media/ranlychan/3E6E20236E1FD28F/Dev/Anomaly-Transformer$ bash ./scripts/SWaT.sh
------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/SWaT
dataset: SWaT
input_c: 51
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 51
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (449919, 51)
train: (495000, 51)
test: (449919, 51)
train: (495000, 51)
test: (449919, 51)
train: (495000, 51)
test: (449919, 51)
train: (495000, 51)
======================TRAIN MODE======================
        speed: 0.1428s/iter; left time: 6612.5713s
        speed: 0.1354s/iter; left time: 6253.0974s
        speed: 0.1352s/iter; left time: 6230.7968s
        speed: 0.1355s/iter; left time: 6233.4215s
        speed: 0.1390s/iter; left time: 6378.5659s
        speed: 0.1403s/iter; left time: 6425.6394s
        speed: 0.1348s/iter; left time: 6161.5534s
        speed: 0.1359s/iter; left time: 6194.7005s
        speed: 0.1356s/iter; left time: 6168.4861s
        speed: 0.1360s/iter; left time: 6174.7164s
        speed: 0.1395s/iter; left time: 6319.8666s
        speed: 0.1389s/iter; left time: 6276.0570s
        speed: 0.1371s/iter; left time: 6183.6705s
        speed: 0.1401s/iter; left time: 6302.9684s
        speed: 0.1351s/iter; left time: 6067.7535s
        speed: 0.1351s/iter; left time: 6050.1461s
        speed: 0.1365s/iter; left time: 6101.5390s
        speed: 0.1378s/iter; left time: 6144.6511s
        speed: 0.1372s/iter; left time: 6104.3526s
        speed: 0.1382s/iter; left time: 6137.7459s
        speed: 0.1381s/iter; left time: 6117.2903s
        speed: 0.1371s/iter; left time: 6061.2191s
        speed: 0.1365s/iter; left time: 6021.1155s
        speed: 0.1345s/iter; left time: 5915.7146s
        speed: 0.1349s/iter; left time: 5921.9364s
        speed: 0.1355s/iter; left time: 5933.2968s
        speed: 0.1354s/iter; left time: 5914.8026s
        speed: 0.1424s/iter; left time: 6210.5447s
        speed: 0.1359s/iter; left time: 5911.6384s
        speed: 0.1374s/iter; left time: 5964.3514s
        speed: 0.1348s/iter; left time: 5838.2409s
        speed: 0.1346s/iter; left time: 5816.1968s
        speed: 0.1346s/iter; left time: 5802.6837s
        speed: 0.1346s/iter; left time: 5788.3701s
        speed: 0.1353s/iter; left time: 5804.4665s
        speed: 0.1394s/iter; left time: 5966.4150s
        speed: 0.1423s/iter; left time: 6074.2209s
        speed: 0.1420s/iter; left time: 6049.9245s
        speed: 0.1345s/iter; left time: 5715.0185s
        speed: 0.1344s/iter; left time: 5698.0888s
        speed: 0.1411s/iter; left time: 5968.1633s
        speed: 0.1415s/iter; left time: 5970.2509s
        speed: 0.1408s/iter; left time: 5928.4565s
        speed: 0.1426s/iter; left time: 5990.2664s
        speed: 0.1357s/iter; left time: 5686.5547s
        speed: 0.1365s/iter; left time: 5704.3444s
        speed: 0.1358s/iter; left time: 5664.3863s
        speed: 0.1392s/iter; left time: 5791.4018s
        speed: 0.1372s/iter; left time: 5693.4794s
        speed: 0.1362s/iter; left time: 5638.4746s
        speed: 0.1376s/iter; left time: 5682.1785s
        speed: 0.1382s/iter; left time: 5695.7126s
        speed: 0.1381s/iter; left time: 5673.8861s
        speed: 0.1428s/iter; left time: 5854.1201s
        speed: 0.1353s/iter; left time: 5535.4870s
        speed: 0.1358s/iter; left time: 5538.6886s
        speed: 0.1359s/iter; left time: 5530.9134s
        speed: 0.1360s/iter; left time: 5520.0075s
        speed: 0.1369s/iter; left time: 5545.7690s
        speed: 0.1353s/iter; left time: 5466.8770s
        speed: 0.1356s/iter; left time: 5465.3225s
        speed: 0.1344s/iter; left time: 5403.5386s
        speed: 0.1346s/iter; left time: 5398.9813s
        speed: 0.1364s/iter; left time: 5455.0500s
        speed: 0.1344s/iter; left time: 5362.2032s
        speed: 0.1346s/iter; left time: 5357.8926s
        speed: 0.1349s/iter; left time: 5356.4118s
        speed: 0.1358s/iter; left time: 5375.5669s
        speed: 0.1393s/iter; left time: 5503.1650s
        speed: 0.1394s/iter; left time: 5492.4838s
        speed: 0.1436s/iter; left time: 5641.9017s
        speed: 0.1382s/iter; left time: 5417.8900s
        speed: 0.1361s/iter; left time: 5320.7390s
        speed: 0.1370s/iter; left time: 5342.9212s
        speed: 0.1406s/iter; left time: 5470.8222s
        speed: 0.1363s/iter; left time: 5289.7796s
        speed: 0.1354s/iter; left time: 5240.4470s
        speed: 0.1352s/iter; left time: 5218.9477s
        speed: 0.1415s/iter; left time: 5446.3757s
        speed: 0.1396s/iter; left time: 5360.8597s
        speed: 0.1403s/iter; left time: 5371.5712s
        speed: 0.1355s/iter; left time: 5174.8979s
        speed: 0.1347s/iter; left time: 5132.1327s
        speed: 0.1356s/iter; left time: 5153.9954s
        speed: 0.1357s/iter; left time: 5144.1700s
        speed: 0.1364s/iter; left time: 5156.1387s
        speed: 0.1370s/iter; left time: 5164.2171s
        speed: 0.1385s/iter; left time: 5208.6199s
        speed: 0.1450s/iter; left time: 5438.2515s
        speed: 0.1492s/iter; left time: 5580.7061s
        speed: 0.1395s/iter; left time: 5203.9755s
        speed: 0.1372s/iter; left time: 5103.0719s
        speed: 0.1402s/iter; left time: 5202.6695s
        speed: 0.1426s/iter; left time: 5274.6904s
        speed: 0.1393s/iter; left time: 5141.1819s
        speed: 0.1393s/iter; left time: 5126.7243s
        speed: 0.1356s/iter; left time: 4976.0627s
        speed: 0.1359s/iter; left time: 4975.5315s
        speed: 0.1387s/iter; left time: 5061.0901s
        speed: 0.1418s/iter; left time: 5162.7360s
        speed: 0.1377s/iter; left time: 4997.3233s
        speed: 0.1360s/iter; left time: 4922.3810s
        speed: 0.1459s/iter; left time: 5267.9064s
        speed: 0.1375s/iter; left time: 4949.4103s
        speed: 0.1393s/iter; left time: 5001.8771s
        speed: 0.1465s/iter; left time: 5245.6815s
        speed: 0.1383s/iter; left time: 4938.3803s
        speed: 0.1372s/iter; left time: 4884.7422s
        speed: 0.1372s/iter; left time: 4871.4987s
        speed: 0.1378s/iter; left time: 4877.6222s
        speed: 0.1345s/iter; left time: 4748.5213s
        speed: 0.1344s/iter; left time: 4730.7891s
        speed: 0.1346s/iter; left time: 4725.0120s
        speed: 0.1359s/iter; left time: 4756.4741s
        speed: 0.1351s/iter; left time: 4715.7454s
        speed: 0.1344s/iter; left time: 4676.1957s
        speed: 0.1359s/iter; left time: 4716.7439s
        speed: 0.1349s/iter; left time: 4668.0923s
        speed: 0.1355s/iter; left time: 4673.7719s
        speed: 0.1369s/iter; left time: 4708.6927s
        speed: 0.1346s/iter; left time: 4615.7443s
        speed: 0.1419s/iter; left time: 4851.2301s
        speed: 0.1420s/iter; left time: 4842.0394s
        speed: 0.1390s/iter; left time: 4726.7629s
        speed: 0.1400s/iter; left time: 4745.2550s
        speed: 0.1417s/iter; left time: 4789.7382s
        speed: 0.1388s/iter; left time: 4677.9071s
        speed: 0.1359s/iter; left time: 4566.2085s
        speed: 0.1362s/iter; left time: 4561.5282s
        speed: 0.1412s/iter; left time: 4717.2056s
        speed: 0.1399s/iter; left time: 4657.4927s
        speed: 0.1398s/iter; left time: 4639.9572s
        speed: 0.1386s/iter; left time: 4586.4466s
        speed: 0.1437s/iter; left time: 4742.9178s
        speed: 0.1414s/iter; left time: 4653.5077s
        speed: 0.1389s/iter; left time: 4555.4527s
        speed: 0.1407s/iter; left time: 4600.7729s
        speed: 0.1353s/iter; left time: 4410.3650s
        speed: 0.1359s/iter; left time: 4415.5378s
        speed: 0.1348s/iter; left time: 4368.9311s
        speed: 0.1359s/iter; left time: 4389.8059s
        speed: 0.1355s/iter; left time: 4362.4974s
        speed: 0.1355s/iter; left time: 4348.9004s
        speed: 0.1363s/iter; left time: 4360.6657s
        speed: 0.1350s/iter; left time: 4307.7063s
        speed: 0.1354s/iter; left time: 4305.2038s
        speed: 0.1359s/iter; left time: 4309.3123s
        speed: 0.1353s/iter; left time: 4276.3862s
        speed: 0.1357s/iter; left time: 4274.6798s
        speed: 0.1354s/iter; left time: 4249.9516s
        speed: 0.1356s/iter; left time: 4245.4502s
        speed: 0.1366s/iter; left time: 4261.3997s
        speed: 0.1374s/iter; left time: 4272.8442s
        speed: 0.1353s/iter; left time: 4194.5101s
Epoch: 1 cost time: 2127.7168984413147
Epoch: 1, Steps: 15466 | Train Loss: -48.4473046 Vali Loss: -47.3290840 
Validation loss decreased (inf --> -47.329084).  Saving model ...
Updating learning rate to 0.0001
        speed: 6.2369s/iter; left time: 192301.2652s
        speed: 0.1412s/iter; left time: 4340.9789s
        speed: 0.1374s/iter; left time: 4207.7656s
        speed: 0.1395s/iter; left time: 4258.3836s
        speed: 0.1371s/iter; left time: 4172.0284s
        speed: 0.1385s/iter; left time: 4202.1003s
        speed: 0.1396s/iter; left time: 4219.4023s
        speed: 0.1378s/iter; left time: 4153.6058s
        speed: 0.1369s/iter; left time: 4110.4457s
        speed: 0.1371s/iter; left time: 4103.2520s
        speed: 0.1402s/iter; left time: 4181.8363s
        speed: 0.1373s/iter; left time: 4081.6900s
        speed: 0.1351s/iter; left time: 4003.4332s
        speed: 0.1390s/iter; left time: 4105.8047s
        speed: 0.1424s/iter; left time: 4190.9918s
        speed: 0.1417s/iter; left time: 4156.5764s
        speed: 0.1403s/iter; left time: 4102.5730s
        speed: 0.1420s/iter; left time: 4138.0644s
        speed: 0.1429s/iter; left time: 4147.9949s
        speed: 0.1402s/iter; left time: 4055.5878s
        speed: 0.1436s/iter; left time: 4141.6097s
        speed: 0.1432s/iter; left time: 4115.6612s
        speed: 0.1436s/iter; left time: 4112.7574s
        speed: 0.1425s/iter; left time: 4066.4701s
        speed: 0.1379s/iter; left time: 3921.2554s
        speed: 0.1488s/iter; left time: 4216.4488s
        speed: 0.1372s/iter; left time: 3873.5076s
        speed: 0.1406s/iter; left time: 3955.3746s
        speed: 0.1341s/iter; left time: 3759.9923s
        speed: 0.1341s/iter; left time: 3745.7212s
        speed: 0.1340s/iter; left time: 3730.7106s
        speed: 0.1341s/iter; left time: 3717.9604s
        speed: 0.1340s/iter; left time: 3703.4859s
        speed: 0.1340s/iter; left time: 3690.3612s
        speed: 0.1340s/iter; left time: 3677.0124s
        speed: 0.1340s/iter; left time: 3662.8140s
        speed: 0.1340s/iter; left time: 3650.3931s
        speed: 0.1340s/iter; left time: 3636.0835s
        speed: 0.1341s/iter; left time: 3624.2639s
        speed: 0.1341s/iter; left time: 3611.4985s
        speed: 0.1391s/iter; left time: 3732.2035s
        speed: 0.1365s/iter; left time: 3649.8741s
        speed: 0.1396s/iter; left time: 3719.0167s
        speed: 0.1363s/iter; left time: 3616.9123s
        speed: 0.1385s/iter; left time: 3659.8609s
        speed: 0.1389s/iter; left time: 3657.7603s
        speed: 0.1378s/iter; left time: 3614.8408s
        speed: 0.1432s/iter; left time: 3741.8558s
        speed: 0.1415s/iter; left time: 3683.3259s
        speed: 0.1433s/iter; left time: 3714.9921s
        speed: 0.1375s/iter; left time: 3552.5034s
        speed: 0.1403s/iter; left time: 3610.4658s
        speed: 0.1388s/iter; left time: 3559.0740s
        speed: 0.1391s/iter; left time: 3551.4850s
        speed: 0.1344s/iter; left time: 3419.0121s
        speed: 0.1383s/iter; left time: 3502.4784s
        speed: 0.1405s/iter; left time: 3544.0996s
        speed: 0.1440s/iter; left time: 3619.6918s
        speed: 0.1463s/iter; left time: 3663.1171s
        speed: 0.1437s/iter; left time: 3582.2442s
        speed: 0.1425s/iter; left time: 3538.2324s
        speed: 0.1430s/iter; left time: 3537.9648s
        speed: 0.1382s/iter; left time: 3405.0349s
        speed: 0.1349s/iter; left time: 3308.9421s
        speed: 0.1352s/iter; left time: 3304.2378s
        speed: 0.1352s/iter; left time: 3289.0986s
        speed: 0.1359s/iter; left time: 3292.5206s
        speed: 0.1351s/iter; left time: 3260.5215s
        speed: 0.1359s/iter; left time: 3266.8184s
        speed: 0.1381s/iter; left time: 3305.8850s
        speed: 0.1421s/iter; left time: 3387.8256s
        speed: 0.1362s/iter; left time: 3233.5133s
        speed: 0.1406s/iter; left time: 3323.1694s
        speed: 0.1390s/iter; left time: 3270.9122s
        speed: 0.1481s/iter; left time: 3470.9108s
        speed: 0.1480s/iter; left time: 3452.5954s
        speed: 0.1434s/iter; left time: 3332.1595s
        speed: 0.1444s/iter; left time: 3340.4601s
        speed: 0.1429s/iter; left time: 3290.3965s
        speed: 0.1418s/iter; left time: 3251.0263s
        speed: 0.1432s/iter; left time: 3269.5059s
        speed: 0.1446s/iter; left time: 3286.1990s
        speed: 0.1414s/iter; left time: 3200.7980s
        speed: 0.1359s/iter; left time: 3061.4477s
        speed: 0.1357s/iter; left time: 3043.1181s
        speed: 0.1374s/iter; left time: 3067.6164s
        speed: 0.1344s/iter; left time: 2988.2089s
        speed: 0.1344s/iter; left time: 2974.6237s
        speed: 0.1345s/iter; left time: 2962.6316s
        speed: 0.1391s/iter; left time: 3051.0582s
        speed: 0.1375s/iter; left time: 3002.8196s
        speed: 0.1360s/iter; left time: 2955.7436s
        speed: 0.1369s/iter; left time: 2962.3081s
        speed: 0.1407s/iter; left time: 3030.2290s
        speed: 0.1372s/iter; left time: 2940.8313s
        speed: 0.1365s/iter; left time: 2911.8519s
        speed: 0.1359s/iter; left time: 2885.3581s
        speed: 0.1359s/iter; left time: 2871.7553s
        speed: 0.1348s/iter; left time: 2835.1846s
        speed: 0.1355s/iter; left time: 2837.3081s
        speed: 0.1349s/iter; left time: 2810.6872s
        speed: 0.1380s/iter; left time: 2860.7424s
        speed: 0.1382s/iter; left time: 2850.8490s
        speed: 0.1380s/iter; left time: 2833.0666s
        speed: 0.1358s/iter; left time: 2775.4763s
        speed: 0.1364s/iter; left time: 2772.4910s
        speed: 0.1417s/iter; left time: 2866.6913s
        speed: 0.1408s/iter; left time: 2834.4955s
        speed: 0.1414s/iter; left time: 2833.6041s
        speed: 0.1386s/iter; left time: 2761.7363s
        speed: 0.1373s/iter; left time: 2722.3473s
        speed: 0.1450s/iter; left time: 2861.6037s
        speed: 0.1379s/iter; left time: 2707.0912s
        speed: 0.1358s/iter; left time: 2652.5041s
        speed: 0.1386s/iter; left time: 2694.2524s
        speed: 0.1394s/iter; left time: 2694.5749s
        speed: 0.1407s/iter; left time: 2706.0995s
        speed: 0.1428s/iter; left time: 2731.5554s
        speed: 0.1428s/iter; left time: 2718.7514s
        speed: 0.1411s/iter; left time: 2670.9368s
        speed: 0.1419s/iter; left time: 2673.1494s
        speed: 0.1371s/iter; left time: 2569.1856s
        speed: 0.1363s/iter; left time: 2540.2892s
        speed: 0.1393s/iter; left time: 2582.3330s
        speed: 0.1404s/iter; left time: 2588.2281s
        speed: 0.1390s/iter; left time: 2547.5025s
        speed: 0.1358s/iter; left time: 2475.4478s
        speed: 0.1347s/iter; left time: 2442.8927s
        speed: 0.1389s/iter; left time: 2504.9989s
        speed: 0.1393s/iter; left time: 2498.3219s
        speed: 0.1385s/iter; left time: 2469.3112s
        speed: 0.1439s/iter; left time: 2551.8338s
        speed: 0.1393s/iter; left time: 2455.4148s
        speed: 0.1361s/iter; left time: 2385.9529s
        speed: 0.1380s/iter; left time: 2405.8499s
        speed: 0.1386s/iter; left time: 2401.7565s
        speed: 0.1405s/iter; left time: 2421.6689s
        speed: 0.1346s/iter; left time: 2305.7041s
        speed: 0.1358s/iter; left time: 2312.4812s
        speed: 0.1366s/iter; left time: 2313.5609s
        speed: 0.1446s/iter; left time: 2433.9046s
        speed: 0.1391s/iter; left time: 2327.8682s
        speed: 0.1357s/iter; left time: 2257.7770s
        speed: 0.1372s/iter; left time: 2267.6875s
        speed: 0.1361s/iter; left time: 2236.3395s
        speed: 0.1362s/iter; left time: 2224.1642s
        speed: 0.1359s/iter; left time: 2205.6602s
        speed: 0.1355s/iter; left time: 2185.4923s
        speed: 0.1360s/iter; left time: 2180.5024s
        speed: 0.1358s/iter; left time: 2164.1690s
        speed: 0.1347s/iter; left time: 2132.9385s
        speed: 0.1351s/iter; left time: 2124.7418s
        speed: 0.1361s/iter; left time: 2127.1795s
        speed: 0.1354s/iter; left time: 2103.8047s
Epoch: 2 cost time: 2142.760348558426
Epoch: 2, Steps: 15466 | Train Loss: -48.7614085 Vali Loss: -47.4089206 
Validation loss decreased (-47.329084 --> -47.408921).  Saving model ...
Updating learning rate to 5e-05
        speed: 6.1778s/iter; left time: 94934.8721s
        speed: 0.1406s/iter; left time: 2146.9605s
        speed: 0.1405s/iter; left time: 2131.2590s
        speed: 0.1406s/iter; left time: 2118.3086s
        speed: 0.1407s/iter; left time: 2105.8900s
        speed: 0.1407s/iter; left time: 2091.9465s
        speed: 0.1406s/iter; left time: 2076.6571s
        speed: 0.1406s/iter; left time: 2062.1350s
        speed: 0.1404s/iter; left time: 2044.6462s
        speed: 0.1415s/iter; left time: 2047.4681s
        speed: 0.1482s/iter; left time: 2129.4256s
        speed: 0.1492s/iter; left time: 2127.9783s
        speed: 0.1365s/iter; left time: 1934.1541s
        speed: 0.1399s/iter; left time: 1968.6168s
        speed: 0.1400s/iter; left time: 1955.1173s
        speed: 0.1475s/iter; left time: 2045.0873s
        speed: 0.1462s/iter; left time: 2012.1008s
        speed: 0.1486s/iter; left time: 2030.5059s
        speed: 0.1392s/iter; left time: 1889.0985s
        speed: 0.1359s/iter; left time: 1830.0555s
        speed: 0.1362s/iter; left time: 1821.0789s
        speed: 0.1380s/iter; left time: 1830.8140s
        speed: 0.1400s/iter; left time: 1842.7255s
        speed: 0.1408s/iter; left time: 1840.0198s
        speed: 0.1404s/iter; left time: 1821.1826s
        speed: 0.1390s/iter; left time: 1788.3899s
        speed: 0.1352s/iter; left time: 1725.9144s
        speed: 0.1353s/iter; left time: 1713.4649s
        speed: 0.1352s/iter; left time: 1699.5077s
        speed: 0.1351s/iter; left time: 1683.8779s
        speed: 0.1349s/iter; left time: 1668.2950s
        speed: 0.1350s/iter; left time: 1656.3314s
        speed: 0.1349s/iter; left time: 1641.9323s
        speed: 0.1358s/iter; left time: 1638.1283s
        speed: 0.1408s/iter; left time: 1684.5690s
        speed: 0.1397s/iter; left time: 1658.0442s
        speed: 0.1364s/iter; left time: 1605.3649s
        speed: 0.1355s/iter; left time: 1581.1990s
        speed: 0.1354s/iter; left time: 1565.9528s
        speed: 0.1353s/iter; left time: 1551.1118s
        speed: 0.1355s/iter; left time: 1539.6940s
        speed: 0.1426s/iter; left time: 1606.9790s
        speed: 0.1432s/iter; left time: 1599.1447s
        speed: 0.1396s/iter; left time: 1544.6739s
        speed: 0.1375s/iter; left time: 1508.0367s
        speed: 0.1357s/iter; left time: 1474.7991s
        speed: 0.1387s/iter; left time: 1493.3745s
        speed: 0.1369s/iter; left time: 1460.2498s
        speed: 0.1450s/iter; left time: 1532.1255s
        speed: 0.1354s/iter; left time: 1417.6454s
        speed: 0.1367s/iter; left time: 1417.3438s
        speed: 0.1383s/iter; left time: 1419.4605s
        speed: 0.1394s/iter; left time: 1417.0308s
        speed: 0.1381s/iter; left time: 1389.8998s
        speed: 0.1359s/iter; left time: 1354.2720s
        speed: 0.1379s/iter; left time: 1360.3001s
        speed: 0.1400s/iter; left time: 1366.8987s
        speed: 0.1357s/iter; left time: 1311.7372s
        speed: 0.1355s/iter; left time: 1296.3165s
        speed: 0.1400s/iter; left time: 1325.6356s
        speed: 0.1396s/iter; left time: 1307.1830s
        speed: 0.1438s/iter; left time: 1332.1646s
        speed: 0.1425s/iter; left time: 1306.2933s
        speed: 0.1405s/iter; left time: 1273.5351s
        speed: 0.1383s/iter; left time: 1240.3545s
        speed: 0.1399s/iter; left time: 1240.1732s
        speed: 0.1349s/iter; left time: 1182.6134s
        speed: 0.1357s/iter; left time: 1176.3826s
        speed: 0.1369s/iter; left time: 1172.9671s
        speed: 0.1396s/iter; left time: 1181.7322s
        speed: 0.1503s/iter; left time: 1257.1756s
        speed: 0.1516s/iter; left time: 1253.5654s
        speed: 0.1357s/iter; left time: 1108.4112s
        speed: 0.1349s/iter; left time: 1088.4797s
        speed: 0.1383s/iter; left time: 1102.0619s
        speed: 0.1379s/iter; left time: 1084.9288s
        speed: 0.1415s/iter; left time: 1098.7484s
        speed: 0.1359s/iter; left time: 1042.0906s
        speed: 0.1380s/iter; left time: 1044.4410s
        speed: 0.1410s/iter; left time: 1052.5578s
        speed: 0.1362s/iter; left time: 1003.2852s
        speed: 0.1407s/iter; left time: 1022.6825s
        speed: 0.1375s/iter; left time: 985.7831s
        speed: 0.1369s/iter; left time: 967.7417s
        speed: 0.1386s/iter; left time: 965.7697s
        speed: 0.1374s/iter; left time: 943.5992s
        speed: 0.1373s/iter; left time: 929.1964s
        speed: 0.1411s/iter; left time: 940.4838s
        speed: 0.1358s/iter; left time: 891.6402s
        speed: 0.1361s/iter; left time: 880.2868s
        speed: 0.1361s/iter; left time: 866.7270s
        speed: 0.1399s/iter; left time: 876.9626s
        speed: 0.1391s/iter; left time: 857.7881s
        speed: 0.1390s/iter; left time: 843.3105s
        speed: 0.1417s/iter; left time: 845.6014s
        speed: 0.1393s/iter; left time: 817.4279s
        speed: 0.1448s/iter; left time: 835.1416s
        speed: 0.1418s/iter; left time: 803.5788s
        speed: 0.1396s/iter; left time: 777.3048s
        speed: 0.1352s/iter; left time: 739.0622s
        speed: 0.1342s/iter; left time: 720.3296s
        speed: 0.1356s/iter; left time: 714.3432s
        speed: 0.1349s/iter; left time: 697.1035s
        speed: 0.1357s/iter; left time: 687.6707s
        speed: 0.1371s/iter; left time: 680.7852s
        speed: 0.1348s/iter; left time: 656.0860s
        speed: 0.1394s/iter; left time: 664.3335s
        speed: 0.1424s/iter; left time: 664.4477s
        speed: 0.1430s/iter; left time: 652.9596s
        speed: 0.1432s/iter; left time: 639.7950s
        speed: 0.1415s/iter; left time: 618.0999s
        speed: 0.1405s/iter; left time: 599.4433s
        speed: 0.1417s/iter; left time: 590.2657s
        speed: 0.1402s/iter; left time: 570.0331s
        speed: 0.1411s/iter; left time: 559.8401s
        speed: 0.1404s/iter; left time: 543.0194s
        speed: 0.1401s/iter; left time: 527.8371s
        speed: 0.1402s/iter; left time: 513.9767s
        speed: 0.1402s/iter; left time: 500.0044s
        speed: 0.1385s/iter; left time: 480.0465s
        speed: 0.1368s/iter; left time: 460.6511s
        speed: 0.1387s/iter; left time: 453.0378s
        speed: 0.1385s/iter; left time: 438.5996s
        speed: 0.1406s/iter; left time: 431.1120s
        speed: 0.1376s/iter; left time: 408.1366s
        speed: 0.1430s/iter; left time: 409.8844s
        speed: 0.1467s/iter; left time: 405.8109s
        speed: 0.1465s/iter; left time: 390.8402s
        speed: 0.1357s/iter; left time: 348.3139s
        speed: 0.1357s/iter; left time: 334.8425s
        speed: 0.1374s/iter; left time: 325.2861s
        speed: 0.1424s/iter; left time: 322.7762s
        speed: 0.1408s/iter; left time: 305.0053s
        speed: 0.1422s/iter; left time: 293.8833s
        speed: 0.1435s/iter; left time: 282.3321s
        speed: 0.1438s/iter; left time: 268.4056s
        speed: 0.1446s/iter; left time: 255.5926s
        speed: 0.1386s/iter; left time: 230.9797s
        speed: 0.1357s/iter; left time: 212.7075s
        speed: 0.1370s/iter; left time: 201.0489s
        speed: 0.1391s/iter; left time: 190.1014s
        speed: 0.1348s/iter; left time: 170.7725s
        speed: 0.1383s/iter; left time: 161.4011s
        speed: 0.1370s/iter; left time: 146.2138s
        speed: 0.1350s/iter; left time: 130.5128s
        speed: 0.1350s/iter; left time: 117.0379s
        speed: 0.1348s/iter; left time: 103.3958s
        speed: 0.1349s/iter; left time: 89.9780s
        speed: 0.1350s/iter; left time: 76.5349s
        speed: 0.1350s/iter; left time: 63.0486s
        speed: 0.1350s/iter; left time: 49.5276s
        speed: 0.1349s/iter; left time: 36.0312s
        speed: 0.1349s/iter; left time: 22.5222s
        speed: 0.1350s/iter; left time: 9.0454s
Epoch: 3 cost time: 2149.257043838501
Epoch: 3, Steps: 15466 | Train Loss: -48.9121188 Vali Loss: -47.4772334 
EarlyStopping counter: 1 out of 3
Updating learning rate to 2.5e-05
------------ Options -------------
anormly_ratio: 0.1
batch_size: 32
data_path: dataset/SWaT
dataset: SWaT
input_c: 51
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 51
pretrained_model: 10
win_size: 100
-------------- End ----------------
test: (449919, 51)
train: (495000, 51)
test: (449919, 51)
train: (495000, 51)
test: (449919, 51)
train: (495000, 51)
test: (449919, 51)
train: (495000, 51)
======================TEST MODE======================
/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0031170047065244427
pred:    (449900,)
gt:      (449900,)
pred:  (449900,)
gt:    (449900,)
Accuracy : 0.9775, Precision : 0.8841, Recall : 0.9371, F-score : 0.9099 

NeurIPS-TS

这个Bencmark比较特殊，需要搭建测试平台:https://github.com/datamllab/tods

使用系统的Python[失败]

按照步骤克隆仓库并运行pip install -e .时出现报错，一些包安装存在问题：

Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [154 lines of output]
      <string>:15: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
      <string>:51: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      <string>:54: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      <string>:51: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      performance hint: statsmodels/tsa/regime_switching/_hamilton_filter.pyx:83:5: Exception check on 'shamilton_filter_log_iteration' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.

问题定位和解决：安装依赖statsmodels==0.11.1出现问题，尝试降低版本，在根目录的setup.py中修改statsmodels==0.11.0rc1，再次执行pip install -e .时成功了。

新的问题：

ERROR: Could not find a version that satisfies the requirement tensorflow==2.4 (from tods) (from versions: 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.6.5, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.7.1, 2.7.2, 2.7.3, 2.7.4, 2.8.0rc0, 2.8.0rc1, 2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.9.0rc0, 2.9.0rc1, 2.9.0rc2, 2.9.0, 2.9.1, 2.9.2, 2.9.3, 2.10.0rc0, 2.10.0rc1, 2.10.0rc2, 2.10.0rc3, 2.10.0, 2.10.1, 2.11.0rc0, 2.11.0rc1, 2.11.0rc2, 2.11.0, 2.11.1, 2.12.0rc0, 2.12.0rc1, 2.12.0, 2.12.1, 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1, 2.14.0rc0, 2.14.0rc1, 2.14.0, 2.14.1, 2.15.0rc0, 2.15.0rc1, 2.15.0, 2.15.0.post1)
ERROR: No matching distribution found for tensorflow==2.4

问题定位和解决：tensorflow 2.4版本找不着，尝试在setup.py中修改为tensorflow==2.5

新的问题：

ERROR: Could not find a version that satisfies the requirement keras-nightly~=2.5.0.dev (from tensorflow) (from versions: none)
ERROR: No matching distribution found for keras-nightly~=2.5.0.dev

问题定位和解决：keras-nightly~=2.5.0.dev也找不着，手动去pypi.org官网下载安装https://pypi.org/project/keras-nightly/#history，我下载了这个的whl：https://pypi.org/project/keras-nightly/2.5.0.dev2021032900/，在终端使用pip install ./keras_nightly-2.5.0.dev2021032900-py2.py3-none-any.whl ，成功。

最后似乎没有完全成功？：

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
spyder 5.3.3 requires pyqt5<5.16, which is not installed.
spyder 5.3.3 requires pyqtwebengine<5.16, which is not installed.
daal4py 2021.6.0 requires daal==2021.4.0, which is not installed.
anaconda-project 0.11.1 requires ruamel-yaml, which is not installed.
pylint 2.14.5 requires typing-extensions>=3.10.0; python_version < "3.10", but you have typing-extensions 3.7.4.3 which is incompatible.
imageio 2.19.3 requires pillow>=8.3.2, but you have pillow 7.1.2 which is incompatible.
conda-repo-cli 1.0.20 requires clyent==1.2.1, but you have clyent 1.2.2 which is incompatible.
conda-repo-cli 1.0.20 requires nbformat==5.4.0, but you have nbformat 5.5.0 which is incompatible.
conda-repo-cli 1.0.20 requires PyYAML==6.0, but you have pyyaml 5.4.1 which is incompatible.
conda-repo-cli 1.0.20 requires requests==2.28.1, but you have requests 2.26.0 which is incompatible.
bokeh 2.4.3 requires typing-extensions>=3.10.0, but you have typing-extensions 3.7.4.3 which is incompatible.
black 22.6.0 requires typing-extensions>=3.10.0.0; python_version < "3.10", but you have typing-extensions 3.7.4.3 which is incompatible.
astroid 2.11.7 requires typing-extensions>=3.10; python_version < "3.10", but you have typing-extensions 3.7.4.3 which is incompatible.
Successfully installed GitPython-3.1.24 absl-py-0.15.0 aiosignal-1.3.1 astunparse-1.6.3 cachetools-5.3.2 combo-0.1.3 custom-inherit-2.3.2 dateparser-1.1.8 flatbuffers-1.12 frozendict-1.2 frozenlist-1.4.0 gast-0.4.0 gitdb-4.0.11 google-auth-2.25.1 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 gputil-1.4.0 grpcio-1.34.1 grpcio-testing-1.32.0 grpcio-tools-1.34.1 h5py-3.1.0 jsonpath-ng-1.5.3 jsonschema-4.0.1 keras-2.4.0 keras-preprocessing-1.1.2 liac-arff-2.5.0 more-itertools-8.5.0 nimfa-1.4.0 numpy-1.19.5 oauthlib-3.2.2 openml-0.11.0 opt-einsum-3.3.0 pandas-1.3.4 pillow-7.1.2 protobuf-3.20.3 pyarrow-14.0.1 pyod-1.0.5 pytypes-1.0b10 pyyaml-5.4.1 ray-2.8.1 requests-2.26.0 requests-oauthlib-1.3.1 rfc3339-validator-0.1.4 rfc3986-validator-0.1.1 rsa-4.9 scikit-learn-0.24.2 scipy-1.7.1 simplejson-3.12.0 six-1.15.0 smmap-5.0.1 statsmodels-0.11.0rc1 stumpy-1.4.0 tamu_axolotl-2021.2.11.1 tamu_d3m-2022.5.23 tensorboard-2.11.2 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorboardX-2.6.2.2 tensorflow-2.5.0 tensorflow-estimator-2.5.0 termcolor-1.1.0 tods-0.0.2 typing-extensions-3.7.4.3 typing-inspect-0.7.1 tzlocal-5.2 webcolors-1.11.1 wrapt-1.12.1 xgboost-2.0.2 xmltodict-0.13.0

使用Conda虚拟环境

在Pycharm中打开项目并新建conda interpreter, python 版本为3.8 (项目要求 Python 3.6 && pip 19+)

将根目录的setup.py更改过的依赖项版本还原

打开终端并确保在tods根目录且使用了conda的虚拟环境python，执行pip install -e .

这次安装无伤速通！总之就是以后再也不要用系统的python interpreter跑项目了！希望有时间能打包个docker镜像造福人类。

在根目录新建test_example.py如下并运行：

import pandas as pd

from tods import schemas as schemas_utils
from tods import generate_dataset, evaluate_pipeline

table_path = 'datasets/anomaly/raw_data/yahoo_sub_5.csv'
target_index = 6 # what column is the target
metric = 'F1_MACRO' # F1 on both label 0 and 1

# Read data and generate dataset
df = pd.read_csv(table_path)
dataset = generate_dataset(df, target_index)

# Load the default pipeline
pipeline = schemas_utils.load_default_pipeline()

# Run the pipeline
pipeline_result = evaluate_pipeline(dataset, pipeline, metric)
print(pipeline_result)

成功输出结果。

{'method_called': 'evaluate',
 'outputs': "[{'outputs.0':      d3mIndex  anomaly"
            '0           0        0'
            '1           1        0'
            '2           2        0'
            '3           3        0'
            '4           4        0'
            '...       ...      ...'
            '1395     1395        0'
            '1396     1396        0'
            '1397     1397        1'
            '1398     1398        1'
            '1399     1399        0'
            ''
            "[1400 rows x 2 columns]}, {'outputs.0':      d3mIndex  anomaly"
            '0           0        0'
            '1           1        0'
            '2           2        0'
            '3           3        0'
            '4           4        0'
            '...       ...      ...'
            '1395     1395        0'
            '1396     1396        0'
            '1397     1397        1'
            '1398     1398        1'
            '1399     1399        0'
            ''
            '[1400 rows x 2 columns]}]',
 'pipeline': '<d3m.metadata.pipeline.Pipeline object at 0x7fcab1e73cd0>',
 'scores': '     metric     value  normalized  randomSeed  fold'
           '0  F1_MACRO  0.708549    0.708549           0     0',
 'status': 'COMPLETED'}

2.6 总结

可见所有数据集与论文第4章Table 1所给数据并无较大出入，且所有F1-Score仍然如Table 1标注所示，领先于其余对比算法。

UCR dataset

下载于：https://compete.hexagon-ml.com/media/data/multi-dataset-time-series-anomaly-detection-39/data.zip

3. 分析与设计

一些思考：能不能跳出时间序列的限制，例如将代码文本作为输入，输出其是否存在异常。首先为了能让代码片段输入，肯定需要进行一定的编码，例如现在大模型流行使用的tokenizer方法。但tokenizer是将单个词映射为定长向量，而一个代码片通常由多个可视为词的符号组成，且词之间具有严密的逻辑关系。

3.1 Anomaly ratio $r$ 及其局限性

论文为每个数据集设定了不同的异常比例$r$，用于确定一个Anomaly Score 阈值$\delta$，使得验证集的异常点占比达到预设的$r$. 这存在需要人工经验取值的问题，况且异常比例$r$在验证集、训练集和测试集的情况很有可能存在不同，我认为这主要是由于时间序列的连续特性无法进行随机采样得到验证集导致的，论文代码中对于验证集的选取也受限于连续特性。另一方面，论文方法是无监督方法，设置阈值是无监督异常检测任务中难以避免的一个操作，而从这个角度进行基于标签数据的有监督学习改进是困难的，因为现实中的时序异常数据很难打标签。

3.1.1 尝试通过改变重建损失计算利用标签数据训练

简单二分策略的使用混合重建损失的结果：

Anomaly Ratio : 50.0
Threshold : 0.0
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.5032, Precision : 0.7028, Recall : 0.2204, F-score : 0.3356 

Threshold : 0.0
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.5032, Precision : 0.7028, Recall : 0.2204, F-score : 0.3356 

3.2 对网络异常检测数据的适用性

3.2.1 在NSL-KDD数据集上训练与测试

简单二分策略

将normal标签设为0，其余均为1，然后在不同的anomaly ratio下测试算法在NSLKDD数据集上的效果。

r=0.5%

(Anomaly-Transformer) ranlychan@ranlychan-ubuntu:/media/ranlychan/3E6E20236E1FD28F/Dev/Anomaly-Transformer$ bash ./scripts/NSLKDD.sh
------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
======================TRAIN MODE======================

------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 122
pretrained_model: 10
win_size: 100
-------------- End ----------------

======================TEST MODE======================
/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.02469959240406773
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.4585, Precision : 0.9481, Recall : 0.0514, F-score : 0.0975 

r=1.0%

(Anomaly-Transformer) ranlychan@ranlychan-ubuntu:/media/ranlychan/3E6E20236E1FD28F/Dev/Anomaly-Transformer$ bash ./scripts/NSLKDD.sh
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TRAIN MODE======================
        speed: 0.1369s/iter; left time: 1601.7550s
        speed: 0.1303s/iter; left time: 1512.0029s
        speed: 0.1306s/iter; left time: 1502.6880s
        speed: 0.1307s/iter; left time: 1490.6938s
        speed: 0.1308s/iter; left time: 1478.8884s
        speed: 0.1308s/iter; left time: 1465.4615s
        speed: 0.1308s/iter; left time: 1452.1905s
        speed: 0.1309s/iter; left time: 1440.6744s
        speed: 0.1313s/iter; left time: 1431.1096s
        speed: 0.1313s/iter; left time: 1418.0322s
        speed: 0.1312s/iter; left time: 1404.4908s
        speed: 0.1313s/iter; left time: 1392.5284s
        speed: 0.1313s/iter; left time: 1378.5787s
        speed: 0.1313s/iter; left time: 1365.6873s
        speed: 0.1312s/iter; left time: 1351.8481s
        speed: 0.1313s/iter; left time: 1339.4321s
        speed: 0.1312s/iter; left time: 1325.7825s
        speed: 0.1312s/iter; left time: 1312.7482s
        speed: 0.1312s/iter; left time: 1299.0108s
        speed: 0.1313s/iter; left time: 1286.7188s
        speed: 0.1312s/iter; left time: 1273.0868s
        speed: 0.1312s/iter; left time: 1260.0926s
        speed: 0.1316s/iter; left time: 1250.9483s
        speed: 0.1335s/iter; left time: 1254.8993s
        speed: 0.1328s/iter; left time: 1235.7682s
        speed: 0.1308s/iter; left time: 1204.1539s
        speed: 0.1309s/iter; left time: 1191.3958s
        speed: 0.1309s/iter; left time: 1178.2278s
        speed: 0.1322s/iter; left time: 1176.9927s
        speed: 0.1315s/iter; left time: 1157.5977s
        speed: 0.1315s/iter; left time: 1144.1452s
        speed: 0.1314s/iter; left time: 1130.5234s
        speed: 0.1314s/iter; left time: 1117.5005s
        speed: 0.1314s/iter; left time: 1104.2098s
        speed: 0.1314s/iter; left time: 1090.8631s
        speed: 0.1315s/iter; left time: 1078.5724s
        speed: 0.1314s/iter; left time: 1064.4714s
        speed: 0.1315s/iter; left time: 1052.1539s
        speed: 0.1353s/iter; left time: 1069.2737s
Epoch: 1 cost time: 517.9919922351837
Epoch: 1, Steps: 3934 | Train Loss: -47.0414631 Vali Loss: -47.4802924 
Validation loss decreased (inf --> -47.480292).  Saving model ...
Updating learning rate to 0.0001
        speed: 0.4840s/iter; left time: 3759.9627s
        speed: 0.1332s/iter; left time: 1021.6347s
        speed: 0.1323s/iter; left time: 1001.5881s
        speed: 0.1319s/iter; left time: 985.3198s
        speed: 0.1345s/iter; left time: 990.9520s
        speed: 0.1343s/iter; left time: 976.3390s
        speed: 0.1390s/iter; left time: 996.3020s
        speed: 0.1346s/iter; left time: 951.4199s
        speed: 0.1360s/iter; left time: 947.8869s
        speed: 0.1341s/iter; left time: 921.2710s
        speed: 0.1379s/iter; left time: 933.7204s
        speed: 0.1326s/iter; left time: 883.9982s
        speed: 0.1394s/iter; left time: 915.8344s
        speed: 0.1355s/iter; left time: 876.6080s
        speed: 0.1428s/iter; left time: 909.5229s
        speed: 0.1436s/iter; left time: 900.0828s
        speed: 0.1437s/iter; left time: 886.3455s
        speed: 0.1433s/iter; left time: 869.4626s
        speed: 0.1445s/iter; left time: 862.6468s
        speed: 0.1449s/iter; left time: 850.2270s
        speed: 0.1418s/iter; left time: 818.0399s
        speed: 0.1367s/iter; left time: 774.7034s
        speed: 0.1367s/iter; left time: 761.4599s
        speed: 0.1367s/iter; left time: 747.4475s
        speed: 0.1364s/iter; left time: 732.2058s
        speed: 0.1368s/iter; left time: 721.0366s
        speed: 0.1367s/iter; left time: 706.8142s
        speed: 0.1367s/iter; left time: 693.1174s
        speed: 0.1367s/iter; left time: 679.1568s
        speed: 0.1367s/iter; left time: 665.7158s
        speed: 0.1368s/iter; left time: 652.5808s
        speed: 0.1367s/iter; left time: 638.4532s
        speed: 0.1362s/iter; left time: 622.4852s
        speed: 0.1364s/iter; left time: 609.4713s
        speed: 0.1364s/iter; left time: 595.9573s
        speed: 0.1364s/iter; left time: 582.4890s
        speed: 0.1364s/iter; left time: 568.5825s
        speed: 0.1364s/iter; left time: 554.9945s
        speed: 0.1365s/iter; left time: 541.9608s
Epoch: 2 cost time: 539.6562712192535
Epoch: 2, Steps: 3934 | Train Loss: -48.5144279 Vali Loss: -48.1329151 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 0.4813s/iter; left time: 1845.6853s
        speed: 0.1364s/iter; left time: 509.2899s
        speed: 0.1362s/iter; left time: 495.1138s
        speed: 0.1364s/iter; left time: 482.0215s
        speed: 0.1365s/iter; left time: 468.8174s
        speed: 0.1364s/iter; left time: 455.0160s
        speed: 0.1364s/iter; left time: 441.3130s
        speed: 0.1364s/iter; left time: 427.6231s
        speed: 0.1364s/iter; left time: 414.0394s
        speed: 0.1364s/iter; left time: 400.4002s
        speed: 0.1363s/iter; left time: 386.2983s
        speed: 0.1363s/iter; left time: 372.6786s
        speed: 0.1364s/iter; left time: 359.5014s
        speed: 0.1364s/iter; left time: 345.8320s
        speed: 0.1364s/iter; left time: 332.1094s
        speed: 0.1363s/iter; left time: 318.1958s
        speed: 0.1364s/iter; left time: 304.7477s
        speed: 0.1362s/iter; left time: 290.8440s
        speed: 0.1366s/iter; left time: 277.9560s
        speed: 0.1363s/iter; left time: 263.7950s
        speed: 0.1364s/iter; left time: 250.2387s
        speed: 0.1364s/iter; left time: 236.6038s
        speed: 0.1363s/iter; left time: 222.8828s
        speed: 0.1363s/iter; left time: 209.2962s
        speed: 0.1364s/iter; left time: 195.7429s
        speed: 0.1363s/iter; left time: 181.9378s
        speed: 0.1364s/iter; left time: 168.4278s
        speed: 0.1362s/iter; left time: 154.6098s
        speed: 0.1360s/iter; left time: 140.7701s
        speed: 0.1364s/iter; left time: 127.5001s
        speed: 0.1359s/iter; left time: 113.5145s
        speed: 0.1362s/iter; left time: 100.1328s
        speed: 0.1363s/iter; left time: 86.5670s
        speed: 0.1362s/iter; left time: 72.8662s
        speed: 0.1361s/iter; left time: 59.1943s
        speed: 0.1363s/iter; left time: 45.6561s
        speed: 0.1361s/iter; left time: 31.9842s
        speed: 0.1363s/iter; left time: 18.3962s
        speed: 0.1364s/iter; left time: 4.7725s
Epoch: 3 cost time: 536.1699142456055
Epoch: 3, Steps: 3934 | Train Loss: -48.7206043 Vali Loss: -48.3033336 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 122
pretrained_model: 10
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TEST MODE======================
/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.007011290364898737
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.4537, Precision : 0.8757, Recall : 0.0468, F-score : 0.0888 

/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0031170047065244427
pred:    (449900,)
gt:      (449900,)
pred:  (449900,)
gt:    (449900,)
Accuracy : 0.9775, Precision : 0.8841, Recall : 0.9371, F-score : 0.9099 

(Anomaly-Transformer) ranlychan@ranlychan-ubuntu:/media/ranlychan/3E6E20236E1FD28F/Dev/Anomaly-Transformer$ bash ./scripts/NSLKDD.sh
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TRAIN MODE======================
        speed: 0.1369s/iter; left time: 1601.7550s
        speed: 0.1303s/iter; left time: 1512.0029s
        speed: 0.1306s/iter; left time: 1502.6880s
        speed: 0.1307s/iter; left time: 1490.6938s
        speed: 0.1308s/iter; left time: 1478.8884s
        speed: 0.1308s/iter; left time: 1465.4615s
        speed: 0.1308s/iter; left time: 1452.1905s
        speed: 0.1309s/iter; left time: 1440.6744s
        speed: 0.1313s/iter; left time: 1431.1096s
        speed: 0.1313s/iter; left time: 1418.0322s
        speed: 0.1312s/iter; left time: 1404.4908s
        speed: 0.1313s/iter; left time: 1392.5284s
        speed: 0.1313s/iter; left time: 1378.5787s
        speed: 0.1313s/iter; left time: 1365.6873s
        speed: 0.1312s/iter; left time: 1351.8481s
        speed: 0.1313s/iter; left time: 1339.4321s
        speed: 0.1312s/iter; left time: 1325.7825s
        speed: 0.1312s/iter; left time: 1312.7482s
        speed: 0.1312s/iter; left time: 1299.0108s
        speed: 0.1313s/iter; left time: 1286.7188s
        speed: 0.1312s/iter; left time: 1273.0868s
        speed: 0.1312s/iter; left time: 1260.0926s
        speed: 0.1316s/iter; left time: 1250.9483s
        speed: 0.1335s/iter; left time: 1254.8993s
        speed: 0.1328s/iter; left time: 1235.7682s
        speed: 0.1308s/iter; left time: 1204.1539s
        speed: 0.1309s/iter; left time: 1191.3958s
        speed: 0.1309s/iter; left time: 1178.2278s
        speed: 0.1322s/iter; left time: 1176.9927s
        speed: 0.1315s/iter; left time: 1157.5977s
        speed: 0.1315s/iter; left time: 1144.1452s
        speed: 0.1314s/iter; left time: 1130.5234s
        speed: 0.1314s/iter; left time: 1117.5005s
        speed: 0.1314s/iter; left time: 1104.2098s
        speed: 0.1314s/iter; left time: 1090.8631s
        speed: 0.1315s/iter; left time: 1078.5724s
        speed: 0.1314s/iter; left time: 1064.4714s
        speed: 0.1315s/iter; left time: 1052.1539s
        speed: 0.1353s/iter; left time: 1069.2737s
Epoch: 1 cost time: 517.9919922351837
Epoch: 1, Steps: 3934 | Train Loss: -47.0414631 Vali Loss: -47.4802924 
Validation loss decreased (inf --> -47.480292).  Saving model ...
Updating learning rate to 0.0001
        speed: 0.4840s/iter; left time: 3759.9627s
        speed: 0.1332s/iter; left time: 1021.6347s
        speed: 0.1323s/iter; left time: 1001.5881s
        speed: 0.1319s/iter; left time: 985.3198s
        speed: 0.1345s/iter; left time: 990.9520s
        speed: 0.1343s/iter; left time: 976.3390s
        speed: 0.1390s/iter; left time: 996.3020s
        speed: 0.1346s/iter; left time: 951.4199s
        speed: 0.1360s/iter; left time: 947.8869s
        speed: 0.1341s/iter; left time: 921.2710s
        speed: 0.1379s/iter; left time: 933.7204s
        speed: 0.1326s/iter; left time: 883.9982s
        speed: 0.1394s/iter; left time: 915.8344s
        speed: 0.1355s/iter; left time: 876.6080s
        speed: 0.1428s/iter; left time: 909.5229s
        speed: 0.1436s/iter; left time: 900.0828s
        speed: 0.1437s/iter; left time: 886.3455s
        speed: 0.1433s/iter; left time: 869.4626s
        speed: 0.1445s/iter; left time: 862.6468s
        speed: 0.1449s/iter; left time: 850.2270s
        speed: 0.1418s/iter; left time: 818.0399s
        speed: 0.1367s/iter; left time: 774.7034s
        speed: 0.1367s/iter; left time: 761.4599s
        speed: 0.1367s/iter; left time: 747.4475s
        speed: 0.1364s/iter; left time: 732.2058s
        speed: 0.1368s/iter; left time: 721.0366s
        speed: 0.1367s/iter; left time: 706.8142s
        speed: 0.1367s/iter; left time: 693.1174s
        speed: 0.1367s/iter; left time: 679.1568s
        speed: 0.1367s/iter; left time: 665.7158s
        speed: 0.1368s/iter; left time: 652.5808s
        speed: 0.1367s/iter; left time: 638.4532s
        speed: 0.1362s/iter; left time: 622.4852s
        speed: 0.1364s/iter; left time: 609.4713s
        speed: 0.1364s/iter; left time: 595.9573s
        speed: 0.1364s/iter; left time: 582.4890s
        speed: 0.1364s/iter; left time: 568.5825s
        speed: 0.1364s/iter; left time: 554.9945s
        speed: 0.1365s/iter; left time: 541.9608s
Epoch: 2 cost time: 539.6562712192535
Epoch: 2, Steps: 3934 | Train Loss: -48.5144279 Vali Loss: -48.1329151 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 0.4813s/iter; left time: 1845.6853s
        speed: 0.1364s/iter; left time: 509.2899s
        speed: 0.1362s/iter; left time: 495.1138s
        speed: 0.1364s/iter; left time: 482.0215s
        speed: 0.1365s/iter; left time: 468.8174s
        speed: 0.1364s/iter; left time: 455.0160s
        speed: 0.1364s/iter; left time: 441.3130s
        speed: 0.1364s/iter; left time: 427.6231s
        speed: 0.1364s/iter; left time: 414.0394s
        speed: 0.1364s/iter; left time: 400.4002s
        speed: 0.1363s/iter; left time: 386.2983s
        speed: 0.1363s/iter; left time: 372.6786s
        speed: 0.1364s/iter; left time: 359.5014s
        speed: 0.1364s/iter; left time: 345.8320s
        speed: 0.1364s/iter; left time: 332.1094s
        speed: 0.1363s/iter; left time: 318.1958s
        speed: 0.1364s/iter; left time: 304.7477s
        speed: 0.1362s/iter; left time: 290.8440s
        speed: 0.1366s/iter; left time: 277.9560s
        speed: 0.1363s/iter; left time: 263.7950s
        speed: 0.1364s/iter; left time: 250.2387s
        speed: 0.1364s/iter; left time: 236.6038s
        speed: 0.1363s/iter; left time: 222.8828s
        speed: 0.1363s/iter; left time: 209.2962s
        speed: 0.1364s/iter; left time: 195.7429s
        speed: 0.1363s/iter; left time: 181.9378s
        speed: 0.1364s/iter; left time: 168.4278s
        speed: 0.1362s/iter; left time: 154.6098s
        speed: 0.1360s/iter; left time: 140.7701s
        speed: 0.1364s/iter; left time: 127.5001s
        speed: 0.1359s/iter; left time: 113.5145s
        speed: 0.1362s/iter; left time: 100.1328s
        speed: 0.1363s/iter; left time: 86.5670s
        speed: 0.1362s/iter; left time: 72.8662s
        speed: 0.1361s/iter; left time: 59.1943s
        speed: 0.1363s/iter; left time: 45.6561s
        speed: 0.1361s/iter; left time: 31.9842s
        speed: 0.1363s/iter; left time: 18.3962s
        speed: 0.1364s/iter; left time: 4.7725s
Epoch: 3 cost time: 536.1699142456055
Epoch: 3, Steps: 3934 | Train Loss: -48.7206043 Vali Loss: -48.3033336 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 122
pretrained_model: 10
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TEST MODE======================
/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.007011290364898737
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.4537, Precision : 0.8757, Recall : 0.0468, F-score : 0.0888 

r=50.0%

------------ Options -------------
anormly_ratio: 50.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 10
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TRAIN MODE======================
        speed: 0.1355s/iter; left time: 5316.9313s
        speed: 0.1356s/iter; left time: 5307.0847s
        speed: 0.1361s/iter; left time: 5313.6697s
        speed: 0.1319s/iter; left time: 5136.0505s
        speed: 0.1325s/iter; left time: 5148.3551s
        speed: 0.1319s/iter; left time: 5109.8676s
        speed: 0.1323s/iter; left time: 5113.9623s
        speed: 0.1322s/iter; left time: 5096.4198s
        speed: 0.1315s/iter; left time: 5054.2963s
        speed: 0.1322s/iter; left time: 5068.8596s
        speed: 0.1324s/iter; left time: 5062.5358s
        speed: 0.1340s/iter; left time: 5111.2675s
        speed: 0.1310s/iter; left time: 4983.8302s
        speed: 0.1316s/iter; left time: 4993.7247s
        speed: 0.1310s/iter; left time: 4958.1371s
        speed: 0.1310s/iter; left time: 4944.1936s
        speed: 0.1310s/iter; left time: 4931.2664s
        speed: 0.1311s/iter; left time: 4920.0523s
        speed: 0.1310s/iter; left time: 4905.3923s
        speed: 0.1311s/iter; left time: 4894.8455s
        speed: 0.1310s/iter; left time: 4879.8760s
        speed: 0.1311s/iter; left time: 4869.0741s
        speed: 0.1310s/iter; left time: 4851.3908s
        speed: 0.1311s/iter; left time: 4841.5871s
        speed: 0.1311s/iter; left time: 4828.5362s
        speed: 0.1311s/iter; left time: 4816.3585s
        speed: 0.1310s/iter; left time: 4800.7019s
        speed: 0.1309s/iter; left time: 4784.6071s
        speed: 0.1310s/iter; left time: 4773.5566s
        speed: 0.1310s/iter; left time: 4761.9220s
        speed: 0.1310s/iter; left time: 4748.1666s
        speed: 0.1310s/iter; left time: 4734.0354s
        speed: 0.1310s/iter; left time: 4720.0640s
        speed: 0.1310s/iter; left time: 4708.9502s
        speed: 0.1310s/iter; left time: 4694.6541s
        speed: 0.1311s/iter; left time: 4685.8344s
        speed: 0.1309s/iter; left time: 4665.9214s
        speed: 0.1309s/iter; left time: 4654.0228s
        speed: 0.1310s/iter; left time: 4642.1672s
Epoch: 1 cost time: 518.1424803733826
Epoch: 1, Steps: 3934 | Train Loss: -46.8131543 Vali Loss: -47.3336469 
Validation loss decreased (inf --> -47.333647).  Saving model ...
Updating learning rate to 0.0001
        speed: 0.4610s/iter; left time: 16276.7726s
        speed: 0.1310s/iter; left time: 4612.5694s
        speed: 0.1310s/iter; left time: 4599.6580s
        speed: 0.1310s/iter; left time: 4587.0812s
        speed: 0.1310s/iter; left time: 4572.4280s
        speed: 0.1312s/iter; left time: 4565.2154s
        speed: 0.1310s/iter; left time: 4546.2637s
        speed: 0.1310s/iter; left time: 4534.1559s
        speed: 0.1310s/iter; left time: 4519.9412s
        speed: 0.1310s/iter; left time: 4506.3366s
        speed: 0.1310s/iter; left time: 4493.2462s
        speed: 0.1309s/iter; left time: 4479.1329s
        speed: 0.1309s/iter; left time: 4465.1925s
        speed: 0.1309s/iter; left time: 4451.1384s
        speed: 0.1309s/iter; left time: 4439.9778s
        speed: 0.1309s/iter; left time: 4425.5404s
        speed: 0.1309s/iter; left time: 4412.6608s
        speed: 0.1309s/iter; left time: 4399.0118s
        speed: 0.1309s/iter; left time: 4386.5066s
        speed: 0.1310s/iter; left time: 4374.8416s
        speed: 0.1309s/iter; left time: 4360.6469s
        speed: 0.1310s/iter; left time: 4348.4842s
        speed: 0.1309s/iter; left time: 4333.9263s
        speed: 0.1309s/iter; left time: 4321.6431s
        speed: 0.1310s/iter; left time: 4309.9051s
        speed: 0.1309s/iter; left time: 4295.7049s
        speed: 0.1309s/iter; left time: 4282.2557s
        speed: 0.1309s/iter; left time: 4268.6591s
        speed: 0.1309s/iter; left time: 4256.5062s
        speed: 0.1309s/iter; left time: 4241.6757s
        speed: 0.1309s/iter; left time: 4228.4744s
        speed: 0.1309s/iter; left time: 4215.9839s
        speed: 0.1309s/iter; left time: 4203.7866s
        speed: 0.1310s/iter; left time: 4192.8665s
        speed: 0.1310s/iter; left time: 4179.2397s
        speed: 0.1310s/iter; left time: 4165.1929s
        speed: 0.1309s/iter; left time: 4151.0590s
        speed: 0.1310s/iter; left time: 4140.1505s
        speed: 0.1310s/iter; left time: 4126.8890s
Epoch: 2 cost time: 515.0924828052521
Epoch: 2, Steps: 3934 | Train Loss: -48.4168298 Vali Loss: -47.9248486 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 0.4590s/iter; left time: 14398.8618s
        speed: 0.1309s/iter; left time: 4094.5120s
        speed: 0.1309s/iter; left time: 4079.1813s
        speed: 0.1309s/iter; left time: 4067.8194s
        speed: 0.1309s/iter; left time: 4054.6873s
        speed: 0.1309s/iter; left time: 4040.4983s
        speed: 0.1309s/iter; left time: 4027.7394s
        speed: 0.1309s/iter; left time: 4015.9503s
        speed: 0.1309s/iter; left time: 4002.8357s
        speed: 0.1319s/iter; left time: 4020.1789s
        speed: 0.1328s/iter; left time: 4032.3251s
        speed: 0.1311s/iter; left time: 3969.1873s
        speed: 0.1311s/iter; left time: 3956.2238s
        speed: 0.1311s/iter; left time: 3943.2807s
        speed: 0.1314s/iter; left time: 3937.5022s
        speed: 0.1317s/iter; left time: 3935.3920s
        speed: 0.1325s/iter; left time: 3944.7383s
        speed: 0.1329s/iter; left time: 3942.8348s
        speed: 0.1313s/iter; left time: 3883.1325s
        speed: 0.1340s/iter; left time: 3950.0506s
        speed: 0.1334s/iter; left time: 3919.4091s
        speed: 0.1309s/iter; left time: 3832.1620s
        speed: 0.1309s/iter; left time: 3818.9089s
        speed: 0.1309s/iter; left time: 3806.7406s
        speed: 0.1309s/iter; left time: 3791.2263s
        speed: 0.1309s/iter; left time: 3779.5752s
        speed: 0.1309s/iter; left time: 3765.7227s
        speed: 0.1309s/iter; left time: 3753.4292s
        speed: 0.1309s/iter; left time: 3739.4836s
        speed: 0.1309s/iter; left time: 3726.6624s
        speed: 0.1309s/iter; left time: 3714.4806s
        speed: 0.1310s/iter; left time: 3703.1740s
        speed: 0.1309s/iter; left time: 3688.7920s
        speed: 0.1309s/iter; left time: 3673.9377s
        speed: 0.1311s/iter; left time: 3667.6697s
        speed: 0.1309s/iter; left time: 3649.3432s
        speed: 0.1310s/iter; left time: 3637.1319s
        speed: 0.1309s/iter; left time: 3622.2054s
        speed: 0.1309s/iter; left time: 3608.6637s
Epoch: 3 cost time: 516.3497984409332
Epoch: 3, Steps: 3934 | Train Loss: -48.6391877 Vali Loss: -48.1122985 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
        speed: 0.4590s/iter; left time: 12595.1135s
        speed: 0.1309s/iter; left time: 3578.3130s
        speed: 0.1309s/iter; left time: 3566.5477s
        speed: 0.1309s/iter; left time: 3552.5281s
        speed: 0.1310s/iter; left time: 3541.1830s
        speed: 0.1309s/iter; left time: 3526.7139s
        speed: 0.1309s/iter; left time: 3514.5533s
        speed: 0.1309s/iter; left time: 3501.3495s
        speed: 0.1310s/iter; left time: 3490.4514s
        speed: 0.1309s/iter; left time: 3474.1579s
        speed: 0.1309s/iter; left time: 3461.5496s
        speed: 0.1309s/iter; left time: 3448.6966s
        speed: 0.1309s/iter; left time: 3434.1434s
        speed: 0.1309s/iter; left time: 3422.2355s
        speed: 0.1309s/iter; left time: 3407.6903s
        speed: 0.1310s/iter; left time: 3396.7607s
        speed: 0.1309s/iter; left time: 3381.6889s
        speed: 0.1309s/iter; left time: 3369.2955s
        speed: 0.1309s/iter; left time: 3355.4160s
        speed: 0.1309s/iter; left time: 3343.8095s
        speed: 0.1309s/iter; left time: 3329.5964s
        speed: 0.1309s/iter; left time: 3316.2136s
        speed: 0.1309s/iter; left time: 3303.9051s
        speed: 0.1309s/iter; left time: 3290.4389s
        speed: 0.1309s/iter; left time: 3277.7411s
        speed: 0.1309s/iter; left time: 3264.3852s
        speed: 0.1310s/iter; left time: 3254.0794s
        speed: 0.1310s/iter; left time: 3241.5225s
        speed: 0.1310s/iter; left time: 3226.8819s
        speed: 0.1310s/iter; left time: 3213.5240s
        speed: 0.1310s/iter; left time: 3201.2858s
        speed: 0.1309s/iter; left time: 3185.9262s
        speed: 0.1309s/iter; left time: 3172.8487s
        speed: 0.1309s/iter; left time: 3160.0894s
        speed: 0.1310s/iter; left time: 3148.5221s
        speed: 0.1309s/iter; left time: 3133.3825s
        speed: 0.1309s/iter; left time: 3121.3162s
        speed: 0.1309s/iter; left time: 3106.9576s
        speed: 0.1309s/iter; left time: 3095.5211s
Epoch: 4 cost time: 514.96653175354
Epoch: 4, Steps: 3934 | Train Loss: -48.7457672 Vali Loss: -48.3127411 
EarlyStopping counter: 3 out of 3
Early stopping
------------ Options -------------
anormly_ratio: 50.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 122
pretrained_model: 20
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TEST MODE======================
/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.5032, Precision : 0.7028, Recall : 0.2204, F-score : 0.3356 

r=60.0%

Threshold : 0.0
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.5284, Precision : 0.6548, Recall : 0.3625, F-score : 0.4666 

------------ Options -------------
anormly_ratio: 60.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 10
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TRAIN MODE======================
        speed: 0.1388s/iter; left time: 5446.8367s
        speed: 0.1314s/iter; left time: 5141.6661s
        speed: 0.1315s/iter; left time: 5133.9673s
        speed: 0.1315s/iter; left time: 5121.2421s
        speed: 0.1365s/iter; left time: 5300.2137s
        speed: 0.1374s/iter; left time: 5322.4496s
        speed: 0.1333s/iter; left time: 5149.1546s
        speed: 0.1323s/iter; left time: 5099.9489s
        speed: 0.1311s/iter; left time: 5038.8110s
        speed: 0.1310s/iter; left time: 5023.0063s
        speed: 0.1445s/iter; left time: 5524.1188s
        speed: 0.1505s/iter; left time: 5740.0504s
        speed: 0.1497s/iter; left time: 5694.0534s
        speed: 0.1498s/iter; left time: 5684.9724s
        speed: 0.1495s/iter; left time: 5657.5665s
        speed: 0.1501s/iter; left time: 5666.0902s
        speed: 0.1508s/iter; left time: 5676.4843s
        speed: 0.1447s/iter; left time: 5432.0509s
        speed: 0.1438s/iter; left time: 5383.0355s
        speed: 0.1459s/iter; left time: 5446.3010s
        speed: 0.1380s/iter; left time: 5138.5641s
        speed: 0.1528s/iter; left time: 5676.7783s
        speed: 0.1533s/iter; left time: 5678.8169s
        speed: 0.1487s/iter; left time: 5494.3238s
        speed: 0.1487s/iter; left time: 5478.8813s
        speed: 0.1354s/iter; left time: 4973.0808s
        speed: 0.1330s/iter; left time: 4874.2198s
        speed: 0.1327s/iter; left time: 4849.0647s
        speed: 0.1334s/iter; left time: 4863.0441s
        speed: 0.1332s/iter; left time: 4840.3917s
        speed: 0.1392s/iter; left time: 5045.4379s
        speed: 0.1484s/iter; left time: 5363.5204s
        speed: 0.1490s/iter; left time: 5370.1684s
        speed: 0.1318s/iter; left time: 4736.2374s
        speed: 0.1317s/iter; left time: 4719.7045s
        speed: 0.1317s/iter; left time: 4705.7837s
        speed: 0.1317s/iter; left time: 4693.7737s
        speed: 0.1380s/iter; left time: 4903.0555s
        speed: 0.1551s/iter; left time: 5496.2793s
Epoch: 1 cost time: 553.4214758872986
Epoch: 1, Steps: 3934 | Train Loss: -47.2930476 Vali Loss: -47.4361793 
Validation loss decreased (inf --> -47.436179).  Saving model ...
Updating learning rate to 0.0001
        speed: 0.5466s/iter; left time: 19300.5711s
        speed: 0.1308s/iter; left time: 4605.8315s
        speed: 0.1311s/iter; left time: 4601.2093s
        speed: 0.1311s/iter; left time: 4589.1366s
        speed: 0.1417s/iter; left time: 4945.4468s
        speed: 0.1421s/iter; left time: 4945.5076s
        speed: 0.1327s/iter; left time: 4604.5278s
        speed: 0.1344s/iter; left time: 4650.3773s
        speed: 0.1396s/iter; left time: 4815.6763s
        speed: 0.1351s/iter; left time: 4649.1411s
        speed: 0.1332s/iter; left time: 4568.2611s
        speed: 0.1401s/iter; left time: 4793.6377s
        speed: 0.1325s/iter; left time: 4520.7700s
        speed: 0.1468s/iter; left time: 4991.2005s
        speed: 0.1412s/iter; left time: 4786.1676s
        speed: 0.1330s/iter; left time: 4496.7579s
        speed: 0.1337s/iter; left time: 4508.0182s
        speed: 0.1333s/iter; left time: 4479.6438s
        speed: 0.1326s/iter; left time: 4442.6618s
        speed: 0.1321s/iter; left time: 4413.8824s
        speed: 0.1310s/iter; left time: 4364.2314s
        speed: 0.1426s/iter; left time: 4734.3299s
        speed: 0.1338s/iter; left time: 4430.3676s
        speed: 0.1325s/iter; left time: 4372.0640s
        speed: 0.1327s/iter; left time: 4367.8091s
        speed: 0.1325s/iter; left time: 4345.4439s
        speed: 0.1327s/iter; left time: 4341.0565s
        speed: 0.1326s/iter; left time: 4324.5873s
        speed: 0.1356s/iter; left time: 4406.5620s
        speed: 0.1464s/iter; left time: 4743.1841s
        speed: 0.1395s/iter; left time: 4506.8946s
        speed: 0.1423s/iter; left time: 4583.7955s
        speed: 0.1461s/iter; left time: 4691.0209s
        speed: 0.1415s/iter; left time: 4530.2409s
        speed: 0.1423s/iter; left time: 4538.9798s
        speed: 0.1390s/iter; left time: 4421.9290s
        speed: 0.1395s/iter; left time: 4421.7746s
        speed: 0.1370s/iter; left time: 4330.0266s
        speed: 0.1368s/iter; left time: 4311.1386s
Epoch: 2 cost time: 539.1221182346344
Epoch: 2, Steps: 3934 | Train Loss: -48.4677824 Vali Loss: -47.9757537 
EarlyStopping counter: 1 out of 3
Updating learning rate to 5e-05
        speed: 0.4911s/iter; left time: 15407.6395s
        speed: 0.1325s/iter; left time: 4144.1985s
        speed: 0.1348s/iter; left time: 4202.4603s
        speed: 0.1361s/iter; left time: 4230.0066s
        speed: 0.1315s/iter; left time: 4074.3328s
        speed: 0.1355s/iter; left time: 4182.4872s
        speed: 0.1483s/iter; left time: 4562.3436s
        speed: 0.1509s/iter; left time: 4627.7957s
        speed: 0.1495s/iter; left time: 4572.1388s
        speed: 0.1501s/iter; left time: 4574.2171s
        speed: 0.1498s/iter; left time: 4548.7989s
        speed: 0.1459s/iter; left time: 4416.7031s
        speed: 0.1436s/iter; left time: 4332.7850s
        speed: 0.1434s/iter; left time: 4311.9022s
        speed: 0.1465s/iter; left time: 4390.5848s
        speed: 0.1476s/iter; left time: 4409.8262s
        speed: 0.1477s/iter; left time: 4398.3314s
        speed: 0.1445s/iter; left time: 4286.7148s
        speed: 0.1463s/iter; left time: 4327.3261s
        speed: 0.1437s/iter; left time: 4235.5724s
        speed: 0.1437s/iter; left time: 4219.7474s
        speed: 0.1462s/iter; left time: 4280.5025s
        speed: 0.1448s/iter; left time: 4223.6321s
        speed: 0.1444s/iter; left time: 4198.9182s
        speed: 0.1446s/iter; left time: 4190.8026s
        speed: 0.1442s/iter; left time: 4164.7457s
        speed: 0.1446s/iter; left time: 4159.9640s
        speed: 0.1440s/iter; left time: 4129.8085s
        speed: 0.1447s/iter; left time: 4133.1227s
        speed: 0.1444s/iter; left time: 4111.4583s
        speed: 0.1449s/iter; left time: 4110.1645s
        speed: 0.1440s/iter; left time: 4072.0241s
        speed: 0.1438s/iter; left time: 4050.8166s
        speed: 0.1444s/iter; left time: 4055.0368s
        speed: 0.1441s/iter; left time: 4031.9231s
        speed: 0.1442s/iter; left time: 4020.3876s
        speed: 0.1445s/iter; left time: 4012.2267s
        speed: 0.1448s/iter; left time: 4007.0133s
        speed: 0.1445s/iter; left time: 3985.0134s
Epoch: 3 cost time: 565.8869743347168
Epoch: 3, Steps: 3934 | Train Loss: -48.6567995 Vali Loss: -48.1764615 
EarlyStopping counter: 2 out of 3
Updating learning rate to 2.5e-05
        speed: 0.5192s/iter; left time: 14247.0282s
        speed: 0.1448s/iter; left time: 3958.9565s
        speed: 0.1448s/iter; left time: 3943.9087s
        speed: 0.1443s/iter; left time: 3916.2532s
        speed: 0.1443s/iter; left time: 3901.8156s
        speed: 0.1440s/iter; left time: 3880.5454s
        speed: 0.1444s/iter; left time: 3874.2425s
        speed: 0.1448s/iter; left time: 3871.7225s
        speed: 0.1443s/iter; left time: 3842.9239s
        speed: 0.1441s/iter; left time: 3824.7977s
        speed: 0.1441s/iter; left time: 3810.2506s
        speed: 0.1442s/iter; left time: 3797.6073s
        speed: 0.1442s/iter; left time: 3782.7356s
        speed: 0.1443s/iter; left time: 3771.7759s
        speed: 0.1444s/iter; left time: 3759.7137s
        speed: 0.1440s/iter; left time: 3736.1705s
        speed: 0.1444s/iter; left time: 3730.0108s
        speed: 0.1440s/iter; left time: 3707.2472s
        speed: 0.1445s/iter; left time: 3706.0638s
        speed: 0.1438s/iter; left time: 3671.9295s
        speed: 0.1442s/iter; left time: 3669.4479s
        speed: 0.1446s/iter; left time: 3664.2420s
        speed: 0.1446s/iter; left time: 3648.3342s
        speed: 0.1446s/iter; left time: 3634.8287s
        speed: 0.1474s/iter; left time: 3691.6503s
        speed: 0.1511s/iter; left time: 3767.4017s
        speed: 0.1494s/iter; left time: 3711.8324s
        speed: 0.1441s/iter; left time: 3564.2086s
        speed: 0.1439s/iter; left time: 3545.6107s
        speed: 0.1468s/iter; left time: 3603.3738s
        speed: 0.1452s/iter; left time: 3548.3801s
        speed: 0.1446s/iter; left time: 3520.4837s
        speed: 0.1441s/iter; left time: 3491.6876s
        speed: 0.1440s/iter; left time: 3476.8049s
        speed: 0.1448s/iter; left time: 3481.9156s
        speed: 0.1466s/iter; left time: 3508.7926s
        speed: 0.1447s/iter; left time: 3450.3449s
        speed: 0.1440s/iter; left time: 3418.7299s
        speed: 0.1439s/iter; left time: 3401.0288s
Epoch: 4 cost time: 569.6859128475189
Epoch: 4, Steps: 3934 | Train Loss: -48.7182953 Vali Loss: -48.2986017 
EarlyStopping counter: 3 out of 3
Early stopping
------------ Options -------------
anormly_ratio: 60.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD
input_c: 122
k: 3
lr: 0.0001
mode: test
model_save_path: checkpoints
num_epochs: 10
output_c: 122
pretrained_model: 20
win_size: 100
-------------- End ----------------
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
test: (22544, 122)
train: (125973, 122)
======================TEST MODE======================
/home/ranlychan/anaconda3/envs/Anomaly-Transformer/lib/python3.6/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Threshold : 0.0
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.5284, Precision : 0.6548, Recall : 0.3625, F-score : 0.4666 

移除训练集异常点

简单去除训练集异常点数据：

#train ar=0.5%
#test ar=60%
Threshold : 8.954137840471525e-22
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.4903, Precision : 0.6855, Recall : 0.1930, F-score : 0.3012 
#train ar=60%
#test ar=60%
Threshold : 1.6401431994555087e-32
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.4899, Precision : 0.6622, Recall : 0.2119, F-score : 0.3210 

对异常点进行KNN插补：

一对多策略 OvR

单独将每个类标为1，其余标为0，每个类的model checkpoint 使用各自的anomaly ratio单独训练。

------------ Options -------------
anormly_ratio: 20.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_0
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.0
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.7100, Precision : 0.2341, Recall : 0.1776, F-score : 0.2020 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_1
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.002423033353406936
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9736, Precision : 0.0107, Recall : 0.0094, F-score : 0.0100 

------------ Options -------------
anormly_ratio: 5.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_2
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 2.4234104793409644e-19
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9197, Precision : 0.0644, Recall : 0.0604, F-score : 0.0623 

------------ Options -------------
anormly_ratio: 5.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_3
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 2.2883160614427485e-21
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9076, Precision : 0.0818, Recall : 0.0675, F-score : 0.0740 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_4
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.006830912414006861
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9606, Precision : 0.0408, Recall : 0.0151, F-score : 0.0221 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_5
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.007120268438011376
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9576, Precision : 0.0259, Recall : 0.0082, F-score : 0.0124 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_6
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.002397903576493261
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9557, Precision : 0.0319, Recall : 0.0123, F-score : 0.0178 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_7
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.6700110692559966
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9985, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_8
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.006463531367480735
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9737, Precision : 0.0124, Recall : 0.0084, F-score : 0.0100 

------------ Options -------------
anormly_ratio: 5.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_9
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 9.709074460615489e-17
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9228, Precision : 0.0520, Recall : 0.0487, F-score : 0.0503 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_10
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.0072950472310184325
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9819, Precision : 0.0127, Recall : 0.0169, F-score : 0.0145 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_11
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.006282573062926521
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9598, Precision : 0.0256, Recall : 0.0088, F-score : 0.0131 

------------ Options -------------
anormly_ratio: 0.1
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_12
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.0748524039611232
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9941, Precision : 0.0108, Recall : 0.0244, F-score : 0.0149 

------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_13
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.028188115973026333
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9874, Precision : 0.0129, Recall : 0.0150, F-score : 0.0139 

------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_14
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.012299377284944485
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9887, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_15
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.7865708318292497
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9984, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_16
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.0023502711369655835
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9731, Precision : 0.0036, Recall : 0.0030, F-score : 0.0033 

------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_17
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.02240190408192611
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9866, Precision : 0.0122, Recall : 0.0142, F-score : 0.0131 

------------ Options -------------
anormly_ratio: 1.0
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_18
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.007351175076328215
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9767, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.5
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_19
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.023260270589962658
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9855, Precision : 0.0115, Recall : 0.0127, F-score : 0.0121 

------------ Options -------------
anormly_ratio: 0.05
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_20
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9965, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.05
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_21
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.10992800116911451
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9961, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.05
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_22
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.13109133851528165
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9963, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_23
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.8262347285803151
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9996, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.05
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_24
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.09242837175354189
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9966, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_25
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.36379067861726466
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9995, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.05
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_26
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.1089880059286936
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9964, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.05
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_27
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.04048785941675266
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9976, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_28
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.6861327379285682
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9990, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_29
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.7651060473798674
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9992, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_30
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.6199230782323712
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9992, Precision : 0.0000, Recall : 0.0000, F-score : 0.0000 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_31
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------
Threshold : 0.5765992528795936
pred:    (22500,)
gt:      (22500,)
pred:  (22500,)
gt:    (22500,)
Accuracy : 0.9993, Precision : 0.0714, Recall : 0.2500, F-score : 0.1111 

------------ Options -------------
anormly_ratio: 0.01
batch_size: 32
data_path: dataset/NSLKDD
dataset: NSLKDD_32
input_c: 122
k: 3
lr: 0.0001
mode: train
model_save_path: checkpoints
num_epochs: 3
output_c: 122
pretrained_model: None
win_size: 100
-------------- End ----------------

x. References

• 论文解读1：https://zhuanlan.zhihu.com/p/553509779

• Detection Adjustment:https://blog.csdn.net/a571625338/article/details/127979281

• 网络安全数据集调研：https://zhuanlan.zhihu.com/p/149130456

• 在KDD99数据集进行入侵检测1：https://cloud.tencent.com/developer/article/1621977

• 代码debug相关

  • CodeBERT代码bug修复：https://juejin.cn/post/7034105242841153550
  • 基于Transformer的微软DeepBug代码bug修复：https://arxiv.org/pdf/2105.09352.pdf
  • 微软代码智能benchmark（含代码纠错Bugs2fix）：https://github.com/microsoft/CodeXGLUE
    • 微软代码智能benchmark中文介绍：https://www.msra.cn/zh-cn/news/features/codexglue

12月22日，有幸听了南京大学周志华教授题为“新型机器学习神经元模型的探索”的讲座。现场热闹非凡，大家像追星一样拿着“西瓜书”找周教授签名。周教授讲得依旧循循善诱，由浅入深，听得我很入迷，故作此记。

周教授首先就人工智能领域火热发展的原因提出了自己的见解，强调了人工智能中基础算法相较于算力的基础性作用：算力提升论或为误解，应当是算法带来基础性突破，然后算力的提升才能在其后一段时间内放大算法突破带来的红利。 周教授随后举例说明，BP算法在深层神经网络里存在着梯度消失问题，而2006年随着Hinton的深层模型问世，深度学习一直在近二十年的时间内火热不已，并随着算力不断提升在不同领域有着越来越亮眼的表现。

接着周教授乘势抛出了两个贯穿本次演讲始终的公式：“神经网络=神经网络模型+学习算法”，而神经网络模型=神经元模型+网络结构。周教授解释道，学习算法指的就是BP算法这种历久弥新的算法，而本次演讲的重点——神经元模型，指的就是受生物神经元接受多个电信号输入，达到阈值后激活并输出的启发，所设计的机器学习神经元数学模型，即著名的M-P神经元模型，形如：

$y = f(\sum_{i=1}^n w_i x_i - \theta)$

周教授指出，关于神经网络的大多数研究都注重在网络结构上做设计，而关于神经元模型的研究甚少，甚至问世近80年的M-P神经元模型到今天仍然遍地在用。此外，近来关于神经元模型的研究又开始有所浮现，讨论能否有别的神经元模型可以使用。在这样的背景下，周教授团队着手了新型神经元模型的相关研究。

周教授首先分享了他们团队在分岔脉冲神经网络 (Bifurcation Spiking Neural Network) 方面的研究^[1]。

首先，脉冲神经网络中一种被广泛研究的神经元模型叫做Leaky integrate and fire (LIF) 模型，LIF神经元模型除了考虑信号的传递和神经元激活，还考虑了信号传递的时间累计过程，其一般形式为：

$\tau \frac{du}{dt} = -u + R f(\bold{I(t)})$

周教授团队从动力系统视角进行分析，发现基于LIF神经元模型的脉冲神经网络的解空间是分开的三部分，由参数$\tau$决定。因此提出了分岔脉冲神经网络（Bifurcation Spiking Neural Network, BSNN），实现了自适应动力系统，将解空间连起来，解决了解空间分岔的问题，使得解空间可达。

周教授进一步分享了他们团队提出的一种生物上合理且具有灵活的突触可塑性的全新神经元模型：Flexible Transmitter (FT) Model ^[2].
这种FT神经元模型参考了生物神经元的神经递质传递过程，尤其是突触的收缩和发育过程，神经递质不仅起到信号传递作用，还会控制突触发育和收缩，使得神经递质的接受量相应增大和缩小，这一过程如下图所示。

FT神经元模型可以表示为：

$(s_t,r_t) = f(wx_t,vr_{t-1})$

用复数进行数学上的抽象可表示为：

$s_t +r_t \bold{i} = f(wx_t,vr_{t-1} \bold{i} )$

周教授强调，这种FT神经元模型的能力更加强大，因为M-P神经元模型只是FT神经元模型的一个子集。

一种简单基于FT神经元模型的FT神经网络 (FTNet) 同样也被展示：

$\bold{s_t} +\bold{r_t i} = f( \bold{Wx}_t, \bold{Vr}_{t-1} \bold{i} )$

周教授认为，这种新型FT神经元模型和基于其上的FT神经网络具有更加强大的能力，可以解决以前基于M-P神经元的神经网络无法解决的问题。周教授团队在一些简单的任务上和常见的神经网络进行对比，例如在MNIST数据集上，和CNN、RNN、基于M-P神经元的FCN、基于脉冲神经网络的SNN等，结果显示基于FT神经元的神经网络具有最高的Accuracy。但这并非没有代价，周教授毫不掩饰地指出了FT神经网络存在的问题，即更多的计算时间。

演讲结束后，老师同学们都很感兴趣，不断提出自己的疑惑并向周教授请教。而周教授也非常耐心、坦诚地回答，整个问答环节持续了超过半小时。

今天下午听了北京大学黄铁军教授的演讲，颇感震撼。黄教授团队的“脉冲视觉”研究从基础理论方面突破，变革了已经沿用两个世纪的传统感光方式，使得每个新型感光元器件可以每个像素单独对光子进行捕获，并可以重现拍摄时任意时刻的清晰图像。也就是说，有了脉冲相机，也许以后都不会再谈帧率了，因为拍摄到的信息将无限接近现实的光的运动，其时域采样率为4万赫兹，这是以前所有动辄上百万的高帧率相机都无法比拟的。此外，脉冲相机的感光元件可以利用现有的CMOS制造技术上进行制造，使得成本可以控制得比较低。这项技术可以说意义非凡，前景广泛。

接着，黄教授还介绍了脉冲相机在高铁检修、涡轮停机检修等方面的应用。我比较感兴趣的是基于脉冲的视觉任务。传统计算机视觉基于的图像、视频等基本是用传统相机拍摄的（当然也不排除用特殊成像方式的），在计算机中图像通常以像素矩阵存储，而脉冲相机拍摄的脉冲序列我认为其实可以看做光子的到达时间序列，提供了更接近物理现实的记录。在脉冲序列上黄教授团队也做了一系列研究，将光流与深度估计、检测跟踪、目标识别等视觉任务基于脉冲序列进行了实现，得到了成倍的速度提升与SOTA的效果。我觉得在脉冲相机的基础上未来可以实现更类脑的神经网络并实现更接近人类视觉能力的方法。

基于脉冲的视觉任务方面，黄教授团队设计并开源了SpikeCV，即在OpenCV的基础上，针对脉冲序列进行了设计，还额外提供了脉冲相机的接口和已经拍摄好的数据集。地址：https://openi.pcl.ac.cn/Cordium/SpikeCV

有同学问了两个比较关心的问题，即脉冲相机的功耗会不会由于高速采样而很高？以及，脉冲相机拍摄的视频占用存储空间会不会很大？黄教授的回答基本上是，首先功耗确实会比正常大，但可以接受。其次存储由于对脉冲进行了编码，其占用带宽在Gbps，存储空间需求虽然大但也可以在现有硬件上可以得到满足。

总之，黄教授的这次演讲干货满满，相信不久的将来可以从前沿研究真正上到我们手机上。研究项目地址：https://spikecv.github.io/

The paper argue that the efficiency and data privacy of Federated Learning are non-orthogonal from the perspective of model training, which means they are restricting each other. So that the paper strictly formulates the problem at first, and designs a cloud-edge-end hierarchical FL system with adaptive control algorithm embedding a two-level Differential Protection method to relieve both the resource and privacy concerns. The design follows the following ideas:

1.Offload part of the model training from resource-limited end devices to the proximate edges by splitting the model into two parts(shallow layers and deep layers) to improve efficiency.

2.Apply two-level differential privacy (DP) noise injection to protect privacy against the honest-but-curious cloud and the edge server, perturbing model updates to cloud and intermediate features to edges. Specifically, the noise function between end and edge is $DP_1(O,\sigma_e)=O+N(0,S_f^2 \sigma_e^2)$, in which the function simply plus the intermediate features O of the end’s shallow layers and noise function N. For the cloud the DP function is $ DP_2(w_i^j)=\zeta \times w_i^j / \parallel w_i^j \parallel + N(0,\zeta^2 \sigma^2) $ , in which the model needs to be performed L2-normalization layer by layer.

3.Adaptively coordinate resource optimization and privacy protection to maximize accuracy under resource budget and privacy budgets. Main adaptive controls are:

• ​ Dynamically schedule local and global aggregations based on resource consumption to minimize loss.
• ​ Adjust device sampling rate based on remaining rounds and privacy risk to avoid early termination.
• ​ Tune offloading decision and local noise intensity to minimize resource consumption since more noise make it harder to coverage.

A prototype of AHFL have been implemented and experiments are conducted on CIFAR-10 and MNIST, showing that AHFL reduces end's computation time by 8.58%, communication time by 59.35%, memory by 43.61% and improves accuracy by 6.34% over state-of-the-art approaches like ltAdap, FLGDP, FEEL.

Strengths

• Base on the assumption that security issues are worth concerning which is relatively less considered in some works.
• Offloading model training tasks by splitting the model into two parts while having a relatively comprehensive consideration on the privacy concerns between end device, edge server and cloud.
• Formulate the problem rigorously with math and UML diagram tools.

Weaknesses

• Balancing between iteration times and noise intensity may not be the optimal way because the model can be evaluated in many aspects and the iteration times only contributes to a part of it.
• Does not evaluate the effect of splitting method at the end device because different splitting may cause different computation and influence the offloading.
• The evaluation on CIFAR-10 and MNIST datasets are relatively simple which make it less convincing.

Comments

The paper mainly focus on privacy and efficiency in a federated learning system and design a hierarchical architecture with adaptive algorithms for balancing. Although the experiments are relatively not that convincing and some key details are not introduced, it’s still a innovative work. Besides, the figure 2 impress me a lot for using a UML lane diagram combining formula to precisely illustrate the workflow.

生活中，许多人似乎都在解决着自己的多目标优化问题。我们即想要这个，又想要那个，有时候目标间甚至相互矛盾，你存我亡。

但其实我们没有严谨的公式和高速计算的能力，去把一个个问题形式化，然后一一求解。

而贪心策略（Greedy Strategy），也就是选择当前最好的，作为一种符合直觉的方式，被广泛地应用到我们生活中，而且很多时候其实我们并不会意识到。

算法课上，一则关于苏格拉底和柏拉图的对话也探讨了贪心策略。柏拉图向苏格拉底问了三个问题，其中第一个问题是：

什么是爱情？

苏格拉底没有直接回答，而让柏拉图去麦田不回头地走一次，并在途中要摘一棵最大最好的麦穗。柏拉图看到最后垂头丧气地空手而归。苏格拉底问为何空手而归，是没有很好的麦穗吗？

柏拉图无奈道：

每次看到的麦穗都不知道是不是最大最好的，又因为只能摘一次，不得已直到走完全程也没有挑到合适的。

苏格拉底说：

这就是爱情！

庞大的麦田有数不清的麦穗，组成巨大的搜索空间。这要如何才能找到“全局最优”的麦穗？然而其映射的道理是，这样的全局最优只存在于理想，而且很容易会错失。

正如开篇所说，作为一个人，我们其实无法做到上帝视角，全面考量。但又往往喜欢使用贪心策略，因而在有的事情上甚至无法得到局部最优解。正所谓捡了芝麻丢了西瓜，向着那存在于理想的收敛目标不断振荡前进以期收敛，然而现实不能简单的梯度下降，世界的模型存在太多变量。

那如果不总是这么贪心，不总是选择最好的
，会如何？这大概就是soft-greedy的一个思想。当然这并非是一个多么高明的优化方法，只是可以确保能够有更多变数。

生命总在与体验，更多的变数是不是意味着更多的体验呢？也许根据贪心策略，你总是能够按时赶上最近一班车，在离目的地最近的站点下车，吃上最适合自己口味且性价比最高的食物，在熟悉的城市从事自己觉得最满意的工作。那如果多一点点小如ε的变动概率，或者说多一点变化会怎样？事情会一团糟，在某些情况下，确实如此。那如果你获得足够的勇气，去放下工作、生活、家庭，也就是说，人的社会性所赋予的一切，去尝试些新的选择，会获得什么样的体验？也许会和《荒野生存》中的Chris一样，在阿拉斯加的山头和旷野，发现自我，长眠于自己极致的理想。

Chris是彻底的理想主义者，有人会批判他是完全反社会的，也有人会羡慕他的决心和勇气。在他人生的某个瞬间，也许属于他的小概率的决定使然，他踏上了他的阿拉斯加之旅。我想，每个人心里都有自己的阿拉斯加。例如有人想徒步318，有人想自驾环游世界，也有人想住进深山。只不过在greedy和soft-greedy之间，许多人无法设置哪怕一个小小的ε，究其一生只能陷在局部最优解的山谷里。