如何將yolov5中的PANet層改為BiFPN詳析

更新時(shí)間：2022年06月27日 16:47:56 作者：m0_56247038

現(xiàn)在yolov5的neck用的是PANet,在efficient論文中提出了BiFPN結(jié)構(gòu),還有更加不錯(cuò)的性能,下面這篇文章主要給大家介紹了關(guān)于如何將yolov5中的PANet層改為BiFPN的相關(guān)資料,需要的朋友可以參考下

一、Add

1.在common.py后加入如下代碼

# 結(jié)合BiFPN 設(shè)置可學(xué)習(xí)參數(shù) 學(xué)習(xí)不同分支的權(quán)重
# 兩個(gè)分支add操作
class BiFPN_Add2(nn.Module):
    def __init__(self, c1, c2):
        super(BiFPN_Add2, self).__init__()
        # 設(shè)置可學(xué)習(xí)參數(shù) nn.Parameter的作用是：將一個(gè)不可訓(xùn)練的類型Tensor轉(zhuǎn)換成可以訓(xùn)練的類型parameter
        # 并且會(huì)向宿主模型注冊(cè)該參數(shù) 成為其一部分 即model.parameters()會(huì)包含這個(gè)parameter
        # 從而在參數(shù)優(yōu)化的時(shí)候可以自動(dòng)一起優(yōu)化
        self.w = nn.Parameter(torch.ones(2, dtype=torch.float32), requires_grad=True)
        self.epsilon = 0.0001
        self.conv = nn.Conv2d(c1, c2, kernel_size=1, stride=1, padding=0)
        self.silu = nn.SiLU()
 
    def forward(self, x):
        w = self.w
        weight = w / (torch.sum(w, dim=0) + self.epsilon)
        return self.conv(self.silu(weight[0] * x[0] + weight[1] * x[1]))
 
 
# 三個(gè)分支add操作
class BiFPN_Add3(nn.Module):
    def __init__(self, c1, c2):
        super(BiFPN_Add3, self).__init__()
        self.w = nn.Parameter(torch.ones(3, dtype=torch.float32), requires_grad=True)
        self.epsilon = 0.0001
        self.conv = nn.Conv2d(c1, c2, kernel_size=1, stride=1, padding=0)
        self.silu = nn.SiLU()
 
    def forward(self, x):
        w = self.w
        weight = w / (torch.sum(w, dim=0) + self.epsilon)  # 將權(quán)重進(jìn)行歸一化
        # Fast normalized fusion
        return self.conv(self.silu(weight[0] * x[0] + weight[1] * x[1] + weight[2] * x[2]))

2.yolov5s.yaml進(jìn)行修改

# YOLOv5 ?? by Ultralytics, GPL-3.0 license
 
# Parameters
nc: 80  # number of classes
depth_multiple: 0.33  # model depth multiple
width_multiple: 0.50  # layer channel multiple
anchors:
  - [10,13, 16,30, 33,23]  # P3/8
  - [30,61, 62,45, 59,119]  # P4/16
  - [116,90, 156,198, 373,326]  # P5/32
 
# YOLOv5 v6.0 backbone
backbone:
  # [from, number, module, args]
  [[-1, 1, Conv, [64, 6, 2, 2]],  # 0-P1/2
   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
   [-1, 3, C3, [128]],
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 6, C3, [256]],
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 9, C3, [512]],
   [-1, 1, Conv, [1024, 3, 2]],  # 7-P5/32
   [-1, 3, C3, [1024]],
   [-1, 1, SPPF, [1024, 5]],  # 9
  ]
 
# YOLOv5 v6.0 BiFPN head
head:
  [[-1, 1, Conv, [512, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 6], 1, BiFPN_Add2, [256, 256]],  # cat backbone P4
   [-1, 3, C3, [512, False]],  # 13
 
   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 4], 1, BiFPN_Add2, [128, 128]],  # cat backbone P3
   [-1, 3, C3, [256, False]],  # 17 (P3/8-small)
 
   [-1, 1, Conv, [512, 3, 2]],  # 為了BiFPN正確add，調(diào)整channel數(shù)
   [[-1, 13, 6], 1, BiFPN_Add3, [256, 256]],  # cat P4 <--- BiFPN change 注意v5s通道數(shù)是默認(rèn)參數(shù)的一半
   [-1, 3, C3, [512, False]],  # 20 (P4/16-medium)
 
   [-1, 1, Conv, [512, 3, 2]],
   [[-1, 10], 1, BiFPN_Add2, [256, 256]],  # cat head P5
   [-1, 3, C3, [1024, False]],  # 23 (P5/32-large)
 
   [[17, 20, 23], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)
  ]

3.修改yolo.py，在parse_model函數(shù)中找到elif m is Concat:語句，在其后面加上BiFPN_Add相關(guān)語句：

# 添加bifpn_add結(jié)構(gòu)
elif m in [BiFPN_Add2, BiFPN_Add3]:
    c2 = max([ch[x] for x in f])

4.修改train.py，向優(yōu)化器中添加BiFPN的權(quán)重參數(shù)

將BiFPN_Add2和BiFPN_Add3函數(shù)中定義的w參數(shù)，加入g1

 # BiFPN_Concat
        elif isinstance(v, BiFPN_Add2) and hasattr(v, 'w') and isinstance(v.w, nn.Parameter):
            g1.append(v.w)
        elif isinstance(v, BiFPN_Add3) and hasattr(v, 'w') and isinstance(v.w, nn.Parameter):
            g1.append(v.w)

然后導(dǎo)入一下這兩個(gè)包

二、Concat

1.在common.py后加入如下代碼

# 結(jié)合BiFPN 設(shè)置可學(xué)習(xí)參數(shù) 學(xué)習(xí)不同分支的權(quán)重
# 兩個(gè)分支concat操作
class BiFPN_Concat2(nn.Module):
    def __init__(self, dimension=1):
        super(BiFPN_Concat2, self).__init__()
        self.d = dimension
        self.w = nn.Parameter(torch.ones(2, dtype=torch.float32), requires_grad=True)
        self.epsilon = 0.0001
 
    def forward(self, x):
        w = self.w
        weight = w / (torch.sum(w, dim=0) + self.epsilon)  # 將權(quán)重進(jìn)行歸一化
        # Fast normalized fusion
        x = [weight[0] * x[0], weight[1] * x[1]]
        return torch.cat(x, self.d)
 
 
# 三個(gè)分支concat操作
class BiFPN_Concat3(nn.Module):
    def __init__(self, dimension=1):
        super(BiFPN_Concat3, self).__init__()
        self.d = dimension
        # 設(shè)置可學(xué)習(xí)參數(shù) nn.Parameter的作用是：將一個(gè)不可訓(xùn)練的類型Tensor轉(zhuǎn)換成可以訓(xùn)練的類型parameter
        # 并且會(huì)向宿主模型注冊(cè)該參數(shù) 成為其一部分 即model.parameters()會(huì)包含這個(gè)parameter
        # 從而在參數(shù)優(yōu)化的時(shí)候可以自動(dòng)一起優(yōu)化
        self.w = nn.Parameter(torch.ones(3, dtype=torch.float32), requires_grad=True)
        self.epsilon = 0.0001
 
    def forward(self, x):
        w = self.w
        weight = w / (torch.sum(w, dim=0) + self.epsilon)  # 將權(quán)重進(jìn)行歸一化
        # Fast normalized fusion
        x = [weight[0] * x[0], weight[1] * x[1], weight[2] * x[2]]
        return torch.cat(x, self.d)

2.yolov5s.yaml進(jìn)行修改

# YOLOv5 ?? by Ultralytics, GPL-3.0 license
 
# Parameters
nc: 80  # number of classes
depth_multiple: 0.33  # model depth multiple
width_multiple: 0.50  # layer channel multiple
anchors:
  - [10,13, 16,30, 33,23]  # P3/8
  - [30,61, 62,45, 59,119]  # P4/16
  - [116,90, 156,198, 373,326]  # P5/32
 
# YOLOv5 v6.0 backbone
backbone:
  # [from, number, module, args]
  [[-1, 1, Conv, [64, 6, 2, 2]],  # 0-P1/2
   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
   [-1, 3, C3, [128]],
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 6, C3, [256]],
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 9, C3, [512]],
   [-1, 1, Conv, [1024, 3, 2]],  # 7-P5/32
   [-1, 3, C3, [1024]],
   [-1, 1, SPPF, [1024, 5]],  # 9
  ]
 
# YOLOv5 v6.0 BiFPN head
head:
  [[-1, 1, Conv, [512, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 6], 1, BiFPN_Concat2, [1]],  # cat backbone P4 <--- BiFPN change
   [-1, 3, C3, [512, False]],  # 13
 
   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 4], 1, BiFPN_Concat2, [1]],  # cat backbone P3 <--- BiFPN change
   [-1, 3, C3, [256, False]],  # 17 (P3/8-small)
 
   [-1, 1, Conv, [256, 3, 2]],
   [[-1, 14, 6], 1, BiFPN_Concat3, [1]],  # cat P4 <--- BiFPN change
   [-1, 3, C3, [512, False]],  # 20 (P4/16-medium)
 
   [-1, 1, Conv, [512, 3, 2]],
   [[-1, 10], 1, BiFPN_Concat2, [1]],  # cat head P5 <--- BiFPN change
   [-1, 3, C3, [1024, False]],  # 23 (P5/32-large)
 
   [[17, 20, 23], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)
  ]

3.修改yolo.py，在parse_model函數(shù)中找到elif m is Concat:語句，在其后面加上BiFPN_Concat相關(guān)語句：

# 添加bifpn_concat結(jié)構(gòu)
elif m in [Concat, BiFPN_Concat2, BiFPN_Concat3]:
    c2 = sum(ch[x] for x in f)

4.修改train.py，向優(yōu)化器中添加BiFPN的權(quán)重參數(shù)

添加復(fù)方式同上（Add）

# BiFPN_Concat
        elif isinstance(v, BiFPN_Concat2) and hasattr(v, 'w') and isinstance(v.w, nn.Parameter):
            g1.append(v.w)
        elif isinstance(v, BiFPN_Concat3) and hasattr(v, 'w') and isinstance(v.w, nn.Parameter):
            g1.append(v.w)

至此，大功告成~~~

reference：

【YOLOv5-6.x】設(shè)置可學(xué)習(xí)權(quán)重結(jié)合BiFPN（Add操作）

【YOLOv5-6.x】設(shè)置可學(xué)習(xí)權(quán)重結(jié)合BiFPN（Concat操作）