通过iptables分析OpenStack 安全组规则

在OpenStack中创建一个实例，同时会生成如下的bridge和port :

·        tap-xxx vm用的端口，配置在libvirt配置文件中的

·        vnet-x 图中有vnet，实际上是没有的，直接用tap插到了qbr上

·        qbr-xxx 虚拟网桥，桥接tap和qvb

·        qvb-xxx 连接br-int的veth端口

·        qvo-xxx qvb-xxx的另一端

例如：

21:qbr14c032e9-bc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueuestate UP
22: qvo14c032e9-bc@qvb14c032e9-bc:<BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1450 qdisc pfifo_fastmaster ovs-system state UP qlen 1000

23:qvb14c032e9-bc@qvo14c032e9-bc: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP>mtu 1450 qdisc pfifo_fast master qbr14c032e9-bc state UP qlen 1000

25:tap14c032e9-bc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdiscpfifo_fast master qbr14c032e9-bc state UNKNOWN qlen 500

查看bridge:

$ brctl show

bridgename         bridgeid                STPenabled        interfaces
qbr14c032e9-bc   8000.b21e7be143d6         no                qvb14c032e9-bc  tap14c032e9-bc

分析Security Group规则

分析FORWARD链

由于规则是配置在host的，所以进出以上端口的规则都只走forward链

删除security group所有规则后，查看iptables

# iptables--list -v

Chain FORWARD (policyACCEPT 0 packets, 0 bytes)
 pkts bytes target     protopt in     out     source              destination        
 348K   19Mneutron-filter-top  all  --  any    any    anywhere             anywhere           
 348K   19Mneutron-openvswi-FORWARD  all  --  any    any    anywhere             anywhere           

Chainneutron-openvswi-FORWARD (1 references)
 pkts bytes target     protopt in     out     source              destination        
  214 25538neutron-openvswi-sg-chain  all  --  any    any    anywhere             anywhere            PHYSDEV match --physdev-outtapc0a350e0-43 --physdev-is-bridged /* Direct trafficfrom the VM interface to the security group chain. */
  248 23854neutron-openvswi-sg-chain  all  --  any    any    anywhere             anywhere            PHYSDEV match --physdev-in tapc0a350e0-43--physdev-is-bridged /* Direct traffic from the VM interface tothe security group chain. */

Chainneutron-openvswi-sg-chain (4 references)
 pkts bytes target     protopt in     out     source              destination        
  214 25538neutron-openvswi-ic0a350e0-4  all  -- any    any     anywhere            anywhere            PHYSDEV match --physdev-out tapc0a350e0-43--physdev-is-bridged /* Jump to the VM specific chain. */
  248 23854neutron-openvswi-oc0a350e0-4  all  -- any    any     anywhere            anywhere             PHYSDEVmatch --physdev-in tapc0a350e0-43--physdev-is-bridged /* Jump to the VM specific chain. */

 --physdev-out tapc0a350e0-43是指tapc0a350e0-43即vm发送到bridge的包

匹配到的包跳转到了neutron-openvswi-ic0a350e0-4

Chainneutron-openvswi-ic0a350e0-4 (1 references)
num   pkts bytes target     prot opt in     out    source              destination        
1     169 20508 RETURN     all  --  any    any    anywhere             anywhere            state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */

2          2   731 RETURN    udp  --  any    any     192.168.1.2         anywhere             udpspt:bootps dpt:bootpc

3       0     0 DROP       all  -- any    any     anywhere            anywhere             stateINVALID /* Drop packets that appear related to an existing connection(e.g. TCP ACK/FIN) but do not have an entry in conntrack. */

4       6  1944 neutron-openvswi-sg-fallback  all  --  any    any    anywhere             anywhere            /* Send unmatched traffic tothe fallback chain. */

Chainneutron-openvswi-sg-fallback (4 references)
num   pkts bytes target     prot opt in     out    source              destination        
1     947 71484 DROP       all  --  any    any    anywhere             anywhere            /* Default drop rule for unmatchedtraffic. */

·        num:1 放行所有已建立连接的包

·        num:2 放行192.168.1.2（dhcp服务器）发过来的udp包

·        num:3 丢弃状态异常的tcp包

·        num:4 丢弃不匹配以上三条的所有包

 --physdev-in tapc0a350e0-43  是指从tapc0a350e0-43即vm发出来的包

匹配到的包跳转到了neutron-openvswi-oc0a350e0-4

Chainneutron-openvswi-oc0a350e0-4 (2 references)
num   pkts bytes target     prot opt in     out    source              destination        
1       2   648 RETURN     udp  --  any    any    default              255.255.255.255     udp spt:bootpc dpt:bootps /* Allow DHCP client traffic. */

2     246 23206 neutron-openvswi-sc0a350e0-4  all  --  any   any     anywhere            anywhere            
3       4  1272 RETURN     udp  --  any    any    anywhere             anywhere            udp spt:bootpc dpt:bootps /* Allow DHCPclient traffic. */

4       0     0 DROP       udp  -- any    any     anywhere            anywhere             udp spt:bootpsdpt:bootpc /* Prevent DHCP Spoofing by VM. */

5     210 19802 RETURN     all  --  any    any    anywhere             anywhere            state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */

6       0     0 DROP       all  -- any    any     anywhere            anywhere             state INVALID /* Droppackets that appear related to an existing connection (e.g. TCP ACK/FIN) but donot have an entry in conntrack. */

7       0     0 neutron-openvswi-sg-fallback  all --  any    any     anywhere            anywhere             /* Sendunmatched traffic to the fallback chain. */Chainneutron-openvswi-sc0a350e0-4 (1 references)
num   pkts bytes target     prot opt in     out    source              destination        
1     246 23206 RETURN     all  --  any    any    192.168.1.12         anywhere            MAC FA:16:3E:C3:EA:D5 /* Allowtraffic from defined IP/MAC pairs. */

2       0     0 DROP       all  -- any    any     anywhere            anywhere             /* Drop trafficwithout an IP/MAC allow rule. */Chain neutron-openvswi-sg-fallback (4 references)
num   pkts bytes target     prot opt in     out    source              destination        
1     947 71484 DROP       all  --  any    any    anywhere             anywhere            /* Default drop rule for unmatchedtraffic. */

·        num1 允许vm发出来的dhcp udp广播包允许源端口是67，目标端口是68端口的数据包通过

·        num2 只允许ip地址为192.168.1.12（vm的分配的ip）通过

·        num3 允许vm（dhcp客户端）发出来的UDP 单播报文

·        num4 禁止vm做dhcp嗅探

·        num5 允许通过所有已建立连接的包通过

·        num6 丢弃所以异常连接的包

·        num7 丢弃不匹配以上任何规则包

说明：

o   bootpc 服务器向67端口(bootpc)广播dhcp回应请求

o   bootps 客户端向68端口(bootps)广播dhcp请求配置

可以看出，在不匹配security规则的情况下，除了dhcp包可以通过之外，其他数据包全部丢弃

配置securitygroup，新增规则后再查看iptables

增加规则1：允许vm发出的所有数据包

查看neutron-openvswi-oc0a350e0-4链

Chainneutron-openvswi-oc0a350e0-4 (2 references)
num   pkts bytes target     prot opt in     out    source              destination        
1       2   648 RETURN     udp  --  any    any    default              255.255.255.255     udp spt:bootpc dpt:bootps /* Allow DHCP client traffic. */

2     246 23206 neutron-openvswi-sc0a350e0-4  all  --  any   any     anywhere            anywhere            
3       4  1272 RETURN     udp  --  any    any    anywhere             anywhere            udp spt:bootpc dpt:bootps /* Allow DHCPclient traffic. */

4       0     0 DROP       udp  -- any    any     anywhere            anywhere             udp spt:bootpsdpt:bootpc /* Prevent DHCP Spoofing by VM. */

5     210 19802 RETURN     all  --  any    any    anywhere             anywhere            state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */

6       0     0 RETURN     all  -- any    any     anywhere            anywhere            
7       0     0 DROP       all  -- any    any     anywhere            anywhere             state INVALID /* Droppackets that appear related to an existing connection (e.g. TCP ACK/FIN) but donot have an entry in conntrack. */

8       0     0 neutron-openvswi-sg-fallback  all --  any    any     anywhere            anywhere             /* Sendunmatched traffic to the fallback chain. */

·        num6 为新增的规则，为放行所有包

增加规则2：允许vm发出的icmp协议包通过

查看neutron-openvswi-ic0a350e0-4链

Chainneutron-openvswi-ic0a350e0-4 (1 references)
num   pkts bytes target     prot opt in     out    source               destination       
1     169 20508 RETURN     all  --  any    any    anywhere             anywhere            state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */

2       2   731 RETURN     udp  --  any    any    192.168.1.2          anywhere            udp spt:bootps dpt:bootpc

3       0     0 RETURN     icmp --  any   any     anywhere            anywhere            
4       0     0 DROP       all  -- any    any     anywhere            anywhere             state INVALID /* Droppackets that appear related to an existing connection (e.g. TCP ACK/FIN) but donot have an entry in conntrack. */

5       6  1944 neutron-openvswi-sg-fallback  all  --  any   any     anywhere            anywhere             /* Send unmatchedtraffic to the fallback chain. */

·        num3 为新增的规则，放行所有icmp协议包

可以看到优化前在实时性方面原始的KVM还是会出现毛刺。在优化后基本上能达到很好的结果。

作者介绍：

曹云涛 ，来自诺云信息系统（上海）有限公司。

相关阅读：

加入中国最活跃的KVM技术讨论QQ群，加群主QQ：502207183，并注明城市、行业、技术方向。