Hi,
I have an issue with a CentOS 6.7 physical implementation on a HP DL360 G9, while running iperf from this server to another server the bandwidth fluctuate instead of producing a stable stream.
You can see below the bandwidth varies wildly instead of sending steadily
We have tried unbonding the NICS (they are in a LACP bond now), but it didn't make any difference, we have also tried to upgrade to the latest Intel ixgbe drivers but it didn't make nay difference either
The NIC ports are plugged into 2 Cisco Nexus 5672up
NICS:
2 x 10Gb port Intel Nic bonded in LACP.
driver: ixgbe
version: 4.0.1-k
firmware-version: 0x800006fd
bus-info: 0000:05:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no
Kernel version
2.6.32-573.26.1.el6.x86_64
See below for iperf results
Code:
[root@localhost ~]# iperf -c server1 -P 5 -i 1
------------------------------------------------------------
Client connecting to server1, TCP port 5001
TCP window size: 171 KByte (default)
------------------------------------------------------------
[ 3] local 10.202.10.51 port 44790 connected with 10.204.10.4 port 5001
[ 5] local 10.202.10.51 port 44787 connected with 10.204.10.4 port 5001
[ 6] local 10.202.10.51 port 44788 connected with 10.204.10.4 port 5001
[ 7] local 10.202.10.51 port 44789 connected with 10.204.10.4 port 5001
[ 4] local 10.202.10.51 port 44791 connected with 10.204.10.4 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 32.5 MBytes 273 Mbits/sec
[ 5] 0.0- 1.0 sec 5.25 MBytes 44.0 Mbits/sec
[ 6] 0.0- 1.0 sec 256 KBytes 2.10 Mbits/sec
[ 3] 1.0- 2.0 sec 81.9 MBytes 687 Mbits/sec
[ 5] 1.0- 2.0 sec 512 KBytes 4.19 Mbits/sec
[ 7] 0.0- 1.0 sec 256 KBytes 2.10 Mbits/sec
[ 4] 0.0- 1.0 sec 256 KBytes 2.10 Mbits/sec
[SUM] 0.0- 1.0 sec 38.5 MBytes 323 Mbits/sec
[ 7] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec
[ 4] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec
[ 4] 2.0- 3.0 sec 10.9 MBytes 91.2 Mbits/sec
[ 6] 1.0- 2.0 sec 640 KBytes 5.24 Mbits/sec
[SUM] 1.0- 2.0 sec 83.2 MBytes 698 Mbits/sec
[ 6] 2.0- 3.0 sec 0.00 Bytes 0.00 bits/sec
[ 7] 2.0- 3.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 4] 3.0- 4.0 sec 67.4 MBytes 565 Mbits/sec
[ 3] 2.0- 3.0 sec 56.2 MBytes 472 Mbits/sec
[ 3] 3.0- 4.0 sec 0.00 Bytes 0.00 bits/sec
[ 5] 2.0- 3.0 sec 896 KBytes 7.34 Mbits/sec
[SUM] 2.0- 3.0 sec 69.1 MBytes 580 Mbits/sec
[ 5] 3.0- 4.0 sec 0.00 Bytes 0.00 bits/sec
[ 6] 3.0- 4.0 sec 768 KBytes 6.29 Mbits/sec
[ 7] 3.0- 4.0 sec 768 KBytes 6.29 Mbits/sec
[SUM] 3.0- 4.0 sec 68.9 MBytes 578 Mbits/sec
[ 3] 4.0- 5.0 sec 10.5 MBytes 88.1 Mbits/sec
[ 7] 4.0- 5.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 5] 4.0- 5.0 sec 768 KBytes 6.29 Mbits/sec
[ 6] 4.0- 5.0 sec 640 KBytes 5.24 Mbits/sec
[ 4] 4.0- 5.0 sec 37.5 MBytes 315 Mbits/sec
[SUM] 4.0- 5.0 sec 50.5 MBytes 424 Mbits/sec
[ 5] 5.0- 6.0 sec 14.0 MBytes 117 Mbits/sec
[ 7] 5.0- 6.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 3] 5.0- 6.0 sec 74.6 MBytes 626 Mbits/sec
[ 6] 5.0- 6.0 sec 384 KBytes 3.15 Mbits/sec
[ 4] 5.0- 6.0 sec 384 KBytes 3.15 Mbits/sec
[SUM] 5.0- 6.0 sec 90.5 MBytes 759 Mbits/sec
[ 5] 6.0- 7.0 sec 82.2 MBytes 690 Mbits/sec
[ 3] 6.0- 7.0 sec 640 KBytes 5.24 Mbits/sec
[ 6] 6.0- 7.0 sec 640 KBytes 5.24 Mbits/sec
[ 7] 6.0- 7.0 sec 640 KBytes 5.24 Mbits/sec
[ 4] 6.0- 7.0 sec 1.62 MBytes 13.6 Mbits/sec
[SUM] 6.0- 7.0 sec 85.8 MBytes 719 Mbits/sec
[ 3] 7.0- 8.0 sec 60.2 MBytes 505 Mbits/sec
[ 7] 7.0- 8.0 sec 6.50 MBytes 54.5 Mbits/sec
[ 4] 7.0- 8.0 sec 768 KBytes 6.29 Mbits/sec
[ 6] 7.0- 8.0 sec 768 KBytes 6.29 Mbits/sec
[ 4] 8.0- 9.0 sec 62.6 MBytes 525 Mbits/sec
[ 5] 7.0- 8.0 sec 7.62 MBytes 64.0 Mbits/sec
[SUM] 7.0- 8.0 sec 75.9 MBytes 636 Mbits/sec
[ 5] 8.0- 9.0 sec 0.00 Bytes 0.00 bits/sec
[ 7] 8.0- 9.0 sec 640 KBytes 5.24 Mbits/sec
[ 3] 8.0- 9.0 sec 6.38 MBytes 53.5 Mbits/sec
[ 4] 9.0-10.0 sec 28.2 MBytes 237 Mbits/sec
[ 4] 0.0-10.0 sec 210 MBytes 176 Mbits/sec
[ 6] 8.0- 9.0 sec 640 KBytes 5.24 Mbits/sec
[SUM] 8.0- 9.0 sec 70.2 MBytes 589 Mbits/sec
[ 6] 9.0-10.0 sec 0.00 Bytes 0.00 bits/sec
[ 6] 0.0-10.0 sec 4.75 MBytes 3.97 Mbits/sec
[ 7] 9.0-10.0 sec 640 KBytes 5.24 Mbits/sec
[ 7] 0.0-10.9 sec 13.0 MBytes 10.0 Mbits/sec
[ 5] 9.0-10.0 sec 24.0 MBytes 201 Mbits/sec
[ 5] 0.0-11.0 sec 135 MBytes 104 Mbits/sec
[ 3] 9.0-10.0 sec 8.25 MBytes 69.2 Mbits/sec
[SUM] 9.0-10.0 sec 61.1 MBytes 513 Mbits/sec
[ 3] 10.0-11.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 0.0-11.3 sec 331 MBytes 247 Mbits/sec
[SUM] 0.0-11.3 sec 694 MBytes 517 Mbits/sec
This is what I would except, this is from a CentOS implementation on a virtual machine running on vSphere. It has 2 x 10 Gb Nics unbonded
Code:
[ ID] Interval Transfer Bandwidth
[ 6] 0.0- 1.0 sec 31.6 MBytes 265 Mbits/sec
[ 3] 0.0- 1.0 sec 23.6 MBytes 198 Mbits/sec
[ 5] 0.0- 1.0 sec 22.5 MBytes 189 Mbits/sec
[ 4] 0.0- 1.0 sec 18.9 MBytes 158 Mbits/sec
[ 7] 0.0- 1.0 sec 16.1 MBytes 135 Mbits/sec
[SUM] 0.0- 1.0 sec 113 MBytes 946 Mbits/sec
[ 6] 1.0- 2.0 sec 27.4 MBytes 230 Mbits/sec
[ 3] 1.0- 2.0 sec 24.5 MBytes 206 Mbits/sec
[ 5] 1.0- 2.0 sec 22.1 MBytes 186 Mbits/sec
[ 4] 1.0- 2.0 sec 24.2 MBytes 203 Mbits/sec
[ 7] 1.0- 2.0 sec 13.1 MBytes 110 Mbits/sec
[SUM] 1.0- 2.0 sec 111 MBytes 934 Mbits/sec
[ 6] 2.0- 3.0 sec 16.9 MBytes 142 Mbits/sec
[ 3] 2.0- 3.0 sec 24.2 MBytes 203 Mbits/sec
[ 5] 2.0- 3.0 sec 22.2 MBytes 187 Mbits/sec
[ 7] 2.0- 3.0 sec 28.2 MBytes 237 Mbits/sec
[ 4] 2.0- 3.0 sec 20.1 MBytes 169 Mbits/sec
[SUM] 2.0- 3.0 sec 112 MBytes 937 Mbits/sec
[ 6] 3.0- 4.0 sec 34.4 MBytes 288 Mbits/sec
[ 3] 3.0- 4.0 sec 20.9 MBytes 175 Mbits/sec
[ 5] 3.0- 4.0 sec 18.0 MBytes 151 Mbits/sec
[ 4] 3.0- 4.0 sec 18.2 MBytes 153 Mbits/sec
[ 7] 3.0- 4.0 sec 19.2 MBytes 161 Mbits/sec
[SUM] 3.0- 4.0 sec 111 MBytes 929 Mbits/sec
[ 6] 4.0- 5.0 sec 21.6 MBytes 181 Mbits/sec
[ 4] 4.0- 5.0 sec 22.8 MBytes 191 Mbits/sec
[ 3] 4.0- 5.0 sec 21.5 MBytes 180 Mbits/sec
[ 5] 4.0- 5.0 sec 21.5 MBytes 180 Mbits/sec
[ 7] 4.0- 5.0 sec 24.4 MBytes 204 Mbits/sec
[SUM] 4.0- 5.0 sec 112 MBytes 937 Mbits/sec
[ 4] 5.0- 6.0 sec 21.0 MBytes 176 Mbits/sec
[ 3] 5.0- 6.0 sec 23.4 MBytes 196 Mbits/sec
[ 5] 5.0- 6.0 sec 21.8 MBytes 182 Mbits/sec
[ 6] 5.0- 6.0 sec 22.5 MBytes 189 Mbits/sec
[ 7] 5.0- 6.0 sec 22.8 MBytes 191 Mbits/sec
[SUM] 5.0- 6.0 sec 111 MBytes 934 Mbits/sec
[ 4] 0.0- 7.0 sec 145 MBytes 174 Mbits/sec
[ 3] 6.0- 7.0 sec 24.1 MBytes 202 Mbits/sec
[ 7] 6.0- 7.0 sec 24.2 MBytes 203 Mbits/sec
[ 6] 6.0- 7.0 sec 20.8 MBytes 174 Mbits/sec
[ 5] 6.0- 7.0 sec 22.6 MBytes 190 Mbits/sec
[ 6] 7.0- 8.0 sec 28.8 MBytes 241 Mbits/sec
[ 3] 7.0- 8.0 sec 24.1 MBytes 202 Mbits/sec
[ 7] 7.0- 8.0 sec 27.5 MBytes 231 Mbits/sec
[ 5] 7.0- 8.0 sec 30.1 MBytes 253 Mbits/sec
[ 3] 8.0- 9.0 sec 27.4 MBytes 230 Mbits/sec
[ 5] 8.0- 9.0 sec 33.2 MBytes 279 Mbits/sec
[ 7] 8.0- 9.0 sec 24.1 MBytes 202 Mbits/sec
[ 6] 8.0- 9.0 sec 26.2 MBytes 220 Mbits/sec
[ 6] 9.0-10.0 sec 30.0 MBytes 252 Mbits/sec
[ 6] 0.0-10.0 sec 260 MBytes 218 Mbits/sec
[ 3] 9.0-10.0 sec 26.1 MBytes 219 Mbits/sec
[ 3] 0.0-10.0 sec 240 MBytes 201 Mbits/sec
[ 7] 9.0-10.0 sec 28.5 MBytes 239 Mbits/sec
[ 7] 0.0-10.0 sec 228 MBytes 191 Mbits/sec
[ 5] 9.0-10.0 sec 26.2 MBytes 220 Mbits/sec
[ 5] 0.0-10.0 sec 240 MBytes 202 Mbits/sec
[SUM] 0.0-10.0 sec 1.09 GBytes 934 Mbits/sec
Any ideas?
Thanks
Regards
