Re: Dom0 bad network performance

[Matthias Petermann <mp%petermann-it.de@localhost>]

Hello Greg,

Am 13.02.21 um 15:50 schrieb Greg Troxel:
>
> Suggestions
>
>    - post the dmesg lines for the network interface (minus mac addr, if
>      you are paranoid) from both GENERIC and XEN3_DOM0.   It is
>      possible that this issue is related to not only Xen but
>      also the driver for the particular network interface
>
>    - run iperf to localhost both ways.  That exercises TCP and the
>      user/kernel path for data, but not the physical network interface
>
>    - For any network issues, I recommend doing "netstat -s" before and
>      after a test, and then diffing the results, trying to understand the
>      origin of all counter changes.   For example in your case, there
>      probably shouldn't be any dropped packets.  So if that increments --
>      or more importantly some other counter none of us have thought
>      about -- that's interesting.
>
>    - Run iperf in UDP mode.  This separates TCP's congestion reaction
>      from ability to push packets.
>
>    - This is hard, and probably not that useful in this case, but
>      graphics/xplot-devel has a script tcpdump2xplot, which probably
>      needs updating for tcpdump format drift, that is really useful for
>      visualizing tcp behavior.  But if netstat -s shows no drops, this
>      isn't so useful here.
>

Thank you for your detailed tips, some of which I tried right away. The 
others I will tackle later today. These are my intermediate results. I 
noticed that the line "re0 interrupting at...." is different in the 
dmesg. Does that mean anything?

Kind regards
Matthias


1.1) ifconfig output with Xen

re0: 
flags=0x8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> 
mtu 1500

capabilities=3f80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
        capabilities=3f80<UDP4CSUM_Rx,UDP4CSUM_Tx>
        enabled=0
        ec_capabilities=3<VLAN_MTU,VLAN_HWTAGGING>
        ec_enabled=1<VLAN_MTU>
        address: xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (1000baseT full-duplex)
        status: active
        inet 192.168.2.50/24 broadcast 192.168.2.255 flags 0x0
        inet6 fe80::96c6:91ff:fe12:9fb5%re0/64 flags 0x0 scopeid 0x2

1.2) dmesg output with Xen

[     1.000003] re0 at pci3 dev 0 function 0: RealTek 8168/8111 PCIe 
Gigabit Ethernet (rev. 0x15)
[     1.000003] re0: interrupting at ioapic0 pin 18
[     1.000003] re0: RTL8168H/8111H (0x5400)
[     1.000003] re0: Ethernet address xx:xx:xx:xx:xx:xx
[     1.000003] re0: using 256 tx descriptors
[     1.000003] rgephy0 at re0 phy 7: RTL8251 1000BASE-T media 
interface, rev. 0
[     1.000003] rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 
1000baseT-FDX, auto

1.3) iperf on localhost with Xen

kallisto$ iperf3 -w 128K -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[  5] local 127.0.0.1 port 65534 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   164 MBytes  1.37 Gbits/sec    0    492 KBytes 

[  5]   1.00-2.00   sec   168 MBytes  1.41 Gbits/sec    0    492 KBytes 

[  5]   2.00-3.00   sec   163 MBytes  1.37 Gbits/sec    0    492 KBytes

2.1) ifconfig output without Xen

re0: 
flags=0x8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> 
mtu 1500

capabilities=3f80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
        capabilities=3f80<UDP4CSUM_Rx,UDP4CSUM_Tx>
        enabled=0
        ec_capabilities=3<VLAN_MTU,VLAN_HWTAGGING>
        ec_enabled=1<VLAN_MTU>
        address: xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (1000baseT full-duplex)
        status: active
        inet 192.168.2.50/24 broadcast 192.168.2.255 flags 0x0
        inet6 fe80::96c6:91ff:fe12:9fb5%re0/64 flags 0x0 scopeid 0x2

2.2) dmesg output without Xen

[     1.032332] re0 at pci3 dev 0 function 0: RealTek 8168/8111 PCIe 
Gigabit Ethernet (rev. 0x15)
[     1.032332] re0: interrupting at msix4 vec 0
[     1.032332] re0: Ethernet address xx:xx:xx:xx:xx:xx
[     1.032332] re0: using 256 tx descriptors
[     1.032332] rgephy0 at re0 phy 7: RTL8251 1000BASE-T media 
interface, rev. 0
[     1.032332] rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 
1000baseT-FDX, auto

2.3) iperf on localhost without Xen

kallisto$ iperf3 -w 128K -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[  5] local 127.0.0.1 port 65534 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   405 MBytes  3.39 Gbits/sec    0    492 KBytes 

[  5]   1.00-2.00   sec   511 MBytes  4.29 Gbits/sec    0    492 KBytes 

[  5]   2.00-3.00   sec   529 MBytes  4.44 Gbits/sec    0    492 KBytes