On Sun, Apr 17, 2005 at 01:40:02PM +0000, Tino Wildenhain wrote:
>  Additional informations: 
>  
>  I added a 3com single card and 2 tlp single cards to the system
>  to compare the drivers. Here is the buslayout (with linux)
>  
>  lspci -tv
>  -[00]-+-00.0  Intel Corp. 440BX/ZX/DX - 82443BX/ZX/DX Host bridge
>        +-01.0-[01]----00.0  ATI Technologies Inc Rage 128 PF/PRO AGP 4x
>  TMDS
>        +-07.0  Intel Corp. 82371AB/EB/MB PIIX4 ISA
>        +-07.1  Intel Corp. 82371AB/EB/MB PIIX4 IDE
>        +-07.2  Intel Corp. 82371AB/EB/MB PIIX4 USB
>        +-07.3  Intel Corp. 82371AB/EB/MB PIIX4 ACPI
>        +-08.0  3Com Corporation 3c905 100BaseTX [Boomerang]
>        +-09.0-[02]--+-04.0  Digital Equipment Corporation DECchip 21140
>  [FasterNet]
>        |            +-05.0  Digital Equipment Corporation DECchip 21140
>  [FasterNet]
>        |            +-06.0  Digital Equipment Corporation DECchip 21140
>  [FasterNet]
>        |            \-07.0  Digital Equipment Corporation DECchip 21140
>  [FasterNet]
>        +-0a.0  Digital Equipment Corporation DECchip 21140 [FasterNet]
>        \-0b.0  Digital Equipment Corporation DECchip 21140 [FasterNet]
>  
>  
>  (in all tests I run only console, no gfx)
>  
>  relevant dmesg output:
>  
>  ex0 at pci0 dev 8 function 0: 3Com 3c905-TX 10/100 Ethernet (rev. 0x0)
>  ex0: interrupting at irq 11
>  ex0: MAC address 00:60:97:83:ab:6d
>  nsphy0 at ex0 phy 24: DP83840 10/100 media interface, rev. 1
>  nsphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>  ppb1 at pci0 dev 9 function 0: Digital Equipment DC21152 PCI-PCI Bridge
>  (rev. 0x03)
>  pci2 at ppb1 bus 2
>  pci2: i/o space, memory space enabled, rd/line, wr/inv ok
>  tlp0 at pci2 dev 4 function 0: DECchip 21140A Ethernet, pass 2.2
>  tlp0: interrupting at irq 5
>  tlp0: Adaptec ANA-6944A, Ethernet address 00:00:d1:1f:09:54
>  nsphy1 at tlp0 phy 1: DP83840 10/100 media interface, rev. 1
>  nsphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>  tlp1 at pci2 dev 5 function 0: DECchip 21140A Ethernet, pass 2.2
>  tlp1: sharing interrupt with tlp0
>  tlp1: Adaptec ANA-6944A, Ethernet address 00:00:d1:1f:09:55
>  nsphy2 at tlp1 phy 1: DP83840 10/100 media interface, rev. 1
>  nsphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>  tlp2 at pci2 dev 6 function 0: DECchip 21140A Ethernet, pass 2.2
>  tlp2: sharing interrupt with tlp0
>  tlp2: Adaptec ANA-6944A, Ethernet address 00:00:d1:1f:09:56
>  nsphy3 at tlp2 phy 1: DP83840 10/100 media interface, rev. 1
>  nsphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>  tlp3 at pci2 dev 7 function 0: DECchip 21140A Ethernet, pass 2.2
>  tlp3: sharing interrupt with tlp0
>  tlp3: Adaptec ANA-6944A, Ethernet address 00:00:d1:1f:09:57
>  nsphy4 at tlp3 phy 1: DP83840 10/100 media interface, rev. 1
>  nsphy4: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>  tlp4 at pci0 dev 10 function 0: DECchip 21140A Ethernet, pass 2.2
>  tlp4: interrupting at irq 9
>  tlp4: Ethernet address 00:80:c8:f6:c9:b3
>  lxtphy0 at tlp4 phy 0: LXT970 10/100 media interface, rev. 1
>  lxtphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>  tlp5 at pci0 dev 11 function 0: DECchip 21140A Ethernet, pass 2.2
>  tlp5: interrupting at irq 10
>  tlp5: Ethernet address 00:80:c8:f6:46:bd
>  lxtphy1 at tlp5 phy 0: LXT970 10/100 media interface, rev. 1
>  lxtphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>  
>  =======================================================================
>  ttcp test with 3com (ex0):
>  -----------------------------------------------------------------------
>  ttcp-t: buflen=8192, nbuf=16384, align=16384/0, port=5001ttcp-t: socket
>  ttcp-t: connect
>  ttcp-t: 134217728 bytes in 12.02 real seconds = 10905.64 KB/sec +++
>  ttcp-t: 134217728 bytes in 5.29 CPU seconds = 24769.54 KB/cpu sec
>  ttcp-t: 16384 I/O calls, msec/call = 0.75, calls/sec = 1363.21
>  ttcp-t: 0.0user 5.2sys 0:12real 43% 0i+0d 0maxrss 0+17228pf 28096+33csw
>  ttcp-t: buffer address 0x8050000
>  
>  =======================================================================
>  ttcp test with AHA-6944A 1. Port (tlp0):
>  -----------------------------------------------------------------------
>  ttcp-t: buflen=8192, nbuf=16384, align=16384/0, port=5001ttcp-t: socket
>  ttcp-t: connect
>  ttcp-t: 134217728 bytes in 30.47 real seconds = 4301.84 KB/sec +++
>  ttcp-t: 134217728 bytes in 6.28 CPU seconds = 20873.48 KB/cpu sec
>  ttcp-t: 16384 I/O calls, msec/call = 1.90, calls/sec = 537.73
>  ttcp-t: 0.0user 6.1sys 0:30real 20% 0i+0d 0maxrss 0+14937pf 38361+17csw
>  ttcp-t: buffer address 0x8050000
>  
>  =======================================================================
>  ttcp test with AHA-6944A 4. Port (tlp3):
>  -----------------------------------------------------------------------
>  ttcp-t: buflen=8192, nbuf=16384, align=16384/0, port=5001ttcp-t: socket
>  ttcp-t: connect
>  ttcp-t: 134217728 bytes in 30.45 real seconds = 4304.97 KB/sec +++
>  ttcp-t: 134217728 bytes in 6.15 CPU seconds = 21312.09 KB/cpu sec
>  ttcp-t: 16384 I/O calls, msec/call = 1.90, calls/sec = 538.12
>  ttcp-t: 0.0user 6.0sys 0:30real 20% 0i+0d 0maxrss 0+14828pf 38473+18csw
>  ttcp-t: buffer address 0x8050000
>  
>  =======================================================================
>  ttcp test with single tlp-Card (tlp4):
>  -----------------------------------------------------------------------
>  ttcp-t: buflen=8192, nbuf=16384, align=16384/0, port=5001ttcp-t: socket
>  ttcp-t: connect
>  ttcp-t: 134217728 bytes in 12.30 real seconds = 10654.03 KB/sec +++
>  ttcp-t: 134217728 bytes in 5.96 CPU seconds = 22000.53 KB/cpu sec
>  ttcp-t: 16384 I/O calls, msec/call = 0.77, calls/sec = 1331.75
>  ttcp-t: 0.0user 5.9sys 0:12real 48% 0i+0d 0maxrss 0+17449pf 22722+42csw
>  ttcp-t: buffer address 0x8050000
>  
>  
>  We can see the performance with the single tlp card is much better.

It may be an issue with the pci-pci bridge on the adaptec adapter.
It would be interesting to see if the linux driver has something special about
it.

Is there a way to dump the PCI registers on linux ? If would be interesting
to compare the values between linux and NetBSD for the pci-pci bridge.

You can also compare the values for the tlp devices on the adaptec adapter and
the single device (with pcictl dump)

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--