[ On Wednesday, January 26, 2005 at 00:38:12 (-0800), List Mail User wrote: ]
> Subject: Re: Should Alpha PCI code manage latency timers?
>
> 	I would expect the Q logic controllers to work since everybody else
> gives them "enough" time.  I'd simply expect them to interfere with other
> devices.  Unfortunately most common the expected failure mode for a bad
> latency setting is that the "bad" device works, just others may not.

Ah, yes, OK that makes sense.

However as I hinted I've not yet tried doing FC and network I/O
simultaneously.  Just one at a time so far.  :-)

On the other hand the only devices that have caused errors or other
major problems so far have been the ohci(4) (with that driver compiled
in the system never boots), and trying to copy from the IDE CDROM to
disk (SCSI, on the adaptec) caused complaints from driver and then,
IIRC, a system hang.


> Do you know what kind of errors you get on the network cards?

I don't actually get any errors with any of the network cards that I've
been able to get running well enough to test.  (that's the DEC OEM
Broadcom with bge(4), various models using sk(4), and various models
using wm(4); I had also tried to find cards compatible with gsip(4) and
stge(4) and ti(4), but nobody seems to make any of them anymore)

I wasn't actually worried about this latency timer setting issue
affecting the ethernet cards until you mentioned the possible
interaction of the qlogic cards.

The problem with the GigE cards is simply one of very poor throughput.

For example under Tru64 I can push an almost respectable 50 MB/s out the
Broadcom card with ttcp, but under NetBSD I can barely push out 20 MB/s
(maybe 25 MB/s with really big buffers and really big socket buffers),
and that's in single-user mode with no other activity on any device or
bus.  Even worse despite the use of all the hardware checksum offloading
features in the NetBSD bge(4) driver the CPU time is nearly the same as
the wall-clock time whereas under Tru64 CPU time is only about 25% of
real time.

IIRC Jason says the problem is that with the amount of RAM I have in the
system (half it's max, just 16GB), the way DMA is done in the average
GigE driver is simply not very efficient.

Unfortunately Jason's own from-scratch wm(4) driver for the Intel cards
doesn't seem to fair any better.  The very best I can get out of what's
supposed to be the fastest Intel model, e.g. i82546GB, is also only
about 20 MB/s, and still with at least 60% CPU utilization.  That
doesn't bode well since he's also the architect of the bus_dma(9)
interface which these drivers use to do their data handling, and I can't
really see from his paper on the DMA framework how there could be any
problem with just the ethernet drivers -- i.e. since the qlogic isp(4)
driver can run at full blast using the same DMA framework the problem
seems to be specific only to the ethernet drivers.  Maybe it has
something to do with DMA to/from mbufs instead of more generic pages.
To read the chipset manual though you'd think the DMA controller could
do damn near anything though -- as it does full scatter-gather DMA with
address mapping to even make 32-bit cards efficient as possible in the
full 40(?)-bit(IIRC) address space; and according to Jason's paper the
bus_dma(9) framework should be able to make full use of it, and indeed
given the performance of the qlogic cards I'd have to believe usually
does, at least for block-mode drivers.  

(all the drivers for all other cards I've tried were derived from
FreeBSD drivers originally authored by Bill Paul, though of course they
all had to go through some changes to adapt to NetBSD's bus_dma(9)
interface, and with Jason's help I further tweaked bge(4) and sk(4) to
at least plod along on big-mem systems instead of crashing everything)


-- 
						Greg A. Woods

H:+1 416 218-0098  W:+1 416 489-5852 x122  VE3TCP  RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>          Secrets of the Weird <woods@weird.com>