Subject: Re: FreeBSD Bus DMA (was Re: AdvanSys board support)
To: Justin T. Gibbs <gibbs@plutotech.com>
From: Jason Thorpe <thorpej@nas.nasa.gov>
List: tech-kern
Date: 06/11/1998 22:03:46
[ Ah, this is better... this is the kind of message I like :-]
On Thu, 11 Jun 1998 21:44:36 -0600
"Justin T. Gibbs" <gibbs@plutotech.com> wrote:
> >Yes, lots of ways. But you don't have to change the interface to
> >bus_dma to do it. The bounce buffer size issue is largely an issue
...actually, I misstated this... you don't have to change the interface to
anything that currently exists in bus_dma to do it. See bullet #2 below.
Obviously bullet #3, while it doens't change bus_dma, changes lots of
other things :-)
> Can you explain how you can remove the need to allocate worse case scenario
> bounce buffer space without adding some way of handing deferred requests?
> I'm not contending the fact that the NetBSD implementation could be smarter
> about copying less data when a bounce has to occur which is easy to do
> within the constraints of the current API.
Yes, being smarter about the bounce is one piece... maybe I'll get around
to that one day.. "maybe" ... really, dealing with dumb ISA devices
isn't my highest priority.. but whatever :-)
Anyhow, a few options for dealing with deferred requests (I at least
agree with you that they're necessary, and will readily acknowledge
that the bus_dma interface doesn't deal with it all that well right now):
(1) The current bus_dma API allows the caller to specify "you
can go to sleep to wait for resources". If it's safe for
the client (I'll use your term :-) to block in this manner,
that's an option. But it's not an option when you're running
in interrupt context, which is a fair amount of the time in
SCSI and network drivers, unfortunately.
(2) You could add a function like the following:
void bus_dmamap_wait_async __P((bus_dma_tag_t tag
bus_dmamap_t map, bus_size_t size,
void (*callback)(void *), void *arg));
This could register on a queue of things waiting for resources
to become available, but that can't sleep, e.g. the command
entry point of a SCSI controller. The controller could simply
add the SCSI request to its queue once registering this callback,
and just return "it's queued". Once the resources become
available, and it's this device's turn, the callback is called
to kick the driver to get it going again.
Obviously, if another request came in for a device which
was already in async wait, it would just queue it up, and
not call the async wait function again (since it needs to
preserve order, and it's waiting for the resources to map
the first request in its queue).
Since per-client order will already be maintained, the
back-end would be able to re-order the callbacks as it
sees fit when resources become available.
I could definitely see the merit in adding this interface
to NetBSD. In fact, it could address something that's been
troubling me regarding loading of DMA maps for mbuf chains
when using SGMAPs on the Alpha.
(3) Preemptive kernel threads. This is where we really want to be,
and is the direction we're actively going in. In this model,
you don't really have "interrupt context" (well, you do, but
drivers don't run in interrupt context ever, except for rare
cases where you don't have much choice).
In this model, an instance of a driver (which is its own thread
in the system) can _always_ block because it always runs in
its own context (except for the interrupt stub that wakes
the thread up when a command has completed, etc.). In this
model, once a resource is relenquished by one thread, a
higher-priority thread that was blocking on the resource can
immediately preempt the running thread and complete its work.
Jason R. Thorpe thorpej@nas.nasa.gov
NASA Ames Research Center Home: +1 408 866 1912
NAS: M/S 258-5 Work: +1 650 604 0935
Moffett Field, CA 94035 Pager: +1 650 428 6939