Excerpts from netbsd: 7-Nov-96 Re: ISA DMA Jason Thorpe@nas.nasa.go (1653)

> I've gotten some really contructive comments from Charles Hannum and
> Matthias Drochner

Thanks for the flowers, I'll try to be even more constructive
in the following sentences:-)

As I wrote in these comments, there are problems passing all
the special parameters needed for a particular bus (VME in my
case) to the bus-independant mapping and access functions.
The solution I'm proposing here consists in a clean separation of
generic, machine-dependant and bus-dependant interfaces.
If one follows this philosophy, this has some impact on the existing
bus mapping interface too.
All this is not only theory - I gained some experience in the
actual implementation of a driver for the Bit3 PCI-VME interface.
(For reference, I'll put current sources for anon ftp to
zelof1.zel.kfa-juelich.de:/pub/NetBSD/pcivme/....)

At first, to clarify some terms and to explain the thoughts
behind the specification issues:

The actual access to the bus where the device to be controlled
resides is done by the functions defined in machine/bus.h,
eg bus_space_read_4(tag, handle, offset).
This interface is common (in theory) to all NetBSD ports;
it's implementation is architecture specific.
Such a bus access is always an access to the processor bus,
commonly called "mainbus". That's why I'd like to narrow
the definitions of "bus_addr_t" and "bus_size_t" to always
referring to the "mainbus", which makes it's definition
architecture-specific (not bus-specific!). IMO, it is not
the Right Thing to use "bus_mem_" types when other busses
are meant - the format of a bus address is naturally bus-specific.

The tag and handle used for these bus access functions is made by
the "mapping functions". These functions convert parameters
defined by the particular bus to parameters which make sense
on the "mainbus" (with possible allocation of mapping registers
or whatever side effects). This means, their interface is bus-specific
and the implementation depends on the "bus adapter". An
implementation would have to map the requestes address to
an address of the parent bus and call the mapping function of
the parent bus with the new address.
An "end user" driver does not call "bus_space_map()" directly;
it calls a bus-specific function which in turn calls the next higher
function... until the mainbus driver is reached.
To complete this thought: an "end user" driver does not get an
architecture-dependant "bus_space_tag" in it's attach arguments
but gets a bus-dependant tag; the mapping functions are used
to create the bus_space_tag which can be passed to the
bus-independent part of the driver.

All this referred to simple, direct bus access. The DMA interface
can be specified following the same principles.
Further, 2 more interfaces are useful:
-DMA engines or "data movers" - data transfer between a bus
  space and the "mainbus" done by an external device.
  (this is called "cntlrdma" in my implementation)
-bus "locking": some interfaces are prone to comcurrency
  problems or deadlock conditions. The VME adapter mentioned
  can not be used for transparent accesses while a "data move"
  is active (if you try, you'll get a PCI hardlock). Call it a broken
  interface, but you'll find other cases where the only means of
  deadlock resolution is a hardware abort at one side, especially
  when busses like VME are involved which don't have a notion
  of "retry" like PCI.

So far for philosophy, now some specification parts:

The bus_dependant tag given to a VME device driver looks like:
typedef struct vme_chipset_tag {
    void *cookie;

    int (*vct_map) __P((void*, vme_addr_t, vme_size_t,
			vme_am_t, vme_datasize_t, vme_swap_t,
			bus_chipset_tag_t*, vme_handle_t*));
    void (*vct_unmap) __P((void*, vme_handle_t*));

    void *(*vct_int_map) __P((void*, int, int,
			      int (*)(void*), void*));
    void (*vct_int_unmap) __P((void*, void*));

    int (*vct_dmamap_create) __P((void*, vme_size_t, int, bus_dma_handle_t*));
    void (*vct_dmamap_destroy) __P((void*, bus_dma_handle_t));

    int (*vct_cntlrdmamap_create) __P((void*, vme_addr_t, vme_size_t,
				       vme_am_t, vme_datasize_t, vme_swap_t,
				       int, bus_cntlrdma_handle_t*));
    void (*vct_cntlrdmamap_destroy) __P((void*, bus_cntlrdma_handle_t));
} *vme_chipset_tag_t;

The functions are accessed via macros like:
#define vme_space_map(vc, vmeaddr, len, am, datasize, swap, tag, handle) \
  (*((vc)->vct_map))((vc)->cookie, (vmeaddr), (len), (am), (datasize), \
  (swap), (tag), (handle))

The handle delivered by the mapping function is:
typedef struct {
    bus_mem_handle_t bmh;
    /* allocate storage for driver private use
     (to avoid dynamic allocation) */
    char _driver_private[VME_DRIVER_PRIVATE_SPACE];
} vme_handle_t;

(This is not really nice. Perhaps the driver private space should be passed
in a separate argument and/or be allocated dynamically. However, the
bus-independant driver part is not affected because only the handle.bmh
part will be passed to it.)

Addresses, lengths and other data related to VME bus are passed to the
driver by these mapping and handle-create functions. If they are not of
use for the driver at this moment, they are stored in the private portion of
the handle. The point is that the bus-independent part of the driver does
not have to handle data of types it does not know about.
(If a VME address is used later, it is always an offset to the base
given here.)

The generic interface to the DMA functions would look like:
typedef struct bus_dma_tag {
    void *cookie;

    int (*bdt_dmamap_load) __P((void*, bus_dma_handle_t, caddr_t, size_t,
			       struct proc*, int));
    void (*bdt_dmamap_unload) __P((void*, bus_dma_handle_t));

    size_t (*bdt_dmamap_extract) __P((void*, bus_dma_handle_t, int, void*));

    void (*bdt_dmamap_sync) __P((void*, bus_dma_handle_t, bus_dmasync_op_t));
} *bus_dma_tag_t;

The bus_dma_tag_t is passed to the driver in the attach arguments.
(It could be part of the vme_chipset_tag shown above.)
The meaning of the functions and their parameters are more or less
like in Jason's proposal, with the following exceptions:
-The bus_dma_handle_t is a void* from the view of the generic interface;
  is is used by the driver only.
-The physical bus addresses to be written into the device's DMA
  registers are extracted by a special function.
This is imo necessary because only the driver knows about the size
of a bit3_dma_segment_t - bus and driver independant code cannot
extract values from an array when it does not know the element size.
(Or: the values could already be stored in mapping tables, then there
is no need to duplicate them.)
One could argue that this extraction is bus-dependant and belongs
into the vme_chipset_tag. This is true in principle, but not done for
efficiency. The busses where a specific chip can be attached to
have probably much in common so that the DMA registers are handled
in a similar way. That's why I left it with the void* return parameter
for now. (more thoughts necessary)

I think this is enough for an overview of my ideas.
The "data mover" and 'lock" interfaces are made in the same fashion.
Their handles are passed via attach args too.
For details, look into the sources :-)
(don't look at the oldest version on zelof1 - it is outdated)

best regards
Matthias Drochner