Excerpts from netbsd: 6-Feb-96 pcmcia stuff: what's next? Ignatios
Souvatzis@cs.un (3059)

> a) use the autoconf stuff for configuration. IMHO, this can easily be
> used for later addition of devices (just reprobe the pcmcia-bus in
> question). 

> c) as for removal of cards: We would add a detach method to the device
> struct of the bus driver, accessible for the drivers through the
> parent pointers. (Christos Zoulas is interested in the PCMCIA
> integration, and was not totally shocked by the idea).

I think I can contribute to these topics. I was playing around with loadable
PCI device drivers dor some time when John Kohl asked for comments
on pcmcia drivers in port-i386. As a summary of experiences collected
so far, I sent him the following:
========================================================
========================================================
First some more philosophical notes:
It is certainly a good idea to use the same probe and attach functions
for statically
configured and loadable drivers (as far as possible - on ISA one has to
take care
of other devices while probing - perhaps the bus code has to maintain a
map of used
mem/io ranges to make this fool-proof).
The most logical way would be, I think, to create a "struct cfdata" -
similar to the
"config" generated - for the new device and call config_attach, what
does all the
bookkeeping and calls the driver's attach function. (One has to find the
parent device
before - more on this later.)
This follows a bit the OSF/1 way - there one includes config information
into the
internal topology tree by calling a special "stanza resolver" function.
OSF/1 tends
to complexity at this point (as on many others - I've written a loadable PCI
driver for an alpha too...) - I hope we can avoid this.

Now some implementation issues:
For the simplest case - a device at the end of the topology tree, and no
need to unload
the driver - there is nothing to change in existing kernel code. To be
able to unload the
dricers cleanly, 2 changes have proven useful for me:
(1) Make the device queue ("alldevs" in subr_autoconf.c) a TAILQ. Besides of
beauty, this allows to remove device entrys without risk to leave the
"static nextp"
in config_attach() dangling around.
(2) Extend the "struct cfdriver" by a "cf_detach" entry point. I use the
following
    interface:
    int (*cd_detach) __P((struct device*));
    Argument is the device softc of the device to be unloaded. Return
value is eg. 1,
    if the device can be removed, and 0, if the device is still needed
by whatever
    open pathes or subdevices.
The first change seems unproblematic - no NetBSD code besides autoconf.c
uses the device list.
The second would affect all drivers (I placed the additional call at the
end of the
struct cfdriver - It will be initialized with 0 and the unloader
interprets this as
"not unloadable" - to save work).
a propos unloader: This is a nearly exact inverse of "config_attach".
Because it
illustrates the concept and is only a short piece of code, I'll quote it
directly:

---------------------------------------------------
void config_detach(dev, callback)
struct cfdata *dev;
void (*callback) __P((struct device*));
{
  struct cfdriver *drv;
  struct device *d;
  int i;

  drv=dev->cf_driver;

  d=alldevs.tqh_first;
  while(d){
    if(/* device uses our driver ? */
       (d->dv_cfdata->cf_driver==drv)
       /* device instance described by this cfdata? */
       &&((dev->cf_fstate==FSTATE_STAR)
	  ||((dev->cf_fstate==FSTATE_FOUND)&&(d->dv_unit==dev->cf_unit)))){

      /* device not busy? */
      /* driver's detach routine decides,
       upper layer (eg bus dependent code) is notified via callback */
      if((drv->cd_detach)&&((*(drv->cd_detach))(d))){
	struct device *help;

	if(callback)(*callback)(d);

	/* remove reference in driver's devicelist */
	drv->cd_devs[d->dv_unit]=NULL;

	/* remove entry in global device list */
	TAILQ_REMOVE(&alldevs, d, dv_next);
	printf("%s removed\n", d->dv_xname);

	/* free memory for dev data (alloc'd in config_make_softc) */ 
	help=d->dv_next.tqe_next;
	free(d, M_DEVBUF);
	d=help;
	continue;
      }
    }
    d=d->dv_next.tqe_next;
  }

  /* driver is not needed anymore? */
  for(i=0;i<drv->cd_ndevs;i++)
    if(drv->cd_devs[i])return;

  /* free devices array (alloc'd in config_make_softc) */
  if(drv->cd_ndevs){
    free(drv->cd_devs, M_DEVBUF);
    drv->cd_ndevs=0;
  }
}
---------------------------------------------

I used the framework as described up to here for my home-grown PCI cards
and (as a proof-of-concept) the "de" PCI ethernet driver. (btw, I used the
PCI code developped by cgd for the alpha port as a base, but this has no
principal impact - it is better structured and it can handle PCI-PCI bridges)

The next step is to load drivers which have subdevices (eg SCSI adapter).
The driver's attach routine usually calls config_search() or config_found().
These functions scan the config-generated cfdata table for potential
children of
a given device. This means, the configuration data base has to be extended.
My first tries were to extend the table directly - allocate memory,
extend the table itself and the parent vector if needed (there is a messing
case if a driver for a PCI-PCI bridge is loaded - the existing PCI bus
entry is parent and child at one time). This was very difficult to maintain
if it came to unloads - even worse if the unloads do not occur in the
inverse load order.
I moved away from this idea and made the config_search- and similar
functions scan not only one table but a TAILQ of tables, whose first is
the config-generated. Loadable Modules can add an own "subtable" and
remove them.
There can be no parent-child-dependencies between tables - this means
in the above mentioned PCI bridge case that a second PCI bus entry has
to be in the "subtable" - referring to the old driver.
This makes things much cleaner. But there are some problems I didn't
address up to now:
-If there is more then 1 "starred" config table entry for the same device
  driver, the device numbering is no more unique. This is easy to resolve:
  use the number of devices as controlled by the driver (cd_devs) as
  a base for the numbering instead the cf_unit member of cfdata.
-The device struct contains a pointer to the cfdata entry. To unload a
subtable,
  we have to be sure that all devices which refer to its table entries
are detached.
  (reference count?)
-perhaps the config_attach should return a pointer to the freshly created
  device softc instead of simply a "1". This would give the parent driver easy
  means to track ressource allocations, which could be automatically freed
  on unload time (triggered by the callback in config_detach).
-We still need a starting point - the parent of the base of the loaded devices.
  (this applies to the no-subdev case too)
  I'm not certain how to to this cleanly. There are (at least) 2 possibilities.
  (a) assume that its cfdriver struct is global - simply use
cfdriver.cf_devs[0..x]
  (b) traverse the list of active devices (alldevs) and compare eg.
       device.dv_devdata->cf_driver->cd_name=="pci"
  The first way is obviously simpler. The second would give us a chance
  to formalize the driver load further so that we only need to know the
parent's
  name to load a driver (I'm thinking about one more member of cfdriver,
  let's call it "cd_reprobe". This could attach subdevices - a little bit like
  the "reprobe_bus" in the SCSI code, but with the cfdata of the potential
  child as an argument).

These are my ideas for the loadable driver support (I have more - automatic
loading, symbol table management ... - but not enough time).
As some last words, some technical details and experiences:
-Some existing drivers simply don't work if loaded at run-time.
  The pms driver's (PS/2 bus mouse) probe doesn't succeed (the driver
  does't work well if statically configured either), I didn't find a reason
  for it. The ncr driver works sporadic, otherwise it complains about
  cache misconfiguration - I'll look at it when I have time.
  Driver writers simply don't think about the possibility to enter a running
  system. (yes, I disable interrupts while attaching)
-ethernet special: my detach routine allows unloading only if there are
  no more addresses assigned to the interface (give the burden of cleanup
  to the user - it would be complicated otherwise) But the inet code
  assings a multicast address at every address addition or change -
  they are never freed (look at in.c:in_ifinit). My driver is forced to
  allow unloading even with multicast addresses remaining - this
   is a memory leak. And: the allocations done in if.c:if_attach
  have to be freed too. This network code is not written with ressource
  freeing in mind...
-If you don't already have DDB support for loadable modules, drop me a note...
========================================================
========================================================

Note that what I call "detach" is a function which detaches a device itself
(stops hardware, frees memory, disconnects from network...), it does
not generally care about parents - there is no need for it in the PCI
framework.
In cgd's pci code, the functions needed for mapping of mem ranges etc are
passed to the PCI device by the pci_attach_args. The device attach code can
save the pointers, and the detach function can use them to unmap the device.

Perhaps my experiences are of use for PCMCIA too.
I have no PCMCIA hardware at all, but I'd like to check your framework
fou usability on PCI before publically visible changes are done.

best regards
Matthias Drochner