Subject: Re: device tree traversal
To: Bill Studenmund <wrstuden@NetBSD.org>
From: Garrett D'Amore <garrett_damore@tadpole.com>
List: tech-kern
Date: 07/10/2006 16:57:10
Bill Studenmund wrote:
> On Tue, Jul 04, 2006 at 09:40:57AM -0700, Garrett D'Amore wrote:
>   
>> Daniel Carosone wrote:
>>     
>>> On Mon, Jul 03, 2006 at 10:22:57PM -0700, Garrett D'Amore wrote:
>>>   
>>> One possible fly in the ointment: there are usb-to-pci bridges (and
>>> many other weird things besides), so you can have a pci bus behind a
>>> hot-deteached usb device.  I've seen them get used for USB video cards
>>> among other things.
>>>
>>> Trying to deal with these might well lead to otherwise-unnecessary
>>> overengineering, of course, but it's worth mentioning.
>>>   
>>>       
>> In such a case, I suspect you _really_ want user interaction _before_
>> the device is removed.  I would imagine a lot of PCI device drivers are
>> not well equipped for a hard device removal in the middle of doing
>> things like DMA.  (From my experience with hotplug drivers mostly on
>> Solaris, it is pretty common to have drivers that want to "clean" the
>> device up for things like removal or power down -- sometimes for example
>> this means waiting for the device to go quiescent -- e.g. waiting for
>> the txfifo on a serial port to drain.)
>>     
>
> The problem is that while we WANT user interaction before, we should have
> a way to handle cases where it doesn't happen. Say your cat is walking
> across your desk and across USB cables. Say now someone makes a loud
> noise, and the cat decides s/he NEEDS to be elsewhere NOW and jumps off
> your desk. Say your USB cables now go flying. Or say some of the things
> you had plugged in went flying instead. These things happen...
>
> You've now had a very unexpected device removal event.
>
> I think we need to figure out how to have the system not explode when this 
> happens.
>   

If it is normal USB, then it shouldn't be a problem, and you don't need
an interrupt to deal with it.  (I.e. I believe USB device drivers
already implicitly deal with device removal and don't rely on a removal
interrupt.)

If it is something other than USB on the other side of the bus, then all
bets are off.  Even with ca_activate (or whatever) there are going to be
a lot of drivers for busses that don't normally support hot plug.  I
doubt we are going to convert them all.

And further, some kinds of removals like this are _bad_, e.g. a device
that is doing DMA across the bus is going to have problems.  In all
likelihood there will be corrupt data structures in DMA memory (e.g.
corrupted descriptor rings) resulting from partial update.  These
drivers/busses were just never intended to be used in a system where
"sudden detach" could occur.

Now, we can argue that calls into question the wisdom of sticking them
on the remote side of a bus that has the sudden detach behavior.... 
Truly, I'm not too worried about those cases anyway, because I consider
them pretty far from the norm.  Most devices that want to be on hotplug
busses are architected appropriately.

And, as I _also_ said before, you simply cannot guarantee a removal
interrupt that e.g. blocks other interrupts.  A lot of hardware just
isn't designed that way.

All this amounts to, I am still opposed to building infrastructure which
relies on a high-level removal interrupt in order to be safe, and I also
don't see a compelling reason to traverse the device tree in interrupt
context.

    -- Garrett
> Take care,
>
> Bil
>   


-- 
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134  Fax: 951 325-2191