Re: Potential problem with reading data from usb devices with ugen(4)

[Brian Buhrow <buhrow%nfbcal.org@localhost>]

	Hello Greg.  Thanks for the reply on this issue.  The problem I'm
trying to solve  is that the ugen(4) driver without the read ahead/write
behind code works completely synchronously.  I want asynchronous reading to
work and the read ahead code gets me most of the way there.  I'd like
asynchronous writing to work as well, but it turns out that this isn't
quite as important is asynchronous reading is.  I'm still working through
the issues, as you can tell from my e-mail, but I have the writing solution
in hand.  It's the reading (from the USB device), that I'm working on at the
moment.
	I know the libimobiledevice and hence the usbmuxd code works on Linux
and Apple OS X.  Both of the afforementioned libraries use libusb1, which
is in our devel tree on the packages collection.  Through reading the OS
specific interface files in the libusb1 library for Linux and OS X, it's
clear the USB drivers are richer on those two OS's than our is in terms of
handling select/poll properly and dealing with asynchronous transactions.
I've modified the NetBSD interface code in the libusb library to handle
more asynchronous reading and writing from the ugen(4) driver, which is how
I got into this situation in the first place.  

	I hope that explanation helps a bit.  

-thanks
-Brian

On Nov 26,  8:57am, Greg Troxel wrote:
} Subject: Re: Potential problem with reading data from usb devices with uge
} --=-=-=
} Content-Type: text/plain
} Content-Transfer-Encoding: quoted-printable
} 
} 
} Brian Buhrow <buhrow%nfbcal.org@localhost> writes:
} 
} > 	The read ahead and write behind code in the ugen(4) driver transfer
} > data from the USB device in question to the user process or from the user
} > process to the USB device respectively.  Each uses a 16K buffer which is
} > filled by a series of reads or writes from or to the USB device by the
} > asynchronous buld read and write routines.  When the data is returned to
} > the user process in the case of a read call or delivered to the USB device
} > in the case of a write call, the ugen(4) does not currently respect any
} > transactional boundaries set up between the user process controlling the
} > USB device and the USB device itself.  For example, if the user process
} > writes two consecutive 512 byte blocks of data to its USB device, the
} > ugen(4) may deliver that data as one 1024 byte USB transaction to the USB
} > device itself.  Similarly, when the datacomes into the ugen(4) driver from
} > the USB device, it may be delivered as two 512 byte USB transactions, but
} > the user process will receive all 1024 bytes from the device in one read(=
} 2)
} > call.  This is a problem for the libimobiledevice library in that it
} > expects to know how much data was transfered from or to the USB device for
} > each USB transaction.
} 
} Originally, this code was for USRP, and we were unconcerned about
} boundary preservation because it was stream semantics, and the userspace
} code was doing big chunks anyway because the point was to get
} efficiency.
} 
} All of this makes me wonder: why are you even trying to use the RAWB
} code?  Are you unable to push enough data to/from your device with a
} userspace thread that does read/write?  Or are you trying to make this
} fit into a select loop that will have data be read before you call read,
} because USB is odd?   Or something else?
} 
} > 	The easiest thing I can think of, is toset up an array of buffers in
} > the endpoint state structure, each of which is the size of the largest USB
} > packet that could come from the USB device, i.e. 16384 bytes.  Then,
} > maintain indices which tell the read ahead code which buffer to use for t=
} he
} > next USB transaction and  the read code which transfers data to the user
} > process which buffers are available for transfer to the user process.
} > Right now, only one buffer is used, but I can't think of a reliable way of
} > marking transactional boundaries from the USB device while only using one
} > buffer.
} 
} An array of indices should work, and that should be about as tricky as
} the array of buffers.
} 
} > 	I think this is a similar approach to how most ethernet drivers work
} > in terms of having an array of receive buffers.  So,
} > my questions are as follows:=20=20
} 
} True, but in ethernet, there is the notion of a packet and that runs
} deep throughout the stack.
} 
} > O  Does this seem like a reasonable approach to the problem?
} 
} It doesn't seem unreasoable, but I don't understand what top-level
} problem you are trying to solve.
} 
} > O  Is it better to pre-allocate the buffers at open time, as is
} > done now for the read ahead and write behind buffers, or is it better to
} > allocate them as read calls happen?
} 
} Generally it seems better to only allocate memory when needed, unless
} you will be in the position of having to allocate at interrupt time when
} you can't, because that reduces memory usage overall.
} 
} > O  Is there a way to do what I want without having to use additional
} > buffers that I just haven't thought of?
} 
} Overall I feel like you are adding complexity in a place where it may
} not be necessary to solve your actual problem.  Could you explain, from
} the beginning, why using the USB driver without trying to use RAWB can't
} do what you want, and what's wrong?  On what other operating systems
} does libimobiledevice work ok, and how?
} 
} --=-=-=
} Content-Type: application/pgp-signature; name="signature.asc"
} 
} -----BEGIN PGP SIGNATURE-----
} Version: GnuPG v1
} 
} iEYEARECAAYFAlZXD+QACgkQ+vesoDJhHiUiDQCaA9il5+Dv+TQFhW0G8OJk3jtO
} bxsAoI5rFv0Kb1Jkthf7TjEOzeSzM7rk
} =neFH
} -----END PGP SIGNATURE-----
} --=-=-=--
>-- End of excerpt from Greg Troxel