audio/video capture timestamping

[Matthew Mondor <mm_lists%pulsar-zone.net@localhost>]

Hello,

Since I have an old Brooktree878 card which NetBSD supports, which I
successfully used in the past with custom software using bktr(4) as
part of a security suite, I thought I'd give it a new life and try to
convert rare VHS which were rotting in a drawer to a digital format.

I tried mencoder and ffmpeg, at first encoding in real-time, and had
a/v sync problems, so I then tried simply capturing the stream to an
interleaved avi file without compression, but unfortunately the issues
still persisted.

With mencoder, the video would often skip a bit to resync with the
smooth audio, and with ffmpeg the audio would often skip to resync with
the smooth video.  Capturing from bktr(4) alone, or from audio(4)
alone, is fine.

If the source was not a tape, it would probably be possible to dump the
video and audio streams separately and multiplex them afterwards, but
this would probably be useless because of the mechanical issues causing
slight speed variations.  Encoding both streams with mencoder and the
same card worked on Linux anyway, and I was then curious as to what was
wrong on NetBSD.  I also tried with another audio chip on NetBSD,
without success.

After reading a thread
(http://www.mail-archive.com/ports%openbsd.org@localhost/msg26418.html) about 
it,
it seems that the main problem would be our capture devices not
supporting timestamps, which if available could be used by an encoder
to more accurately synchronize audio/video.

I'd like to know if someone already thought about those issues on
NetBSD or already started some work to allow this.  Indeed, on Linux
with the same card and software the synchronization is better, and the
timestamps are probably the reason.  ALSA appears to support querying
the timestamp for a recorded buffer, and v4l2 also seems to support
timestamps.

We do have a v4l2 partial video(4) implementation, although I didn't
yet try capture with a uvideo(4) through it, and didn't yet read enough
of its code to see if timestamps are supported.  In any case, it
probably means that it'd be possible to eventually hook bktr(4) though
video(4) as well via video(9) and provide timestamps for userland to
use...

As for audio(4), I don't know how it could safely be extended to
support timestamps without backwards compatibility issues, other than
perhaps allowing an ioctl(2) to be used to request that timestamps be
enabled, and yet another ioctl(2) to request the latest timestamp for
the latest read buffer, which might also be considered quite hackish.
Was there a WIP for another audio interface already for NetBSD, which
also supports timestamps?  Or does anyone have suggestions on how
audio(4) could provide timestamps decently?

I honestly have no idea on how much time I myself could put working on
this, but it'd already be nice to determine what is really wanted in
this area for the future, so that if I (or any other coder) has enough
interest and time, progress could be made...  so I'm asking for
opinions and ideas.

Thanks,
-- 
Matt