Re: flock(2): locking against itself?

[Mouse <mouse%Rodents-Montreal.ORG@localhost>]

Back on 2023-03-30, I posted to tech-kern on a thread that had drifted
off-topic for tech-kern, into tech-userlevel territory.  I was writing
about bad interactions between libcurses and nonblocking I/O.

In particular, I wrote

> [...].  I have a program that uses curses for output but wants to do
> non-blocking input.  So I set stdin nonblocking and threw fd 0 into
> the poll() loop.

> But, in normal use, stdin and stdout come from the same open on the
> session's tty, so setting stdin nonblocking also sets stdout
> nonblocking, which curses is not prepared to handle, leading to large
> screen updates getting partially lost.

Today, I tried to fix this.

So far, I've tried setting up an output copier process, creating a pipe
which the main process uses for output, leaving that pipe blocking and
creating a copier process which copies from the pipe to the original
stdout in a way that is prepared to handle nonblocking output.

It doesn't work.  libcurses insists on getting the tty size from the
output side, and of course the pipe has no size setting.  (It might
well work if the tty size equaled the default size from the termcap
database for the terminal type in use, but that shouldn't be necessary,
and in my tests it isn't so.)

I considered using an input copier, instead, but (a) that won't fix it
all, because libcurses also pokes at its input (the code is lousy with
tcsetattr and tcgetattr on ->infd), and (b) it is significantly harder
to make an input copier process die when the main process does than it
is for an output copier.

I mentioned, in the tech-kern thread, the possibility of using
libcurses's newterm() with a funopen()ed stream for output.  That won't
work either, for basically the same reason an output copier process
doesn't, though with different details: curses uses fileno() on its
output stream and tries to get the size from the result.  With a pipe,
fileno() works but it doesn't have a window size; with funopen(),
fileno() will return -1, which of course isn't a valid descriptor and
thus doesn't support getting its window size.

It seems to me that there are multiple issues with libcurses here.

(1) It does not get along with input and/or output being set
non-blocking.

(2) newterm() takes FILE *s rather than file descriptors, but requires
for correct operation that they be connected to file descriptors anyway
(it uses fileno() all over the place).

(3) Even if the FILE *s _are_ connected to file descriptors, it does
not work correctly unless those descriptors are connected directly to
tty devices.

(4) There's no way, without going under stdio's hood, to create a stdio
stream that operates like a funopen()ed stream but allows the caller to
arrange for fileno() to return something useful.

The only solution I've come up with that stands a real chance of
working right without getting into the guts of libcurses is to allocate
a new pty and use that for output, watching for size changes on the
"real" tty and pushing them to the pty.  That seems outrageously
heavyweight for the purpose.

At this point, I'm tempted to pull the screen-updating smarts out of
libcurses (that's the only part I want to use, here) and make it
available under an API that has absolutely nothing OS-dependent about
it, possibly excepting terminal type capabilitiy lookup.  So, I'm
writing to ask:

(a) Does anyone have any other ideas, not mentioned above, for how to
address these issues?

(b) Does anyone know of any work done towards pulling the screen
updater out of curses, so it can be used without all the baggage tied
to the OS that libcurses imposes?

(c) If not, would there be any interest in such a thing?

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse%rodents-montreal.org@localhost
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B