lib/58615: incomplete c8rtomb, c16rtomb handles NUL termination wrong

[campbell+netbsd%mumble.net@localhost]

>Number:         58615
>Category:       lib
>Synopsis:       incomplete c8rtomb, c16rtomb handles NUL termination wrong
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Aug 17 19:55:00 +0000 2024
>Originator:     Taylor R Campbell
>Release:        current
>Organization:
The NetBS\0 Foundation
>Environment:
>Description:
C11 says of c16rtomb(3):

> If c16 is a null wide character, a null byte is stored, preceded by any shift sequence needed to restore the initial shift state; the resulting state described is the initial conversion state.

Likewise C23 of c8rtomb(3).

This seems to suggest that if an incomplete UTF-8 or UTF-16 sequence is followed by a null, it should discard that incomplete sequence (can't imagine anything else legitimate to do with it), optionally output a shift sequence, and output a NUL byte.

However, our implementation -- and every other one I've looked at, in OpenBSD, FreeBSD, and glibc -- fails with EILSEQ in this case and produces no output.
>How-To-Repeat:
c16rtomb(buf, 0xd83d, &s);    /* should succeed and return 0 */
c16rtomb(buf, L'\0', &s);     /* should succeed, write one NUL byte to buf[0], and return 1 */
>Fix:
Yes, please!