Subject: Re: lib/6314: Bug in libc with `double' => `long long' conversions
To: der Mouse <mouse@Rodents.Montreal.QC.CA>
From: Krister Walfridsson <cato@df.lth.se>
List: netbsd-bugs
Date: 10/18/1998 01:54:57
> > Compile the following program and execute it.  It will print
> > 280379776630786 instead of 280379776630785 as it should.  The bug
> > seems to affect several NetBSD platforms.  It also happens on i386
> > and mac68k.
> 
> > long long f (double x) { return x; }
> > main () { printf ("%qd\n", f (280379776630785.0)); }
> 
> It also behaves as described on my SS1+ (the sparc port).  I have not
> yet investigated further; if-&-when I do, I will report here.

This is due to a bug in __fixunsdfdi (libc/quad/fixunsdfdi.c). The culprit
is

        if (x < 0) {
                t.ul[H]--;
                x += ULONG_MAX;
        }
        if (x > ULONG_MAX) {
                t.ul[H]++;
                x -= ULONG_MAX;
        }
        t.ul[L] = (u_long)x;

that obviously should be something like

        if (x < 0) {
                t.ul[H]--;
                x += ULONG_MAX + 1;
        }
        if (x > ULONG_MAX) {
                t.ul[H]++;
                x -= ULONG_MAX + 1;
        }
        t.ul[L] = (u_long)x;

But I'm not sure this is good enough. For example, what happens if 
ULONG_MAX < x < ULONG_MAX + 1? Can it happen? And I don't understand 
why we are doing

        toppart = (x - ONE_HALF) / ONE;

instead of

        toppart = x / ONE;

since we have to compensate for too big/small toppart anyway...

I have glanced at some of the other floating point functions in libc, 
and my intuition says that they may loose accuracy too.

I have fixing this on my list of things I should look into sometime,
but I don't have enough knowledge about numerical analysis, so I need
to spend some time studying first...

   /Krister