tech-userlevel archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: long double losing mantissa bits



> Date: Tue, 19 Jan 2021 22:16:32 +0000
> From: Taylor R Campbell <riastradh%NetBSD.org@localhost>
> 
>    input: 72057594037927937 -> scanf: 72057594037927937.000000
>    frac: 0.500000 0x8.00000000000008p-4 sexp: 57
>    uexp: b9
>    ufrac: 0080000000000000         
>    56   : 00ffffffffffffff         
>    Unexpected result: 5c80 0000 0000 0000
>      expected       : 5c80 0000 0000 0001
>    056200: sign:  0 uexp:  b9 ufrac: 00 0000 0000 0000
> 
> Here frac = 0x8.00000000000008p-4 = (1 + 2^56)/2^57, so
> frac*72057594037927936 = frac * 2^56 = 0x8.00000000000008p52 =
> 0x80000000000000.8p52.
                    ^^^
Excuse me -- the exponent at the very end was a typo.  It should read:

> Here frac = 0x8.00000000000008p-4 = (1 + 2^56)/2^57, so
> frac*72057594037927936 = frac * 2^56 = 0x8.00000000000008p52 =
> 0x80000000000000.8.

Or, frac * 2^56 = (1 + 2^56)/2 = 1/2 + 2^55.


Home | Main Index | Thread Index | Old Index