Port-amd64 archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Changing x87 precision to full 63bit as default



On 12 Nov, 2013, at 18:17 , Dennis Ferguson 
<dennis.c.ferguson%gmail.com@localhost> wrote:

> which, assuming dc isn't broken, is actually closer to the
> "wrong" answer of 1.0000000000000004 than it is to the "correct"
> answer of 1.0000000000000002.  Of course the rounding of the
> answers to 16 digits exaggerates the difference; the correct value
> of the ratio is almost exactly in the middle of those two values,
> but when I print them with many more digits the precise value
> is still closer to the "wrong" answer (in the 23rd decimal place).
> 
> The definition of "correct" and "wrong" being applied in this case
> is bizarre.

Sigh, I didn't print enough digits.  1.0000000000000002 is the
correct rounded-to-nearest 53 bit answer, but the difference
is in the 31st decimal digit.

$ dc
40k
1 2 2 52 ^ / + p
1.0000000000000004440892098500626161694526
3002399751580332 3002399751580331 / p
1.0000000000000003330669073875469251492345
- p
.0000000000000001110223024625156910202181
3002399751580332 3002399751580331 / p
1.0000000000000003330669073875469251492345
1 1 2 52 ^ / + p
1.0000000000000002220446049250313080847263
- p
.0000000000000001110223024625156170645082
- p
.0000000000000000000000000000000739557099

Dennis Ferguson


Home | Main Index | Thread Index | Old Index