Subject: fpsetmask / IEEE 754 arithmetic
To: None <port-alpha@netbsd.org>
From: Paul Mather <paul@gromit.dlib.vt.edu>
List: port-alpha
Date: 12/17/1999 00:59:13
I've been messing around a little tonight trying to get the R stat
package to run on my NetBSD/alpha 1.4L DEC 3000/300.  (The package
compiles, but dumps core with a floating point exception in
"arithmetic.c" upon starting up.

It seems the culprit is that R expects the mathematics library to behave
like IEEE 754: specifically, that (double) 0.0/(double) 0.0 returns NaN
instead of generating an error.  There is some murmuring in the INSTALL
doc. about FreeBSD requiring a "fpsetmask(0)" to set the correct IEEE
754 signalling behaviour, so I figured this might be good for NetBSD,
too.

However, after inserting the appropriate statement, still no luck.  Upon
further investigation, though, I couldn't get fpsetmask to do much of
anything.  According to the man page, it is supposed to return the value
of the previously-set fp exception mask, yet all the time it returns 0,
even when I set it to a non-zero value (e.g., FP_X_INV, etc.) and
even query the value via fpgetmask.

BTW, am I right in thinking that "fpsetmask(0)" will clear all the
exception types, meaning no FPEs will be generated, or do I need to do
"fpsetmask(FP_X_DZ)" to disable divide by zero exceptions?  (I find the
man page a little unclear.)  Either way, it doesn't explain the bad
return values I'm getting all the time (and I've tried it under both
interpretations and get FPEs both ways).

I notice there is a bug report (#6805) filed for something akin to this
problem going back to NetBSD/alpha 1.3I.  I don't get a core dump from
the fpsetmask() call as in the bug report, but I do always get FPEs no
matter what the mask I use.

If anyone could offer any hints; I'd really like to get R running on my
alpha (since PSPP appears to have bad math).

Cheers,

Paul.

e-mail: paul@gromit.dlib.vt.edu

"Without music to decorate it, time is just a bunch of boring production
 deadlines or dates by which bills must be paid."
        --- Frank Vincent Zappa