Re: VAX floating point formats & tuning GCC for CPU models (was Re: Some more patches for GCC...)

[Jake Hamby <jehamby420%me.com@localhost>]

> On Apr 6, 2016, at 14:35, Jake Hamby <jehamby420%me.com@localhost> wrote:
> 
> Other than those differences, you can convert between IEEE single and VAX F_float, or IEEE double and VAX G_float, by word-swapping the 16-bit words within the value (VAX uses big-endian 16-bit words for its FP formats, a PDP-11 artifact), and by adding or subtracting 2 from the exponent bias (can be done with integer add/subtract on the word containing the exponent).
> 
> It took me several hours of confusion before I finally figured out why the VAX docs say they use "excess 128" and excess 1024 encoding for the exponent, when you have to add/subtract 129 or 1025 to get the real exponent (IEEE uses excess 127 for singles and 1023 for doubles). On VAX, the mantissa range is from 0.5 to 0.9999..., and on IEEE, the range is from 1.0 to 1.9999..., with a hidden one bit in both cases. So if you interpret the mantissa in IEEE terms, the "real" exponent bias for VAX FP formats is 129 (F and D_float) or 1025 (G_float).

To try to clarify a point that's really confusing, VAX floating-point numbers in all formats are stored with the exponent and sign bit in the first 16-bit (little endian) word, followed by 16-bit chunks of the mantissa in big-endian order. So the bytes within the 16-bit words are little-endian, but the words within the 32-bit or 64-bit (or 128-bit for H_float) number are in big-endian order (most significant bits first).

In order to convert between VAX and IEEE (which uses the native big or little-endian byte order with the sign & exponent in the most significant bits) format, the easiest method is to reverse the 16-bit words within the number, but don't swap the bytes within the 16-bit words. The VAX assembly-language math routines do a lot of this, and the fdlibm routines (whether in C or asm) for IEEE do the same thing.

Since the number of bits of exponent and mantissa are identical between IEEE single & VAX F, or IEEE double & VAX G, it is possible to convert between them as I described, but the difference in the exponent bias means that IEEE can hold larger values (higher max exponent). VAX can hold smaller values without denormalization, but because IEEE FP supports denormal numbers, the max negative exponent is increased by 23 or 53 by using the bits in the mantissa to encode the leading 1 bit instead of the normal implicit leading 1. Since VAX doesn't support denormal numbers, its additional range of 2 for negative exponents due to the different exponent bias is helpful to avoid underflow, but not an advantage over IEEE.

Regards,
Jake