VAX <--> IEEE FP

[Jason Thorpe <thorpej%me.com@localhost>]

(Cross-posted to port-vax because this might be of interest there…)

So, I was adding emulation for some FIX extension insns to the opDec trap handler and noticed that the alpha FP completion code accepts the VAX FP opcodes, but then uses the (IEEE) soft float library to execute them.  This seems Not Very Good.

So, using the known-working libvaxdata plus other random snippets easily found by web search as a reference, I have written some routines that can convert between IEEE <--> VAX formats as they are represented in an alpha FP register.  The conversion isn’t super complicated:

1- Different exponent bias.
2- Lack of Inf / NaN on VAX (those are converted to +/- extreme values and an FP_X_INV exception raised when converting to VAX).
3- Some IEEE values can overflow on VAX due to the difference in exponent bias (converted to +/- extreme and FP_X_OFL exception raised).
4- Lack of subnormal representation on VAX … those silently underflow to VAX true-zero.
5- Lack of -0.0 on VAX (reserved operand on VAX, converted to VAX true-zero).
6- Need to convert VAX dirty-zero to +0.0 in IEEE.

So my question for the DEC hive mind is: When we get a VAX FP software completion trap, is it reasonable to:

A- Convert the in-register VAX format to IEEE.
B- Then use the IEEE soft float library to execute the operation.
C- Then convert the in-register IEEE format back to VAX.

?  Are there any other exception rules I need to think about?

Of course, I don’t expect to really ever encounter this in the wild, but it’s not a lot of extra code, so figured it might be worth doing anyway, just to be thorough?

-- thorpej