control that floating point [was: Re: single precision]

[Ashwin Hirschi <deery@...>]

Ah, floating point control words... that brings back some memories... Bad memories!

So, here's some information of special interest to anyone using Lua to develop for good ol' Windows. Brett, if you're doing PS2 development[?] this probably doesn't concern your particular situation. But since you're bringing up Lua numbers and control words I reckon this is as good a time as any to relate my little (true) story.

Here goes...

Not so long ago we were testing one of our Lua-based products on several Windows configurations. Everything worked like a charm, right uptil the point I discovered our application would sometimes behave very strangely on one of the new test machines. On that particular machine our software would start out fine, but all of a sudden Lua would throw us alerts complaining about nil values being used to index tables, simple calculations that could not be performed, strings that suddenly disappeared and a wide variety of other (very unwelcome) SFX.

Quite to my dismay it turned out the software could even forgo the trouble of displaying a messagebox (with the Lua alert) and simply crash. Now, my runtime crashes *very* rarely [not due to any of my talents, but because of the high quality of Lua! :-)], so I was really alarmed. It was all the more baffling since everything had been running so smoothly for months on all the other machines...

To make a long story not too long: after a weekend of low-level debugging [using the excellent OllyDBG] I found the cause of the problem:

     Some Windows API functions do NOT necessarily preserve the FP control word.

Yes, it's true: they're (probably?) supposed to, but that doesn't make it so! In my case the offending function turned out to be PlaySound [or more likely: some flaky sound driver used in the underlying sound system, hidden behind PlaySound]. But some subsequent web searching turned up a host of other known "possible offenders" [like a couple of the common dialogs].

How can a changed FP control word cause trouble for your Lua-based software? Well, depending on which bits were changed simple FP computations may not produce the expected numerical results. And many Lua operations are based on these results, with doubles/floats used for Lua numbers. Also, fairly innocent (and common) FP exceptions that are usually masked (like under/overflow situations or somesuch) may suddenly reach the application, causing it to crash.

So, what's the moral of the story? I guess if you're simply aware of this particular issue you're half-way there. Incidental/indirect changes to the FP control word can cause Lua to (seemingly) act very crazy, since there are a lot of floating points being used behind the scenes.

If you think you've run into this problem, check to see if any of your crashes are actually FP exceptions, since that may be an indication. In any case, dig up your trusty debugger and keep a close eye on the value of your FP control word. If you find the Windows API function that allows / does the control word changing, you're there!

Ashwin.
--
no signature is a signature