Hello,

I'm taking a course in Computer Architecture, and one of the topics is on binary representations of numbers in the computer. One of the points that they make is that floating-point arithmetic is not associative nor distributive, but I just don't understand how (especially the associative part). Could someone please explain this to me?

Actually floating point arithmetic is associative and distributive.

Floating point numbers have a fixed length which means that round off errors are common. You have to give some consideration to the order of your calculations and the magnitude of your intermediate results to minimize the effect of round off errors. If you do a sequence of calculations in several different orders you can get several different final results depending on how fast the round off errors propagate through the various intermediate results. All of this is a consequence of round off errors and does not mean that floating point arithmetic is non-associative or non-distributive.

-----------------------
Steve Stites

Actually, that is exactly what it means. Associative means that A*(B*C) = (A*B)*C, but given roundoff errors and number magnitudes etc this can be easily shown to not be true in all cases with computer floating point arithmetic. Hence, computer fp arithmetic is not associative.

Similarly, distributive property says that (A+B)*C = A*C + B*C. Again, it is simple to show that this is not true using computer fp arithmetic.

The fact is that the breakdown of these rules is due to a defect in the floating point arithmetic scheme, where word size turns out to be very important. But the existence of the defect leads to the consequence.

jiml8

Semantically what you say is correct but it is misleading. Suppose a student programmer solves a complex calculation using two different valid arithmetic sequences and gets two different answers. When he asks what is wrong are you going to tell him that floating point arithmetic does not follow the rules of algebra or are you going to explain the effects of roundoff errors and number magnitudes on intermediate results?

---------------------
Steve Stites

I am going to tell him that because of the limitations of the hardware, the numerical approximations to the mathematical functions are not associative or distributive, and therefore he has to pay attention to the order of his operations.

From the practical point of view, the main thing is to keep track of the progression of errors.

In computer sci, you use the rationals and not the reals usually.

Formally floating point arithmetic is actually being done over a "set that is larger than the set of rationals".

Q - field of rationals

Define a reflexive and symmetric relation T on Q via

$(x,y)\,\in\,T$ iff <your limitation on bounds based condition>

Now the set of "maximal mutually T-related elements" will actually be larger than Q

When two 'floating point numbers' are being added, then such "maximal mutually T-related elements" are being added.

Best

A. Mani

Thanks jiml8, that's kind of what I thought, but I was (and still am a little) confused over the exact reason. What confuses me even more is that if you take floating-point arithmetic in the real world (outside of the confines of a digital circuit), then normal mathematical concepts apply. aaarrrggghhh!