#### Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

# 0.3 does not equal 0.3?!? Codea bug? Lua bug? Ambush bug?

Posts: 557

Can anyone tell me how to fix this? Apparently I've done something to make three not equal three.

Here's my code:

``````function wrapXAroundY(X, Y)
if X < Y then
return X
end
return Y - (X - Y)
end

function tests()
local base, given, intendedResult, calculation = 0.0, 0.0, 0.0
base = 0.35
given = 0.40
intendedResult = 0.30
calculation = wrapXAroundY(given, base)
print("calculation is: ", calculation)
print("intendedResult is: ", intendedResult)
print("calculation and result are equal: ", intendedResult == calculation)
end
``````

Here's the output:

``````calculation is: 0.3
intendedResult is: 0.3
calculation and result are equal: false
``````

??????
!!!!!!!
??????

Tagged:

• Posts: 35

This is a subtle technical detail that's present in almost all programming languages. Internally the numbers are stored as floating point (possibly as doubles - not sure for Lua). Technically it is unsafe to compare floating point values for equality unless certain conditions are met (those conditions hold for whole numbers, for example).

Internally the number of bits are limited and certain fractional values end up being stored as approximations. This means there might be multiple ways to represent 0.3 in terms of the actual bits being used under the hood so 0.3 is not always exactly equal to 0.3.

I don't know what best practice would be for Lua here, but one workaround would be to convert them to strings and compare the strings.

• Posts: 557
Do you know of a way to test whether or not your speculation is correct?

I know that floats have all sorts of weird behavior, but not all languages collapse in the face of them. For example I don't think I could quite as easily get Swift to claim with a straight face that 0.3 doesn't equal 0.3. I certainly can't do it with the identical operations--I tried it out and Swift handles those just fine.
• Posts: 35

Swift:

• Posts: 557
Ha, you got me there! I had written the code on a little Swift-simulator on my iPhone, and I thought I had done that exact thing, but I must not have.
• Posts: 35

Converting to strings before comparing will probably work, but it wouldn't be super efficient. Better to use a function that uses an epsilon to compare floats or just try to avoid having to do this sort of comparison altogether.

• edited September 2017 Posts: 175

@UberGoober As @BigZaphod said, this is a typical bug that occurs when testing whether floating point numbers are exactly equal. It occurs because floating point numbers have limited precision and are not exact.

I've modified you're code slightly so that you can see what's going on.

``````function wrapXAroundY(X, Y)
if X < Y then
return X
end
return Y - (X - Y)
end

function setup()
local base, given, intendedResult, calculation = 0.0, 0.0, 0.0
base = 0.35
given = 0.40
intendedResult = 0.30
calculation = wrapXAroundY(given, base)
print(string.format("calculation is: %.20f",calculation))
print(string.format("intendedResult is: %.20f",intendedResult))
print("calculation and result are equal: ", intendedResult == calculation)
end
``````

The way you fix this is the same approach as other languages. Instead of testing for exact equality, you have to test for approximate equality within a certain precision.

``````function setup()
print(floatequal(0.101,0.100,0.1))
print(floatequal(0.101,0.100,0.01))
print(floatequal(0.101,0.100,0.001))
end

function floatequal(left,right,precision)
local diff = math.abs(left-right)
return diff < precision
end
``````

Edit: Just a word of caution about the above approach. Depending on you're context it can be good enough but this is not always the case. The following blog post and stack overflow explains it better than I could hope to.

https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/

https://stackoverflow.com/questions/17333/what-is-the-most-effective-way-for-float-and-double-comparison

• Posts: 557
Oy all that makes my head hurt.

A system for representing numbers that makes simple math unreliable seems like a very odd thing for the world of computer science to embrace.

I know it's beyond the scope of these forums to discuss questions this broad, but it really leaves me wondering, why are these things a good idea in the first place?

What if I just turned everything into ints by multiplying everything by 100000 or so? That should easily be enough precision for me, and then all I have to do is good old int math, easy as pie. :-)
• Mod
Posts: 8,199

Here's an example that shows different values of .3 and how to do a compare with .3 .

``````function setup()
z=.3
print(string.format("%.30f",z))

z=.1+.1+.1
print(string.format("%.30f",z))

print("z =",z)

print("z - .3 = ",z-.3)

if math.abs(z-.3)<.00000001 then
end
end
``````
• Posts: 175

@UberGoober Think of it along the same lines as a data type like int or float having a minimum and maximum representable value. There's a limited number of bits used to represent it in binary. So in the case of floating point types an approximation is as good as it gets, but it's good enough.

Also if you start multiplying everything by large numbers then you could run into overflows instead which would cause similar problems. It's a surprising complex problem depending on what your context is but I wouldn't worry too much unless you're doing something really specialised.

• Mod
Posts: 8,199

@UberGoober See this link if you want to see the results of what a value is and what can be represented. Enter .3 in the area `You entered`, then press return to see the actual value stored.

``````https://www.h-schmidt.net/FloatConverter/IEEE754.html
``````
• Mod
Posts: 884

Maybe one of them became string?

• Posts: 557

So yeah okay sure whatever sure. So floats can't exactly represent certain values, so getting really really really close has to do, okay fine. So even then, those really really really close numbers can't be expected to be consistent from one float to the next. So yeah okay sure whatever sure.

Why the heck does every language pretend that they're accurate representations? Why show me 0.3 and 0.3 instead of 0.300000-whatever and 0.300000-slightly-different-whatever? It seems like unaccountably intentional befuddlement.

• Mod
Posts: 8,199

@UberGoober They are accurate for the amount of bits that are used to represent the numbers. For example, if I said show me \$14.23 using just \$10 bills, the closest you could come is 1 \$10. If you used \$10 and \$5 bills, you could get closer with 1 \$10 bill and 1 \$5 bill. If you used \$10 or \$5 and \$1 bills, you could get even closer. The smaller the denomination, the closer you can get. So depending on the number of bits a language uses depends on how close they can get to a specific number. Codea uses 64 bits. Other languages could use even more bits.

• Posts: 557
Exactly my point @dave1707.

If I said show me x in denomination y, and you did, you would be responding correctly to an explicit request.

If I said show me all the money in your pockets, and you had \$3.02 but you only pulled out \$3.00, you would be responding dishonestly to an explicit request.

If I said here, use this big fancy abacus and tell me what it says 4 minus 1 is, and you went and did all the clicky-clacky-clicky-clacky, and the answer came up 3.00000000002, and you came back and said to me "the abacus says 4 minus 1 is 3," you would be responding dishonestly to an explicit request.

Representing floats inaccurately is a traditionally unacknowledged yet omnipresent obfuscation that causes lots of trouble, trouble that simple honesty would avoid--lots of mistakes, and lots of otherwise-unnecessary lectures by CS professors, and lots of otherwise-unnecessary forum threads like this one.

I don't understand the purpose of displaying one thing as another when it's not. It seems like an emperor without clothes. Just sayin'.
• Mod
Posts: 8,199

@UberGoober The way floats are represented aren't unacknowledged. The majority of programmers know that floating point numbers aren't exact. When they start using them, they realize that fact just as you are now. Normally when you do a print(.3) it prints .3 because the print function will round to about 6 or 7 places to the right of the decimal point. I'm sure that if you did do a print(.3) you would want it to print .3 and not .300000011920928955078125 . Below is a little program that prints the numbers from 0 to 4 in .1 interval. This shows what Codea thinks the numbers are to 25 digits past the decimal point.

``````displayMode(FULLSCREEN)

function setup()
textMode(CORNER)
end

function draw()
background(0)
fill(255)
c=0
for z=0,4,.1 do
c=c+18
text(string.format("%.1f     %.25f",z,z),250,HEIGHT-c-20)
end
end
``````
• Posts: 557

@dave1707, sorry to be annoying, but it seems like you keep making the opposite of the point you think you're making--my point, in fact.

The majority of programmers know...

Yes, this is a great definition of obscurity: something you have to know in order to know it. That a group of people, who are already insiders, share a common understanding is, I think, what they call trade knowledge, and it's by definition the opposite of something plainly discoverable.

Normally when you do a print(.3) it prints .3 because the print function will round to about 6 or 7 places to the right of the decimal point.

Exactly, dave! The print function performs a rounding that it never tells you about. This is exactly my abacus example. If I say "tell me what the abacus comes up with for ___", and you just say "it came up with 3," you're lying to me. You don't have to say "it came up with .300000011920928955078125" every time, but you could at least say "it came up with 3 plus shavings," or something like that.

To my limited powers of recall, it seems like standard practices in Objective-C used to call for adding an `f` after all floating-point numbers, so instead of `3` you'd write `3.0f`. It was an optional stylistic choice, not required by the language, and I used to hate it. When they announced that it was now preferred to not use the f, I was very happy. And I continued to be happy that the coinage didn't even exist in Swift (AFAIK). Older and wiser, I now see the wisdom behind having some obvious sign that a number has been rounded or is incapable of being precisely represented. I never thought I'd say it, but I miss the `f`.

If you took all the man hours that have been lost to the world because of the lack of an obvious signifier that a floating-point number is not a precise number... uh... you'd have a whole lot of hours!

• Mod
edited September 2017 Posts: 8,199

@UberGoober Before you started to write programs, did you know how a 'for' loop worked, or how a 'class' worked. No. You learned how to use them just as you learned how to use everything else throughout your life. You either read about it, somebody told you about it, or you learned thru trial and error. The same thing goes for using `floating point numbers`. You now know that floating point numbers can't be held in memory as exact numbers. Apparently when you started to use them, you didn't know how they worked. You found out that something was different about them thru trial and error and now you are finding out why. There are a lot of things in Codea you don't know about, so are you going to say that there's something wrong with it. No, you're going to learn how to use them just as everyone else that uses Codea will do including me. The next version of Codea uses Craft. I don't know anything about Craft (well I know a little since I have the beta version) but I'll learn more just as everyone else will. That's the fun of programming.

``````https://en.m.wikipedia.org/wiki/Arbitrary-precision_arithmetic
``````
• Posts: 557

@dave1707 so, yeah, I get it, you think it's a thing that's just part of normal learning, and that my points are all dismissable by "that's what it means to learn things."

I'm pretty sure that failure to plan for floating-point inaccuracies is a thing that is a thing, you know? I'm not complaining about `for` loops here. I'm talking about a very common mistake that fouls up even the most experienced programmers from time to time.

Would I like this mistake to be easier to avoid? Yes. Am I fine with shrugging and moving on? You betcha.

• Mod
Posts: 8,199

@UberGoober I'm not trying to dismiss anything you're saying. I just trying to explain that you can't express an infinite number of values using only 64 bits. Using only 64 bits also affects integer values too. Try the example below.

``````function setup()
val=1000000000000000000

a=val*9
for z=1,5 do
a=a+1
print(a)
end

print()

a=val*10
for z=1,5 do
a=a+1
print(a)
end
end
``````
• Posts: 557

@dave1707 yes, I understand, and it seems we're talking at cross-purposes. You're emphasizing the limits of expressing any kind of numerical value in a bit-constrained system. I'm addressing how those limits are conveyed or not conveyed to the user.

• Mod
Posts: 8,199

@UberGoober I guess it is hard to look up something when you don't know that you have to.

• Posts: 35

I don't think anyone should make excuses for the way floating point has historically been handled in programming languages. It is really weird and unexpected and it is "trade knowledge" and it is pretty crappy and constantly bites people in the ass. It would be better if all of this wasn't true or there were warnings or other mechanisms to surface the situation to learners, but it is what it is at this moment in time.

Engineering is about making tradeoffs in the face of imperfect realities and the tradeoffs between storage size, performance, and precision is an especially difficult one. Decisions were made decades ago. Standards were written. Conventions established. It's not an unchangeable situation, but the first step is knowing what you're up against.

• Mod
Posts: 884

`tointeger()`? `.. ""`? Tryed already?

• edited September 2017 Posts: 154

I had never thought floating point numbers and their inexact representation in programming languages implementations was a "trade knowledge". I think it is something you learned pretty early on, if you started with C, Pascal or even BASIC. And frankly, it only ever bite you in the ass once. After that one time you should know better. Perhaps today, when newer languages are further removed from hardware, it is less common that you would bump into accuracy of floating point numbers.

There are also an entire classs of numbers that you cannot represent accurately in binary or in decimal form even theoretically - irrational numbers. So, for comparing floating point numbers you would typically do something like:

``````local accuracy = 0.00001
if math.abs(a - b) < accuracy then
print("close enough")
end
``````

Tweaking the accuracy accordingly to your needs.

@UberGoober , i know you are arguing a broader point, and not asking for code snippets, but really i think @dave1707 is exactly right - some things you don't know and then you learn, there is no conspiracy here

• Posts: 557
@juce, may I kindly point out that you said you didn’t think it was trade knowledge and then went on to describe it as exactly that.

It’s Standard transmission vs Automatic, as it always is in these discussions. Just know what side you’re on!
• Posts: 154

@UberGoober, i see what you mean, and i guess you're right to a certain extent... but then pretty much anything related to programming could be classified as "trade knowledge". How would you define that term exactly?

• Mod
Posts: 8,199

@UberGoober Trade knowledge can refer to anything. I'm sure doctors know a lot of things about surgery that I don't. You said above, `That a group of people, who are already insiders, share a common understanding is, I think, what they call trade knowledge, and it's by definition the opposite of something plainly discoverable`. I'm sure if I wanted to be a surgeon I would have to know a lot more than I do now. And even if I learned enough to be a surgeon, there would still be things that I wouldn't know until I actually did surgery. The same applies to programming. There are things you won't know until you start programming and actually run into them. You can't expect to be told everything that you don't know. Some thing you have to learn by doing it.

• Posts: 194

That's really weird. Lua's console does deceive you, in a way. Trying something similar in Python and JavaScript, I found that both perform the same arithmetic error, but they print out the inaccuracy to the console properly.
JavaScript console output:
`Calculation is: 0.29999999999999993 Intended result is: 0.3 Calculation and intended result are equal: false`
Maybe Lua should do something similar.

• Mod
Posts: 8,199

The print function in Codea rounds the result to 14 digits which would print .3 . By now everyone should know how floating point numbers are stored, how they print out, and that you won't always get an equal comparison in some situations. That's the way it works and it won't change until Codea goes to 128 bit math which I doubt it ever will. To get a larger print result, use string.format which overrides the normal print function.

• edited September 2017 Posts: 557

I walked into an art store once and I asked the counter guy "hey how long do these different paints last?"

He said "what?"

I said "These different kinds of paint, the brands and types, do you know how long it takes for them to lose their color or fade or distort or whatever?" These are paints for art, not for house painting, of course.

The guy says "I can't do everything for you! It's like you're coming in here and asking me to make a painting for you!"

...which, you know, I wasn't at all. I wasn't doing anything of the kind.

But this was clearly an art school kid, frustrated with the dumb questions people come in off the street and ask, and my particular question--while not related to anything he was talking about--set off his "why should I have to do everything for you people" alarm. So I got the "why should I have to do everything for you people" reaction, even though my question regarded some very specialized knowledge that it wouldn't be easy for just anyone to pick up on their own.

So, if you like, you can see my objection to the way floats are communicated to users as being in the "why should I have to do everything for you people" category. That's a perspective you get to assert, and that's what it seems like when you start to say things like "everything is trade knowledge". Which is fine.

Parenthetically, when you say "By now everyone should know" I'm amused to speculate what you mean by the terms "by now" and "everyone". Do you mean ever since all public broadcasting has been playing an hour of Codea tutorials every day since 2005? I agree! Those tutorials are really well done! People who don't know Lua by now have only themselves to blame!

Anyway, I personally don't think this float thing falls under "why should I have to do everything for you people" category.

You do, I don't, what are you gonna do? Peoples is peoples.

So when you write your book on coding in Codea, maybe you won't take any special care to warn people about how Codea inaccurately represents floats in print statements (although does accurately represent them if you watch them as a parameter, btw). And when I write mine, maybe I'll do a chapter on it. Yay, everybody's happy, no big whoop.

• Posts: 557

@em2, see that makes sense to me. Of the two approaches, the Lua one and the Python one, the Python one is the one I would expect Swift to use, and I'm surprised it doesn't.

• Mod
Posts: 8,199

@UberGoober When I said, `By now everyone should know`, I meant everyone reading this discussion. I still stick with my point that nobody knows everything and you’re going to run into things you don’t know. How you handle those situations are up to you. Are you going to say that someone should have told you about them before you ran into them, or are you going to say, that doesn’t work the way I expected and figure out why.

• Posts: 154

@UberGoober , i don't think your example about art store counter guy is relevant. No one here was responding to you with that sort of attitude.

• edited September 2017 Posts: 557

@juce I agree, no one had an irrational flip out in a cartoonish way. The attitude wasn't the point.

Just so we're all on the same page, the stuff we're arguing about is what some people's whole careers are based on trying to figure out. To make a language robust, accessible, powerful, parse-able, and feature-competitive is hard.

These questions constitute some people's entire work life. We're not going to resolve this is a you're right, I'm wrong way. If it was that easy, we'd never be having the argument in the first place.

You're right enough and I'm right enough that this could go on forever. Can we just say, hey, look, we're in an argument that could go on forever, let's not?

• Posts: 557

The guy who wrote the old Balance of Power game, wayyyyyyy back in the day, wrote a blog post called "Why Everyone Should Learn to Program," a long long time ago. He called the first rule of programming "try to figure it out for yourself." In other words, if you want to learn to program, before you ask anyone else how to solve a problem, see if you can do it on your own.

That's what I think is true, and I think that's what you guys are saying. I agree with that.

• Posts: 557

Oh and, @dave1707 , thanks for clarifying what you meant by `By now everyone should know.` I honestly didn't know what you meant, it makes more sense now.

• Posts: 154

True. This lovely conversation could go on and on...
As so elegantly put by an XKCD comic here: https://xkcd.com/386/

Forgive me for another long post. And yes, it is not black and white, but I do feel that you're more wrong than right. Let me make one last attempt to explain why. Then i will shut up.

I completely agree with you about language design. It is hard, and it is also why there are so many programming languages around. Some are better designed and some are quite awful.

However, the choice of particular implementation of numbers is typically about hardware, not about language design. The way numbers are represented in computers is one of the most fundamental things in programming and in computers science and electrical engineering in general. If you were doing any sort of scientific calculations, solving equations with numerical methods, etc. - you would pretty much instantly learn about many various aspects of not just floating point numbers but integers too. But Codea is not about math, it has a different focus: it's about graphical ideas and games. The brilliance of Codea is that people without any CS background or prior programming experience can start using it and be able to create something right away. And they don't even need to think about numbers, bits and bytes.

However, in the end of the day, Codea is an application running on a computer, with a particular architecture. Lua is the programming language used by Codea, and like any other programming language it needs to handle numbers in a way that is memory-efficient and fast on a particular kind of hardware that it is running.

Just to give one example:

Let's take integers, because there is nothing complicated about them, right? Wrong. You can fall into all sorts of traps. Integers can be represented with fixed number of bytes or with a varying number of bytes. They may or may not use highest bit as an indicator of negative value. A size of "int" may be defined by hardware or it may be defined by the language spec itself (as in Java). For instance, let's say an "int" as a 4-byte integer, which represents both negative and positive values, and there is a specific combination of bits for the value of 0. This means that you can only represent values in this range: [-2147483648, 2147483647]. What happens when you calculation produces a value of 2.2billion? It depends. You may run in to an OverflowError or the value may simply be truncated. Neither is a good outcome, so in the situations where you do need to deal with big numbers, you may be able to switch to another fixed-width data type that uses more bytes (8-byte integer), or sometimes that is not possible, because the values are just too large. In that case, there are data types specifically designed for that: such as big integers in Python - bigger values use more bits, as needed. But, such specialized data types have their pros and cons - operations on them don't map directly into hardware instructions, so math involving such numbers is slower.

Anyways, my point is that there are scenarios where neither floats nor integers can be viewed as simple black boxes that magically always work as you expect. Most often they do, but in some cases, you must take hardware into account.

Should that all be explained in detail in Codea documentation? Frankly, I don't see the need. It's not unique to Codea or Lua, so i think that it is totally reasonable to expect users to do some research to find out what's going on, if/when they bump into "strange" behaviours.

• Posts: 154

@UberGoober , heh.. took me too long to write my previous post!
You already added some good messages meanwhile, which kinda make my post unnecessary

• Posts: 557

I am glad to have you guys sharing your knowledge here. Thank you.

So the thing that I'm not clear on is the thought process, the logic, behind these representations.

Let's say I have two groups of people. I don't know their exact numbers, but I know there are three families in each group.

Now, if I want to order Winnebagos for each group, and I can assume each family fits in a single Winnebago, I know that I can order 3 Winnebagos for one group and 3 Winnebagos for the other group. Each needs the same amount of Winnebagos.

However, if I want to order hats for each group, I actually have no idea how many hats to order, because I don't know the size of each family. Not only am I unable to say how many hats one group needs, I am completely unable to say that they need the same amount of hats.

Right?

So let's say Lua and I were discussing this situation.

``````Me: How many Winnebagos in group 1?

Lua: 3

Me: And how many Winnebagos in group 2?

Lua: 3

Me: Ok. So group 1 and group 2 have the same amount of Winnebagos.

Lua: Incorrect.

Me: Incorrect? What? Why?

Lua: Number of hats needed for each group is not known.

``````

That doesn't make sense, right? It's inconsistent.

But that seems like the method that the language-makers chose for reporting floats: talk in terms of Winnebagos in all cases except equivalence, and in the case of equivalence talk about hats.

I'm curious what the thought process is, what the logic is, behind that decision.

• Mod
edited September 2017 Posts: 8,199

The IEEE Standard for Floating-Point Arithmetic (IEEE 754) is a technical standard for floating-point computation established in 1985 by the Institute of Electrical and Electronics Engineers (IEEE). The standard addressed many problems found in the diverse floating point implementations that made them difficult to use reliably and portably. Many hardware floating point units now use the IEEE 754 standard.

As you can see above, there is a standard for floating point. By using this standard, floating point values in different languages will be represented the same. How the floating point value is displayed will vary by language based on who wrote the language, basically the print function. Some languages may not truncate the value and print the full number, while other languages will round at a certain number of digits and only print those digits. The way floating point values are stored isn’t going to change anytime soon. Their values aren’t going to change until more bits are used to store them. And no matter how many bits are used, there will always be floating point numbers that can’t be represented exactly. That’s the way it is and it’s not going to change. It doesn’t matter how many Winnebagos are made.

• Posts: 557
As you say, it’s not going to change. If I’ve given the impression in my last post that I want to change it, I’ve given the wrong impression.
• Mod
Posts: 8,199

@UberGoober I didn’t get the impression that you wanted to change it. I’m just saying it’s not likely to change anytime soon so you might as well get used to the way it works. I suggest you google floating point numbers and read about it so you understand what you might run into as you use them.

• Posts: 557

I do often I suppose give the impression I know nothing about all this, but it's not so much the case. And just so, the act of questioning any entrenched system leaves one open to being told to google it. That's fair enough.

I did initially ask the question "what are they doing it this way" with the intended subtext of "that's dumb and they shouldn't do it that way." I am not anymore and perhaps I didn't make that clear.

I am sure I don't need to explicate the Winnebago/hat analogy for you, but I think I may have seemed to be trying to ridicule the practice, and I wasn't. I was before, but now I'm actually asking out of sincere dispassionate curiosity. It's a logical inconsistency to quantify imprecise quantities as Winnebagos in some cases and as hats in others, and I'm actually asking what the rationale for that is. In describing the IEEE standard, and the varying approaches to the print function, you seem to be putting forth the viewpoint that it doesn't really matter how different languages approach the print function, because they're all talking about the same underlying storage system, and understanding that system is what really matters.

Of course, to have the deepest facility with floats, you're absolutely right--that system is what really matters. That said, I'm curious to understand the logic being applied to how they're represented in the print function, because a lot of thought usually goes into these things. At this point I'm neither critical of it nor ignorant of it--I'm just curious about it.