[MacRuby] #521: Float problem with 8.1 and 9.1
#521: Float problem with 8.1 and 9.1 ------------------------------------+--------------------------------------- Reporter: macruby@… | Owner: lsansonetti@… Type: defect | Status: new Priority: major | Milestone: MacRuby 0.5 Component: MacRuby | Keywords: ------------------------------------+--------------------------------------- MacRuby 0.5b2 seems to have a problem with the floats 8.1 and 9.1. Here is what 'macirb' does: irb(main):005:0> 9.1 => 9.09999999999999 irb(main):006:0> 8.1 => 8.09999999999999 I get the same results from 'macruby'. The strange thing is that if I do the following: irb(main):007:0> 9.1 + 1 => 10.1 irb(main):008:0> 10.0 - 0.9 => 9.09999999999999 very strange - AYAL -- Ticket URL: <http://www.macruby.org/trac/ticket/521> MacRuby <http://macruby.org/>
#521: Float problem with 8.1 and 9.1 ------------------------------------+--------------------------------------- Reporter: macruby@… | Owner: lsansonetti@… Type: defect | Status: new Priority: major | Milestone: MacRuby 0.5 Component: MacRuby | Keywords: ------------------------------------+--------------------------------------- Comment(by emoy@…): Actually, this should be fixed with the float changes in #159. This problem stems from the last two bits of the floating point number being used as flags, so there is a lose in precision. Worse is that in 32-bit mode, floating point is done with 32-bit floats, so the two bit lose in precision is even worse: {{{ % macruby -e 'puts 8.1' 8.09999999999999 % arch -i386 macruby -e 'puts 8.1' 8.09999847412109 }}} With the changes in #159: {{{ % miniruby -e 'puts 8.1' 8.1 % arch -i386 miniruby -e 'puts 8.1' 8.1 }}} -- Ticket URL: <http://www.macruby.org/trac/ticket/521#comment:1> MacRuby <http://macruby.org/>
participants (1)
-
MacRuby