[MacRuby-devel] difference in float behaviour

[Emil Tin]

ok. 
it confused me a bit that d prints as "1.0", but d==1.0 evaluates to false. but sprintf shows that it's just float#to_s that is truncating the output:

z = 0.034228469424
r = Math.sqrt z*z
d = z/r  
puts d        			# => 1.0
puts d==1.0                	# => false
puts sprintf "%.20f",d	# =>  "1.00000000000000088818"


what is the precision of macruby floats?



On 07/06/2010, at 04.35, Laurent Sansonetti wrote:

> Hi Emil,
> 
> This is sort-of expected. MacRuby floats have less precision than MRI because we pack them as immediate values, for performance reasons. As a more generic issue, floating point arithmetic is almost never accurate.
> 
> Laurent
> 
> On Jun 5, 2010, at 9:59 AM, Emil Tin wrote:
> 
>> 
>> hey, 
>> thanks for all the great work with macruby! 
>> 
>> i'm experiencing a difference in floating point behaviour between mri and the latest macruby 0.7 nightly:
>> 
>> ~$ ruby -v
>> ruby 1.8.7 (2009-06-08 patchlevel 173) [universal-darwin10.0]
>> 
>> ~$ ruby -e "z = 0.034228469425; puts Math.sqrt(z*z) < z"
>> false
>> 
>> ~$ ruby -e "z = 0.034228469424; puts Math.sqrt(z*z) < z"
>> false
>> 
>> 
>> ~$ macruby -v
>> MacRuby 0.7 (ruby 1.9.2) [universal-darwin10.0, x86_64]
>> 
>> ~$ macruby -e "z = 0.034228469425; puts Math.sqrt(z*z) < z"
>> false
>> 
>> ~$ macruby -e "z = 0.034228469424; puts Math.sqrt(z*z) < z"
>> true
>> 
>> 
>> notice how that last script outputs true instead of false. also i'm seeing this difference with floats:
>> 
>> ~$ ruby -e "puts 8.1"
>> 8.1
>> 
>> ~$ macruby -e "puts 8.1"
>> 8.09999999999999
>> 
>> 
>> 
>> 
>> this difference is causing trouble because asin() might fail even when the input seems to be 1.0:
>> 
>> z = 0.034228469424
>> r = Math.sqrt z*z
>> puts d = z/r # => 1.0
>> puts d>1.0 # false with mri, but true with macruby
>> puts Math.asin d # => 1.5707963267949 with mri, but error with macruby (since d>1.0)
>> 
>> 
>> 
>> -emil
>> 
>> _______________________________________________
>> MacRuby-devel mailing list
>> MacRuby-devel at lists.macosforge.org
>> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
> 
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1920 bytes
Desc: not available
URL: <http://lists.macosforge.org/pipermail/macruby-devel/attachments/20100608/10778e09/attachment.bin>