[MacRuby] #1211: (120 * 0.2).floor should be 24, but MacRuby rounds it down to 23

MacRuby ruby-noreply at macosforge.org
Sun Apr 10 21:47:47 PDT 2011


#1211: (120 * 0.2).floor should be 24, but MacRuby rounds it down to 23
------------------------------------+---------------------------------------
 Reporter:  mrada@…                 |       Owner:  lsansonetti@…        
     Type:  defect                  |      Status:  new                  
 Priority:  blocker                 |   Milestone:                       
Component:  MacRuby                 |    Keywords:                       
------------------------------------+---------------------------------------

Comment(by mrada@…):

 I've dug into the issue some more and found that the MacRuby code for
 floats is fairly different from CRuby. In this specific case, MacRuby has
 a bitwise and operation that removes some precision from a double before
 coercing the data to a fixnum.

 When I skip the bitwise and step then I fix my rounding error, but a ton
 of spec tests fail (I can run them all because one of the hangs).

 The function doing the coercion is coerce_ptr_to_double in
 include/ruby/ruby.h:184

-- 
Ticket URL: <http://www.macruby.org/trac/ticket/1211#comment:3>
MacRuby <http://macruby.org/>



More information about the macruby-tickets mailing list