[MacRuby] #502: Marshalling large integers returns the wrong value.

[MacRuby]

#502: Marshalling large integers returns the wrong value.
------------------------------------+---------------------------------------
 Reporter:  jens.nockert@…          |       Owner:  lsansonetti@…        
     Type:  defect                  |      Status:  new                  
 Priority:  minor                   |   Milestone:                       
Component:  MacRuby                 |    Keywords:                       
------------------------------------+---------------------------------------

Comment(by jhemmelg@…):

 There were two issues with the loading of bignums in marshal.  The first
 was that the calculation of the size of the bignum in the call to
 rb_big_resize() was wrong.  The original code was calculating a zero size
 for the bignum with this test case.  Therefore the loop that translates
 the saved bits wasn't even entered and a zero value resulted.

 The second issue was the commented out call to MEMZERO.  With the size of
 the bignum now correct, this call should be reinstated to clear the bits
 that were not initialized from the marshalled stream.

 The third issue was in the loop that translated the bits.  Each byte was
 cast to long before being shifted into place in the bignum digit.  This
 caused a 32-bit value to be shifted by up to 56 bits with predictably
 whacky results.  Changing the cast from (long) to (BDIGIT) fixed this.

 Cosmetic: There were a lot of magic number "2"s scattered around this
 code.  As far as I can see these are actually SIZEOF_SHORT so I changed
 them.

 Side note: I found a difference in the behavior of ruby 1.9.1 and macruby
 when trying to Marshal.dump(1000000000).  1.9.1 dumps this as a fixnum and
 macruby dumps it as a bignum.  I think the difference is that macruby is
 now using 2 bits in a VALUE instead of 1.  I believe this is due to
 including floats as intrinsics?  I don't think this is a big problem
 because they interoperate correctly.  However, if we want to change it,
 line 658 in marshal.c:

         if (RSHIFT((long)obj, 31) == 0 || RSHIFT((long)obj, 31) == -1) {

  is the place.  Changing the 31's to 32's makes it work the same as 1.9.1.
 I didn't change it because I wasn't sure how to distinguish whether 1 or 2
 bits are being used to distinguish a pointer from an intrinsic.

-- 
Ticket URL: <http://www.macruby.org/trac/ticket/502#comment:2>
MacRuby <http://macruby.org/>