#502: Marshalling large integers returns the wrong value. ------------------------------------+--------------------------------------- Reporter: jens.nockert@… | Owner: lsansonetti@… Type: defect | Status: new Priority: minor | Milestone: Component: MacRuby | Keywords: ------------------------------------+--------------------------------------- Comment(by jhemmelg@…): There were two issues with the loading of bignums in marshal. The first was that the calculation of the size of the bignum in the call to rb_big_resize() was wrong. The original code was calculating a zero size for the bignum with this test case. Therefore the loop that translates the saved bits wasn't even entered and a zero value resulted. The second issue was the commented out call to MEMZERO. With the size of the bignum now correct, this call should be reinstated to clear the bits that were not initialized from the marshalled stream. The third issue was in the loop that translated the bits. Each byte was cast to long before being shifted into place in the bignum digit. This caused a 32-bit value to be shifted by up to 56 bits with predictably whacky results. Changing the cast from (long) to (BDIGIT) fixed this. Cosmetic: There were a lot of magic number "2"s scattered around this code. As far as I can see these are actually SIZEOF_SHORT so I changed them. Side note: I found a difference in the behavior of ruby 1.9.1 and macruby when trying to Marshal.dump(1000000000). 1.9.1 dumps this as a fixnum and macruby dumps it as a bignum. I think the difference is that macruby is now using 2 bits in a VALUE instead of 1. I believe this is due to including floats as intrinsics? I don't think this is a big problem because they interoperate correctly. However, if we want to change it, line 658 in marshal.c: if (RSHIFT((long)obj, 31) == 0 || RSHIFT((long)obj, 31) == -1) { is the place. Changing the 31's to 32's makes it work the same as 1.9.1. I didn't change it because I wasn't sure how to distinguish whether 1 or 2 bits are being used to distinguish a pointer from an intrinsic. -- Ticket URL: <http://www.macruby.org/trac/ticket/502#comment:2> MacRuby <http://macruby.org/>