macruby produces strings with encodings that differ from MRI
Code ======== ABC='ABC' puts "ABC[0] encoding is #{ABC[0].encoding}" puts "?\\xff encoding is #{?\xff.encoding}" Output ======== MRI output ABC[0] encoding is US-ASCII ?\xff encoding is ASCII-8BIT macruby output ABC[0] encoding is UTF-8 ?\xff encoding is UTF-8 The encodings reported above did not seem to be effected by the encoding of the source file. I tried both ASCII and UTF-8. When the same code is executed in (mac)irb the results are the same for macirb as they are for macruby. irb for MRI however produces UTF-8 strings in both cases! This seemed very odd but I'm fairly sure it's because I have an environment variable: LANG=GB.UTF-8 When I changed to LANG=GB.US_ASCII irb for MRI rendered 'abc'[0] with US_ASCII encoding. macirb still used UTF-8. (I discovered this when trying to get ruby-mysql to work with macruby. It doesn't work as-is but seems to work with a few mods that use force_encoding to make MRI and macruby produce the same outputs). I abandoned my earlier attempts to use postgres with macruby via the pg gem. It failed regularly but in unpredictable ways associated, as far as I could tell, with memory management problems.
Hi, Maybe related to http://www.macruby.org/trac/ticket/742. MacRuby ignore magic-comment, and uses default encoding UTF8. Thanks. 2011/9/18 Steve Clarke <steve@sclarkes.me.uk>:
Code ========
ABC='ABC' puts "ABC[0] encoding is #{ABC[0].encoding}" puts "?\\xff encoding is #{?\xff.encoding}"
Output ========
MRI output
ABC[0] encoding is US-ASCII ?\xff encoding is ASCII-8BIT
macruby output
ABC[0] encoding is UTF-8 ?\xff encoding is UTF-8
The encodings reported above did not seem to be effected by the encoding of the source file. I tried both ASCII and UTF-8.
When the same code is executed in (mac)irb the results are the same for macirb as they are for macruby. irb for MRI however produces UTF-8 strings in both cases! This seemed very odd but I'm fairly sure it's because I have an environment variable: LANG=GB.UTF-8 When I changed to LANG=GB.US_ASCII irb for MRI rendered 'abc'[0] with US_ASCII encoding. macirb still used UTF-8.
(I discovered this when trying to get ruby-mysql to work with macruby. It doesn't work as-is but seems to work with a few mods that use force_encoding to make MRI and macruby produce the same outputs). I abandoned my earlier attempts to use postgres with macruby via the pg gem. It failed regularly but in unpredictable ways associated, as far as I could tell, with memory management problems.
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
Yes, looks like the same issue as ticket 742. I did look at tickets but failed to spot that. The comment on the ticket re only UTF-8 being required may be true - it certainly is for me. Sadly the ruby-mysql gem works in such a way that the difference between MRI and macruby breaks it. Steve On 18 Sep 2011, at 06:07, Watson wrote:
Hi,
Maybe related to http://www.macruby.org/trac/ticket/742. MacRuby ignore magic-comment, and uses default encoding UTF8.
Thanks.
2011/9/18 Steve Clarke <steve@sclarkes.me.uk>:
Code ========
ABC='ABC' puts "ABC[0] encoding is #{ABC[0].encoding}" puts "?\\xff encoding is #{?\xff.encoding}"
Output ========
MRI output
ABC[0] encoding is US-ASCII ?\xff encoding is ASCII-8BIT
macruby output
ABC[0] encoding is UTF-8 ?\xff encoding is UTF-8
The encodings reported above did not seem to be effected by the encoding of the source file. I tried both ASCII and UTF-8.
When the same code is executed in (mac)irb the results are the same for macirb as they are for macruby. irb for MRI however produces UTF-8 strings in both cases! This seemed very odd but I'm fairly sure it's because I have an environment variable: LANG=GB.UTF-8 When I changed to LANG=GB.US_ASCII irb for MRI rendered 'abc'[0] with US_ASCII encoding. macirb still used UTF-8.
(I discovered this when trying to get ruby-mysql to work with macruby. It doesn't work as-is but seems to work with a few mods that use force_encoding to make MRI and macruby produce the same outputs). I abandoned my earlier attempts to use postgres with macruby via the pg gem. It failed regularly but in unpredictable ways associated, as far as I could tell, with memory management problems.
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
Hi Steve, It would be nice to know what exactly in ruby-mysql causes the problem. If you can reduce the problem to a simple code snippet or point us to the ruby-mysql source code it would be great. Thanks Laurent On Sun, Sep 18, 2011 at 8:38 AM, Steve Clarke <steve@sclarkes.me.uk> wrote:
Yes, looks like the same issue as ticket 742. I did look at tickets but failed to spot that.
The comment on the ticket re only UTF-8 being required may be true - it certainly is for me. Sadly the ruby-mysql gem works in such a way that the difference between MRI and macruby breaks it.
Steve
On 18 Sep 2011, at 06:07, Watson wrote:
Hi,
Maybe related to http://www.macruby.org/trac/ticket/742. MacRuby ignore magic-comment, and uses default encoding UTF8.
Thanks.
2011/9/18 Steve Clarke <steve@sclarkes.me.uk>:
Code ========
ABC='ABC' puts "ABC[0] encoding is #{ABC[0].encoding}" puts "?\\xff encoding is #{?\xff.encoding}"
Output ========
MRI output
ABC[0] encoding is US-ASCII ?\xff encoding is ASCII-8BIT
macruby output
ABC[0] encoding is UTF-8 ?\xff encoding is UTF-8
The encodings reported above did not seem to be effected by the encoding of the source file. I tried both ASCII and UTF-8.
When the same code is executed in (mac)irb the results are the same for macirb as they are for macruby. irb for MRI however produces UTF-8 strings in both cases! This seemed very odd but I'm fairly sure it's because I have an environment variable: LANG=GB.UTF-8 When I changed to LANG=GB.US_ASCII irb for MRI rendered 'abc'[0] with US_ASCII encoding. macirb still used UTF-8.
(I discovered this when trying to get ruby-mysql to work with macruby. It doesn't work as-is but seems to work with a few mods that use force_encoding to make MRI and macruby produce the same outputs). I abandoned my earlier attempts to use postgres with macruby via the pg gem. It failed regularly but in unpredictable ways associated, as far as I could tell, with memory management problems.
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
Hi Laurent, The problem is in the self.eof_packet? method of protocol.rb. The statement that causes the problem is: data[0] == ?\xfe && data.length == 5 When run under macruby this is what happens: Encoding of data[0] is ASCII-8BIT. Encoding of ?\xfe is UTF-8 The compare fails in macruby but in MRI it succeeds (because in MRI the encoding of ?\xfe is ascii - actually it's US_ASCII, but that seems to compare equal) If I use force_encoding UTF-8 then things work OK - as far as I've tested. Steve Begin forwarded message:
From: Laurent Sansonetti <laurent.sansonetti@gmail.com> Subject: Re: [MacRuby-devel] macruby produces strings with encodings that differ from MRI Date: 18 September 2011 10:00:13 GMT+01:00 To: "MacRuby development discussions." <macruby-devel@lists.macosforge.org> Reply-To: "MacRuby development discussions." <macruby-devel@lists.macosforge.org>
Hi Steve,
It would be nice to know what exactly in ruby-mysql causes the problem. If you can reduce the problem to a simple code snippet or point us to the ruby-mysql source code it would be great.
Thanks Laurent
On Sun, Sep 18, 2011 at 8:38 AM, Steve Clarke <steve@sclarkes.me.uk> wrote:
Yes, looks like the same issue as ticket 742. I did look at tickets but failed to spot that.
The comment on the ticket re only UTF-8 being required may be true - it certainly is for me. Sadly the ruby-mysql gem works in such a way that the difference between MRI and macruby breaks it.
Steve
On 18 Sep 2011, at 06:07, Watson wrote:
Hi,
Maybe related to http://www.macruby.org/trac/ticket/742. MacRuby ignore magic-comment, and uses default encoding UTF8.
Thanks.
2011/9/18 Steve Clarke <steve@sclarkes.me.uk>:
Code ========
ABC='ABC' puts "ABC[0] encoding is #{ABC[0].encoding}" puts "?\\xff encoding is #{?\xff.encoding}"
Output ========
MRI output
ABC[0] encoding is US-ASCII ?\xff encoding is ASCII-8BIT
macruby output
ABC[0] encoding is UTF-8 ?\xff encoding is UTF-8
The encodings reported above did not seem to be effected by the encoding of the source file. I tried both ASCII and UTF-8.
When the same code is executed in (mac)irb the results are the same for macirb as they are for macruby. irb for MRI however produces UTF-8 strings in both cases! This seemed very odd but I'm fairly sure it's because I have an environment variable: LANG=GB.UTF-8 When I changed to LANG=GB.US_ASCII irb for MRI rendered 'abc'[0] with US_ASCII encoding. macirb still used UTF-8.
(I discovered this when trying to get ruby-mysql to work with macruby. It doesn't work as-is but seems to work with a few mods that use force_encoding to make MRI and macruby produce the same outputs). I abandoned my earlier attempts to use postgres with macruby via the pg gem. It failed regularly but in unpredictable ways associated, as far as I could tell, with memory management problems.
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
participants (3)
-
Laurent Sansonetti
-
Steve Clarke
-
Watson