[MacPorts] #43145: Perl PortGroup fails to reinplace non-ascii Makefiles

[MacPorts]

#43145: Perl PortGroup fails to reinplace non-ascii Makefiles
-----------------------------------+--------------------------------
  Reporter:  mojca@…               |      Owner:  macports-tickets@…
      Type:  defect                |     Status:  new
  Priority:  Normal                |  Milestone:
 Component:  ports                 |    Version:  2.2.1
Resolution:                        |   Keywords:
      Port:  p5-test-base p5-yaml  |
-----------------------------------+--------------------------------

Old description:

> When building p5-yaml on 10.8 and 10.9, sed fails with
> {{{
> DEBUG: Executing reinplace: /usr/bin/sed {/^CCFLAGS *=/s/$/ /} <
> /opt/local/var/macports/build/_opt_mports_dports_perl_p5-yaml/p5.16-yaml/work/YAML-0.90/Makefile
> >@ file14
> DEBUG: sed: RE error: illegal byte sequence
> }}}
> (see https://build.macports.org/builders/buildports-mavericks-
> x86_64/builds/2454) because the `Makefile` contains a non-ascii character
> (in Latin 1 encoding, invalid UTF).
>
> This comes from the perl PortGroup:
> {{{
> # CCFLAGS can be passed in to "configure" but it's not necessarily
> inherited.
> # LDFLAGS can't be passed in (or if it can, it's not easy to figure out
> how).
> post-configure {
>     fs-traverse file ${configure.dir} {
>         if {[file isfile ${file}] && [file tail ${file}] eq "Makefile"} {
>             ui_info "Fixing flags in [string map "${configure.dir}/ {}"
> ${file}]"
>             reinplace "/^CCFLAGS *=/s/$/ [get_canonical_archflags cc]/"
> ${file}
>             reinplace "/^OTHERLDFLAGS *=/s/$/ [get_canonical_archflags
> ld]/" ${file}
>         }
>     }
> }
> }}}
>
> Suggestions from IRC:
>
> neverpanic:
> > set LC_ALL to C
> > I think that's a common issue with ill-formatted UTF-8 files
> > /usr/bin/sed changed on 10.8 and above
> > You can either patch that to be valid UTF-8 or set configure.env
> LC_ALL=C in the Portfile
> > It is kind of weird to use the locale settings for that.
> > I mean you could set the correct locale for the file, but then any
> messages to your terminal would be broken…
>
> anddam:
> > I'd escape that dollar sign
>
> Dar1us:
> > it refuses to process files which aren't validly encoded given the
> current LC settings
> > .. and by default that is UTF-8

New description:

 When building p5-yaml on 10.8 and 10.9, sed fails with
 {{{
 DEBUG: Executing reinplace: /usr/bin/sed {/^CCFLAGS *=/s/$/ /} <
 /opt/local/var/macports/build/_opt_mports_dports_perl_p5-yaml/p5.16-yaml/work/YAML-0.90/Makefile
 >@ file14
 DEBUG: sed: RE error: illegal byte sequence
 }}}
 (see https://build.macports.org/builders/buildports-mavericks-
 x86_64/builds/2454) because the `Makefile` contains a non-ascii character
 (in Latin 1 encoding, invalid UTF).

 This comes from the perl PortGroup:
 {{{
 # CCFLAGS can be passed in to "configure" but it's not necessarily
 inherited.
 # LDFLAGS can't be passed in (or if it can, it's not easy to figure out
 how).
 post-configure {
     fs-traverse file ${configure.dir} {
         if {[file isfile ${file}] && [file tail ${file}] eq "Makefile"} {
             ui_info "Fixing flags in [string map "${configure.dir}/ {}"
 ${file}]"
             reinplace "/^CCFLAGS *=/s/$/ [get_canonical_archflags cc]/"
 ${file}
             reinplace "/^OTHERLDFLAGS *=/s/$/ [get_canonical_archflags
 ld]/" ${file}
         }
     }
 }
 }}}

 Suggestions from IRC:

 neverpanic:
 > set LC_ALL to C
 > I think that's a common issue with ill-formatted UTF-8 files
 > /usr/bin/sed changed on 10.8 and above
 > You can either patch that to be valid UTF-8 or set configure.env
 LC_ALL=C in the Portfile
 > It is kind of weird to use the locale settings for that.
 > I mean you could set the correct locale for the file, but then any
 messages to your terminal would be broken…

 Dar1us:
 > it refuses to process files which aren't validly encoded given the
 current LC settings
 > .. and by default that is UTF-8

--

Comment (by and.damore@…):

 The portgroup cannot possibly know what encoding will the produced
 Makefile use.

 Port p5-yaml is using ExtUtils::MakeMaker that in turn relies on Pod::Man.
 Pod::Man can produce utf8 Makefiles, [http://perldoc.perl.org/Pod/Man.html
 search utf8 in in this page] but seems that MakeMaker doesn't have any
 option to specify that, see
 [http://article.gmane.org/gmane.comp.lang.perl.modules.extutils-
 makemaker.devel/2074/ this request].


 Since the portgroup is patching Makefiles we could sanitize the file by
 adding a new variable specifying a charset to convert from. In this case
 the portfile writer would specify something like
 {{{
 perl5.makefile_encoding    iso8859-1
 perl5.setup         YAML 0.90
 }}}

 and the post-configure phase would take care of convert the file to UTF-8
 (or any other target encoding depending on the darwin release) before
 reinplaces.

-- 
Ticket URL: <https://trac.macports.org/ticket/43145#comment:4>
MacPorts <http://www.macports.org/>
Ports system for OS X