#52420: neomutt: Umlauts, accents are wrongly displayed ------------------------------------+------------------------ Reporter: michele.marcionelli@… | Owner: leonardo@… Type: defect | Status: new Priority: Normal | Milestone: Component: ports | Version: 2.3.4 Resolution: | Keywords: Port: neomutt | ------------------------------------+------------------------ Comment (by leonardo.schenkel@…): That's very odd. I use all sorts of accented characters as well and I never experienced any issue. I'll need some more details: - Does it happen with all e-mails or just some e-mails? - Does it happen in the UI as well? Try typing a mutt command with accents. - Which terminal do you use? The built-in Terminal? iTerm? Something else? - Could you please run the command `locale` and post the output? - Could you please run the command `mutt -v` and post the output? -- Ticket URL: <https://trac.macports.org/ticket/52420#comment:2> MacPorts <https://www.macports.org/> Ports system for the Mac operating system

#52420: neomutt: Umlauts, accents are wrongly displayed ------------------------------------+------------------------ Reporter: michele.marcionelli@… | Owner: leonardo@… Type: defect | Status: new Priority: Normal | Milestone: Component: ports | Version: 2.3.4 Resolution: | Keywords: Port: neomutt | ------------------------------------+------------------------ Comment (by michele.marcionelli@…): Just for your information: the old mutt-devel (Mutt 1.6.0 (2016-04-01)) was working well with the default settings, i.e. with the "set locale variables automatically" setting for Terminal (or iTerm) which correspond to following {{{locale}}}: {{{ LANG= LC_COLLATE="C" LC_CTYPE="UTF-8" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL= }}} Now with the new neomutt (NeoMutt 20160916 (1.7.0) - just installed today) I get following results: 1) with default {{{locale}}} (see above): - in the mail listing I read the subject "Z??rich" instead of "Zürich" - while reading the email I see in the header "Subject: Z??rich" and in the body "Z\303\274rich" 2) with {{{unset LC_CTYPE}}} I get this output of {{{locale}}} {{{ LANG= LC_COLLATE="C" LC_CTYPE="C" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL= }}} and I see: - in the mail listing I read the subject "Zrich" instead of "Zürich" - while reading the email both header and body is OK 3) with {{{export LC_ALL=en_US.UTF-8}}} I get this output of {{{locale}}} {{{ LANG= LC_COLLATE="en_US.UTF-8" LC_CTYPE="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_ALL="en_US.UTF-8" }}} and everything seems to work fine; also {{{export LC_ALL=en_US}}} oder {{{export LC_ALL=de_CH}}} work find. What is the difference with (1)? Why (1) don't work correctly with neomutt? I also noticed that not setting the LC_ALL but all other LC_* event to different values mutt will display umlauts and accents correctyle; but if I just unset one of the LC_* mutt will fail: {{{ LANG= LC_COLLATE="de_AT" LC_CTYPE="de_DE" LC_MESSAGES="en_US" LC_MONETARY="de_CH" LC_NUMERIC="fr_FR" LC_TIME="en_GB" LC_ALL= }}} ==> is ok for mutt {{{ LANG= LC_COLLATE="de_AT" LC_CTYPE="de_DE" LC_MESSAGES="en_US" LC_MONETARY="C" LC_NUMERIC="fr_FR" LC_TIME="en_GB" LC_ALL= }}} ==> is no more ok for mutt HTH ;-) Bests, Michele -- Ticket URL: <https://trac.macports.org/ticket/52420#comment:4> MacPorts <https://www.macports.org/> Ports system for the Mac operating system

#52420: neomutt: Umlauts, accents are wrongly displayed ------------------------------------+------------------------ Reporter: michele.marcionelli@… | Owner: leonardo@… Type: defect | Status: new Priority: Normal | Milestone: Component: ports | Version: 2.3.4 Resolution: | Keywords: Port: neomutt | ------------------------------------+------------------------ Comment (by leonardo.schenkel@…): This is what I suspected, but I just wanted to confirm before blaming your set-up. Unfortunately the LC_CTYPE="UTF-8" that Terminal.app sets by default is not a valid recognized locale by a lot of software (the locale is supposed to be something like "de_CH.UTF-8"). The macOS userland can cope with it (and many BSD as well), but if you ever ssh to a Linux box for example you'll instantly get this warning: `setlocale: LC_CTYPE: cannot change locale (UTF-8)` because the value is rejected by the GNU userland. Mutt 1.7 is now being affected by this. Before 1.7 it didn't pay attention to your environment's locale (only to the locale you set in the configuration file), but that setting was removed and it is now determing the locale and the charset based on the environment variables, like virtually all other software does. Unfortunately it got bitten by the broken macOS defaults. This is so annoying in general that on my shell initialization scripts I have the following: {{{ if [ "${LC_CTYPE}" == "UTF-8" ]; then # OS X likes to set LC_CTYPE="UTF-8". In general this is supported by BSD # systems, but not Linux (which will become a problem when ssh'ing to # foreign systems). Check if it is possible to unset LC_CTYPE without # changing the charset, and unset it if that is true. if [ "$(LC_CTYPE= locale charmap)" == "UTF-8" ]; then unset -v LC_CTYPE fi fi }}} If you change your shell initialization scripts to contain `export LANG=de_CH.UTF-8` (or whatever is your locale of choice) *before* the snippet above, then LC_CTYPE will be automatically unset and you should not have any more of these issues. Note that as "cal" have correctly said, the change of behavior that you have experienced was introduced vanilla Mutt, not NeoMutt neither this port. All I can do in this situation is to suggest you to change your settings. -- Ticket URL: <https://trac.macports.org/ticket/52420#comment:6> MacPorts <https://www.macports.org/> Ports system for the Mac operating system

#52420: neomutt: Umlauts, accents are wrongly displayed ------------------------------------+------------------------ Reporter: michele.marcionelli@… | Owner: leonardo@… Type: defect | Status: new Priority: Normal | Milestone: Component: ports | Version: 2.3.4 Resolution: | Keywords: Port: neomutt | ------------------------------------+------------------------ Comment (by leonardo.schenkel@…): I don't touch LANG as well. In all Macs I have ever owned, in all OS X versions I can remember, Terminal.app sets the locale like this: {{{ LANG=<locale from regional settings>.UTF-8 LC_CTYPE=UTF-8 }}} I never saw plain "C" for LANG, though, which is curious. Why it uses "UTF-8" for LC_CTYPE and not a well-formed locale name is a mystery to me, especially since it can do that just well for LANG — and since LANG is already defined as a UTF-8 locale, there's no need to set LC_CTYPE at all. I'm not an expert, but my understanding is that the relevant specs say that all locales must be in the form of `lang_country_variant.charset` with notable exceptions ("C", "POSIX"). Even though "UTF-8" exists as a locale in the Mac, it's not supposed to be a valid locale name. I even tried to create such a locale via `locale-gen` on Linux systems and it refuses to do so. -- Ticket URL: <https://trac.macports.org/ticket/52420#comment:8> MacPorts <https://www.macports.org/> Ports system for the Mac operating system

#52420: neomutt: Umlauts, accents are wrongly displayed ------------------------------------+------------------------ Reporter: michele.marcionelli@… | Owner: leonardo@… Type: defect | Status: new Priority: Normal | Milestone: Component: ports | Version: 2.3.4 Resolution: | Keywords: Port: neomutt | ------------------------------------+------------------------ Comment (by larryv@…): Replying to [comment:8 leonardo.schenkel@…]:

I don't touch LANG as well. In all Macs I have ever owned, in all OS X versions I can remember, Terminal.app sets the locale like this: {{{ LANG=<locale from regional settings>.UTF-8 LC_CTYPE=UTF-8 }}}

I have seen the opposite: I have never had `LC_CTYPE` set automatically. My environment has always contained `LANG` alone. If I had to, I would guess that the system doesn’t try to guess a country if you tweak the system settings in a way that doesn’t explicitly denote one. But I have no idea. UTF-8 is obviously just a character encoding without any locale information at all. -- Ticket URL: <https://trac.macports.org/ticket/52420#comment:10> MacPorts <https://www.macports.org/> Ports system for the Mac operating system