Re: i18n problem: citrus bug?

["Takehiko NOZAKI" <takehiko.nozaki%gmail.com@localhost>]

hi,

> I can't reproduce this problem on a 4.0_STABLE i386 system using the
> same version of Mutt, i.e. reading the mail works fine there.
>
> Any hints?

it seems that the problem occurs when this multi-part messages's
MIME charaset(ks_c_5601-1983) doesn't supported by iconv_open(3)
and return (iconv_t)-1 case.

at a grance of mutt's code:

1. iconv_open(3)'s argument is implementation defined.
you don't expect that all IANA Character Registry's charset name
(such as ks_c_5601-1983) are accepted. so that mutt should have
IANA <-> iconv translation table itself.
please maintain charset.c in mutt's distribution.

--- charset.c.orig      2008-11-19 19:45:11.000000000 +0900
+++ charset.c   2008-11-19 19:44:41.000000000 +0900
@@ -57,6 +57,7 @@
 }
 PreferredMIMENames[] =
 {
+  { "ks_c_5601-1987",  "euc-kr"        },
   { "ansi_x3.4-1968",  "us-ascii"      },
   { "iso-ir-6",                "us-ascii"      },
   { "iso_646.irv:1991",        "us-ascii"      },


or for temporary workround, add iconv alias following step:

    # cd /usr/share/i18n/esdb
    # echo "ks_c_5601-1983 EUC-KR" >>esdb.dir
    # mkesdb -m -o esdb.dir.db esdb.dir

2. hangup is for the sake of infinity loop in mutt_filter_unprintable():mbyte.c
let's blame of misusage of mbrtowc(3).

if iconv_open(3) failed, no codeset conversion may happen.
so that whole multibyte character(decoded from base64) is junk &&
illegal byte sequence.
but mutt guess them as current locale's codeset in mutt_filter_unprintable().
thus mbrtowc(3) may *always* fail.

in addition, SUSv3 said if mbrtowc(3) fail, mbstate_t's state is "undefined".
http://www.opengroup.org/onlinepubs/009695399/functions/mbrtowc.html
snip
...
(size_t)-1
    If an encoding error occurs, in which case the next n or fewer bytes
do not contribute to a complete and valid character (no value is stored).
In this case, [EILSEQ] shall be stored in errno and the conversion
state is undefined.
...
/snip

so you have to re-initialize mbstate_t,  but mutt never ;-<

--- mbyte.c.orig        2008-11-19 19:27:28.000000000 +0900
+++ mbyte.c     2008-11-19 19:40:42.000000000 +0900
@@ -496,6 +496,7 @@
     {
       k = 1;
       wc = replacement_char();
+      memset (&mbstate1, 0, sizeof (mbstate1));
     }
     if (!IsWPrint (wc))
       wc = '?';

this patch solved the problem in my environment, 5.0_STABLE + mutt
1.5.18 in pkgsrc

very truly yours.
-- 
Takehiko NOZAKI<takehiko.nozaki%gmail.com@localhost>