Re: bin/60496: Locale-dependent UTF-8 issue over SSH session

["Robert Elz via gnats" <gnats-admin%NetBSD.org@localhost>]

The following reply was made to PR bin/60496; it has been noted by GNATS.

From: Robert Elz <kre%munnari.OZ.AU@localhost>
To: Thomas Dreibholz <dreibh%simula.no@localhost>
Cc: gnats-bugs%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost, riastradh%NetBSD.org@localhost
Subject: Re: bin/60496: Locale-dependent UTF-8 issue over SSH session
Date: Tue, 22 Sep 2026 00:29:10 +0700

 There is progress on this issue, it hasn't been forgotten!
 
 Given the test program:
 
 #include <stdio.h>
 #include <string.h>
 #include <unistd.h>
 #include <locale.h>
 
 int
 main(int argc __unused, char **argv __unused)
 {
         char buf[129];
 
         setlocale(LC_ALL, "");
         memset(buf, 0, sizeof buf);
 
         sprintf(buf, "%'d\n", 123456789);
         write(1, buf, strlen(buf));
 
         sprintf(buf, "%'40.4f\n", 123456789012.76543);
         write(1, buf, strlen(buf));
 
         return 0;
 }
 
 [Just using the sprintf/write combination as it was easier for debugging
  - the changes are all buried inside the __vfprintf_unlocked_l or
  __vfwprintf_unlocked_l function (as appropriate) and the subordinate
  routines it calls, which are used for all *printf() variations, so this
  should be the same for any kind of printf().]
 
 When built to make (the imaginatively named) binary) "t" I now get:
 
 LC_ALL=nb_NO.UTF-8 ./t
 123Â 456Â 789
                  123Â 456Â 789Â 012,7654
 
 LC_ALL=nb_NO.UTF-8 ./t | hexdump
 0000000 3231 c233 34a0 3635 a0c2 3837 0a39 2020
 0000010 2020 2020 2020 2020 2020 2020 2020 3120
 0000020 3332 a0c2 3534 c236 37a0 3938 a0c2 3130
 0000030 2c32 3637 3435 000a
 
 That is, the grouping characters are properly C2A0
 aka \u00A0 (the unbreakable space) which I believe
 is what is intended (it is what is in our locale data
 anyway, if incorrect that can be a different change).
 
 
 In my fake test locale, I get:
 
 LC_ALL=x1_X1.UTF-8 ./t | hexdump
 0000000 3231 d933 34ac 3635 d937 38ac 0a39 2020
 0000010 2020 2020 2020 2020 2020 3120 d932 33ac
 0000020 acd9 3534 d936 37ac 3938 d930 31ac e232
 0000030 8e97 3637 3435 000a
 
 There the grouping characters are D9AC (\u066c)
 and the decimal point is E2978E (\u25ce) - not
 because they're particularly likely values to
 ever exist, but just because they're not impossible
 either:
 
 LC_ALL=x1_X1.UTF-8 ./t
 123Ù¬4567Ù¬89
              12Ù¬3Ù¬456Ù¬7890Ù¬12â??7654
 
 (and yes, this fake locale also uses a rather odd grouping
 specification, not just every N digits).
 
 I am still working on improving the ATF tests before this
 gets committed, but I believe the libc parts are all OK
 (wrt this issue anyway) now.   The decimal point change is
 identical to (obtained from) the FreeBSD changes to fix
 the same problem, the grouping changes are done differently,
 FreeBSD's implementation has diverged from ours in other
 ways, and it was (for just this) easier to simply reimplement
 the changes in ways that fit our current sources.
 
 kre