Re: Weird 8859-1 charset

[RVP <rvp%SDF.ORG@localhost>]

On Sun, 9 Aug 2026, Robert Elz wrote:

> While looking, I looked in .../src/share/locale/ctype/charset and for
> the purposes of this e-mail anyway, the Latin-1 file in particular.
>
> In that we see (inter alia):
>
> 	ALPHA		0x40 - 0x56 0x58 - 0x76 0x78 - 0x7f
> 	LOWER		0x60 - 0x76 0x78 - 0x7f
> 	PUNCT		0x21 - 0x3f 0x57 0x77
> 	UPPER		0x40 - 0x56 0x58 - 0x5f
>
> What's with the Ws ???   (0x57 and 0x77)

I don't know why that is, but, this, OTOH, is clearly some typo:

```
$ cat wctrans-test.c
/**

for L in $(locale -a)
do      test -f /usr/share/locale/"$L"/LC_CTYPE && skip=false || skip=true
        $skip && [ "$L" != C ] && [ "$L" != POSIX ] && continue
        LANG="$L" ./wctrans-test || echo 1>&2 "$L: failed."
done

 */

#include <err.h>
#include <locale.h>
#include <wctype.h>

static wint_t mytrans(wint_t wc, char* map) {
        wctrans_t t;

        if ((t = wctrans(map)) == (wctrans_t)0)
                return WEOF;
        return towctrans(wc, t);
}

int main(void) {
        wint_t wc = L'A';

        setlocale(LC_ALL, "");
        if (towupper(towlower(wc)) != wc)
                errx(1, "std. failed");
        if (mytrans(mytrans(wc, "tolower"), "toupper") != wc)
                errx(1, "mytrans failed");
        return 0;
}

$ for L in $(locale -a)
do      test -f /usr/share/locale/"$L"/LC_CTYPE && skip=false || skip=true
        $skip && [ "$L" != C ] && [ "$L" != POSIX ] && continue
        LANG="$L" ./wctrans-test || echo 1>&2 "$L: failed."
done
wctrans-test: std. failed
kk_KZ.PT154: failed.
wctrans-test: std. failed
zh_CN.GB18030: failed.

$
```

and, sure enough:

src/share/locale/ctype/kk_KZ.PT154.src:
     26 MAPLOWER        <'A' - 'Z' : 'a'>
     27 MAPUPPER        <'A' - 'Z' : 'A'>

src/share/locale/ctype/zh_CN.GB18030.src:
     40 MAPLOWER        < 0x61 - 0x7a : 0x41 >
     41 MAPUPPER        < 0x41 - 0x5a : 0x61 >


(compare with src/share/locale/ctype/en_US.ASCII.src)

> And while I am here, two related questions, more just to satisfy
> my curiosity.
>
> First, the Latin-1 file also contains:
>
> 	CHARSET		",A"
>
> which a comment suggests is used after an ESC.   Used for what?
> Anything that matters?   Who or what assigned the ",A" ?
> (Other files in that dir have similar definitions, with different
> values.)

I *think* that's used when transcoding from ISO-8859-1 to multibyte ISO-2022
(CN, JP, KR):

```
$  printf 'hello \xc0 world' | iconv -f iso_8859-1 -t iso-2022-jp-1 | hexdump -C
00000000  68 65 6c 6c 6f 20 1b 24  28 44 2a 22 1b 28 42 20  |hello .$(D*".(B |
00000010  77 6f 72 6c 64                                    |world|
00000015

$ printf 'hello '$'\u00c0\u30f0 ''world' | iconv -f utf-8 -t iso-2022-jp-1 | hexdump -C
00000000  68 65 6c 6c 6f 20 1b 24  28 44 2a 22 1b 24 42 25  |hello .$(D*".$B%|
00000010  70 1b 28 42 20 77 6f 72  6c 64                    |p.(B world|
0000001a
```

See the `Character set designations' and associated table here:

https://en.wikipedia.org/wiki/ISO/IEC_2022

NetBSD's implementation uses seems to follow footnote (h).

HTH,

-RVP