CVS commit: src/sbin/efi

["Robert Elz" <kre%netbsd.org@localhost>]

Module Name:    src
Committed By:   kre
Date:           Wed Jul  8 02:13:38 UTC 2026

Modified Files:
        src/sbin/efi: utils.c

Log Message:
sbin/efi - avoid generating invalid UTF-8

Strings in UEFI are all made up of 16 bit chars, which are defined in the
UEFI standard to be UCS-2 (despite that format having been deprecated for
decades - replaced by UTF-16).

UCS-2 allowed for just 65536 code points (hence 16 bit storage).  UTF-16
defines a block of 8192 of the available code-points to be surrogates, 4096
of them high surrogates, the other 4096 low surrogates, always to be used
as a pair, high then low.

Each surrogate of a pair provides 10 data bits, combining them together gives
a 20 bit value, which is used by UTF-16 to represent the code point values
from 0x10000 -> 0x10FFFF.  By doing that they (for all time) reserved the
8192 surrogate code points (those with values from 0xD800 to 0xDFFF (incl)),
regardless of the format in which they're encoded, as being invalid, except
when used as part of a surrogate pair in UTF-16.

So, since UCS-2 had no surrogates (not really) and is (was) defined to
represent only the BMP (the first 65536 code points), none of which can be
represented by a surrogate pair, and since none of the surrogate code points
will ever (alone) represent any character, if one happens, somehow, to appear
in some string of UCS-2 characters being converted to UTF-8 (which are what
is used for presentation) don't simply encode the surrogate as the "obvious"
UTF-8 string, that makes invalid UTF-8, instead replace the surrogate by the
UCS2_REPLACEMENT_CHARACTER (which is also used when converting UTF-8 to UCS-2
if the UTF-8 is either invalid, or represents a value UCS-2 cannot store).
This way only valid UTF-8 is generated.

Similarly, if UTF-8 being converted to UCS-2 contains (invalid) UTF-8 which
would represent a surrogate code point, replace that by the replacement
character, just as is done for other invalid UTF-8 byte sequences.

The chances of anyone ever seeing this happen in practice are close to zero.
Hence no pullups planned.


To generate a diff of this commit:
cvs rdiff -u -r1.4 -r1.5 src/sbin/efi/utils.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.