Re: pkg/58875: python: terminal colours broke in some applications

["Taylor R Campbell via gnats" <gnats-admin%NetBSD.org@localhost>]

The following reply was made to PR pkg/58875; it has been noted by GNATS.

From: Taylor R Campbell <riastradh%NetBSD.org@localhost>
To: Christos Zoulas <christos%zoulas.com@localhost>
Cc: gnats-bugs%NetBSD.org@localhost, pkgsrc-bugs%NetBSD.org@localhost,
	Aleksey Cheusov <cheusov%NetBSD.org@localhost>,
	Christos Zoulas <christos%NetBSD.org@localhost>, Jonathan Perkin <jperkin%mnx.io@localhost>
Subject: Re: pkg/58875: python: terminal colours broke in some applications
Date: Thu, 5 Dec 2024 17:44:37 +0000

 > Date: Thu, 05 Dec 2024 11:36:23 -0500
 > From: Christos Zoulas <christos%zoulas.com@localhost>
 >=20
 > On 2024-12-05 10:29 am, Taylor R Campbell wrote:
 > > The attached patch resolves the problem by using libreadline
 > > unconditionally.  Maybe in the future if libedit is changed to support
 > > colours (which christos tells me it does not right now) we can put the
 > > option back, but until then I think we should have applications work
 > > out of the box.
 >=20
 > I think it does support colors in the same way readline does according=20
 > to this:
 > https://wiki.hackzine.org/development/misc/readline-color-prompt.html
 
 Interesting.  If I patch Mercurial to surround the prompt by \001 and
 \002, the prompt comes out just fine with libedit and libreadline:
 
 diff -r 47eacd67ce16 mercurial/ui.py
 --- a/mercurial/ui.py	Thu Dec 05 13:48:22 2024 +0000
 +++ b/mercurial/ui.py	Thu Dec 05 17:18:57 2024 +0000
 @@ -1745,7 +1745,7 @@
              self.flush()
              prompt =3D b' '
          else:
 -            prompt =3D self.label(prompt, b'ui.prompt') + b' '
 +            prompt =3D b'\001' + self.label(prompt, b'ui.prompt') + b'\002=
 ' + b' '
 =20
          # prompt ' ' must exist; otherwise readline may delete entire line
          # - http://bugs.python.org/issue12833
 
 But these are not required by libreadline.
 
 It looks like the difference is that libedit simply discards wide
 characters with wcwidth <=3D 0, like ESC:
 
      85 	w =3D wcwidth(end[1]);	/* column width of the visible char */
      86 	*wp =3D (int)w;
      87=20
      88 	if (w <=3D 0)		/* we require something to be printed */
      89 		return 0;
 ...
     100 	for (n =3D 0, i =3D 0; i < len; i++)
     101 		n +=3D ct_encode_char(b + n, (size_t)(w - n), buf[i]);
     102 	n +=3D ct_encode_char(b + n, (size_t)(w - n), end[1]);
     103 	b[n] =3D '\0';
 
 https://nxr.netbsd.org/xref/src/lib/libedit/literal.c?r=3D1.5#85
 
 In contrast, I think libreadline preserves them, and just calculates
 the column widths differently.  (The logic in
 https://git.savannah.gnu.org/cgit/readline.git/tree/display.c?h=3Dreadline-=
 8.2&id=3Df7a382fd09319b20ef4435b9b554183b605468c1
 is a lot more complicated, though, and I haven't stepped through it
 carefully.)
 
 On the one hand, it might be appropriate for Mercurial to use \001 and
 \002 around any terminal-specific escape sequences it uses that take
 up zero width -- particularly since, without that, readline might
 compute the column widths wrong for strings like "\033[0;33m", where
 it will treat the "\033" part as zero-width but the "[0;33m" as
 nonzero-width.
 
 On the other hand, I don't think Python really exposes this reasonably
 as part of the documented and reliable builtin input() or readline
 semantics.  (There are macros RL_PROMPT_START/END_IGNORE in readline.h
 but Python doesn't expose them.  It's hard to search for applications
 that handle this because what am I going to search for, the text
 `\001' or `\x01' or something?)  And what libedit is doing here isn't
 quite the same as libreadline, so there is a compatibility issue.
 
 So my inclination is still to switch back to unconditional libreadline
 until we've had more time to digest this -- after the branch.