bin/60177: vi crash when using cedit

["Mouse via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60177
>Category:       bin
>Synopsis:       vi crash when using cedit
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Apr 06 00:45:00 +0000 2026
>Originator:     Mouse
>Release:        NetBSD 9.1
>Organization:
	Dis-
>Environment:
First seen by an IRC acquaintance on, according to them, 10.1/amd64; I
have not personally verified that.  I have verified it present on
9.1/amd64, 5.2/amd64, 5.2/i386, and 5.2/shark.  Does not occur (at
least not with this how-to-repeat) on 1.4T/sparc, 1.4T/i386, or
4.0.1/i386.  On 1.4T and 4.0.1, vi was modified by renaming vi-internal
O_* symbols to o_* (motivated by collisions with O_DIRECTORY, which I
introduced separately) and removing trailing whitespace from source
code, but was otherwise unmodified.  On 5.2, vi was unmodified except
for trailing whitespace removal.  9.1 vi was unmodified except for
removing all CVS directories.  I don't *know*, but I feel moderately
sure the 10.1 vi was totally stock.
>Description:
	vi crashes upon a specific input sequence (invoking cedit
	support).  See How-To-Repeat.

	Given how longstanding this is, it clearly is not a problem for
	very many people, which is why I'm classifying it as low
	priority.  Please reclassify if/as appropriate.
>How-To-Repeat:
	My test case is, as a user with no ~/.exrc, type these
	characters (starting at a shell prompt).  I'm giving hexdump
	output to be completely unambiguous:
00000000  76 69 0a 69 0a 1b 3a 73  65 74 20 63 65 64 69 74  |vi.i..:set cedit|
00000010  3d 16 12 0a 3a 12                                 |=...:.|
00000016
	Representing control characters with ^ notation, this means
	typing

	vi
	i
	^[:set cedit=^V^R
	:^R

	The crash occurs shortly after the last ^R is typed.  On slow
	machines (eg, the shark), the delay can be human-perceptible;
	on fast machines, such as multi-GHz amd64, it appears instant
	to me.  Under 5.2, vi dies with a SIGSEGV; on 9.1, SIGABRT.  My
	IRC correspondent reports "Abort trap" on 10.1, so presumably
	SIGABRT there too.

	My $TERM was mterm.  Under 5.2/amd64 (I haven't checked
	others), it also occurs with $TERM set to vt100 or window-v2.
	My lnext setting was 0x16 (^V); I have not investigated whether
	changing that affects the above in any way.

	Looking at the core under 5.2, it appeared to be due to
	vs_paint() recursing enough to run the process out of stack.
	Manual binary search in gdb ("frame N" for various N) let me
	discover that the stack was 7670 frames deep, with $sp in the
	top frame being 0x7f7fffffd7a0 but in the bottom (crashing)
	frame 0x7f7fffdffff0; proc.$$.rlimit.stacksize.soft in the
	invoking shell prints 2097152, which is 0x200000, which matches
	well enough with the $sp values to make me reasonably confident
	the crash symptom is indeed due to stack exhaustion.  The
	bottom few frames are
#0  0x00007f7ffdc19dfa in _cursesi_addwchar () from /usr/lib/libcurses.so.6
#1  0x00007f7ffdc1a6eb in __waddbytes () from /usr/lib/libcurses.so.6
#2  0x0000000000405fbf in addstr4 ()
#3  0x0000000000440859 in vs_line ()
#4  0x0000000000442a3f in vs_paint ()
#5  0x00000000004435ab in vs_paint ()
#6  0x00000000004435ab in vs_paint ()
and the top few are
#7664 0x00000000004435ab in vs_paint ()
#7665 0x00000000004435ab in vs_paint ()
#7666 0x00000000004435ab in vs_paint ()
#7667 0x000000000044412f in vs_refresh ()
#7668 0x000000000043f713 in vi ()
#7669 0x0000000000427df7 in editor ()
#7670 0x00000000004068fe in main ()
	I didn't check all 7657 intermediate frames to be sure they're
	all vs_paint, but the bottom two thousand, except for the
	bottom four, are, and all the ones I probed in my manual binary
	search are, so I'm moderately confident they all are.
>Fix:
	Unknown.  I haven't looked at the code at all; because I don't
	use cedit myself, fixing it is relatively low priority for me.