Re: bin/53164: vi coredump when viewing binary file

[Rin Okuyama <rokuyama%rk.phys.keio.ac.jp@localhost>]

The following reply was made to PR bin/53164; it has been noted by GNATS.

From: Rin Okuyama <rokuyama%rk.phys.keio.ac.jp@localhost>
To: gnats-bugs%NetBSD.org@localhost, gnats-admin%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost
Cc: 
Subject: Re: bin/53164: vi coredump when viewing binary file
Date: Sat, 7 Apr 2018 21:39:08 +0900

 On 2018/04/07 21:00, tobiasu%tmux.org@localhost wrote:
 > 711     done_cursor:
 > 712             /*
 > 713              * Sanity checking.  When the repainting code messes up, the usual
 > 714              * result is we don't repaint the cursor and so sc_smap will be
 > 715              * NULL.  If we're debugging, die, otherwise restart from scratch.
 > 716              */
 ...
 > 724             if (vip->sc_smap == NULL) {
 > 725                     if (F_ISSET(sp, SC_SCR_REFORMAT))
 > 726                             abort(); /* XXX */
 > 727                     F_SET(sp, SC_SCR_REFORMAT);
 > 728                     return (vs_paint(sp, flags));
 > 729             }
 ...
 > Line 726 is new (rev 1.7) and not present in the latest nvi nor in Free/Open.
 > The new behavior contradicts the comment after done_cursor.
 
 Well, this does not actually contradicts to the comment. When we fails
 to repaint, we try again from scratch by specifying SC_SCR_REFORMAT
 flag. This flag is switched off if we succeed. Therefore, it means that
 we're falling into an infinite recursion if we reach this code segment
 with this flag turned on.
 
 > Playing around, there seem to be other issues as well.
 > 
 > The modeline cursor column (curcol) goes wildly out of range,
 > scrolling stops working somewhat, the screens isn't redrawn properly,
 > the vip pointer gets corrupted resulting in a segfault from time to time,
 > etc.
 >
 > How-To-Repeat:
 > Open /usr/bin/gdb in vi and quickly scroll around with j/k/page up/down.
 > I'm using urxvt over ssh.
 
 I could reproduce this problem with LC_CTYPE=ja_JP.UTF-8, whereas
 I couldn't with LC_CTYPE=C. I guess that we fail to repaint screen
 when we encounter some invalid byte sequence in non-ASCII encoding.