kern/60696: genfb(4) restores the wrong palette when leaving WSDISPLAYIO_MODE_MAPPED

["raytran08%gmail.com@localhost via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60696
>Category:       kern
>Synopsis:       genfb(4) restores the wrong palette when leaving WSDISPLAYIO_MODE_MAPPED
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Sep 08 10:40:00 +0000 2026
>Originator:     Ray Tran
>Release:        11.0
>Organization:
Diamond Creek Digital
>Environment:
NetBSD/mac68k 11.0
Tested on: Macintosh IIcx (68030, NuBus "Toby" video), Centris 650 (68040, onboard "Wombat" DAFB)
>Description:
genfb keeps one copy of the palette, scp->sc_cmap_red/green/blue[256].

  - genfb_putcmap(), the WSDISPLAYIO_PUTCMAP path, writes into it
    (dev/wsfb/genfb.c, "memcpy(&scp->sc_cmap_red[index], ...)").
  - genfb_restore_palette() reads out of it and pushes it to hardware.
  - WSDISPLAYIO_SMODE calls genfb_restore_palette() on the transition back
    to WSDISPLAYIO_MODE_EMUL.

So when a program maps the framebuffer, programs its own colours through
WSDISPLAYIO_PUTCMAP and then exits, genfb "restores" that program's
palette onto the console.  The console's own palette is gone: nothing
kept a copy of it.  The restore is close to a no-op -- it rewrites the
hardware with the values already there.

This affects any genfb attachment whose bus backend supplies a colormap
callback, since without one genfb_putpalreg() does nothing and the
hardware LUT is never touched at all.  On an 8-bit display the effect is
obvious: after leaving X the console is drawn with X's colour cube.
>How-To-Repeat:
On a machine whose genfb attachment provides a colormap callback, at
depth 8: note the console colours, run an X server that allocates
colours (i.e. without a static colormap), exit it, and look at the
console.  It is drawn with the palette X last installed rather than the
one it had before X started.
>Fix:
Keep a copy of the palette when leaving MODE_EMUL and put it back when
returning, so the console's colours survive a mapped-mode program.

The change is contained in the WSDISPLAYIO_SMODE case plus three arrays
in struct genfb_private, and does not affect a system with no userland
palette changes.  The order of the mode callback relative to the restore
is unchanged.

Tested on a Macintosh Centris 650 (DAFB, 640x480x8, colormap callback
provided) running NetBSD 11.0, with an X server that programs the CLUT
and a console driver that installs its own palette.  Before, the
console came back wearing X's colours; after, its own palette returns.

Applies with "patch -p1" from the top of usr/src; verified with -F0
(no fuzz) against NetBSD-current 11.99.8 (20260830003849Z) and 11.0.

--- a/sys/dev/wsfb/genfb.c
+++ b/sys/dev/wsfb/genfb.c
@@ -93,6 +93,14 @@
 	u_char sc_cmap_red[256];
 	u_char sc_cmap_green[256];
 	u_char sc_cmap_blue[256];
+	u_char sc_cmap_saved_red[256];
+	u_char sc_cmap_saved_green[256];
+	u_char sc_cmap_saved_blue[256];
 	bool sc_want_clear;
 #ifdef SPLASHSCREEN
 	struct splash_info sc_splash;
@@ -508,11 +516,33 @@
 			return error;
 
 		if (new_mode != scp->sc_mode) {
+			if (scp->sc_mode == WSDISPLAYIO_MODE_EMUL) {
+				memcpy(scp->sc_cmap_saved_red,
+				    scp->sc_cmap_red, 256);
+				memcpy(scp->sc_cmap_saved_green,
+				    scp->sc_cmap_green, 256);
+				memcpy(scp->sc_cmap_saved_blue,
+				    scp->sc_cmap_blue, 256);
+			}
 			scp->sc_mode = new_mode;
 			if (scp->sc_modecb != NULL) {
 				scp->sc_modecb->gmc_setmode(sc, scp->sc_mode);
 			}
 			if (new_mode == WSDISPLAYIO_MODE_EMUL) {
+				memcpy(scp->sc_cmap_red,
+				    scp->sc_cmap_saved_red, 256);
+				memcpy(scp->sc_cmap_green,
+				    scp->sc_cmap_saved_green, 256);
+				memcpy(scp->sc_cmap_blue,
+				    scp->sc_cmap_saved_blue, 256);
 				genfb_restore_palette(sc);
 				vcons_redraw_screen(ms);
 			}