port-mac68k/60719: mac68k: ascaudio(4) capture gain wraps, causing noise recorded audio above a quarter of full scale

["raytran08%gmail.com@localhost via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60719
>Category:       port-mac68k
>Synopsis:       mac68k: ascaudio(4) capture gain wraps, causing noise recorded audio above a quarter of full scale
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-mac68k-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Sep 11 23:20:00 +0000 2026
>Originator:     Ray Tran
>Release:        11.0
>Organization:
Diamond Creek Digital
>Environment:
Hardware: Apple Macintosh Centris 650
Kernel: NetBSD 11.0 and CURRENT
>Description:
In the capture half of ascaudio_intr() each byte from FIFO A is read
into an int8_t, flipped to signed, and scaled:

	val = val * sc->sc_recvol / 64;
	*sc->sc_rptr++ = val;

val is int8_t and sc_recvol is uint8_t.  The product is computed as an
int and then stored back into the int8_t.  sc_recvol defaults to 255
(the mixer's inputs.dac), which makes the scale factor almost four: an
input of 33 becomes 131, which stores as -125, and an input of -100
becomes -398, which stores as 114.  Every sample whose magnitude is
above 32 comes out with the wrong sign and a random-looking magnitude,
so anything but near-silence records as noise.  Only a gain of 64 or
less avoids it, and the mixer's default is the worst case.
>How-To-Repeat:
Record anything with a real signal at the default gain:

	audiorecord -t 5 -e slinear_be -P 8 -s 22050 -c 1 out.au

and listen, or look at the samples.  Loud passages are garbage; only
quiet ones survive.  mixerctl -w inputs.dac=64 makes it record cleanly,
which shows the arithmetic rather than the hardware is at fault.
>Fix:
Applies to sys/arch/mac68k/obio/ascaudio.c.  It is independent of the
other ascaudio PRs from this machine (port-mac68k/60701 and 60702),
which change other parts of the file.

On a Centris 650 (EASC 0xbb) with nothing in the microphone jack the
FIFO delivers a constant 0x7e, and a capture at gain 64 stores it as -2
while one at gain 255 stores -7, as the expression reads.  A signal
large enough to show the wrap was not to hand, so the defect rests on
the arithmetic above.  The patch does not change the result for any
sample that did not overflow.  The patched driver builds
under NetBSD 11.0.  The same patch applies to -current.

Applies with "patch -p1" from the top of usr/src; verified with -F0
(no fuzz) against NetBSD-current 11.99.8 (20260830003849Z) and 11.0.

--- a/sys/arch/mac68k/obio/ascaudio.c
+++ b/sys/arch/mac68k/obio/ascaudio.c
@@ -811,7 +811,7 @@
 	struct ascaudio_softc *sc = arg;
 	uint8_t status;
 	int8_t val;
-	int loc_a, loc_b, total, count, i;
+	int loc_a, loc_b, total, count, i, scaled;
 
 	if (!sc)
 		return;
@@ -857,8 +857,19 @@
 				val = bus_space_read_1(sc->sc_tag,
 				    sc->sc_handle, loc_a);
 				val ^= 0x80;
-				val = val * sc->sc_recvol / 64;
-				*sc->sc_rptr++ = val;
+				scaled = val * sc->sc_recvol / 64;
+				if (scaled > INT8_MAX)
+					scaled = INT8_MAX;
+				else if (scaled < INT8_MIN)
+					scaled = INT8_MIN;
+				*sc->sc_rptr++ = scaled;
 				if (loc_b) {
 					(void)bus_space_read_1
 					    (sc->sc_tag, sc->sc_handle, loc_b);