kern/55175: Interpretation of AUDIO_FORMAT_LINEAR is wrong for 8-bit samples

[nia <nia%NetBSD.org@localhost>]

>Number:         55175
>Category:       kern
>Synopsis:       Interpreation of AUDIO_FORMAT_LINEAR is wrong for 8-bit samples
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Apr 14 12:10:00 +0000 2020
>Originator:     nia <nia%NetBSD.org@localhost>
>Release:        NetBSD 9.0_STABLE
>Organization:
>Environment:
System: NetBSD r 9.0_STABLE NetBSD 9.0_STABLE (GENERIC) #13: Mon Apr 13 15:12:50 IST 2020 nia@r:/home/nia/netbsd-9/sys/arch/amd64/compile/obj/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
illumos/Solaris/SunOS define AUDIO_ENCODING_LINEAR as follows:

#define AUDIO_ENCODING_LINEAR (3) /* Signed Linear PCM encoding */

However, in NetBSD, it's an alias for AUDIO_ENCODING_PCM16, which is deprecated.

#define AUDIO_ENCODING_PCM16            3 /* signed linear PCM, obsolete */
#define AUDIO_ENCODING_LINEAR           AUDIO_ENCODING_PCM16 /* SunOS compat */

Note that both definitions say the encoding is signed...

In the kernel, PCM16 is interpreted as unsigned if the samples are 8-bit. This seems unexpected and wrong.

I'm not sure of the origin of the definition of PCM16 and as far as I can tell it was never documented in NetBSD, but far more code in the wild uses LINEAR, and even our own headers say it's signed.
>How-To-Repeat:
Attempt to do audio I/O with precision=8 and AUDIO_FORMAT_LINEAR.
>Fix:
Probably this:

Index: audio.c
===================================================================
RCS file: /cvsroot/src/sys/dev/audio/audio.c,v
retrieving revision 1.65
diff -u -r1.65 audio.c
--- audio.c	26 Mar 2020 13:32:03 -0000	1.65
+++ audio.c	14 Apr 2020 11:35:13 -0000
@@ -6086,12 +6086,8 @@
 {
 
 	/* Convert obsoleted AUDIO_ENCODING_PCM* */
-	/* XXX Is this conversion right? */
 	if (p->encoding == AUDIO_ENCODING_PCM16) {
-		if (p->precision == 8)
-			p->encoding = AUDIO_ENCODING_ULINEAR;
-		else
-			p->encoding = AUDIO_ENCODING_SLINEAR;
+		p->encoding = AUDIO_ENCODING_SLINEAR;
 	} else if (p->encoding == AUDIO_ENCODING_PCM8) {
 		if (p->precision == 8)
 			p->encoding = AUDIO_ENCODING_ULINEAR;