Re: port-amd64/57236 (Gude, 9.8, https://netbsd.org/docs/guide/en/chap-x.html#chap-x-xdm is wrong)

["David H. Gutteridge" <david%gutteridge.ca@localhost>]

The following reply was made to PR xsrc/57236; it has been noted by GNATS.

From: "David H. Gutteridge" <david%gutteridge.ca@localhost>
To: matthew green <mrg%eterna.com.au@localhost>, gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: port-amd64/57236 (Gude, 9.8,
 https://netbsd.org/docs/guide/en/chap-x.html#chap-x-xdm is wrong)
Date: Thu, 02 Mar 2023 00:18:40 -0500

 On Sat, 2023-02-25 at 06:28 +1100, matthew green wrote:
 > > =C2=A0The issue is basically that as we presently distribute xdm, it
 > > won't
 > > =C2=A0interoperate properly with some modern X applications, because th=
 e
 > > =C2=A0authorization mechanism it defaults to doesn't work. This was
 > > surprising
 > > =C2=A0to me as well (I don't use xdm or any other display manager), as
 > > the
 > > =C2=A0error messages aren't clear here about what's happening. I was ab=
 le
 > > to
 > > =C2=A0reproduce this myself and test the configuration tweak.
 > > =C2=A0
 > > =C2=A0Other BSDs and Linux distros dealt with this years ago; e.g.,
 > > FreeBSD
 > > =C2=A0changed the default in the configuration to be what I suggested,
 > > Debian
 > > =C2=A0changed xdm's compilation settings to fundamentally disable the x=
 dm
 > > =C2=A0authorization method, etc. I feel NetBSD should do something of t=
 he
 > > sort
 > > =C2=A0as well, but that probably requires discussion among developers.
 > > So, for
 > > =C2=A0now, I've adjusted chapter 9.8 in the guide to add this detail.
 >=20
 > i'm confused.=C2=A0 the default should be:
 >=20
 > =C2=A0=C2=A0 "XDM-AUTHORIZATION-1 MIT-MAGIC-COOKIE-1"
 >=20
 > so i'm not sure why the xdm-config setting is needed.=C2=A0 or is it
 > that "XDM-AUTHORIZATION-1" method doesn't work at all, and it
 > can only be "MIT-MAGIC-COOKIE-1"?=C2=A0 this seems to be the default,
 > since libXdcmp default to including XdmcpWrap.c, which means
 > xdm finds it and enables xdm auth methods.
 
 Yes, that's right, when XDM-AUTHORIZATION-1 is included (first, in the
 default), it is what gets used, and then things can break. The problem
 is that applications can do things like reload X11 libraries after init,
 and this causes the cookie state to get duplicated. The underlying code
 used by many clients (found these days in libxcb) is problematic. It
 uses a function-local static variable to generate a simple nonce for the
 cookie, which can get unintentionally reset, and mixes that with the PID
 and system time to the second for "uniqueness".
 
 This is 100% reproducible for me with 10.99.2 and Xfce from pkgsrc
 2022Q4. All that needs to be done is to start xdm (from xbase) with the
 default configuration, and set "exec startxfce4" in .xsession. (While it
 is wrong, per upstream, to start xfce4-session directly instead, I also
 tested with that, same result.) In these examples, dbus is not enabled
 before startup. I'm assuming that's how the reporter configured things.
 
 Having ktraced what's happening to see what goes on with this specific
 example, I see that xfce4-session tries to start a dbus session if one
 isn't already enabled. It does so by calling 'execvp("dbus-launch",...'
 dbus itself (by default in pkgsrc) is linked against X11 libraries,
 including libxcb, like xfce4-session.
 
 So, when the process starts, it loads libxcb, calculates "PID+nonce=3D0+
 seconds since the epoch" for use by xdm authorization. Then it calls
 execvp(), re-loading libxcb in the process, which resets the nonce to
 0, and the PID is the same, as is the seconds since epoch value. The
 authorization mechanism then sees that the same cookie has been
 generated twice and bombs.
 
 Since so many downstream consumers of potentially affected software
 long ago worked around this one way or another (or, indeed, stopped
 documenting or recommending anyone use xdm with any DEs), I guess this
 doesn't come up much anymore. There are references in older bug reports
 to non-DE applications like SDL-based games also encountering this
 problem. Nothing other than xdm itself uses this authorization method,
 AFAIK.
 
 Given this is ugly, and the general approach with other downstream
 projects not to use xdm authorization by default, I figured it was
 simplest to explicitly recommend the configuration override for now.
 
 Dave