Here are some of my thoughts about PAM as an authentication system:

Too complex than needs to be. Why loadable libraries as part
of the design? Why not a client/server model? Or even a simpler exec
chain model such as checkpassword or CVM? In a client/server design, the
modules (executables) could all conform to the same simple interface for
passing credentials to the server. This would allow the modules to be
either statically or dynamically linked; a sysadmin would have a
choice. Debugging would be trivial, the modules themselves would be much
smaller in size, and the API could be very simple.

Why loadable modules again? Because the designers tried to avoid
fork/exec overhead? Or a message over socket overhead? Is the overhead
really that high to warrant such a design? Measure it, you'll note some
other syscalls take far more CPU than fork/exec.

About static linking. Static linking, by the way results in faster fork/exec.
Up to several times faster. This is thus important for busy fork/exec servers.
And with dynamic linking, sharing too many libraries can put a strain on VM.

Another issue is PAM in its current form is becoming mandatory. If
you look at FreeBSD for example, /usr/bin/login and others are already
linked with PAM, and forced onto sysadmin. Did we universally choose to
use this system? I think it should be up to administrators which
authentication mechanism to use. This is why it should be a simple
interface rather than something as complex as PAM.

With the exec chain it would even be trivial. Example: /usr/bin/login
gets username/password pair, writes it to file descriptor 3, fork/execs
authentication module which reads the pair from descriptor 3, and if
successful execs /usr/bin/login-finish, which does everything which needs
to be done before running shell, and then execs the shell.

The modules could be doing anything you want, e.g. an
LDAP server search. Static, dynamic, up to you. 
This is http://cr.yp.to/checkpwd.html. Also see
http://www.untroubled.org/cvm/ for a similar, but a more feature-rich
set of tools.
Also http://www.nimh.org/code/kchuid/ is a Linux specific piece which
changes uid/gid of a running process instead of suid.

Also http://www.superscript.com/ucspi-ipc/intro.html for another way to
avoid suid.

Also I find PAM configuration files confusing, and the whole system very
hard to debug. I doubt the files are easily parseable, and editable by
software too. Simplicity and ease of use definitely weren't design goals
there, IMO.

Authenticatiion is the easy part, because all you need is request
username/password, give to the authentication server, and get OK/FAIL.
The problem spot is the NSS library. How to make all the getpw* be
sysadmin-configurable without too much fuss? I for example am looking
into ways to make getpw* functions pull everything out of an LDAP
directory, and can't see how to do this without much fuss in any of the
BSDs. Linux has nss-ldap which seems to work OK with its NSS library,
but what if I have a BSD machine on the network? nss-ldap is only
supported through some BIND library on *BSDs, and it's something I
wouldn't trust. Writing this support for the current NSS would also mean
linking the whole LDAP enchilada into it statically, no?