Subject: Re: Best practice to stack LKMs (export a service hook)?
To: None <tech-kern@NetBSD.org>
From: Bill Studenmund <wrstuden@netbsd.org>
List: tech-kern
Date: 12/19/2005 16:22:09
--UHN/qo2QbUvPLonB
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Dec 19, 2005 at 10:35:39PM +0100, Gerhard Sittig wrote:
> [ in case you remember, this is for my NetBSD port of www.dazuko.org,
> current work is done under NetBSD 2.0.2 ]
>=20
> The situation is this: There are two LKMs, one providing a service for
> another -- or vice versa, one hooking into the other by means of a
> callback. What is the correct way to export the callback hook from the
> first module for the second module?
>=20
> The alternatives I can see: A function pointer variable which simply
> gets set and unset from the second module. Some register and unregister
> functions. If so, how to export them? Introduce new syscalls? Or just
> export the routines? Although there is only one "user" module for the
> exporting module. No need for chains or something along these lines.
Just export the routine.
Actually, export the routine and also an addref/decrec routine.
> Here is what I did, which works but I'm just not sure if it's an
> acceptable way. I took the first, simple approach.
>=20
> The base LKM named dazukofs(4) (a VFS module, hooking all appropriate
> vnode ops and "asking" the upper module for permission for the access)
> has a declaration in its src/miscfs/dazukofs/dazuko.h header file:
>=20
> enum dazukofs_opcode {
> DAZ_OP_OPEN, DAZ_OP_CREATE, DAZ_OP_CLOSE, ...
> };
>=20
> union dazukofs_vop_extra { ... };
>=20
> extern int (*dazukofs_vop_hook) __P((
> enum dazukofs_opcode code,
> struct proc *procp,
> struct ucred *credp,
> struct vnode *vp,
> const char *fn,
> union dazukofs_vop_extra *extra));
>=20
> and an appropriate variable in its implementation .c file:
>=20
> int (*dazukofs_vop_hook) __P(... params ...) =3D NULL;
>=20
> The unload() routine refuses to unload the base LKM when the hook is
> set:
>=20
> if (dazukofs_vop_hook !=3D NULL) {
> return(EBUSY);
>=20
>=20
> The client LKM named dazuko(4) (a DEV module implementing access control
> and providing a character device to the user space for Dazuko enabled
> daemons) implements the callback (or better: upcall) routine:
>=20
> static int netbsd_dazuko_vn_hook(... params ...);
>=20
> and simply registers and unregisters this routine by means of a simple
> assignment to the base LKM's variable:
>=20
> dazukofs_vop_hook =3D netbsd_dazuko_vn_hook;
> ...
> dazukofs_vop_hook =3D NULL;
>=20
> As stated above there is only this one dazuko(4) module hooking into
> dazukofs(4). Multiple registered hooks are not supported. Neither in
> parallel (in a table) nor in sequence (in a chain).
>=20
>=20
> Using the variable from within the base dazukofs(4) module is rather
> straight forward:
>=20
> if (dazukofs_vop_hook !=3D NULL) {
> ...
> rc =3D dazukofs_vop_hook(...);
> ...
> }
>=20
> Although now that I cite the code I noticed that the pointer might get
> unset between the check and the call, so I may have to insert some kind
> of locking here. And export the lock as well as the pointer variable so
> the hooking module will take part in the synchronization.
As above, swap them around and just have a register/deregister use=20
routine.
> Which again suggests that an "accessor" function is more appropriate
> than I thought above. So I would have to export a set, a query, a lock
> and an unlock routine instead of the variable itself and the lock.
>=20
> The most important question here is: Can it happen that the dazuko(4)
> module gets unloaded while in parallel a vnode op takes place? That's
> when the pointer might get dereferenced while it's NULL.
Yes. Unlikely, since the administrator has to undo it, but it can happen.=
=20
Thus the need for a "don't unload me" routine.
> PS: Yes, I noticed that the CVS repo suggests that the NetBSD project
> now prefers ANSI C style over K&R declarations. I will gladly switch to
> ANSI style later (actually have been there first but switched to K&R
> after seeing the 2.0.2 source tree :) ), but I want to get the function
> right before doing style improvements.
>=20
> Another PS: I did try to put both parts into a single LKM. This worked
> in principle, but the modstat(8) output with only one LKM type spec does
> not look half as nice as the current setup with one VFS and one DEV
> module separately listed. Plus I wanted to keep the VFS hooks and the
> access control parts split if possible.
Sounds reasonable. I had a custom vfs and device driver when I did the=20
nastore work.
> Yet another PS: The above approach needs the -s switch for modload(8)
> when loading dazukofs(4). Not exporting the module's symbols will
> prevent the dazuko(4) module from loading. This is not too nice for
> users, but maybe could be solved by means of /etc/lkm.conf (haven't
> checked yet).
We're still improving this part of things. :-)
Take care,
Bill
--UHN/qo2QbUvPLonB
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (NetBSD)
iD8DBQFDp06xWz+3JHUci9cRAl0+AJ9b7wPs0gEdg+VtOKDB6mI/Pd6PYQCfTKi/
Zfzu9KRWywYykZ6q2OmaSAc=
=LcQI
-----END PGP SIGNATURE-----
--UHN/qo2QbUvPLonB--