Typically with a character device, the kmod can get unloaded while an ioctl
is being executed on it.
That's solvable. Conceptually I think the main stumbling block is that
there are two layers at play which need to reconciled: specfs and devsw. It
could also be an opportunity to lessen the distinction between block and
character devices, there's no real need for cached access from userspace,
that would simplify things too.
When it comes to syscalls, I haven't looked
closely, but the issue is likely the same.
It's atomic and side effect free if done correctly. We have pleasing
examples of this. This is hard to get right though, so naturally mistakes
are made.