task queues

[Taylor R Campbell <riastradh%NetBSD.org@localhost>]

We don't have a nice easy lightweight way for a driver to ask that a
task be executed asynchronously in thread context.  The closest we
have is workqueue(9), but each user has to do bookkeeping for a
different workqueue and incurs a long-term kthread, or ncpu long-term
kthreads, that are mostly idle -- most users of workqueue(9) create a
single struct work for the workqueue and use it for a single purpose.

For USB drivers, we have usb_task, but the API is limited and, for
most users, broken: there's no way to wait for a task to complete,
e.g. before detaching a device and unloading its driver module.
There's also no reason this is peculiar to USB drivers.

The other month I threw together an facility for scheduling
lightweight tasks to be executed in thread context on the current CPU,
supporting cancellation, and without requiring a long-term kthread per
user.

All you need to do to call my_task_action asynchronously is allocate a
struct task object, e.g. in your device softc, and do

        task_init(task, &my_task_action);
        task_schedule(task);

static void
my_task_action(struct task *task)
{

        printf("My task ran!\n");
}

If you want scheduling fairness between tasks, you can create your own
task queue to run tasks in a separate kthread from everyone else's,
but the kthreads are pooled so that if some task queues are idle they
won't hang onto kthreads.

There's an introduction to the facility here:

https://www.NetBSD.org/~riastradh/tmp/20140517/task/tasks.txt

The code is untested -- it is just an idea that was kicking around in
my head -- but it and some man pages can be found in the directory at:

https://www.NetBSD.org/~riastradh/tmp/20140517/task/

The idea has undergone some private discussion (thanks, apb@,
jakllsch@, matt@, mlelstv@, mrg@, nick@, rmind@), and is loosely
inspired by Linux workqueues and FreeBSD taskqueues.

Thoughts?  Comments?  Questions?  Worth pursuing this further and
making it work and replacing broken USB tasks by it?