Subject: Re: soc zfs: taskqueue / workqueue
To: Oliver Gould <ogould@olix0r.net>
From: Juan RP <juan@xtrarom.org>
List: tech-kern
Date: 07/10/2007 00:37:01
On Mon, 9 Jul 2007 16:34:06 -0400
Oliver Gould <ogould@olix0r.net> wrote:

> Hello-
> 
> I've been looking in more detail at FreeBSD's taskqueue(9) [1], which
> provides several predefined task queues [1]:
> 
> 	The system provides four global taskqueues, taskqueue_fast,
> 	taskqueue_swi, taskqueue_swi_giant, and taskqueue_thread.
> 	...
> 
> Particularly, ZFS uses the thread queue:
> 
> 	The thread queue can be used, for instance, by interrupt level
> 	routines that need to call kernel functions that do things that
> 	can only be done from a thread context.  (e.g., call malloc with
> 	the M_WAITOK flag.)
> 
> Taskqueue is used to queue up 'tasks', which consist of a function pointer
> and some other data:
> 
>     struct task {
> 	STAILQ_ENTRY(task)       ta_link;	/* link for queue */
> 	u_short			 ta_pending;	/* count times
> queued */ u_short		     	 ta_priority;	/*
> priority of task in queue */ task_fn_t		 ta_func;	/*
> task handler */ void			*ta_context;	/*
> argument for handler */ }
> 
> Whereas, our workqueue(9) seems to be a framework for queueing up 'work'
> (data) to be passed to a specified handler function within a newly created
> thread for each workqueue.
> 
> In the kernel module loader, zfs_start() (which in turn creates several
> kernel threads) is enqueued [2:1802] in the dedicated taskqueue thread.
> This is the only place where I have found FreeBSD's ZFS to use taskqueue;
> so, I think that it's best to avoid using workqueue(9) and simply use
> kthread_create1() with zfs_start passed to it.
> 
> I think the general idea (as I have inferred) is that we don't want the
> lkm load operation to take as long zfs_start() may take, so it is thusly
> offloaded to a worker thread.  Is it worth 'wasting' a kthread on this
> one task?  Are there any other options?

You can also use sysmon_task_queue_sched() from
sys/dev/sysmon/sysmon_taskq*.

But Matt Thomas told me some time ago that it's better to use workqueue(9)
because using a dedicated kthread consumes more resources.

-- 
Juan Romero Pardines	- The NetBSD Project
http://plog.xtrarom.org	- NetBSD/pkgsrc news in Spanish