I'm writing a USB device driver for a system based on PowerPC/Sandpoint, 
and implemented read/write by using SIMPLEQ(3) as a fifo queue. Each 
queue element has a pointer to a buffer, and kernel memory is allocated 
to the buffer using malloc(9) and free(9). For arguments of the 
functions, I used the following.

malloc(size, M_USBDEV, M_WAITOK)
free(p, M_USBDEV)

When I input data of size 2MB into the queue, as I checked the memory 
status with vmstat(8) - with my kernel having KMEMSTATS turned on - free 
memory indicated by 'fre' decreased as expected amount, around 2MB. 
Also, when checked with -m option, 'Memory statistics by type' shows 2MB 
are used for the USB device.

But when I had flushed the queue, and thereby freed the allocated 
memory, 'Memory statistics by type' showed that the USB device was not 
using the allocated memory any longer, but still 'fre' was in the 
decreased size.

And if I push data continuously to the queue, then kernel panics with 
the following message.

panic: ``malloc: out of space in kmem_map''

Well I'm completely lost. Is there a some cautious point in using 
malloc(9) and free(9) comparing to malloc(3) and free(3)?

The following is some vmstat results for your reference.

----
* before bulk_read
# vmstat
vmstat: No drives attached.
 procs   memory     page                         faults   cpu
 r b w   avm   fre  flt  re  pi   po   fr   sr   in   sy  cs us sy id
 0 0 0  2440 114416    3   0   0    0    0    0   20    3   4  0  0 100
# vmstat -m
...
Memory statistics by type                           Type  Kern
         Type  InUse MemUse HighUse  Limit Requests Limit Limit Size(s)
        devbuf    34    11K     11K 19661K       38    0     0  
16,32,64,128,256,512,4096,8192
...


* after bulk_read
# vmstat
vmstat: No drives attached.
 procs   memory     page                         faults   cpu
 r b w   avm   fre  flt  re  pi   po   fr   sr   in   sy  cs us sy id
 0 0 0  2440 111932    4   0   0    0    0    0  130    4   4  0  0 100
# vmstat -m
...
Memory statistics by type                           Type  Kern
         Type  InUse MemUse HighUse  Limit Requests Limit Limit Size(s)
        devbuf    34    11K     11K 19661K       38    0     0  
16,32,64,128,256,512,4096,8192
...
    USB device 63172  2468K   2468K 19661K    63172    0     0  16,64
...

* real input: 2021447 bytes = 1974KB
* allocated memory: 2484KB

*after bulk_write
# vmstat
vmstat: No drives attached.
 procs   memory     page                         faults   cpu
 r b w   avm   fre  flt  re  pi   po   fr   sr   in   sy  cs us sy id
 0 0 0  2512 110040    2   0   0    0    0    0   47   22   2  0  0 100
# vmstat -m
...
Memory statistics by type                           Type  Kern
         Type  InUse MemUse HighUse  Limit Requests Limit Limit Size(s)
        devbuf    34    11K     11K 19661K       38    0     0  
16,32,64,128,256,512,4096,8192
...
    USB device     0     0K   2468K 19661K    63172    0     0  16,64
...

* real output: 2021447 bytes
* allocated memory: 4376KB
----