Subject: Re: Kernel memory allocation
To: Jaromir Dolecek <jdolecek@NetBSD.org>
From: Park Chan-youn <phygeeks@gmail.com>
List: tech-kern
Date: 02/07/2005 11:32:13
Jaromir Dolecek wrote:
> Park Chan-youn wrote:
> 
>>I'm writing a USB device driver for a system based on PowerPC/Sandpoint, 
>>and implemented read/write by using SIMPLEQ(3) as a fifo queue. Each 
>>queue element has a pointer to a buffer, and kernel memory is allocated 
>>to the buffer using malloc(9) and free(9). For arguments of the 
>>functions, I used the following.
>>
>>malloc(size, M_USBDEV, M_WAITOK)
>>free(p, M_USBDEV)
>>
>>When I input data of size 2MB into the queue, as I checked the memory 
>>status with vmstat(8) - with my kernel having KMEMSTATS turned on - free 
>>memory indicated by 'fre' decreased as expected amount, around 2MB. 
>>Also, when checked with -m option, 'Memory statistics by type' shows 2MB 
>>are used for the USB device.
>>
>>But when I had flushed the queue, and thereby freed the allocated 
>>memory, 'Memory statistics by type' showed that the USB device was not 
>>using the allocated memory any longer, but still 'fre' was in the 
>>decreased size.
> 
> 
> Most likely problem is that you are not actually freeing all
> allocated elements in the queue. If you access the queue from
> interrupt context, the queue might have been corrupted.
> 
> Doing splusb()/splx() around all queue operations might
> fix this - worth trying at least.
> 
> I'd try to add some intrumentation to detect possible queue item
> leak, such as have some counter incremented/decremented when items
> are added/removed.
> 
> Jaromir

I've tried to wrap all queue operations with spl, but it didn't worked. 
And I've already implemented a queue counter, and it shows the expected 
results.

It will be true, as you expects, that I'm doing some wrong 'freeing' - 
but it's not evident what is wrong, because vmstat(8) tells USB device 
does not have the allocated memory any longer after freeing the memory.

So, some questions are

1) What is the meaning of 'fre' memory shown when called vmstat(8) with 
no option? Man page tells it is the size of the free list, but I can't 
understand what the free list is. Is it different from the free memory 
available?

2) I've found that, through some experiments, a kernel panic occurs if 
the amount of the memory used by the USB device driver - total amount 
that have used from the beginning regardless of whether the allocated 
memory has freed or not - is over some pre-designated size. that is,

Memory statistics by type                           Type  Kern
          Type  InUse MemUse HighUse  Limit Requests Limit Limit Size(s)
            USB 63172  2468K   2468K 19661K    63172    0     0  16,64

The 'HighUse' accumulates the memory usage, and if it exceeds the 
'Limit', then kernel panics. Of course, after freeing the memory, 
'InUse' and 'MemUse' go to zero, but 'HighUse' does not. What is 
'HighUse' and 'Limit', and why they behave in such ways?

3)And again, could you tell me where can I find some good examples using 
   malloc(9) and free(9)?

Thanx in advance.