pool_cache_get behavior

[Adam Hamsik <haaaad%gmail.com@localhost>]

Hi folks,

During work on zfs I'm repeatedly hitting one problem, where zfs can't allocate 
object by pool_cache_get from pool on a machine with >4GB(amd64). Usually I can 
hit this problem right after the boot. Adding KM_SLEEP flag to  allocator flags 
helps and everything works, but that is hack. Because solaris guys really know 
why they do not use KM_SLEEP flags in problematic parts of zfs, where we are 
crashing because of this bug.[1]

I talked with Andrew about this problem and he said that there is some sort of 
a lock in our uvm which if it is locked can make pool_cache_get return NULL 
even if there is enough memory available. My question is do someone has any 
idea where problem can be ? 

Probably easiest solution would be preallocate some number of elements in a 
pool and use object from them first if global uvm lock is locked and we can't 
allocate in non SLEEPABLE mode now.

[1] Simple example where I see crashes because of this bug is zfs arc code. 

Regards

Adam.