Re: Temporary memory allocation from interrupt context

[Jason Thorpe <thorpej%me.com@localhost>]

> On Nov 11, 2020, at 1:44 AM, Martin Husemann <martin%duskware.de@localhost> wrote:
> 
> In a perfect world we would avoid the interrupt allocation all together, but
> I have not found a way to rearrange things here to make this feasible.
> 
> Is kmem_intr_alloc(9) the best way forward?

While softints are backed by threads these days, kmem_intr_alloc() is the API to use in this scenario.  (As an aside, kmem_alloc() is itself actually a wrapper around kmem_intr_alloc() that merely asserts that you're not in hard- or soft-interrupt context ... historically, there used to be a real distinction, because they were backed by different VM maps with different locking protocols ... these days, it's all backed by a vmem arena).

However, it is worth noting that softint threads are ONLY allowed to sleep if blocking on a mutex or rwlock; sleeping for memory allocation, or a condvar or whatever is not allowed (this is a policy decision rooted in the fact that any given softint thread can only be processing one softint at a time, and we want to prevent starvation).  So, because you can't sleep, you must pass the KM_NOSLEEP flag to kmem_intr_alloc() (there's already an assertion to ensure the caller has passed exactly one of KM_SLEEP *or* KM_NOSLEEP, but we should probably add an ASSERT_SLEEPABLE() in the KM_SLEEP case to catch errors like this).

If the size fits into one of the kmem_cache or kmem_cache_big buckets, the allocation *will* come out of a pool_cache, and figuring out which pool cache to use is pretty quick, so you're not being penalized too badly here for not knowing the size ahead of time.

-- thorpej