Subject: Re: uvm_km_alloc/free from interrupt in netbsd-3
To: Bill Studenmund <wrstuden@netbsd.org>
From: Manuel Bouyer <bouyer@antioche.eu.org>
List: tech-kern
Date: 07/29/2006 23:15:00
On Fri, Jul 28, 2006 at 11:55:27AM -0700, Bill Studenmund wrote:
> On Fri, Jul 28, 2006 at 01:03:59PM +0200, Manuel Bouyer wrote:
> > Hi,
> > I'm trying to backport the twa driver to netbsd-3, and I'm running in
> > issues with uvm_km_alloc/free.
> 
> I don't think this should be too hard. The driver started life on a 3.99.5 
> kernel.
> 
> Also, you might want to chat w/ Jordan Rhody (jordanr at wasabisystems dot 
> com) about the driver. He's its primary keeper.
> 
> > First, is it safe to use uvm_km_alloc()/free() on kmem_map from interrupt
> > context in -current (I guess so because we have a UVM_KMF_NOWAIT flag) ?
> 
> It's questionable. If you aren't above splvm(), it's probably ok.

OK, so in a block device driver it's probably OK.

> 
> > Is it safe in netbsd-3, where we can't pass UVM_KMF_NOWAIT to uvm_km_alloc() ?
> > 
> > Here is the code in current:
> > 			s = splvm();
> > 			tr->tr_data = (void *)uvm_km_alloc(kmem_map, 
> > 			    tr->tr_length, 512, UVM_KMF_NOWAIT|UVM_KMF_WIRED);
> > 			splx(s);
> > 
> > I translated this in netbsd-3 to:
> > 
> > 			tr->tr_copy_length = tr->tr_length + 511UL;
> > 			s = splvm();
> > 			tr->tr_copy_data = (void *)uvm_km_alloc(kmem_map,
> > 			    tr->tr_copy_length);
> > 			splx(s);
> > 			tr->tr_data = (void *)
> > 				(((u_long)tr->tr_copy_data + 511UL) & ~511UL);
> > 
> > but with this I get random panics apparently due to data corruption after
> > doing some I/O to the twa. I suspect this is because of this alloc/free
> > for unaligned requests, because I got panic only when working on the raw
> > device (e.g. newfs, disklabel, etc ...). Once I get past this I can work
> > on the mounted filesystem without troubles, but I think requests from
> > a mounted filesystem are always aligned on 512 bytes.
> 
> Assuming that you mean that the data buffers are 512-byte aligned, yes.
> 
> Also, I don't see all of your changes, but you are freeing 
> tr->tr_copy_data, correct? Not tr->tr_data? And you're freeing the 
> increased length?

I think so, I checked this several times. I've put the code at
ftp://asim.lip6.fr/outgoing/bouyer/ if someone wants to have a look.

Also, I have code which does something similar in the xen3 block front-end
(see xbd_map_align/xbd_unmap_align in arch/xen/xen/xbd_xenbus.c). Works
fine in current, but it has also problems in netbsd-3, with similar panics.
I suspect it's the same issue. The xen2 front-end uses uvm_km_kmemalloc1()
for this, I'll have to try this instead.

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--