Subject: using an uvm aobj with MFS - page fault trap problems.
To: None <tech-kern@netbsd.org>
From: Simon Burge <simonb@netbsd.org>
List: tech-kern
Date: 05/27/2000 22:30:55
Folks,

I'm playing with copying an MFS into a uvm aobj so that the user-land
process isn't needed after the mount is finished.  This is in part based
on a background project I've had going for a little while on writing a
swapfs and the recent MFS hassles.

I'm allocating the aobj and some scratch space to uvm_map to with

	mfsp->mfs_data = uao_create(round_page(mfsp->mfs_size), 0);
	mfsp->mfs_uvmpage = uvm_km_valloc_wait(kernel_map, MAXBSIZE);

then I copyin and (for non-zero pages) copy to the aobj.  Hmm, I had
some problems earlier if I didn't pre-setup mfs_uvmpage, but it seems to
work fine...  Anyways, the body of mfs_strategy is basically

	error = uvm_map(kernel_map, &mfsp->mfs_uvmpage,
	    round_page(bp->b_bcount), mfsp->mfs_data,
	    trunc_page(bp->b_blkno << DEV_BSHIFT),
	    UVM_MAPFLAG(UVM_PROT_RW, VM_PROT_ALL, UVM_INH_NONE,
	    UVM_ADV_NORMAL, 0));
	error = uvm_fault_wire(kernel_map, mfsp->mfs_uvmpage,
	    round_page(mfsp->mfs_uvmpage + bp->b_bcount),
	    VM_PROT_READ|VM_PROT_WRITE);

	if (bp->b_flags & B_READ)
		memcpy(bp->b_data, (caddr_t)mfsp->mfs_uvmpage, bp->b_bcount);
	else
		memcpy((caddr_t)mfsp->mfs_uvmpage, bp->b_data, bp->b_bcount);

	uvm_unmap(kernel_map, mfsp->mfs_uvmpage, round_page(bp->b_bcount));

Longer term, I'd planned to add a cache of recently mapped pages to
hopefully result in fewer calls to uvm_map/uvm_fault_wire.

This seems to work fine for the first few (around 40 to 60) calls to
mfs_strategy, then I get a

	uvm_fault(0xc996b8a0, 0x0, 0, 1) -> 1
	kernel: page fault trap, code=0
	Stopped in sync at      0:uvm_fault(0xc996b8a0, 0xbfc00000, 0, 1) -> 1
	      kernel: page fault trap, code=0
	Stopped in sync at      db_disasm+0x1b: movl    PTmap(%eax),%eax

panic at the top of uvm_fault_wire() - at the "pmap = vm_map_pmap(map);"
I guess (this is on i386 as you can guess by that last bit).  Each call
to uvm_map results in mfs_uvmpage increasing by a page or two, depending
on bp->b_bcount.

Now the question - am I doing something silly with mapping in the pages
of the aobj?

Also, will this approach result in the same TLB unfriendliness of
Jason's orignal approach of uvm_io()ing to the mount_mfs process's
address space?

I'll make the complete patches available for anyone who wants a peek.

TIA,
Simon.