Re: Portmasters alert: was => Re: uvm physseg - put the lid on it.

[Cherry G. Mathew <cherry%zyx.in@localhost>]

>>>>> "Cherry" == Cherry G Mathew <cherry%zyx.in@localhost> writes:
    Cherry> Hi Everyone, Here's a patch to put the vm physseg stuff
    Cherry> behind bars.

    Cherry> I've made some pretty intrusive changes, esp. to the boot
    Cherry> time msgbuf stuff, so if portmasters could have a quick look
    Cherry> at this, I'll be able to go ahead and change the equivalents
    Cherry> on other ports.

    Cherry> I have a bunch of tests that actually test these functions
    Cherry> in userland via rump. It's pretty cool :-)

    Cherry> Ultimately, I want to change the backing implementation to
    Cherry> use an rb tree rather than a static array, so that the
    Cherry> plug/unplug is dynamic. Obviously, this patch is not any
    Cherry> attempt at an "API" - it's just code reorg, to make the next
    Cherry> steps easier.

    > PS: Please reply everyone for public replies, since some on the
    > Cc: list may not be subscribed to tech-kern@

    > Hi Everyone,

    > Thanks for the feedback to folks who sent them in.

    > Here's some further patches. I've broken them out for your
    > convenience:

    > i) atf(7) tests for the new consolidated "API". These are with the
    > caveat that they will change along with the implementation code,
    > when the actual unplug API hits the road. Santhosh (Cc:ed)
    > collaborated closely with me for these.

    > ftp://ftp.netbsd.org/pub/NetBSD/misc/cherry/uvm_physmem/cvs.tests.sys.uvm.diff

    > ii) uvm(9) changes. The #ifdef _KERNEL_OPT is required for the
    > rump(9) testing code (see (ii) below:

    > ftp://ftp.netbsd.org/pub/NetBSD/misc/cherry/uvm_physmem/cvs.sys.uvm.diff

    > iii) port specific changes. I have only tested i386 and amd64. I
    > have compiled all ports in these diffs.

    > *Portmasters*, it would be immensely helpfull if you could test
    > them on your respective ports please.

    > ftp://ftp.netbsd.org/pub/NetBSD/misc/cherry/uvm_physmem/cvs.sys.arch.diff

    > Please, no KNF comments yet - I will clean up in a final pass -
    > I'm really looking for testing and reporting on function (or
    > breakage!)

    > I will aim to cut a branch in CVS RSN.

    > Many Thanks, -- ~cherry

Have updated the patches with some more stuff:

i) Reverted to the current (imho broken) use of
   start/avail_start,end/avail_end in boot time segment "stealing". 
   This was to minimise divergence with the current implementation.

ii) Changed the assumption of "handles" to memory segments from mutable
    to immutable. A corresponding test was added (which fails). This is
    in preparation for changing from the current array based
    implementation to a more dynamic implementation where the handle
    pointing to backing nodes will be immutable.

iii) Introduced specific error constants for specific error conditions
     in segment node handle traversal.

iv) Fixed a bug introduced in calculating the number of backing struct
    vm_page;s allocated per segment.


I've changed my mind about cutting a new branch. The rbtree backed node
management code is wip (it passes all the userland tests, but needs boot
time memory alloc sorted ). I will cut the new branch with the rbtree
code. This set of patches will have tests which will very likely remain
unchanged wrt the newer implementation with rbtrees.

Santhosh is now working on managing backing pgs[] in the case of
unplug(). This may involve some changes to extent(9). We're gravitating
towards using extent(9) instead of vmem(9) since it provides a much more
flexible backend memory allocation (the caveat of alloc on fragmenting
free()s can be addressed by using pool(9), iiuc right now - still under
design)

Those are my unfinished thoughts for now.

Cheers,

Cherry.