Subject: Re: My experience with the _RCs so far
To: None <port-i386@NetBSD.org>
From: Chapman Flack <flack@cerias.purdue.edu>
List: port-i386
Date: 10/10/2004 10:29:49
Snipped from many messages, many posters:

> Finding the roots of the umass problem will be hard (it's memory
> fragmentation)

-snip-

> What does any of this have to do with umass? This discussion is about
> "umass0: no buffer memory" when plugging in USB mass storage devices,
> not Sigmatel IrDAs or ath2s.

-snip-

> A temporary fix isn't that hard, and can be implemented in a day.
> Aallocate a chunk of contiguous memory (using bus_dmamem_alloc)
> directly at attach time in uhci. If a later memory allocation
> ...
> Now, this is not guaranteed to work, you could still attach an
> 8-port hub and attach an umass device to each port. But that's
> not a very likely scenario.

I'm concerned that there's a discussion here on port-i386 that is duplicating
a quite recent discussion on tech-kern but is overlooking information from
that earlier thread.

For starters, it's right to expand the discussion beyond umass, because the
underlying issue is not umass-specific, but affects every kind of device that
may come along and request a chunk of contiguous memory.

Finding the root of the problem isn't hard.  The memory manager needs to be
able to page out or relocate mapped pages that are in the way of a kernel
contiguous allocation request.

Reserving in advance a chunk of contiguous memory is an acceptable short-term
fix, but not if it is limited only to uhci (because the _problem_ _isn't_),
and not if it is based on guesses about what is a "likely" configuration.
If a hard-coded reservation is used, there needs to be a way for the user
to specify the size of the reservation, most simply via options(4).  That
means while everything may not work perfectly out-of-the-box for every user,
every user does have a perfectly straightforward way of making it work
correctly regardless of whether his or her configuration is typical or
unusual.  The default setting of the option, now, could be based on typical
expectations.  To assist the user in sizing the reservation, it would be
helpful if allocation failure messages included the requested size.

As soon as there is a configurable option and a wired-down contiguous
reservation, the problem has been addressed.  Boom.  What's left is further
refinement of the solution.  Can the default reservation be sized more
intelligently, perhaps by figuring in device counts from the configuration?
Should the MM be able to map read-only pages in the reserved space, which
can be quickly discarded without flushing, so that the space is not wasted
before it is needed?  Should the MM be able to map writable pages there and
relocate them when needed?  How much MM work is tolerable at the spl's of
the affected attach routines?  If either a read-only or read-write soft
reservation policy is workable, is it feasible to make default reservations
so large that user configuration is hardly ever needed, and still get good
performance?  All those questions and more can be tried out as later
refinements of the basic idea of reserving a chunk in advance.  But the
most primitive form--just reserve a chunk of config-specified size, and
don't use it for anything else--should be quick to put in place, and it
doesn't even have to be called a temporary hack.  It's an obvious member
of the solution family that includes the more sophisticated ideas, and
it can grow up into them over time.

-Chap