Subject: Re: NetBSD1.6 UVM problem? - Problem restated!
To: Sylvain Fontaine <sfontaine@hyperchip.com>
From: Charu Pakar <charlie007000@yahoo.ca>
List: tech-kern
Date: 12/09/2002 10:00:44
From the description it would be better to restate the
problem so that we can arrive at a generic solution.

Problem :

Irrespective of wether there is swap or not. How can
one prevent the failure of a process, due to failuer
to get real memory. Hence the use of calloc() in the
test program.

Comments :

1) Since RAM + SWAP is always finite in space, only
processes which fit into the space provided will be
supported by the OS.

2) To alleviate the problems observed all UVM
thersholds can be tweaked to move the window where the
problem occurs eg increasing target free, but cannot
be eliminated.

3) The hidden question I suspect is "How do I prevent
my process from dying due to a OS resource crunch ?".
In most systems the expected behaviour is that some
process will exit a some point in time thus release
resources which will be made available. However what
if my application is a bunch of processes which do not
exit. Under such circumstances starting another
process or an existing process would fail on a calloc
if no resources are available.

4) IMHO there is no generic kernel based solution if
(3) is valid. There has to be some sort of strategy
which is implemented across kernel+userland. When a
resource crunch is detected the kernel needs to
indicate to the processes to reduce their core size,
thereby making space of the new process. This scheme
can be evolved depending on the operational
environment of the application.

The BSD gurus please correct anything incorrect.

Charu Palkar
--

 --- Sylvain Fontaine <sfontaine@hyperchip.com> wrote:
> 
> I found a behavior with the UVM that seems to be
> wrong.
> My system has no swap space. 1G of RAM.
> 
> Here's what I see:
> - Create a file of 100M on a MFS of 100M.  Then 200M
> are removed from free,
> 100M active, 100M file.
>   I'm left with about 730M of free mem now, but with
> 100M file memory
> available when needed.
> - I start a program that calloc (not malloc) 800M. 
> I expect file memory to
> be returned to me.
>   The test program does callocation by 100M slices.
> - The page deamon does not react until its
> free_target is not under.  About
> ~~500k.
> - When the page deamon finds the free_target not to
> be met, it starts
> recovering pages, 
>   but there's a race condition freeing/allocating
> and my test app wins...
> resulting in faulting
>   and being killed by the UVM when uvm_fault returns
> ENOMEM.  
> 
> 
> I found two solutions:
> 1) In this condition, when I know there is still
> memory I can retrieve from
> file memory,
>     I call uvm_wait, and tell the pagedeamon to
> overlook the free_target and
> to free 
>     some pages because I need some, even if the
> free_target is met.
> 2) Increraese the free target to 1/20 of my memory
> ~50M... This way the race
> begins sooner
>     And the pagedeamon wins.
> 
> Sylvain
> 
>   
> 
> 
>  

=====
Charu Palkar
--
Email:charlie007000@yahoo.ca
      cpalkar@sympatico.ca

______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca