Subject: Re: more lfs problems
To: Petter Lindquist <peli@ebi.nu>
From: Jesse Off <joff@gci-net.com>
List: current-users
Date: 11/22/2000 09:59:38
What looks like is happening here is that you're running out of memory. 
lfs_cleanerd can use lots of memory as it has to malloc() space for
every segment it reads.  It looks like your segment size is huge.  Your
output size below says segment 7 contains
almost 20 Mb of active bytes?  I would guess your segment size is at
least 32 megabytes?  That seems a little unreasonable.

It should be noted that the cpg field in your disklabel does not mean
the same thing
to newfs_lfs as it does to newfs_ffs.  Default segment size is derived
by shifting
the block size by the cpg.  For instance, if the block size in your disk
label is 8192,
and you have a cpg of 16, your segment size would be 8192 << 16, or
536870912 bytes.  Thats crazy.

The ideal segment size is very much disk dependent, but the idea is that
you want it
at a value where seek time between segments is small enough to be a
non-issue when compared to the time taken to contiguously write each
one.  

Various papers on lfs have found 384k-1M to be the ideal segment size at
the time they were written.  I personally use a 1Mb segment size, with
8192 bytes sized blocks, and 512 byte fragments. (Lfs does not have the
same limits on frag/block shift as FFS does)
On some large hard drives, you will have to use a 2mb segment size due
to a limitation
that currently exists in newfs_lfs. (although 2mb is probably more
appropriate for the larger drives anyway)

Also, there were some memory leaks in lfs_cleanerd that are going to
ship in 1.5 and are in 1.5BETA2.  If at all possible, you should use
-current sources for the LFS kernel code and userspace utilities.  There
have been a lot of fixes I've sent to Konrad since 1.5BETA2 that aren't
going to appear in 1.5.

//Jesse Off


Petter Lindquist wrote:
> 
> after some pleasant usage of lfs the cleanerd startet chewing all free cpu
> time.
> 
> peli# /usr/libexec/lfs_cleanerd -d /lfs
> lfs_cleanerd[426]: Cleaner starting on filesystem /lfs
> lfs_cleanerd[426]: Cleaner Running  at Wed Nov 22 14:23:24 2000
>  (5 of 11 segments available, avail = 5652138, bfree = 10073193)
> lfs_cleanerd[426]: Cleaning segment 7 (of 1 choices)
> lfs_cleanerd[426]: adding segment 7: contains 20911104 bytes
> lfs_cleanerd[426]: mmap_segment: malloc failed: Cannot allocate memory
> Segmentation fault (core dumped)
> 
> I guess this is because of a bad disklabel, I'll try with some more sane
> values (7 instead of 16 as it is now)
> 
> (1.5_BETA2)
> 
> --
> Pållen - http://www.astrakan.hig.se/~pollen/