>I did some tests on the ccd partition again.  I filled it up with small 
>size files to about 90% and did rm -rf in that filesystem:


>the deletion of files and directories was pretty fast all the way down to 
>about 40% and it crawed after that... from 40% to current 7% it took over 
>3 hours now..  doesn't seem normal.

That's a bug.  A performance bug.  That's almost certainly enough to
let soemeone else  and replicate the bug, and thus find it.  But that
job would be much easier if you could do a kernel profiling run.
That would  tell us at least where the time is being mis-spent:


* take your existing kernel  config.
* create a new kernel directory.
* run `config' there with your usual arguments, but with the `-p' flag, too.
  (if you want to regularly keep around a kernel with profiling,
  adding a profile option to the config file is probably safer).

* Boot the kernel with profiling enabled.

* Run long enough to get to a point where the performance bug
  is showing up.
  In this case, it's definitely happened by the time you've
  removed 40% of the files from your ccd filesystem.

* Do kgmon -r -b when you get to that point.
  (in this case, 40% free).

* Do kgmon -p after several minutes, preferably about half an hour,
  if possible.

*  Then send the output of `gprof /netbsd gmon.out' with a
   PR describing the problem, and a description or copy  of
   your kernel config file.

That would make the problem *much* easier to debug.  Whoever looks at
this could build a profiling kernel themselves, but if there's
something odd about your setup -- like the specific performance
mismatch of disks in your CCD, or an unusual set of config options --
the problem may not be easy to duplicate.

Tom, this isn't a complaint about how you've described your bug: your
report of the performance anomaly was great and to the point.  But
maybe this will help you and others produce even better bug reports
about future performance problems.

And of course, better bug reporting means quicker fixes :).