Subject: Re: LFS(?) ioflush/vnlock issues
To: Bill Stouder-Studenmund <wrstuden@netbsd.org>
From: Anil Gopinath <anil_public@yahoo.com>
List: tech-kern
Date: 03/09/2007 09:43:23
We are running into a similar issue with LFS where
ioflush is taking up a lot of cpu. I obtained the
following profiled information when this was
happening. Is there a scope for optimization?
Note: the disk write cache is turned off
index % time self children called name
<spontaneous>
[1] 64.5 19.11 0.00 Xspllower
[1]
-----------------------------------------------
<spontaneous>
[2] 28.3 0.02 8.37
lfs_putpages [2]
2.47 4.43 6893704/6893704
check_dirty [3]
0.01 1.11 2156/2273
lfs_writeseg [15]
0.06 0.18 6893704/6914521
preempt [28]
0.01 0.08 2156/2156
genfs_putpages [43]
0.00 0.02 2156/2273
lfs_updatemeta [105]
0.00 0.00 2156/2273
lfs_release_finfo [665]
0.00 0.00 2156/2273
lfs_acquire_finfo [662]
-----------------------------------------------
2.47 4.43 6893704/6893704
lfs_putpages [2]
[3] 23.3 2.47 4.43 6893704
check_dirty [3]
1.52 0.14 74835280/75136474
pmap_clear_attrs [12]
1.39 0.00 81726828/81849018
uvm_pagelookup [13]
1.37 0.01 74835280/74835674
pmap_page_remove [14]
-----------------------------------------------
[4] 7.8 0.02 2.29 10181+30552 <cycle 2
as a whole> [4]
0.01 2.25 5092
cgdstart <cycle 2> [5]
0.00 0.03 10184
spec_strategy <cycle 2> [88]
0.01 0.00 10184
VOP_STRATEGY <cycle 2> [171]
0.00 0.00 10181
dk_start <cycle 2> [569]
-----------------------------------------------
5092
dk_start <cycle 2> [569]
[5] 7.6 0.01 2.25 5092 cgdstart
<cycle 2> [5]
0.00 2.25 5092/5092
cgd_cipher [8]
0.00 0.00 2930/8764
malloc [196]
0.00 0.00 5092/10182
iostat_busy [568]
0.00 0.00 5092/10182
disk_busy [567]
0.00 0.00 5092/16110
getiobuf1 [556]
0.00 0.00 5092/5092
getiobuf_nowait [612]
0.00 0.00 2156/28410
getmicrouptime [524]
5092
VOP_STRATEGY <cycle 2> [171]
-----------------------------------------------
0.00 2.25 260578/260578
cgd_cipher [8]
[6] 7.6 0.00 2.25 260578
cgd_cipher_aes_cbc [6]
0.01 2.24 260578/260578
cgd_cipher_uio_cbc [7]
-----------------------------------------------
0.01 2.24 260578/260578
cgd_cipher_aes_cbc [6]
[7] 7.6 0.01 2.24 260578
cgd_cipher_uio_cbc [7]
0.01 2.23 521156/521156
aes_cbc_enc_int [9]
-----------------------------------------------
0.00 2.25 5092/5092
cgdstart <cycle 2> [5]
[8] 7.6 0.00 2.25 5092
cgd_cipher [8]
0.00 2.25 260578/260578
cgd_cipher_aes_cbc [6]
0.00 0.00 265670/407642
memset [454]
-----------------------------------------------
thanks,
Anil
--- Bill Stouder-Studenmund <wrstuden@netbsd.org>
wrote:
> On Thu, Mar 08, 2007 at 08:34:21PM +1100, Paul Ripke
> wrote:
> > On Thu, Mar 01, 2007 at 08:19:49PM +0100, Edgar
> Fu? wrote:
> > > I've no idea whether this problem is actually
> LFS related or
> > > I was just hitting a coincidence.
> > >
> > > We have performance problems on our mySQL
> server. As analysis
> > > showed that disk I/O, in particular seeks, were
> the problem,
> > > I thought it might be worth to try LFS. So we
> dumped the
> > > database, set up an LFS partition on a
> development server and
> > > fed the dump into mysql. First, everything went
> as expected
> > > (mysqld being CPU bound). Then, as most of the
> database seems
> > > to have been written, we observed ioflush
> consuming 100% CPU
> > > load. From then on, any attempt to access the
> LFS partition
> > > (ls, du) locked up in vnlock state.
> >
> > IMHO: LFS really isn't suitable for databases -
> every random
> > write operation can fragment the file on disk. Do
> a few
> > thousand random writes and the database files will
> be very
> > fragmented.
> >
> > While it'll still all work, I'm sure this can't be
> efficient.
>
> Yes. FFS and LFS each have different "bad"
> workloads. For LFS, one is
> overwriting blocks in files. That's exactly what a
> database does a lot.
> :-(
>
> Take care,
>
> Bill
>
____________________________________________________________________________________
Don't pick lemons.
See all the new 2007 cars at Yahoo! Autos.
http://autos.yahoo.com/new_cars.html