Subject: CCD performance tuning
To: NetBSD pmax <port-pmax@netbsd.org>
From: Uwe Lienig <Uwe.Lienig@fif.mw.htw-dresden.de>
List: port-pmax
Date: 01/02/2002 15:12:50
This is a multi-part message in MIME format.
--------------4FF2EDACA783F54F9E47907F
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Hi all,
first of all - Happy New Year to everybody!
I do have a question that isn't tightly related to pmax but I use NetBSD-pmax so
that's why I'm asking this question in the pmax community.
I'm running a 3MAXPLUS (DEC 5000/240) with the standard scsi host adapter and a
additional tc card with the similar chip as the standard built-in (NCR53C94).
There are 2 RZ57 and 2 RZ58 connected to the built-in scsi host adapter. The tc
card connects to 2 RZ57 and 4 RZ58. To setup the drives for maximum I/O
performance I decided to use the RZ57's for system and the like. Therefor I
partitioned the disks and allocated /, /var, /usr, /tmp and those somehow system
related filesystems to the RZ57's. The RZ58's are bundeled in a ccd to get a
large logical disk for a large filesystem.
Before putting a filesystem on this ccd I ran some tests to evaluate how the
configuration will impact on the I/O performance. I wouldn't use RAID since the
filesystem will be backed up every night so filesystem failover isn't a concern.
The RZ58's are labeled:
# /dev/rsd12c:
type: unknown
disk: rz58
label:
flags:
bytes/sector: 512
sectors/track: 85
tracks/cylinder: 15
sectors/cylinder: 1275
cylinders: 2112
total sectors: 2698061
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
8 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 20 0 unused 1024 8192 # (Cyl. 0 - 0*)
c: 2698061 0 unused 1024 8192 # (Cyl. 0 - 2116*)
d: 2698041 20 ccd # (Cyl. 0*- 2116*)
The d-partitiones are bundeled into the ccd. Then I wrote a simple script to
measure the throughput. The script is as follows:
------------------------------- Start of Script ------------------------
#!/bin/sh
PATH=/bin:/usr/bin:/sbin:/usr/sbin
exec 1>/dev/null 2>&1
: MakeCCDLabelFile ccdsize labelfile
MakeCCDLabelFile ()
{
cyl=`expr $1 '/' 2048`
logger -p daemon.info -t "$0" "Size of CCD= $1, Cylinders of CCD= $cyl"
cat > $2 << _EOF_
# /dev/rccd1c:
type: ccd
disk: ccd
label: default label
flags:
bytes/sector: 512
sectors/track: 2048
tracks/cylinder: 1
sectors/cylinder: 2048
cylinders: $cyl
total sectors: $1
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
3 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 2048 0 4.2BSD 1024 8192 16
c: $1 0 4.2BSD 1024 8192 16
_EOF_
}
CCDDEVS="/dev/sd2d /dev/sd10d /dev/sd3d /dev/sd11d /dev/sd12d /dev/sd13d"
ccdlabel=/root/ccd.disklabel
InterLeaveStart=16
InterLeaveIncrement=8
InterLeaveEnd=1024
InterLeaveCurrent=$InterLeaveStart
echo "InterLeave Times" >$HOME/OptimizeCCD.out
while [ $InterLeaveCurrent -le $InterLeaveEnd ]
do
logger -p daemon.info -t "$0" "Optimizing CCD performance: interleave=
$InterLeaveCurrent"
ccdsize=`ccdconfig -cv ccd1 $InterLeaveCurrent 0 $CCDDEVS | awk '{print
$10}'`
MakeCCDLabelFile $ccdsize $ccdlabel
disklabel -R -r ccd1 $ccdlabel
sleep 2
ccdconfig -u ccd1
sleep 2
ccdconfig -cv ccd1 $InterLeaveCurrent 0 $CCDDEVS
disklabel -R -r ccd1 $ccdlabel
sleep 2
newfs /dev/rccd1c
mount -t ffs /dev/ccd1c /mnt/ccd1
cd /mnt/ccd1
count=`expr $ccdsize '/' 10`
count=`expr $ccdsize '-' $count`
count=`expr $count '/' 512`
logger -p daemon.info -t "$0" "Writing $count block of 256k to CCD"
time=`/usr/bin/time dd if=/dev/zero ibs=256k count=$count of=/mnt/ccd1/outfile
2>&1 | sed -e '1,3d'`
echo "$InterLeaveCurrent $time" >> $HOME/OptimizeCCD.out
cd $HOME
InterLeaveCurrent=`expr $InterLeaveCurrent '+' $InterLeaveIncrement`
umount /mnt/ccd1
dd if=/dev/zero ibs=1k of=/dev/rccd1c count=1
ccdconfig -u ccd1
done
rm $ccdlabel
======================= End of Script =====================================
Basically the script creates and newfs the ccd writes a large file of zeros to
the filesystem umounts the ccd and zeros the disklabel. The ccd configuration is
done with different parameters for the interleave.
After all the test was running for nearly two weeks. This was not only a
performance test but a burn-in for the disks as well. Everything worked like a
charm with no problem.
At the end I got the results in a file. The data obtained should give me the
right impression how the interleave parameter will work regarding the I/O
performance. But I'm not sure if this procedure will lead me to the right
decision. So I'm looking forward for your comments.
The results are included as well. Fell free to interpret the results.
Thanks to everybody.
Uwe
Results of the ccd test:
InterLeave Times from 'time dd if=/dev/zero of=/mnt/ccd1/outfile ...'
16 5466.17 real 225.55 user 5171.84 sys
24 5346.58 real 221.02 user 5001.81 sys
32 5194.05 real 293.49 user 4840.44 sys
40 5080.78 real 228.25 user 4791.76 sys
48 4963.36 real 201.45 user 4703.75 sys
56 4976.27 real 203.13 user 4716.13 sys
64 4950.32 real 201.77 user 4693.63 sys
72 5031.55 real 208.39 user 4762.87 sys
80 4900.33 real 224.94 user 4620.50 sys
88 5101.53 real 201.38 user 4607.20 sys
96 4876.96 real 222.32 user 4597.14 sys
104 4912.48 real 220.09 user 4635.70 sys
112 4948.27 real 222.04 user 4667.80 sys
120 4884.45 real 224.00 user 4590.70 sys
128 5039.86 real 224.26 user 4734.84 sys
136 4854.80 real 202.74 user 4576.81 sys
144 4887.54 real 235.46 user 4586.71 sys
152 5049.50 real 218.72 user 4756.72 sys
160 4841.04 real 233.44 user 4547.84 sys
168 4833.69 real 203.77 user 4570.00 sys
176 4925.14 real 273.68 user 4592.38 sys
184 4826.60 real 202.62 user 4565.71 sys
192 4778.59 real 202.94 user 4516.97 sys
200 5004.52 real 259.45 user 4681.03 sys
208 4878.41 real 206.16 user 4611.29 sys
216 5051.96 real 219.10 user 4537.68 sys
224 4813.62 real 260.70 user 4491.69 sys
232 4809.22 real 220.70 user 4529.75 sys
240 4827.75 real 237.60 user 4526.85 sys
248 4878.86 real 219.77 user 4598.48 sys
256 4905.86 real 229.41 user 4604.39 sys
264 4935.42 real 208.16 user 4662.78 sys
272 4752.71 real 202.70 user 4489.05 sys
280 5062.92 real 214.94 user 4773.53 sys
288 4803.35 real 202.74 user 4539.51 sys
296 4935.47 real 208.99 user 4654.14 sys
304 4765.10 real 202.98 user 4501.06 sys
312 4796.90 real 200.57 user 4534.39 sys
320 4956.78 real 244.48 user 4621.62 sys
328 4827.56 real 206.29 user 4496.95 sys
336 4902.94 real 208.50 user 4629.10 sys
344 5150.95 real 249.90 user 4603.60 sys
352 4791.52 real 218.15 user 4511.84 sys
360 4744.10 real 203.39 user 4479.34 sys
368 4854.74 real 284.48 user 4507.29 sys
376 4830.23 real 198.41 user 4549.11 sys
384 4913.54 real 226.96 user 4585.87 sys
392 4777.93 real 203.39 user 4511.07 sys
400 4947.27 real 263.07 user 4621.57 sys
408 5051.07 real 300.06 user 4681.39 sys
416 4871.50 real 284.73 user 4521.41 sys
424 4751.74 real 204.01 user 4483.65 sys
432 4738.95 real 202.54 user 4473.49 sys
440 4757.50 real 202.21 user 4491.99 sys
448 4825.84 real 204.99 user 4561.25 sys
456 4997.86 real 284.92 user 4648.50 sys
464 4858.46 real 219.24 user 4574.74 sys
472 4993.50 real 202.57 user 4494.46 sys
480 4757.07 real 203.14 user 4481.00 sys
488 5140.83 real 224.05 user 4852.57 sys
496 4976.61 real 257.40 user 4655.07 sys
504 5102.97 real 235.32 user 4802.48 sys
512 4940.51 real 265.72 user 4615.38 sys
520 4769.27 real 204.87 user 4501.99 sys
528 4762.19 real 204.37 user 4495.42 sys
536 4845.34 real 224.09 user 4559.15 sys
544 4873.50 real 225.15 user 4586.03 sys
552 4984.68 real 220.66 user 4701.64 sys
560 4759.82 real 203.50 user 4491.97 sys
568 4795.30 real 201.40 user 4530.35 sys
576 4749.18 real 203.61 user 4483.86 sys
584 4748.02 real 201.72 user 4482.52 sys
592 4750.48 real 202.01 user 4483.57 sys
600 4794.47 real 222.15 user 4510.72 sys
608 5056.86 real 220.83 user 4568.00 sys
616 4932.25 real 223.90 user 4643.51 sys
624 4852.58 real 233.45 user 4555.06 sys
632 4744.05 real 202.87 user 4476.92 sys
640 4991.74 real 344.92 user 4582.99 sys
648 4794.31 real 199.47 user 4530.08 sys
656 4849.64 real 204.97 user 4579.20 sys
664 4837.88 real 202.35 user 4570.71 sys
672 4762.25 real 202.91 user 4492.04 sys
680 4932.99 real 219.99 user 4649.25 sys
688 4908.06 real 226.39 user 4616.25 sys
696 4782.56 real 222.40 user 4495.78 sys
704 4759.18 real 201.88 user 4483.22 sys
712 5031.31 real 303.82 user 4664.03 sys
720 4807.96 real 214.99 user 4528.70 sys
728 4747.34 real 205.90 user 4478.74 sys
736 4976.48 real 201.81 user 4475.49 sys
744 4761.18 real 203.62 user 4494.12 sys
752 4920.34 real 289.59 user 4565.82 sys
760 4881.64 real 273.85 user 4543.65 sys
768 5005.44 real 204.22 user 4735.62 sys
776 4746.04 real 202.19 user 4480.26 sys
784 4796.31 real 222.10 user 4509.75 sys
792 5022.64 real 202.65 user 4750.59 sys
800 4790.21 real 221.17 user 4495.83 sys
808 4826.30 real 220.89 user 4540.06 sys
816 4749.05 real 200.82 user 4483.58 sys
824 4823.03 real 221.48 user 4537.61 sys
832 4905.59 real 244.52 user 4595.58 sys
840 4862.14 real 203.51 user 4592.90 sys
848 4780.67 real 220.58 user 4496.23 sys
856 5007.13 real 318.94 user 4622.54 sys
864 5109.57 real 231.27 user 4576.40 sys
872 5012.23 real 297.95 user 4647.77 sys
880 5047.59 real 235.86 user 4745.66 sys
888 4829.78 real 199.64 user 4565.48 sys
896 4873.19 real 239.24 user 4525.77 sys
904 4798.42 real 203.43 user 4522.38 sys
912 4898.52 real 241.28 user 4589.39 sys
920 4976.93 real 219.07 user 4685.00 sys
928 4818.79 real 239.86 user 4506.29 sys
936 4781.70 real 224.14 user 4493.80 sys
944 4881.93 real 206.60 user 4613.96 sys
952 4775.18 real 223.64 user 4492.08 sys
960 4893.08 real 242.20 user 4579.64 sys
968 4748.16 real 202.75 user 4481.47 sys
976 4821.95 real 229.02 user 4530.17 sys
984 4739.91 real 203.51 user 4471.59 sys
992 5042.08 real 214.87 user 4526.68 sys
1000 4831.54 real 202.97 user 4564.06 sys
1008 4904.07 real 243.85 user 4595.76 sys
1016 4847.16 real 220.01 user 4562.26 sys
1024 4746.71 real 200.51 user 4472.36 sys
--------------4FF2EDACA783F54F9E47907F
Content-Type: text/x-vcard; charset=us-ascii;
name="Uwe.Lienig.vcf"
Content-Transfer-Encoding: 7bit
Content-Description: Card for Uwe Lienig
Content-Disposition: attachment;
filename="Uwe.Lienig.vcf"
begin:vcard
n:Lienig;Uwe
tel;fax:(+49 351) 462 3476
tel;work:(+49 351) 462 2780
x-mozilla-html:FALSE
org:Forschungsinstitut Fahrzeugtechnik -FiF-;Computer Aided Design and Advanced Simulation Technology
adr:;;Friedrich-List-Platz 1;Dresden;Saxony;01069;Germany
version:2.1
email;internet:uwe.lienig@fif.mw.htw-dresden.de
note;quoted-printable:... have you ever seen MicroSoft users smiling ...=0D=0A=0D=0ASysAdmin
x-mozilla-cpt:;-6720
fn:Uwe Lienig
end:vcard
--------------4FF2EDACA783F54F9E47907F--