--oplxJGu+Ee5xywIT
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Jan 17, 2007 at 06:05:19AM +0100, Nino Dehne wrote:
> 3) I'm on SMP kernel again. Start 2 instances of
>=20
>       gzip -9c </dev/zero | gzip -dc >/dev/null
>=20
>    i.e. 4 gzip processes are running.
> 4) Additionally, run
>=20
>       while true; do dd if=3D/dev/rcgd0d bs=3D65536 count=3D1024 2>/dev/n=
ull | md5
>=20
>    and
>=20
>       while true; do dd if=3D/dev/rcgd0d bs=3D65536 count=3D1024 skip=3D1=
23456 2>/dev/null | md5
>=20
>    concurrently. After 100 runs each, not a single mismatch occurred.

Hmm.  I can only really suspect the filesystem at this point.  How large is=
 it?

>    After that, I killed the second dd and tried different blocksizes
>    but was faced with serious trouble:
>=20
>    While the first dd was running I tried a
>=20
>       dd if=3D/dev/rcgd0d bs=3D123405 count=3D1024 skip=3D56454
>=20
>    This gave me:
>=20
>    dd: /dev/rcgd0d: Invalid argument

Yeah, bigger than MAXPHYS on a raw device, probably. =20

>    Then I tried
>=20
>       dd if=3D/dev/rcgd0d bs=3D32000 count=3D1024 skip=3D56454 | md5
>=20
>    This panicked the box!

Oops. Not good at all but probably a separate issue (non power-of-two
read from raw device).

>    cgd0: error 22
>    uvm_fault(0xc03c62c0, 0xca596000, 1) -> 0xe

EINVAL, followed by something else that hasn't taken an error path
=66rom there.  All yours, Roland :)

> 5) A watt meter showed 175W usage during 2)-4) for a whole bunch of
>    hardware including the server. The hardware minus the server is
>    drawing ~42W, i.e. the server was drawing around 133W during these
>    tests. The power supply is only some weeks old and is a bequiet
>    BQT E5-350W.
>=20
> After I was done with that, I mounted cgd0a and hashed the usual file
> in a loop. Result: mismatch at the 3rd try. This was on an idle box, i.e.
> no 3) or 4) running.

Hrm.  I suspect the filesystem at this point, since you seem to have
eliminated seeking power and the cgd device.

Can you tell us more about the fs?  ufs1, ufs2? lfs? :-)
What size, block and frag size, etc?

Another thing you might do is compare sucessive dump(8)s of the
filesystem. =20

> >  dd from raid and a concurrent fsck -n of the cgd filesystem
> >=20
> >  multiple concurrent fsck -n's, to see if they ever report different
> >  errors.  -n is especially important here, both because of the
> >  concurrency and if they're going to find spurious errors
>=20
> This was actually not possible:
>=20
> # fsck -n /home
> ** /dev/rcgd0a (NO WRITE)
> ** File system is clean; not checking
> # fsck -pn /home
> NO WRITE ACCESS
> /dev/rcgd0a: UNEXPECTED INCONSISTENCY; RUN fsck_ffs MANUALLY.

Oh, you wanted -fn not -pn.

> I also don't want to mess with the filesystem further. I'm already trying
> to minimize access to it in fear of permanent corruption.

Valid fear, and hence -n.

> Now I'm not sure what to make of this. The cgd/raid panic looks creepy but
> I'm not sure how to interpret it.
>
> Does this help you?
>=20
> In either case, thanks a lot for your help and best regards,

You're most welcome, and have found at least one concrete problem
already for your methodical efforts.  I'm very curious now what the
other problem might be, keep at it..

--
Dan.

--oplxJGu+Ee5xywIT
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (NetBSD)

iD8DBQFFrbOsEAVxvV4N66cRAjbJAJ9/FjIlhtta+Yp5hZL3DngeKUSIUgCg61X1
dRQZlzIo2L1gW8AuufubbYE=
=RExs
-----END PGP SIGNATURE-----

--oplxJGu+Ee5xywIT--