installing into RAID-1: HOWTO and example

To: port-xen%netbsd.org@localhost
Subject: installing into RAID-1: HOWTO and example
From: Greg Troxel <gdt%ir.bbn.com@localhost>
Date: Wed, 01 Mar 2006 20:45:58 -0500
I recently set up a machine with 2 x 400G SATA disks in RAID-1 with
Xen.  Previous postings were helpful, and I tried to document what I
did carefully.

I learned that when one is going to use Xen, and hence at least for
now grub, that one should not fight the PC way and use disklabel with
no fdisk partitions.  While I hear one can make grub put the stage1.5
elsewhere, the normal place is in the slack sectors following the MBR.

This example has overlay partitions for root (for booting with grub)
and swap (for dumps), and /usr and /n0 (to make sure it all lines up).
swap in RAID is small (0.75+G ) since my dom0 will have limited
memory.  This machine is dedicated to Xen and will have a half-dozen
domUs.

I have not tested all the boot options.


$Id: adroit-xen.txt,v 1.6 2006/03/01 21:03:41 gdt Exp $

references:

http://mail-index.netbsd.org/port-xen/2005/11/02/0006.html
http://mail-index.netbsd.org/port-xen/2005/11/10/0000.html



----------------------------------------
Intel 915 chipset with 4GB RAM

BIOS set to limit VGA memory reservation to smallest values
need to investigate why 768 is reserved (pci hole?) after xen

2 400G SATA drives

stock NetBSD-current

RAID-1 on two drives

----------------------------------------
/etc/raid0.conf
--------------------
START array
1 2 0

START disks
/dev/wd0a
/dev/wd1a

START layout
128 1 1 1

START queue
fifo 100
--------------------
PHYSICAL DISK LABELING (wd0, wd1 same)

PLAN:

one DOS primary partition (grub needs dos partitioning)

a: whole disk within partition (63-)
c: NetBSD portion: same as a
d: whole disk (0-)
# boot, dump, sanity check, sanity check
e: raid0a
f: raid0b
g: raid0e
h: raid0f

ACTUAL

#> fdisk wd0
Disk: /dev/rwd0d
NetBSD disklabel disk geometry:
cylinders: 775221, heads: 16, sectors/track: 63 (1008 sectors/cylinder)
total sectors: 781422768

BIOS disk geometry:
cylinders: 1023, heads: 255, sectors/track: 63 (16065 sectors/cylinder)
total sectors: 781422768

Partition table:
0: NetBSD (sysid 169)
    start 63, size 781422705 (381554 MB, Cyls 0-48641/81/1)
1: <UNUSED>
2: <UNUSED>
3: <UNUSED>

#> disklabel wd0
# /dev/rwd0d:
type: RAID
disk: raid
label: fictitious
flags:
bytes/sector: 512
sectors/track: 128
tracks/cylinder: 8
sectors/cylinder: 1024
cylinders: 763108
total sectors: 781422592
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0 

8 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a: 781422529        63       RAID                     # (Cyl.      0*- 763107)
 c: 781422705        63     unused      0     0        # (Cyl.      0*- 763108+)
 d: 781422592         0     unused      0     0        # (Cyl.      0 - 763107)
 e:    491520       127     4.2BSD   2048 16384 25648  # (Cyl.      0*-    480*)
 f:   1585152    491647       swap                     # (Cyl.    480*-   2028*)
 g:   8388608   2076799     4.2BSD      0     0     0  # (Cyl.   2028*-  10220*)
 h: 770957184  10465407     4.2BSD      0     0     0  # (Cyl.  10220*- 763107*)
disklabel: partitions a and e overlap
disklabel: partitions a and f overlap
disklabel: partitions a and g overlap
disklabel: partitions a and h overlap
disklabel: partition c: partition extends past end of unit

----------------------------------------
RAID0 DISKLABEL

* PLAN
/    240M # starting at 0
swap 774M
/usr 4G
/n0  rest (kernel build, xen images)

* ACTUAL

#> disklabel rai0
# /dev/rraid0d:
type: RAID
disk: raid
label: fictitious
flags:
bytes/sector: 512
sectors/track: 128
tracks/cylinder: 8
sectors/cylinder: 1024
cylinders: 763107
total sectors: 781422464
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0 

6 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:    491520         0     4.2BSD   1024  8192 40960  # (Cyl.      0 -    479)
 b:   1585152    491520       swap                     # (Cyl.    480 -   2027)
 d: 781422464         0     unused      0     0        # (Cyl.      0 - 763107*)
 e:   8388608   2076672     4.2BSD      0     0     0  # (Cyl.   2028 -  10219)
 f: 770957184  10465280     4.2BSD   2048 16384 28848  # (Cyl.  10220 - 763107*)


----------------------------------------
/grub/menu.lst:

# Grub config file for NetBSD/xen.
default=0
fallback=4
# boot after 10s the default entry if the user didn't hit keyboard
timeout=10
# use vga
terminal console
#
# Xen/NetBSD with 256 MB
title Xen 2.0 / NetBSD - WD0
  root(hd0,e)
  kernel (hd0,e)/xen.gz dom0_mem=262144
  module (hd0,e)/netbsd-XEN0.gz root=/dev/hda1 ro console=pc
title Xen 2.0 / NetBSD.ok - WD0
  root(hd0,0)
  kernel (hd0,e)/xen.gz dom0_mem=262144
  module (hd0,e)/netbsd-XEN0.ok.gz root=/dev/hda1 ro console=pc
title Xen 2.0 / NetBSD - WD1
  root(hd1,e)
  kernel (hd1,e)/xen.gz dom0_mem=262144
  module (hd1,e)/netbsd-XEN0.gz root=/dev/hda1 ro console=pc
title Xen 2.0 / NetBSD.ok - WD1
  root(hd1,0)
  kernel (hd1,e)/xen.gz dom0_mem=262144
  module (hd1,e)/netbsd-XEN0.ok.gz root=/dev/hda1 ro console=pc
#
# Load the NetBSD bootloader, letting it load the NetBSD/i386 kernel.
title NetBSD chain - WD0
  root (hd0,0)
  chainloader +1
title NetBSD chain - WD1
  root (hd1,0)
  chainloader +1
#
# Regular NetBSD - does not get boot options
title NetBSD - WD0
  root (hd0,e)
  kernel --type=netbsd /netbsd-GENERIC.gz
title NetBSD - WD1
  root (hd1,e)
  kernel --type=netbsd /netbsd-GENERIC.gz
#
# Regular NetBSD - does not get boot options
title NetBSD.ok - WD0
  root (hd0,e)
  kernel --type=netbsd /netbsd-GENERIC.ok.gz
title NetBSD.ok - WD1
  root (hd1,e)
  kernel --type=netbsd /netbsd-GENERIC.ok.gz
## end of grub config file.
----------------------------------------

GRUB setup

# mkdir /grub
# cp -p /usr/pkg/lib/grub/i386-/* /grub

setup menu.lst as above

# grub --no-floppy

grub> root (hd1,e)
 Filesystem type is ffs, partition type 0xa9

grub> setup (hd1)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/ffs_stage1_5" exists... yes
 Running "embed /grub/ffs_stage1_5 (hd1)"...  14 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd1) (hd1)1+14 p (hd1,0,e)/grub/stage2 
/grub/menu.lst"...
 succeeded
Done.

grub> root (hd0,e)
 Filesystem type is ffs, partition type 0xa9

grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/ffs_stage1_5" exists... yes
 Running "embed /grub/ffs_stage1_5 (hd0)"...  14 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd0) (hd0)1+14 p (hd0,0,e)/grub/stage2 
/grub/menu.lst"...
 succeeded
Done.

----------------------------------------
In domU: fstab

# $Id: adroit-xen.txt,v 1.6 2006/03/01 21:03:41 gdt Exp $
/dev/raid0a     /       ffs     rw              1 1
/dev/raid0b     none    swap    sw              0 0
/dev/raid0b     /tmp    mfs     rw,-s=1049328
/dev/raid0e     /usr    ffs     rw              1 2
/dev/raid0f     /n0     ffs     rw              1 3
/dev/wd0f       none    none    dp      
kernfs          /kern   kernfs  rw
procfs          /proc   procfs  rw,noauto
/dev/cd0a       /cdrom  cd9660  ro,noauto

----------------------------------------
In all domU, /etc/rc.conf has powerd=YES to invoke shutdown from
shutdown of dom0.
Follow-Ups:
- Re: installing into RAID-1: HOWTO and example
  - From: Manuel Bouyer
- Re: installing into RAID-1: HOWTO and example
  - From: Florian Heigl
Prev by Date: Re: BUFQ_FCFS for XENU kernels?
Next by Date: Re: installing into RAID-1: HOWTO and example
Previous by Thread: BUFQ_FCFS for XENU kernels?
Next by Thread: Re: installing into RAID-1: HOWTO and example
Indexes:
Home | Main Index | Thread Index | Old Index