Subject: Re: Hardware RAID Wrong Size
To: Chuck Swiger <cswiger@mac.com>
From: the carlos <speed_daemon@the-carlos.net>
List: netbsd-users
Date: 01/12/2007 16:19:17
Thanks Chuck!  Here's the rest.  A quick search of the mailing lists
gives me the impression that GPT will be in 4.0 (http://mail-
index.netbsd.org/source-changes/2006/10/15/0096.html)

I really would prefer not having to split my volume.  The 3ware bios
will let me split the array into 2 logical drives easily, but my
application would favor the bigger single logical drive.  If there's
no other way maybe I can get ccd to join the two logical drives...

Slightly off-topic, is backing up a huge raid array to another raid
array an acceptable backup strategy?

<<carlos>>
<<nvllvmmagnvmingenivmsinemixtvradementiae>>


# /dev/ld0d:
type: ld
disk: unknown
label: fictitious
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 272321
total sectors: 79869569
rpm: 7200
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0

0 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]



Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
2005, 2006
     The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
     The Regents of the University of California.  All rights reserved.

NetBSD 4.0_BETA (CDROM_GENERIC) #3: Wed Sep 27 20:06:29 CEST 2006
         root@vmware:/usr/src/sys/arch/i386/compile/GENERIC_CD
multiboot: Information structure flags: 0x000007e7
multiboot: Boot loader: GNU GRUB 0.97
multiboot: Command line: /netbsd.gz
multiboot: 636 KB lower memory, 2096064 KB upper memory
total memory = 2047 MB
avail memory = 2003 MB
timecounter: Timecounters tick every 10.000 msec
timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
timecounter: selected timecounter "i8254" frequency 1193182 Hz
quality 100
BIOS32 rev. 0 found at 0xf0010
mainbus0 (root)
cpu0 at mainbus0: (uniprocessor)
cpu0: AMD Unknown K7 (Athlon) (686-class), 1993.78 MHz, id 0x20f12
cpu0: features
178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu0: features 178bfbff<PGE,MCA,CMOV,PAT,PSE36,MPC,MMX>
cpu0: features 178bfbff<FXSR,SSE,SSE2,HTT>
cpu0: features2 1<SSE3>
cpu0: "Dual Core AMD Opteron(tm) Processor 270"
cpu0: AMD Power Management features: TS FID VID TTP
cpu0: AMD Cool`n'Quiet Technology 2000 MHz
cpu0: available frequencies (Mhz): 1800 2000
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
ppb0 at pci0 dev 6 function 0: Advanced Micro Devices AMD8111 I/O Hub
(rev. 0x07)
pci1 at ppb0 bus 3
pci1: i/o space, memory space enabled
ohci0 at pci1 dev 0 function 0: Advanced Micro Devices AMD8111 USB
Host Controller (rev. 0x0b)
ohci0: interrupting at irq 10
ohci0: OHCI version 1.0, legacy support
ohci0: SMM does not respond, resetting
usb0 at ohci0: USB revision 1.0
uhub0 at usb0
uhub0: Advanced Micro OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1 at pci1 dev 0 function 1: Advanced Micro Devices AMD8111 USB
Host Controller (rev. 0x0b)
ohci1: interrupting at irq 10
ohci1: OHCI version 1.0, legacy support
usb1 at ohci1: USB revision 1.0
uhub1 at usb1
uhub1: Advanced Micro OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
satalink0 at pci1 dev 5 function 0
satalink0: Silicon Image SATALink 3114 (rev. 0x02)
satalink0: 33MHz PCI bus
satalink0: bus-master DMA support present
satalink0: using irq 10 for native-PCI interrupt
atabus0 at satalink0 channel 0
atabus1 at satalink0 channel 1
atabus2 at satalink0 channel 2
atabus3 at satalink0 channel 3
vga1 at pci1 dev 6 function 0: ATI Technologies Rage XL (rev. 0x27)
wsdisplay0 at vga1 kbdmux 1: console (80x25, vt100 emulation)
wsmux1: connecting to wsdisplay0
fxp0 at pci1 dev 8 function 0: i82550 Ethernet, rev 16
fxp0: interrupting at irq 11
fxp0: Ethernet address 00:e0:81:41:74:c2
inphy0 at fxp0 phy 1: i82555 10/100 media interface, rev. 4
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
pcib0 at pci0 dev 7 function 0
pcib0: Advanced Micro Devices AMD8111 LPC Controller (rev. 0x05)
viaide0 at pci0 dev 7 function 1
viaide0: Advanced Micro Devices AMD8111 IDE Controller (rev. 0x03)
viaide0: bus-master DMA support present
viaide0: primary channel configured to compatibility mode
viaide0: primary channel interrupting at irq 14
atabus4 at viaide0 channel 0
viaide0: secondary channel configured to compatibility mode
viaide0: secondary channel ignored (disabled)
Advanced Micro Devices AMD8111 SMBus Controller (SMBus serial bus,
revision 0x02) at pci0 dev 7 function 2 not configured
amdpm0 at pci0 dev 7 function 3: Advanced Micro Devices AMD8111 ACPI
Controller (rev. 0x05)
timecounter: Timecounter "amdpm0" frequency 3579545 Hz quality 1000
timecounter: selected timecounter "amdpm0" frequency 3579545 Hz
quality 1000
amdpm0 24-bit timer
iic at amdpm0 not configured
amdpm0: random number generator enabled (apprx. 13ms)
ppb1 at pci0 dev 10 function 0: Advanced Micro Devices AMD8131 PCI-X
Tunnel (rev. 0x12)
pci2 at ppb1 bus 2
pci2: memory space enabled
bge0 at pci2 dev 9 function 0: Broadcom BCM5704C Dual Gigabit Ethernet
bge0: interrupting at irq 15
bge0: ASIC BCM5704 A3 (0x2003), Ethernet address 00:e0:81:41:75:00
brgphy0 at bge0 phy 1: BCM5704 1000BASE-T media interface, rev. 0
brgphy0: using BCM5704 DSP patch
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
bge1 at pci2 dev 9 function 1: Broadcom BCM5704C Dual Gigabit Ethernet
bge1: interrupting at irq 9
bge1: ASIC BCM5704 A3 (0x2003), Ethernet address 00:e0:81:41:75:01
brgphy1 at bge1 phy 1: BCM5704 1000BASE-T media interface, rev. 0
brgphy1: using BCM5704 DSP patch
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
Advanced Micro Devices AMD8131 IO Apic (interrupt system, interface
0x10, revision 0x01) at pci0 dev 10 function 1 not configured
ppb2 at pci0 dev 11 function 0: Advanced Micro Devices AMD8131 PCI-X
Tunnel (rev. 0x12)
pci3 at ppb2 bus 1
pci3: i/o space, memory space enabled
3ware 9550SX series: (rev. 0x00)
twa0 at pci3 dev 3 function 0: 3ware Apache
twa0: interrupting at irq 15
twa0: 8 ports, Firmware FE9X 3.04.00.005, BIOS BE9X 3.04.00.002
twa0: Monitor BL9X 3.02.00.001, PCB Rev 032 , Achip 1.70    , Pchip 1.60
twa0: port 0: WDC WD3200YS-01PGB0                      305245 MB
twa0: port 1: WDC WD3200YS-01PGB0                      305245 MB
twa0: port 2: WDC WD3200YS-01PGB0                      305245 MB
twa0: port 3: WDC WD3200YS-01PGB0                      305245 MB
twa0: port 4: WDC WD3200YS-01PGB0                      305245 MB
twa0: port 5: WDC WD3200YS-01PGB0                      305245 MB
twa0: port 6: WDC WD3200YS-01PGB0                      305245 MB
twa0: port 7: WDC WD3200YS-01PGB0                      305245 MB
twa0: AMCC    9550SX-8LP DISK 3.04D3148279946A0B0018E1
ld0 at twa0 unit 0
ld0: 2086 GB, 272321 cyl, 255 head, 63 sec, 512 bytes/sect x
4374845440 sectors
Advanced Micro Devices AMD8131 IO Apic (interrupt system, interface
0x10, revision 0x01) at pci0 dev 11 function 1 not configured
pchb0 at pci0 dev 24 function 0
pchb0: Advanced Micro Devices AMD64 HyperTransport configuration
(rev. 0x00)
pchb1 at pci0 dev 24 function 1
pchb1: Advanced Micro Devices AMD64 Address Map configuration (rev.
0x00)
pchb2 at pci0 dev 24 function 2
pchb2: Advanced Micro Devices AMD64 DRAM configuration (rev. 0x00)
pchb3 at pci0 dev 24 function 3
pchb3: Advanced Micro Devices AMD64 Miscellaneous configuration (rev.
0x00)
pchb4 at pci0 dev 25 function 0
pchb4: Advanced Micro Devices AMD64 HyperTransport configuration
(rev. 0x00)
pchb5 at pci0 dev 25 function 1
pchb5: Advanced Micro Devices AMD64 Address Map configuration (rev.
0x00)
pchb6 at pci0 dev 25 function 2
pchb6: Advanced Micro Devices AMD64 DRAM configuration (rev. 0x00)
pchb7 at pci0 dev 25 function 3
pchb7: Advanced Micro Devices AMD64 Miscellaneous configuration (rev.
0x00)
isa0 at pcib0
lpt0 at isa0 port 0x378-0x37b irq 7
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
attimer0 at isa0 port 0x40-0x43: AT Timer
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker (CPU-intensive output)
sysbeep0 at pcppi0
isapnp0 at isa0 port 0x279: ISA Plug 'n Play device support
npx0 at isa0 port 0xf0-0xff
npx0: using exception 16
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
pcppi0: attached to attimer0
isapnp0: no ISA Plug 'n Play devices found
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
Kernelized RAIDframe activated
satalink0: port 0: device present, speed: 1.5Gb/s
wd0 at atabus0 drive 0satalink0: port 1: device present, speed: 1.5Gb/s
: <WDC WD3200KS-00PFB0>
wd0: drive supports 16-sector PIO transfers, LBA48 addressing
wd0: 298 GB, 620181 cyl, 16 head, 63 sec, 512 bytes/sect x 625142448
sectors
satalink0: port 2: device present, speed: 1.5Gb/s
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd0(satalink0:0:0): using PIO mode 4, Ultra-DMA mode 6 (Ultra/133)
(using DMA)
wd1 at atabus1 drive 0: <WDC WD3200KS-00PFB0>
wd1: drive supports 16-sector PIO transfers, LBA48 addressing
wd1: 298 GB, 620181 cyl, 16 head, 63 sec, 512 bytes/sect x 625142448
sectors
wd1: 32-bit data port
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd1(satalink0:1:0): using PIO mode 4, Ultra-DMA mode 6 (Ultra/133)
(using DMA)
wd2 at atabus2 drive 0: <WDC WD3200KS-00PFB0>
wd2: drive supports 16-sector PIO transfers, LBA48 addressing
wd2: 298 GB, 620181 cyl, 16 head, 63 sec, 512 bytes/sect x 625142448
sectors
wd2: 32-bit data port
wd2: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd2(satalink0:2:0): using PIO mode 4, Ultra-DMA mode 6 (Ultra/133)
(using DMA)
atapibus0 at atabus4: 2 targets
cd0 at atapibus0 drive 0: <PIONEER DVD-RW  DVR-107D, CLDL001215WL,
1.03> cdrom removable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
cd0(viaide0:0:0): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33)
(using DMA)
uhub2 at uhub1 port 1
uhub2: Mitsumi Electric Hub in Apple Extended USB Keyboard, class
9/0, rev 1.10/1.22, addr 2
uhub2: 3 ports with 2 removable, bus powered
uhidev0 at uhub2 port 1 configuration 1 interface 0
uhidev0: Mitsumi Electric Apple Extended USB Keyboard, rev 1.10/1.22,
addr 3, iclass 3/1
ukbd0 at uhidev0
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub2 port 1 configuration 1 interface 1
uhidev1: Mitsumi Electric Apple Extended USB Keyboard, rev 1.10/1.22,
addr 3, iclass 3/0
uhidev1: 3 report ids
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
uhid1 at uhidev1 reportid 3: input=3, output=0, feature=0
uhidev2 at uhub1 port 2 configuration 1 interface 0
uhidev2: American Power Conversion Smart-UPS 750 FW:651.12.D USB FW:
4.2, rev 1.10/0.06, addr 4, iclass 3/0
uhidev2: 54 report ids
uhid2 at uhidev2 reportid 1: input=0, output=0, feature=1
uhid3 at uhidev2 reportid 2: input=0, output=0, feature=1
uhid4 at uhidev2 reportid 3: input=0, output=0, feature=1
uhid5 at uhidev2 reportid 4: input=0, output=0, feature=1
uhid6 at uhidev2 reportid 5: input=0, output=0, feature=1
uhid7 at uhidev2 reportid 6: input=0, output=0, feature=2
uhid8 at uhidev2 reportid 7: input=2, output=0, feature=2
uhid9 at uhidev2 reportid 8: input=3, output=0, feature=3
uhid10 at uhidev2 reportid 9: input=0, output=0, feature=2
uhid11 at uhidev2 reportid 10: input=0, output=0, feature=2
uhid12 at uhidev2 reportid 11: input=0, output=0, feature=2
uhid13 at uhidev2 reportid 12: input=1, output=0, feature=1
uhid14 at uhidev2 reportid 13: input=2, output=0, feature=2
uhid15 at uhidev2 reportid 14: input=0, output=0, feature=2
uhid16 at uhidev2 reportid 15: input=0, output=0, feature=1
uhid17 at uhidev2 reportid 16: input=0, output=0, feature=2
uhid18 at uhidev2 reportid 17: input=0, output=0, feature=1
uhid19 at uhidev2 reportid 18: input=0, output=0, feature=2
uhid20 at uhidev2 reportid 19: input=0, output=0, feature=3
uhid21 at uhidev2 reportid 20: input=1, output=0, feature=1
uhid22 at uhidev2 reportid 21: input=2, output=0, feature=2
uhid23 at uhidev2 reportid 22: input=1, output=0, feature=1
uhid24 at uhidev2 reportid 23: input=1, output=0, feature=1
uhid25 at uhidev2 reportid 24: input=0, output=0, feature=2
uhid26 at uhidev2 reportid 25: input=0, output=0, feature=2
uhid27 at uhidev2 reportid 26: input=2, output=0, feature=2
uhid28 at uhidev2 reportid 27: input=1, output=0, feature=1
uhid29 at uhidev2 reportid 28: input=1, output=0, feature=1
uhid30 at uhidev2 reportid 29: input=0, output=0, feature=2
uhid31 at uhidev2 reportid 30: input=0, output=0, feature=2
uhid32 at uhidev2 reportid 31: input=0, output=0, feature=2
uhid33 at uhidev2 reportid 32: input=0, output=0, feature=2
uhid34 at uhidev2 reportid 33: input=1, output=0, feature=1
uhid35 at uhidev2 reportid 34: input=1, output=0, feature=1
uhid36 at uhidev2 reportid 35: input=2, output=0, feature=2
uhid37 at uhidev2 reportid 36: input=2, output=0, feature=2
uhid38 at uhidev2 reportid 37: input=2, output=0, feature=2
uhid39 at uhidev2 reportid 38: input=2, output=0, feature=2
uhid40 at uhidev2 reportid 39: input=0, output=0, feature=2
uhid41 at uhidev2 reportid 40: input=0, output=0, feature=2
uhid42 at uhidev2 reportid 41: input=0, output=0, feature=3
uhid43 at uhidev2 reportid 42: input=0, output=0, feature=2
uhid44 at uhidev2 reportid 43: input=0, output=0, feature=2
uhid45 at uhidev2 reportid 44: input=0, output=0, feature=2
uhid46 at uhidev2 reportid 45: input=2, output=0, feature=2
uhid47 at uhidev2 reportid 46: input=0, output=0, feature=1
uhid48 at uhidev2 reportid 47: input=0, output=0, feature=1
uhid49 at uhidev2 reportid 48: input=0, output=0, feature=1
uhid50 at uhidev2 reportid 49: input=0, output=0, feature=1
uhid51 at uhidev2 reportid 50: input=0, output=0, feature=1
uhid52 at uhidev2 reportid 51: input=2, output=0, feature=2
uhid53 at uhidev2 reportid 52: input=0, output=0, feature=1
uhid54 at uhidev2 reportid 53: input=0, output=0, feature=2
uhid55 at uhidev2 reportid 54: input=1, output=0, feature=1
raid0: RAID Level 1
raid0: Components: /dev/wd1e /dev/wd2e component2[**FAILED**]
raid0: Total Sectors: 584176512 (285242 MB)
boot device: wd0
root on cd0a
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)
raid0: Error re-writing parity!
uid 0, pid 204, command dd, on /dev: file system full

On Jan 12, 2007, at 5:36 PM, Chuck Swiger wrote:

> On Jan 11, 2007, at 11:00 PM, the carlos wrote:
>> I have a 3ware 9550SX-8-LP (twa) and in netbsd 3.1 or 4.0beta amd64,
>> the 3ware setup correctly detects the size of the raid array (around
>> 2TB) in dmesg, but disklabel reports the number of sectors as being
>> far less (and doesn't let me correct the label with the dmesg
>> number).  I end up with a ~30GB (albeit oddly functional) array!  The
>> dmesg sector count is apparently slightly off, but disklabel rejected
>> the corrected number (4374836865) as well.  Attached I have what
>> disklabel -e gives me, and the obligatory dmesg.  I also have dmesg
>> output from 3.1, but since I already attached it to my other post,
>> "SATA Controllers Conflict," I figured I'd be frugal.
>
> Regretfully, the NetBSD mailing lists have had to take anti-spam
> measures which tend to strip out MIME attachments, so the output of
> those commands and your dmesg are missing.
>
> Anyway, even without that data I can say that it is likely that you
> have a RAID volume configured which is 2TB + 30GB in size, and you
> are seeing a wraparound issue over the 2TB boundary.  I don't know
> whether NetBSD supports something called GPT (for "Giant Partition
> Table"); it's a new partition table layout intended for very large
> volumes and might work better for your intended purposes if it is
> available.  Otherwise, consider splitting your RAID volume into two
> separate LUNs so that they are smaller than 2TB-- the 3ware BIOS
> ought to support doing this.
>
> --
> -Chuck
>
>