Re: wedges and raidframe recovery

[Steven Bellovin <smb%cs.columbia.edu@localhost>]

On Jun 14, 2010, at 6:38 38PM, Greg Oster wrote:

> On Mon, 14 Jun 2010 18:34:01 -0400
> Steven Bellovin <smb%cs.columbia.edu@localhost> wrote:
> 
>> I'm setting up three 2TB disks in a RAID-5 array, on amd64/5.1_rc2.
>> During testing, I ran into what I suspect is a serious problem.
>> 
>> Because of the size of the disks, as best I can tell I have to use
>> wedges instead of disklabel.  After poking around and fighting it a
>> bit, I managed to set up dk0, dk1, and dk2 as the three wedges, each
>> of which is all of one disk.  I then successfully configured the
>> raid1 device across those three dk devices, and configured a fourth
>> wedge, dk3, to describe all of raid1.  I was then able to build a
>> file system:
>> 
>> # raidctl -s raid1
>> Components:
>>            /dev/dk0: optimal
>>            /dev/dk1: optimal
>>            /dev/dk2: optimal
>> No spares.
>> Component label for /dev/dk0:
>>   Row: 0, Column: 0, Num Rows: 1, Num Columns: 3
>>   Version: 2, Serial Number: 2010061200, Mod Counter: 187
>>   Clean: No, Status: 0
>>   sectPerSU: 64, SUsPerPU: 1, SUsPerRU: 1
>>   Queue size: 100, blocksize: 512, numBlocks: 3907028992
>>   RAID Level: 5
>>   Autoconfig: Yes
>>   Root partition: No
>>   Last configured as: raid1
>> Component label for /dev/dk1:
>>   Row: 0, Column: 1, Num Rows: 1, Num Columns: 3
>>   Version: 2, Serial Number: 2010061200, Mod Counter: 187
>>   Clean: No, Status: 0
>>   sectPerSU: 64, SUsPerPU: 1, SUsPerRU: 1
>>   Queue size: 100, blocksize: 512, numBlocks: 3907028992
>>   RAID Level: 5
>>   Autoconfig: Yes
>>   Root partition: No
>>   Last configured as: raid1
>> Component label for /dev/dk2:
>>   Row: 0, Column: 2, Num Rows: 1, Num Columns: 3
>>   Version: 2, Serial Number: 2010061200, Mod Counter: 187
>>   Clean: No, Status: 0
>>   sectPerSU: 64, SUsPerPU: 1, SUsPerRU: 1
>>   Queue size: 100, blocksize: 512, numBlocks: 3907028992
>>   RAID Level: 5
>>   Autoconfig: Yes
>>   Root partition: No
>>   Last configured as: raid1
>> Parity status: clean
>> Reconstruction is 100% complete.
>> Parity Re-write is 100% complete.
>> Copyback is 100% complete.
>> # df
>> Filesystem 1024-blocks       Used      Avail %Cap Mounted on
>> /dev/raid0a   302652118   29210740  258308774  10% /
>> kernfs                1          1          0 100% /kern
>> ptyfs                 1          1          0 100% /dev/pts
>> procfs                4          4          0 100% /proc
>> /dev/dk3     3787858122          2 3598465214   0% /shared
>> 
>> During some testing involving removing cables, dk2 was perceived as
>> "failed" by the RAIDfrom code.  Ah -- a perfect opportunity to test
>> recovery.  The problem is that I couldn't make it work; no matter
>> what I tried, I could not induce the kernel to start recovery on that
>> wedge.  dmesg showed complaints about being unable to open the
>> device, with 16 -- EBUSY -- as the error code.
> 
> What was the raidctl command you used?  I'd have expected
> 
> raidctl -R /dev/dk2 raid1
> 
> to have worked. (i.e. to rebuild that 'disk' in place...)

Yes, that's what I tried, without success.  It silently did nothing, even with 
-v.

                --Steve Bellovin, http://www.cs.columbia.edu/~smb