join project on Improving RAIDframe parity check after unclean shutdown.

[鏖马 <maao129%gmail.com@localhost>]

Hello guys,
I'm a master student majoring in File and Storage System at Institute of Computing Technology, Chinese Academy of Sciences. I am proposing to work in the open source project on Improving RAIDframe parity check after unclean shutdown. Here, I introduce my comprehension and plan for such project.

Goal: 
The sudden shut down would cause the incoherence of data and parity in RAID system. The traditional method is to check the whole disk and recalculate the incoherence parity and replace the old one. Our aim is to use the bitmap to record these recent modified stripes and only check this area after shutdown.

My rough Plan:
We use bitmap to represents stripe as log-info to reduce the cost. The log-info is stored in front of each disk, just behind the component label.  We need to modify the RAID in several ways:
I/O operation
When a new write come, it first sets the corresponding bitmap to 1 in memory. Only after the bitmap info is flushed down to disk, can the data I/O be handled with.
After the data is flushed down to disk, the bitmap is set to 0 in memory.
After a certain time or some thing triggers, the deamon flush the modified bitmap info to disk.
Memory
We allocate some memory to store bitmap. Therefore, there are always two types of bitmap in memory: the used ones, the free ones.
We use a deamon to release the used bitmap. I mean, the trigger condition for flush bitmap is: a timer and a threshold of left number of free bitmap.
We could use LRU or LFU to flush bitmap.


There are also several points we could discuss and make some Optimizations:
1.bitmap flush: when a new write comes, it requests to set the bitmap to 1 and flush bitmap down to disk. We could flush down several bitmap as 1 on the assumption that write in most cases comes in sequence.
2.Since the write could not be handled until get the call back of bitmap flush, we could try to flush bitmap to another different disk from the target disk for I/O data. Because the bitmap info is always set at the front of disk. Then a flush of bitmap and data for the same disk cause a significant time loss due to disk head rotational delay.
3.Tradeoff for the algorithm of bitmap replacing:  choose to flush the bitmap page with most 1 means save much time when check the incoherence. On the other hand, that bitmap also means a frequent access right now. So there are much interesting things to do. I suggest we just implement the simplest LFU and LRU algorithm and optimize that if time is enough.

Get rid of this, I'm wondering whether we could go further to enhance the reliability of RAID.  I mean since the probability of latent sector error is much higher than it is imagined, the traditional RAID system suffers sever data corruption at RAID reconstruction. A new method called Scrub has been proposed to address such problem. Unlike traditional RAID passively restores data after some disks are failure, scrub uses read operation to find sector error and use RAID mechanism to restore data promptly, therefore reducing the hazard of more than one error in a single RAID stripe. I suggest adding such mechanism into our RAID system.
This original plan may change after our further discussion.

On the other hand, I am fairly confident about my quantitative background: experienced Linux kernel programming, a thorough understanding of VFS, comprehensive OS and mathematical background several project experiences in block level. All of these can be ways of help, and you may see this from my CV blow.

Since I am graduating this summer and will continue my Ph. D study at U.S, there is a vacancy till the opening ceremony in September. So I plan to join in this project. 



Ao Ma

PERSONAL DETAILS
 
Address:    
Institute of Computing Technology,
Chinese Academy of Sciences,
No. 6 South Kexueyuan Road,
Beijing, China
Zip Code:    100190
Gender:        Male
Date of Birth:    10/23/1983
Citizenship:    China
E-mail:        maao129%gmail.com@localhost
Phone:        +86-13426305760
 


EDUCATION
September 2006 – present:
M.S. in file and storage systems (expected in 06/2009)
National Research Center for High Performance Computer
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
Thesis: "High-reliability RAID system"

September 2002 – July 2006:
B.S. in School of Computer Science and Engineering
University of Electronic Science & Technology of China, Chengdu, China
GPA:     major 89/100        overall 85/100
Dissertation: "Design and implement Hidden Markov model-based speech recognition"
    
HONORS & AWARDS
1.The only student who is admitted by ICT, Chinese Academy of Sciences, with entry examination waived for excellent academic achievements in college
2.Excellent Bachelor’s Thesis Award                                 2006
3.Outstanding Graduate                                                       2006 
4.The First-class scholarship, top 3%                                 from 2004 to 2005
5.The Second-class scholarship, top 10%                         from 2003 to 2004
6.The Special-class scholarship, 3 out of 558 students    from 2002 to 2003

PUBLICATIONS
1.Ao Ma, Yang Yin, Wenwu Na, Qingzhong Bu and Lu Xu “Scrubbing in virtualization platform for long-term backup application”  Accepted by ARES’2009
2.Yang Yin, Ao Ma, Zhenjun Liu, Shuo Feng and Lu Xu “C-Aware: A cache management algorithm considering cache media speed characteristic”  In preparation to submit


PROJECTS 
1. High-reliability RAID system             August 2008 - now
I combined disk scrub for latent sector error and intra-disk redundancy to enhance the reliability of RAID system. I also proposed a novel deterministic scrub strategy basing on the spatial and temporal locality of latent sector error. A new model was developed to evaluate the scrubbing effect.

2. A block-level snapshot system        November 2007 - July 2008
A snapshot system including COW, ROW and writable snapshots was implemented as a kernel driver for Linux 2.6.18 version. My responsibility was to implement the COW, writable snapshots, the copy-back thread for ROW and the metadata swap between memory and disk with LRU and LFU.
This production was purchased by PROWARE, a storage Corp. in Taiwan.

3. Storage virtualization in SAN environment    March 2007 - October 2007
A typical storage virtualization system called Allocate-on-demand Storage Device (ASD) was designed and implemented for the SAN environment as a kernel driver for Linux 2.6.18 version. My responsibility was to implement the physical resource management, including multiple resource pools, a B+ tree for search and the parts of ASD in user-mode. 

TECHNICAL SKILLS
Programming: C++/C, Perl
Operating system: Linux, Windows XP/2000/NT, UNIX (AIX, Solaris)
Science software: MATLAB, LaTeX

AFFILIATIONS
Member of CCF Young Computer Scientists & Engineers Forum for Graduate Students
Reviewer for ICACT’2009

ENGLISH ABILITY
GRE          1350    Quantitative 800; Verbal 550; writing 4.0            
TOEFL IBT      98    Reading 28; Listening 28; Speaking 18; Writing 24


-- 
Ao Ma
email:    maao129%gmail.com@localhost
phone:   86-13426305760
National Research Center for High Performance Computers
Institute of Computing Technology
Chinese Academy of Sciences