Re: [GSoC 2013] Defragmentation for FFS

[Manuel Wiesinger <manuel%bsdstammtisch.at@localhost>]

On 07/24/13 12:46, Alan Barrett wrote:

> Does the defragmentation takes place while the file system is
> mounted and in use by other processes?
No. This would have been too risky for a successful GSoC project.
But, I bear in mind while coding and designing, to keep things 
extensible for an online defrag version.

I have not many technical toughs about the online version yet. Expect 
that some black magic has to be done, to redirect the I/O operations to 
the main memory, or other free parts on the hard disk. To my 
understanding, VFS has no support for things like that. So this will be 
hard.

However, I am prepared to work on that in another project. Maybe for 
another GSoC, a university project, just out of interest (when time 
allows), or even as paid job (which I don't expect).

> If I start a defragmentation operation, and then kill it (or the machine
> crashes) before the operation completes, and then I start it again, and
> it is killed before completion again, will it eventually complete?  From
> your description, I guess that if it is killed during the analysis phase
> then it will not make any progress, but if it is killed during the later
> phases then it will make progress.
Exactly. When it crashes during the analysis part, the analysis data is 
lost. I thought about saving it for a moment, but this would not make 
sense. There is no guarantee, that it has not been mounted, or changed 
in the meantime. The other (and more important) point is a fresh 
analysis just does not detect the blocks that have already been done, 
because they are already fine.

When it crashes during a later phase, it depends on the exact moment of 
the crash. My idea is like following:

1. Set the fs_clean to false.
2. copy data blocks
3. update metadata
4. flush to disk
5. Set fs_clean to true
6. continue with the next data move operation, i.e. goto 1.

When a crash happens during 1., well this is easy.
during 2: no harm, there are just some wrong blocks
during 3:
this is a bigger problem, however, when the system crashes during a 
rename(2) call, there is a way to do it, such that fsck can fix it. I'll 
probably will take the same way (something soft-update like). Although I 
have not looked into that in detail yet.

4. Also here, I'll look into what soft-updates do, to guarantee 
something fixable.

So when it crashes, you either have to restart the analysis part, or do 
an fsck(8) and start it again. My tool strictly refuses to work on 
non-clean filesystems.

Manuel