Subject: Fwd: A JBD file-system (generic Journal file for ext2)
To: None <tech-kern@netbsd.org>
From: Adam Hamsik <haaaad@gmail.com>
List: tech-kern
Date: 09/20/2007 01:20:36
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm moving this to public tech-kern because there are better  
developers than me.


>
>
> Hello,
>       Could you send me the browsing links again because like a  
> moron I deleted
> the email with the links you sent me for the project....Also didn't  
> bookmark
> those.
>
> Another thing was I was planning on starting to code a,
>
> 1.) Multi-threaded implementation of ufs_tans.. where by a copy of  
> pending
> transactions are made at commit and ufs_add is allowed to proceed  
> from fs.
> While commit takes place concurrently. This is where its easier to  
> have the
> commit happen by the ufs_trans itself rather than the fs because,  
> if fs is
> making the commit then it needs to be multi-threaded doesn't it  
> since the user
> apps are asking for stuff at the same time a commit needs to be  
> made, which
> further increases complexity, I see this as the main reason for  
> linux to make
> all commits from the jbd itself (/usr/src/fs/jbd/commit.c).
>
> 2.) I also will look at the check-pointing code this I think is  
> much easier to
> deal with (implement). Cause, once the the commits are all in it is  
> just a
> matter of checking in ufs_add if the asked for number of blocks for  
> the
> transactions are exceeding the journal blocks available and if so  
> hold the
> adding for now.. snip the tail of the log and flush it onto the  
> disk..at the
> app places. Same during un-mounting except no checking required for  
> number of
> blocks exceeding the number of blocks free in journal.
>
> Please note: Currently I am not considering the scenarios where a  
> required
> transaction (like write,delete) which might require huge number of  
> blocks
> always exceeding the journal block limit would need to be broken  
> down into
> smaller transactions.
>
> regards,
>
> Quoting Adam Hamsik <haaaad@gmail.com>:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>>
>> On Sep 19, 2007, at 4:10 AM, Avinash Malik wrote:
>>
>>> Hello,
>>>       I have been going through the source and have a few questions,
>>> 1.) Current ufs_trans implementation uses number of trans (10) to
>>> make a commit.
>>> Linux uses a timer to do the same. Is there a reason for this  
>>> change?
>> No I think that this is type of easy to write and test change. ufs-
>> trans is WIP and therefore is not done yet. I think that we can go
>> with linux approach but we have to keep GPL out of our kernel. So be
>> careful with re-implementing linux code.
>>> 2.) During commit stage in ufs_commit the transaction obtains a
>>> lock, thereby
>>> preventing any further changes to the transaction-list by fs. Linux
>>> (I think)
>>> (either) makes a copy of the transaction list before committing it
>>> (or) makes a
>>> new transaction list. This allows for less unnecessary delays for  
>>> the
>>> file-system. As the fs can still make calls (ufs_add) to the
>>> journal while
>>> journal is committing. Are there plans to take the linux approach?
>> This looks for me like RCU. I don't know if we have RCU
>> implementation in our kernel.
>> We can discuss this here :) I'm not expert for this.
>>
>>> 3.) The commit functionality in NETBSD is completely implemented by
>>> each fs
>>> rather than by the ufs_commit. Why is that, commit would involve
>>> flushing the
>>> meta-data onto the log and then writing the sector-tag in the log
>>> which is same
>>> for all the file systems isn't it.
>> But there are differences between log types. FFS can use other
>> journal architecture then JBD/EXT3. But as I already said, when we
>> will have functional ext3 :)(or at least we will understand to log) I
>> will separate jbd code from ext2 and create ufs-journal library with
>> support for more than one type of journal.
>>
>>> 4.) Lastly, by check-pointing I meant flushing the log into the
>>> real disk
>>> places. In linux this is done when fs calls journal requiring
>>> number of blocks
>>> which exceeds the number of current blocks available in the
>>> journal. Or at
>>> unmount point. Also in linux since the log is flushed onto the disk
>>> only at
>>> these points, only the newest meta-data for each block-group needs
>>> to be
>>> flushed (at least in the default writeback mode), I am unable to
>>> see this
>>> functionality anywhere in source. What am I missing.
>> After I read that kerneltrap article i understand what checkpointing
>> means.
>> This is missing I think. We haven't implemented this yet:).
>>
>>>
>>> Can I download the source for the branches from this website. I
>>> have never
>>> before used monotone so any guidance would be appreciated. Is it
>>> like git?
>>>
>> Do you use netbsd ? or what OS. We have monotone[1] in pkgsrc devel/
>> monotone.
>> It is distributed source code management tool.[2]
>>
>> for download:
>>
>> mtn db init --db [path to db]
>> mtn genkey [your mail addr or somethink@hostname]
>> mtn pull ufs-trans.mtn-host.prjek.net [\* for all or branch name for
>> branch source]
>> mtn checkout -b [branch name] directory
>>
>>>
>>> [snip]
>>
>> [1] monotone.ca
>> [2]http://www.venge.net/mtn-wiki/EssentialMonotone
>> Regards
>> - -----------------------------------------
>> Adam Hamsik
>> jabber: haad@jabber.org
>> icq: 249727910
>>
>> Proud NetBSD user.
>>
>> We program to have fun.
>> Even when we program for money, we want to have fun as well.
>> ~ Yukihiro Matsumoto
>>
>>
>>
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.7 (Darwin)
>>
>> iD8DBQFG8QBPlIxPgX3Go0MRAtE5AKD2pcdl5GeiW4ZQPEdFxcOBfUhlHgCglE+N
>> s6/QhWrYdc5l/b+3LiUHrAg=
>> =df1i
>> -----END PGP SIGNATURE-----
>>
>
>
>

Regards
- -----------------------------------------
Adam Hamsik
jabber: haad@jabber.org
icq: 249727910

Proud NetBSD user.

We program to have fun.
Even when we program for money, we want to have fun as well.
~ Yukihiro Matsumoto




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iD8DBQFG8a7ElIxPgX3Go0MRArNXAKCr0ndw7KL01AxGWdAgjMDdymChuQCfYAY/
9tIVU2WsUqkTzyhEZFn6bvU=
=uSnD
-----END PGP SIGNATURE-----