Subject: Re: Fwd: A JBD file-system (generic Journal file for ext2)
To: Adam Hamsik <firstname.lastname@example.org>
From: Avinash Malik <email@example.com>
Date: 09/20/2007 16:36:59
after thinking about the various journaling requirements this is what I can
come up with, correct me if I am wrong
1.) We still call the commit process from ufs_commit to fs_commit, but on a
writer thread. But before calling this fs_commit function we make a copy of
trans_attr and use the new copy for fs-transactions. The old copy is used for
committing. Now since, the jbd thread is calling the fs_commit function using a
func-pointer, from what I understand the fs process will not hang since fs
itself is not calling the commit functionality.
Also in the commit function is where we also do check-pointing if the required
number of blocks exceeds the number of free blocks in the log.
If what I understand is correct (about fs not supsending-see above) then, we can
have same functionality as linux but with support for multiple Journaling
PS: I will switch to jabber soon enough, I am behind a firewall currently which
bans me from using jabber client (by charging excessive amount of money for
Quoting Adam Hamsik <firstname.lastname@example.org>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> I'm moving this to public tech-kern because there are better
> developers than me.
> > Hello,
> > Could you send me the browsing links again because like a
> > moron I deleted
> > the email with the links you sent me for the project....Also didn't
> > bookmark
> > those.
> > Another thing was I was planning on starting to code a,
> > 1.) Multi-threaded implementation of ufs_tans.. where by a copy of
> > pending
> > transactions are made at commit and ufs_add is allowed to proceed
> > from fs.
> > While commit takes place concurrently. This is where its easier to
> > have the
> > commit happen by the ufs_trans itself rather than the fs because,
> > if fs is
> > making the commit then it needs to be multi-threaded doesn't it
> > since the user
> > apps are asking for stuff at the same time a commit needs to be
> > made, which
> > further increases complexity, I see this as the main reason for
> > linux to make
> > all commits from the jbd itself (/usr/src/fs/jbd/commit.c).
> > 2.) I also will look at the check-pointing code this I think is
> > much easier to
> > deal with (implement). Cause, once the the commits are all in it is
> > just a
> > matter of checking in ufs_add if the asked for number of blocks for
> > the
> > transactions are exceeding the journal blocks available and if so
> > hold the
> > adding for now.. snip the tail of the log and flush it onto the
> > disk..at the
> > app places. Same during un-mounting except no checking required for
> > number of
> > blocks exceeding the number of blocks free in journal.
> > Please note: Currently I am not considering the scenarios where a
> > required
> > transaction (like write,delete) which might require huge number of
> > blocks
> > always exceeding the journal block limit would need to be broken
> > down into
> > smaller transactions.
> > regards,
> > Quoting Adam Hamsik <email@example.com>:
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >> On Sep 19, 2007, at 4:10 AM, Avinash Malik wrote:
> >>> Hello,
> >>> I have been going through the source and have a few questions,
> >>> 1.) Current ufs_trans implementation uses number of trans (10) to
> >>> make a commit.
> >>> Linux uses a timer to do the same. Is there a reason for this
> >>> change?
> >> No I think that this is type of easy to write and test change. ufs-
> >> trans is WIP and therefore is not done yet. I think that we can go
> >> with linux approach but we have to keep GPL out of our kernel. So be
> >> careful with re-implementing linux code.
> >>> 2.) During commit stage in ufs_commit the transaction obtains a
> >>> lock, thereby
> >>> preventing any further changes to the transaction-list by fs. Linux
> >>> (I think)
> >>> (either) makes a copy of the transaction list before committing it
> >>> (or) makes a
> >>> new transaction list. This allows for less unnecessary delays for
> >>> the
> >>> file-system. As the fs can still make calls (ufs_add) to the
> >>> journal while
> >>> journal is committing. Are there plans to take the linux approach?
> >> This looks for me like RCU. I don't know if we have RCU
> >> implementation in our kernel.
> >> We can discuss this here :) I'm not expert for this.
> >>> 3.) The commit functionality in NETBSD is completely implemented by
> >>> each fs
> >>> rather than by the ufs_commit. Why is that, commit would involve
> >>> flushing the
> >>> meta-data onto the log and then writing the sector-tag in the log
> >>> which is same
> >>> for all the file systems isn't it.
> >> But there are differences between log types. FFS can use other
> >> journal architecture then JBD/EXT3. But as I already said, when we
> >> will have functional ext3 :)(or at least we will understand to log) I
> >> will separate jbd code from ext2 and create ufs-journal library with
> >> support for more than one type of journal.
> >>> 4.) Lastly, by check-pointing I meant flushing the log into the
> >>> real disk
> >>> places. In linux this is done when fs calls journal requiring
> >>> number of blocks
> >>> which exceeds the number of current blocks available in the
> >>> journal. Or at
> >>> unmount point. Also in linux since the log is flushed onto the disk
> >>> only at
> >>> these points, only the newest meta-data for each block-group needs
> >>> to be
> >>> flushed (at least in the default writeback mode), I am unable to
> >>> see this
> >>> functionality anywhere in source. What am I missing.
> >> After I read that kerneltrap article i understand what checkpointing
> >> means.
> >> This is missing I think. We haven't implemented this yet:).
> >>> Can I download the source for the branches from this website. I
> >>> have never
> >>> before used monotone so any guidance would be appreciated. Is it
> >>> like git?
> >> Do you use netbsd ? or what OS. We have monotone in pkgsrc devel/
> >> monotone.
> >> It is distributed source code management tool.
> >> for download:
> >> mtn db init --db [path to db]
> >> mtn genkey [your mail addr or somethink@hostname]
> >> mtn pull ufs-trans.mtn-host.prjek.net [\* for all or branch name for
> >> branch source]
> >> mtn checkout -b [branch name] directory
> >>> [snip]
> >>  monotone.ca
> >> http://www.venge.net/mtn-wiki/EssentialMonotone
> >> Regards
> >> - -----------------------------------------
> >> Adam Hamsik
> >> jabber: firstname.lastname@example.org
> >> icq: 249727910
> >> Proud NetBSD user.
> >> We program to have fun.
> >> Even when we program for money, we want to have fun as well.
> >> ~ Yukihiro Matsumoto
> >> -----BEGIN PGP SIGNATURE-----
> >> Version: GnuPG v1.4.7 (Darwin)
> >> iD8DBQFG8QBPlIxPgX3Go0MRAtE5AKD2pcdl5GeiW4ZQPEdFxcOBfUhlHgCglE+N
> >> s6/QhWrYdc5l/b+3LiUHrAg=
> >> =df1i
> >> -----END PGP SIGNATURE-----
> - -----------------------------------------
> Adam Hamsik
> jabber: email@example.com
> icq: 249727910
> Proud NetBSD user.
> We program to have fun.
> Even when we program for money, we want to have fun as well.
> ~ Yukihiro Matsumoto
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (Darwin)
> -----END PGP SIGNATURE-----