-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Sep 20, 2007, at 6:36 AM, Avinash Malik wrote:

> Hello,
>      after thinking about the various journaling requirements this  
> is what I can
> come up with, correct me if I am wrong
>
> 1.) We still call the commit process from ufs_commit to fs_commit,  
> but on a
> writer thread. But before calling this fs_commit function we make a  
> copy of
> trans_attr and use the new copy for fs-transactions. The old copy  
> is used for
> committing. Now since, the jbd thread is calling the fs_commit  
> function using a
> func-pointer, from what I understand the fs process will not hang  
> since fs
> itself is not calling the commit functionality.
>
> Also in the commit function is where we also do check-pointing if  
> the required
> number of blocks exceeds the number of free blocks in the log.
>
> If what I understand is correct (about fs not supsending-see above)  
> then, we can
> have same functionality as linux but with support for multiple  
> Journaling
> systems.
>
Ok I think that we can do it this way, it may increase our ufs-trans  
performance.

> PS: I will switch to jabber soon enough, I am behind a firewall  
> currently which
> bans me from using jabber client (by charging excessive amount of  
> money for
> internet connections).
>
Did you checkouted sources from that mnt repository ? let me know  
when you write and commit some code to your local repo. I will add  
your pub key to that public repo.

You should commit your changes to new branch e.g. avi-ufs-trans. we  
can easily merge changes between branches in mtn.

> regards,
>
> Quoting Adam Hamsik <haaaad@gmail.com>:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> I'm moving this to public tech-kern because there are better
>> developers than me.
>>
>>
>>>
>>>
>>> Hello,
>>>       Could you send me the browsing links again because like a
>>> moron I deleted
>>> the email with the links you sent me for the project....Also didn't
>>> bookmark
>>> those.
>>>
>>> Another thing was I was planning on starting to code a,
>>>
>>> 1.) Multi-threaded implementation of ufs_tans.. where by a copy of
>>> pending
>>> transactions are made at commit and ufs_add is allowed to proceed
>>> from fs.
>>> While commit takes place concurrently. This is where its easier to
>>> have the
>>> commit happen by the ufs_trans itself rather than the fs because,
>>> if fs is
>>> making the commit then it needs to be multi-threaded doesn't it
>>> since the user
>>> apps are asking for stuff at the same time a commit needs to be
>>> made, which
>>> further increases complexity, I see this as the main reason for
>>> linux to make
>>> all commits from the jbd itself (/usr/src/fs/jbd/commit.c).
>>>
>>> 2.) I also will look at the check-pointing code this I think is
>>> much easier to
>>> deal with (implement). Cause, once the the commits are all in it is
>>> just a
>>> matter of checking in ufs_add if the asked for number of blocks for
>>> the
>>> transactions are exceeding the journal blocks available and if so
>>> hold the
>>> adding for now.. snip the tail of the log and flush it onto the
>>> disk..at the
>>> app places. Same during un-mounting except no checking required for
>>> number of
>>> blocks exceeding the number of blocks free in journal.
>>>
>>> Please note: Currently I am not considering the scenarios where a
>>> required
>>> transaction (like write,delete) which might require huge number of
>>> blocks
>>> always exceeding the journal block limit would need to be broken
>>> down into
>>> smaller transactions.
>>>
>>> regards,
>>>
>>> Quoting Adam Hamsik <haaaad@gmail.com>:
>>>
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>>
>>>>
>>>> On Sep 19, 2007, at 4:10 AM, Avinash Malik wrote:
>>>>
>>>>> Hello,
>>>>>       I have been going through the source and have a few  
>>>>> questions,
>>>>> 1.) Current ufs_trans implementation uses number of trans (10) to
>>>>> make a commit.
>>>>> Linux uses a timer to do the same. Is there a reason for this
>>>>> change?
>>>> No I think that this is type of easy to write and test change. ufs-
>>>> trans is WIP and therefore is not done yet. I think that we can go
>>>> with linux approach but we have to keep GPL out of our kernel.  
>>>> So be
>>>> careful with re-implementing linux code.
>>>>> 2.) During commit stage in ufs_commit the transaction obtains a
>>>>> lock, thereby
>>>>> preventing any further changes to the transaction-list by fs.  
>>>>> Linux
>>>>> (I think)
>>>>> (either) makes a copy of the transaction list before committing it
>>>>> (or) makes a
>>>>> new transaction list. This allows for less unnecessary delays for
>>>>> the
>>>>> file-system. As the fs can still make calls (ufs_add) to the
>>>>> journal while
>>>>> journal is committing. Are there plans to take the linux approach?
>>>> This looks for me like RCU. I don't know if we have RCU
>>>> implementation in our kernel.
>>>> We can discuss this here :) I'm not expert for this.
>>>>
>>>>> 3.) The commit functionality in NETBSD is completely  
>>>>> implemented by
>>>>> each fs
>>>>> rather than by the ufs_commit. Why is that, commit would involve
>>>>> flushing the
>>>>> meta-data onto the log and then writing the sector-tag in the log
>>>>> which is same
>>>>> for all the file systems isn't it.
>>>> But there are differences between log types. FFS can use other
>>>> journal architecture then JBD/EXT3. But as I already said, when we
>>>> will have functional ext3 :)(or at least we will understand to  
>>>> log) I
>>>> will separate jbd code from ext2 and create ufs-journal library  
>>>> with
>>>> support for more than one type of journal.
>>>>
>>>>> 4.) Lastly, by check-pointing I meant flushing the log into the
>>>>> real disk
>>>>> places. In linux this is done when fs calls journal requiring
>>>>> number of blocks
>>>>> which exceeds the number of current blocks available in the
>>>>> journal. Or at
>>>>> unmount point. Also in linux since the log is flushed onto the  
>>>>> disk
>>>>> only at
>>>>> these points, only the newest meta-data for each block-group needs
>>>>> to be
>>>>> flushed (at least in the default writeback mode), I am unable to
>>>>> see this
>>>>> functionality anywhere in source. What am I missing.
>>>> After I read that kerneltrap article i understand what  
>>>> checkpointing
>>>> means.
>>>> This is missing I think. We haven't implemented this yet:).
>>>>
>>>>>
>>>>> Can I download the source for the branches from this website. I
>>>>> have never
>>>>> before used monotone so any guidance would be appreciated. Is it
>>>>> like git?
>>>>>
>>>> Do you use netbsd ? or what OS. We have monotone[1] in pkgsrc  
>>>> devel/
>>>> monotone.
>>>> It is distributed source code management tool.[2]
>>>>
>>>> for download:
>>>>
>>>> mtn db init --db [path to db]
>>>> mtn genkey [your mail addr or somethink@hostname]
>>>> mtn pull ufs-trans.mtn-host.prjek.net [\* for all or branch name  
>>>> for
>>>> branch source]
>>>> mtn checkout -b [branch name] directory
>>>>
>>>>>
>>>>> [snip]
>>>>
>>>> [1] monotone.ca
>>>> [2]http://www.venge.net/mtn-wiki/EssentialMonotone
>>>> Regards
>>>> - -----------------------------------------
>>>> Adam Hamsik
>>>> jabber: haad@jabber.org
>>>> icq: 249727910
>>>>
>>>> Proud NetBSD user.
>>>>
>>>> We program to have fun.
>>>> Even when we program for money, we want to have fun as well.
>>>> ~ Yukihiro Matsumoto
>>>>
>>>>
>>>>
>>>>
>>>> -----BEGIN PGP SIGNATURE-----
>>>> Version: GnuPG v1.4.7 (Darwin)
>>>>
>>>> iD8DBQFG8QBPlIxPgX3Go0MRAtE5AKD2pcdl5GeiW4ZQPEdFxcOBfUhlHgCglE+N
>>>> s6/QhWrYdc5l/b+3LiUHrAg=
>>>> =df1i
>>>> -----END PGP SIGNATURE-----
>>>>
>>>
>>>
>>>
>>
>> Regards
>> - -----------------------------------------
>> Adam Hamsik
>> jabber: haad@jabber.org
>> icq: 249727910
>>
>> Proud NetBSD user.
>>
>> We program to have fun.
>> Even when we program for money, we want to have fun as well.
>> ~ Yukihiro Matsumoto
>>
>>
>>
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.7 (Darwin)
>>
>> iD8DBQFG8a7ElIxPgX3Go0MRArNXAKCr0ndw7KL01AxGWdAgjMDdymChuQCfYAY/
>> 9tIVU2WsUqkTzyhEZFn6bvU=
>> =uSnD
>> -----END PGP SIGNATURE-----
>>
>
>
>

Regards
- -----------------------------------------
Adam Hamsik
jabber: haad@jabber.org
icq: 249727910

Proud NetBSD user.

We program to have fun.
Even when we program for money, we want to have fun as well.
~ Yukihiro Matsumoto




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iD8DBQFG8vuRlIxPgX3Go0MRAiqzAJ4nH8LqVtAPnFWDUUsVxpgIwoC+1gCfTgj5
XrZx98y3YE1269sGepM+9w8=
=pGf3
-----END PGP SIGNATURE-----