Re: Regarding summer of code 2008(writing device drivers)

[David Young <dyoung%pobox.com@localhost>]

On Fri, Mar 28, 2008 at 01:45:24PM +0530, pankaj gupta wrote:
> Sir,
>      I am Pankaj gupta, a student of computer engineering in India.
> 
> I have an idea for a project which I named "WAN optimization through
> application independent caching".
> 
> The idea is to reduce the bandwidth consumption by avoiding
> re-transmission of packets between two gatways/nodes.

Pankaj,

This is an interesting project that you propose.

What you propose is a kind of packet compression, don't you think?
Perhaps your proposal is just the first of several packet compression
facilites that you can add to NetBSD.

Delivery of IP packets is not guaranteed.  It seems that you will need
some feedback from the receiver, in order to know that the sender and
receiver have the same contents in their cache.  Is that so?

Is your technique capable of opportunistically accelerating peer-to-peer
filesharing apps such as BitTorrent?  Two instances of the same
application may segment the same stream of data differently; does your
technique compensate for differences in segmentation, or does it miss
opportunities for compression if the segmentation is different?

You mention the transport layer.  I am not precisely sure where your
solution will reside in the kernel.  Will you say some more about that?

Do you use only the checksum to detect duplicate packets?  It seems
that there is a risk of a stream being corrupted by chance.  Also,
the technique seems susceptible to data injection.  What do you think?

Do endpoints who are using the packet-cache technique automatically
detect each other?

Have you thought about getting routers involved?

Dave

> repetetive.
>  I have deviced a way to avoid this re-transmission of  packet by
> developing a networking device driver (at transport layer) using cache
> structure in driver.
> 
> This would not be like normal browser cache. It would be a system wide
> cache. Any application accessing data over network would use the cache
> without knowing that it is using cache. If one application has
> accessed a data once, that would be available locally to any other
> application accessing the same data over network, this is another plus
> point of implementing a cache  at driver level.
> 
> The  driver will be installed on  both the communicating machines, and
> cache will be maintained at both the communicating machines.
> Cache consists of most recently used data packets( incoming packet or
> outgoing packet) passed over network stack.
>  While storing the packet in cache, its MD5 checksum will be
> calculated and saved along with the packet in cache. Packets will be
> stored in a hash based on the MD5 checksum of packet.
> Now, whenever a packet is sent from server side it will be captured by
> our driver(module) and its MD5 checksum will be calculated and would
> be looked-up in cache hash table.
>  If a match occurs, it means packet has been sent earlier and sending
> it again can be avoided if other side (which also have the same
> coherent cache) can be informed  that this  particular packet which
> was recieved earlier and currently is in your cache.
>  So, after finding a match of MD5 checksum in its cache, server would,
> instead of sending whole 1480 bytes size packet it would send only
> 16bytes of MD5 checksum to the other receiving machine.And other
> machine would look up that particular packet in its own cache using
> that MD5 checksum.
> 
> I have completed major portion of this project as my final year
> project which is on WINDOWS platform. I am very eager to implement
> this on UNIX platform also.
> 
> I have not sent a proposal for this project but I am waiting for your
> worthful reply.
> 
> I am looking forward to your reply.
> 
> 
> 
> -- 
> Regards
> Pankaj Gupta
> AIT Pune

-- 
David Young             OJC Technologies
dyoung%ojctech.com@localhost      Urbana, IL * (217) 278-3933 ext 24