tech-kern archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: netbsd version of compressed caching




[added tech-kern]

LKM/module or not shouldn't really be a question for a pseudo device, AFAIU the new modules framework will allow to easily compile code into the kernel or use as a module. I wouldn't worry too much there.

About compressino algorithm, have a look at the vnd(4) driver, which has an option to use images that are compressed with in the "cloop" format, which internally uses the gzip algorithm (IIRC).

I think the most interesting part will be management of "free" space, as you can't assume the compressed blocks to be of a fixed size - if you replace one (compressed) "page" with another one, the new one may not fit as it's too big. vnd(4) avoids this by insisting that compressed images need to be read-only (and compressed in userland, see vndcompress(1)).


 - Hubert


On Wed, 10 Dec 2008, Selcuk AYA wrote:
Hi,
I found about the compressed caching project on the netbsd projects
page.  The page shows tech-embed mailing list as the mailing list to
contact. I have no NETBSD experience but I have been poking around the
code to get an understanding of uvm interfaces and pseudo drivers and
I think I can get the job done. I have also looked at the Linux
implementation of compressed caching. I belive there are three ways to
implement compressed caching in NETBSD:

a) as a pseudo device inside the kernel. This pesuodo device would be
used as a swap partition. Behind the scenes, it would compress pages
which need to be swapped out and store them in memory.
b) similar to a) but this would be a pseudo driver implemented as an
loadable kernel module.

For these two options, I found out that md.c (ram disk driver) in the
kernel serves as a good starting point to write the pseudo driver.

c)Inside the kernel, swap pager would need to be made aware of
compressed caching. I imagine swap pager keeps a map to keep track of
where swapped out pages are on the swap device. Some of these entries
would point to in memory compressed pages.

For all these options, compressed caching will have to manage its own
memory and keep track of variable sized chunks (I do not have a
detailed idea about this yet). I would prefer to go with options a) or
b) as I do not see a an advantage of making the swap pager aware of
compressed caching. Would an lkm be OK for an embedded system?


Being new to the open source world, I am worried about the licensing
issues. Linux compressed caching uses LZO algorithm for compression.
LZO is released under GPL so I guess we cannot use it in NETBSD. But
there are are other alternatives which do not bear the sign of GPL
(http://www.cs.utexas.edu/users/oops/compressed-caching/index.html) .
If I use a non GPL licenced compress/decompress algorithm and
re-implement compressed caching as above for NETBSD, do I avoid
licencing issues?


thanks.



Home | Main Index | Thread Index | Old Index