--ABTtc+pdwF7KHXCz
Content-Type: multipart/mixed; boundary="IiVenqGWf+H9Y6IX"
Content-Disposition: inline


--IiVenqGWf+H9Y6IX
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Hello-

I've put together a proposal [attached] for the ZFS-port SoC project.
I'd very much like to do this project, so any feedback that can help
strengthen the proposal is absolutely appreciated.

Thanks,
  - Oliver

--IiVenqGWf+H9Y6IX
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="ZFS_PROPOSAL.txt"
Content-Transfer-Encoding: quoted-printable

Porting ZFS to NetBSD
Proposal for Google Summer of Code 2007
Oliver Gould <ogould (at) olix0r.net>


 0 Background
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Sun Microsystem's Zettabyte Filesystem (ZFS), since Nov 2005, has proven to=
 be a
major breakthrough in filesystem technology, supporting features such as di=
sk
pools, snapshots, "unlimited" scalability, and various other performance
improvements over prior art.  Such advancements will make ZFS a major force=
 in
industry in coming years.

A BSD-licensed implementation of ZFS is utterly impractical within the
constraints of Google's Summer of Code (SoC).  However, porting the CDDL'd =
ZFS
implementation present in OpenSolaris is feasible.  In particular, Pawel Ja=
kub
Dawidek has been able to port a significant portion [1] of ZFS to FreeBSD. =
 His
work provides a good basis for porting ZFS to NetBSD.

Furthermore, OpenSolaris has published a high-level roadmap [3] for porting=
 ZFS
to other platforms.  This outlines a path for success, which I shall follow.


 1 Objectives
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

While Pawel's work provides a valuable starting point for much of my work,
numerous FreeBSD interfaces differ from those in NetBSD, including the virt=
ual
memory (vm(9)/uvm(9)), scheduler(9), driver(9), and file system (vfs(9)) ke=
rnel
interfaces.  However, most userland tools require a minimal amount of work =
to
bring into NetBSD.

 1.1 zpool and testing functionality
 -----------------------------------

Sun's storage pool abstraction, zpool, offers significant advantages over
classical logical volume interfaces such as NetBSD's ld(4) and ccd(4).  Por=
ting
this component is the first and foremost objective of this project.  It
comprises roughly 80% of the kernel-land code necessary for a functional ZFS
implementation.

Unfortunately, Pawel's port of zpool relies on FreeBSD's geom(4) interface,
which is not present in NetBSD, for virtual device (VDEV) functionality.  W=
hile
NetBSD's VDEV implementation will be fairly straightforward compared to
FreeBSD's, it also requires a significant portion of newly-written code.  T=
he
VDEV component shall be extended to behave similarly to NetBSD's vnd(4) vir=
tual
disk device.

The ZFS I/O Pipeline (ZIO) and Adaptive Replacement Cache (ARC) require work
with NetBSD's virtual memory and mutex interfaces.

The ztest, and therefore zdb, userland components will enable me to accurat=
ely
test throughout the remainder of the project. =20

 1.2 Userland management tools
 -----------------------------

ZFS's userland management tools, zpool(1) and zfs(1), rely on the /dev/zfs
device.  The kernel device will differ from prior implementations, however =
the
userland tools should not.

 1.3 ZFS Emulated Volume
 -----------------------

The ZFS Emulated Volume component (ZVOL) enables raw devices from the stora=
ge
pool to be presented to userland device consumers.  It therefore requires a
NetBSD-specific device driver.

 1.4 Filesystem Internals
 ------------------------

The ZFS Posix Layer (ZPL) is significantly nuanced and OS-specific.  It is a
very small and very difficult component when compared with the rest of ZFS.=
  It
is unrealistic to expect that the ZPL will be complete by the end of the
project.  Instead, a detailed blueprint, in the form of a development plan =
(as
described below) will be composed so that minimal work will be required to
complete the ZFS filesystem layer.

 1.2 Development Plans
 ---------------------

As the adage goes- when one fails to plan, he plans to fail.  Throughout the
entire project, I shall author an iterative development plan.  This plan wi=
ll
serve as a blueprint for the development of each phase of the project, with=
 the
goal of identifying functionality that _must_ be implemented for developmen=
t to
continue (dependencies) and functionality that can remain unimplemented sho=
uld
parts of the project run over-schedule (contingencies).

Development plans will include the following:
  - Function-call diagrams (as needed)
  - Updated Gantt charts illustrating dependencies and contingencies in the
    scope of the project timeframe
  - Required functionality upon completion
  - Optional functionality upon completion
  - File- and function-level summaries of work needed to successfully port =
each
    portion of code
  - Other specific documentation requested by mentors

Iterative development plans, at the end of the project, shall outline the w=
ork
that has been completed and will thusly provide future ZFS maintainers with
detailed documentation of NetBSD-specific features and bugs.


 2 Deliverables
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

In addition to project milestones, I shall maintain a project journal that
outlines day-to-day goals and achievements.   Weekly summaries based on my
written logs and CVS commit messages will enable the project mentors to pro=
vide
meaningful feedback regarding my work and prevent potential problems from g=
oing
unnoticed.

 2.1 Milestones
 --------------

2007/05/28:	[project start]

2007/07/24:	initial development plan including dependency diagrams & gantt
		charts outlining further development.

2007/06/30:	libzpool, ztest, and zdb complete

2007/07/08:	phase 2 development plan outlining userland management tools
		including:
		- zfs(1)
		- zpool(1)
		- /dev/zfs

2007/07/09:	[project midterm]

2007/07/17:	phase 2 complete

2007/07/22:	phase 3 development plan outlining zvol

2007/08/09:	phase 3 complete

2007/08/18:	'final' development plan blueprinting remainder of zfs port
		including zpl (filesystem internals)

2007/08/20:	project summary and lessons learned
		[project complete]


 3 About Me
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

I received my bachelors degree in Computer Science from Stevens Institute of
Technology (SIT), where I am currently pursuing a masters degree in Systems
Engineering while working in the School of Sciences and Arts academic system
administration team.  My current resume is available here:
	http://www.issa.stevens.edu/~ogould/resume.pdf

I have been tracking NetBSD-current for over a year, and follow many NetBSD
lists regularly.  Furthermore, I have taken courses with Jan Schaumann and
Hubert Feyrer, and I worked under Jan for over a year and with Thor Simon w=
hile
he consulted for SIT.  I have contacted each of these NetBSD developers whi=
le
preparing this proposal.

Recently, I have begun to port Fabrice Bellard's KQEMU LKM from FreeBSD to
NetBSD.  Through this project, I have gained familiarity with NetBSD's lkm(=
9),
driver(9), uvm(9), and scheduler(9) interfaces.  At this point, my port of =
KQEMU
has been delayed due to changing customer (SIT) priorities.


  4 References
  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

[1] Pawel Dawidek's initial post to freebsd-current regarding his ZFS port
    http://lists.freebsd.org/pipermail/freebsd-current/2006-August/065306.h=
tml

[2] NetBSD's ZFS SoC project description
    http://www.netbsd.org/contrib/soc-projects.html#zfs-port

[3] ZFS Porting page
    http://opensolaris.org/os/community/zfs/porting/

[4] ZFS Source Tour
    http://www.opensolaris.org/os/community/zfs/source/

[5] Solaris ZFS Administration Guide
    http://docs.sun.com/app/docs/doc/819-5461

[6] ZFS On-Disk Specification [Draft]
    http://www.opensolaris.org/os/community/zfs/docs/ondiskformat0822.pdf


$Id: ZFS_PROPOSAL.txt,v 1.6 2007/03/25 14:57:26 ogould Exp $

--IiVenqGWf+H9Y6IX--

--ABTtc+pdwF7KHXCz
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (NetBSD)

iQEVAwUBRgaS3fBcMTqzwkq9AQL+PAgApsC4WJ7sd/N/xZ9BSm+I+itA/z6Kxw0i
QjBudqPFMW+lH27SbvUyHbNxe3tPoI6ysj/QWWFW+xplyfQ8tKg6Qe+DBwJ1XrPH
uQ51Eu0fZUEHa+jz8PK+kMZlcOAHfU2idrZvisKWezg0/UXC4K4mdNJXxlMG+/MW
22y8iCea/+vdwhpNOata6V1UKOSAPMV/pl0agISF/R4QjVZrVK1kHPNZZLzendPO
t1n/7Fi+qh3QhcVFmarQP7AX8/POcJ/bc0EIBaP8lGrnAfpF0nFBi7kIFdxyXjB0
euRBNWjOLSd0lw036DljTySUfoMlnQ9QUc/jI56qFoj9nZ93JTVU6A==
=QBgK
-----END PGP SIGNATURE-----

--ABTtc+pdwF7KHXCz--