> -----Original Message-----
> From: tech-kern-owner@NetBSD.org [mailto:tech-kern-owner@NetBSD.org]On
> Behalf Of Reinoud Zandijk
> Sent: Tuesday, June 28, 2005 6:29 AM
> To: tech-kern@netbsd.org
> Subject: Re: CVS commit: src
>
>
> Hi,
>
> On Sun, Jun 26, 2005 at 07:07:14PM -0700, Jason Thorpe wrote:
> > >how about something-@ostype-@osrelease ?
> >
> > You really should be using subdirectories :-)
>
> yup...  though i have a similar naming problem as has HFS+. On UDF a file
> can have an associated streamdir with it basicly turning it into a
> composite of a file and a directory with named streams.
> Directories though
> can also have an associated streamdir with it. (!)

See Solaris attropen() and openat (..., X_ATTR).

>
> These streamdir's are defined to accomodate named extended
> attributes next
> to UDFs normal extended attributes (like various extra timestamps
> as there
> are a.o. last backup time, valid-from, valid-until) or DVD's css
> info. Some
> are meant to be invisible, others are to be explicitly visible.
>
> Examples of use for the named streams :
> 	* Mac's resource fork
> 	* NT and/or Unix ACL's
> 	* indexing tables, derived data, preview images etc. etc.

Subfiles (named streams) are the right way to think about #1 and #3.
But not #2. Subfiles are manipulated with read/write. Attributes
(including ACLs) are manipulated using get/set. The system needs
to check an ACL data structure to be sure it is correct before
applying it to a file. Can't do that with multiple write()s.
This issue has come up in NFSv4 also.

Subfiles and attributes are different animals. Don't let
the accidental misnaming of them, in some circles, confuse you.

>
> If such a file or directory is moved/copied, all associated data is then
> copied with it though it still reads/writes like an ordinary file.

Solaris uses -@ to basic file utilities enable handling/inclusing
of subfiles.

> Note
> that these files can not be hardlinked outside the file.

That is consistent with Solaris model of subfiles (and I
suspect NetApp's and NT's).

>
> To fit those two concepts into the Unix filingsystem is a bit
> tricky since
> VFS doesn't understand this concept too well.

True. VFS will have to be fixed. I don't know exactly
how Solaris did this but they had to do something.

> HFS+ has some
> similar issues
> so maybe it would be good to think of a way of adressing such
> named streams
> and the file's associated attributes. The chflags(1) interface is the
> closest matching interface but far too limited.

Linux getxattr() and setxattr() are better.

<EXPERT>It is easy to try and pair getxattr() with
NFSv4 OPENATTR because of the textual names. This is
not the case. Attributes require rigorous definitions
so that they can be marshalled between systems. RPC
structures must be defined. NFSv4 OPENATTR deals with
opaque data and, therefor, makes it appropriate for
subfiles. New attributes can be readily defined for
NFSv4. The realization of all this came late in the
NFSv4 design cycle, long after the poorly chosen
name OPENATTR.</EXPERT>

>
> Luckily, in UDF named streams are not allowed to have subdirectries.

That is consistent with Solaris, NetApp, and NT. NFSv4 left
the door open thought most think it a bad idea to go through it.

>
> Now i can see a couple of options on how to deal with it:
>
> option 1:
> 	ignore all, but preserve on operations like move etc. when not
> 	leaving the filingsystem otherwise discard them or raising an
> 	error. (simple, filingsystem internal)
>
> option 2:
> 	only allow these sub-files to be looked up when using open() but
> 	never list them. A naming scheme like "dirname/fname$attr" or
> 	"dirname/dirname$attr" could be used for this. (not intrusive but
> 	not consistent from user's perspective)
>
> option 3:
> 	also list the files using the naming scheme but hide by
> 	default in `ls' unless otherwise instructed. (bit intrusive)
>
>
>
> Apart from these named streams, other extended attributes like the
> timestamps could be looked up by a new VOP primitive or also be presented
> as a named stream. UDF uses the Ecma-167 its based on: a 32 bit integer
> indicating wich times are defined for the file followed by a series of
> timestamps in increasing associated bitnumber.
>
> The VOP would allowing other filingsystems like FFS to support various
> extra times too without modifying struct stat. Extending/modifying struct
> stat is most likely not wanted though. Applications for such
> times could be
> backup managers searching of files that need to be backed up,
> filingsystem
> purgers to purge old logfiles or even userdirs indicated to be removed
> after a time etc etc.

Linux getxattr() and setxattr().

>
> Thoughts ?
> Reinoud

It's very easy to get myopic about this stuff. How does
this impact NFS? What about the tar(1) format? What about
interchanging tarballs? There are already answers to those
questions. Not just for subfiles, but also for additional
attributes.

These issues are looming. No avoiding them. It would be,
IMO, sad to have these issues create yet more divergence
in UNIX-type systems. Such divergence is a barrier to
acceptance and deployment.

On the other hand, these issues present a rare opportunity
for convergence. There are already good ideas for these
things out there. We have a rare opportunity to converge
as the file model jumps up one more quantum step. It
would go a long way to enhancing UNIX-type acceptance
and deployment.

Please see:

     http://www.nasconf.com/pres04/waidhofer.pdf

Regards,
	-gww