--da4uJneut+ArUgXk
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Thu, Jun 30, 2005 at 11:18:17AM -0700, Gordon Waidhofer wrote:
> Reinoud wrote
> > Apart from its on-disc structures i think it would be wise to use or at
> > least study its [UDF] semantics and definitions as a starting reference
> for its
> > at least a coherent specification that has been `out there in the
> > wild' for quite some time.
> 
> It should be included in a proposal as part of a survey. Time on
> the planet is relavent, but deployment and interoperability
> are weightier issues. NT, NetApp, and Solaris semantics come to mind.
> And expect change.

Its deployed on various platforms though i only have access to a Roxio 
implementation for MS-windows wich only records mere trivial attributes on 
test discs i created. Haven't tried a ACLs yet though :)

> > Interestingly it is pretty like the proposed linux/solaris mix we
> > came to. I'll try to give a summary of UDF attributes/subfile
> > support and handling to give an indication.
> > --snip--
> 
> Nice summary on UDF model. I didn't know UDF was so clear
> about time updates. It is interesting how UDF partitions
> spaces based on size. The "invisible to userland" option
> in space 4 entries is a good trick for very large attributes.

not entirely only on size though but also motivated by update speed 
considerations. The in-node attribute space is very easily updated every 
time you need to write the node anyway and they also don't require reading 
in extra sectors (possibly at a distance) when accessing a file thus 
speeding up the process of searching and processing. Bare in mind that its 
also used on WORM devices... you DONT want to rewrite more than strickly 
nessisary :-D

> > Furthermore UDF preferably stores NT ACLs and UNIX ACLs (most likely
> > refering to posix ACLs) in space 4 under fixed names for version 2.60+.
> > This is not optimal IMHO since it thus will cost at least 2
> > discblocks for
> > each annotated file extra (!) They dont have to be exported as inodes
> > though but still. Are NT/posix ACL's that lengthy at times that its worth
> > such spillage? Big bummer is that the (lengthy?) ACL's can not be shared
> > between files since hardlinking in subfiles is NOT allowed.
> 
> POSIX ACLs tend to be fairly limited size, like 1k or 2k limits.
> Often smaller. NT ACLs can be a monolithic 64k data structure.
> Interoperability tests (NetApp's, I think) already test implementations
> to handle max ACL size. These show up at plugfests.

hmm plugfests... well UDF has its regular plugfests but they are allways on 
virtually the other side of the planet for me and though not expensive to 
attent to financially completely impossible due to travel and staying costs 
since i'll have to pay for them myselve and my budget just doesn't allow 
such escapades. They are ormally held in an Ambassy hotel somewhere in the 
USA, somewhere in California AFAIR.

> > If ACL's are short they prolly will also be allowed in one of the other
> > streams but i haven't explicitly seen a reference to it yet. I'll ask
> > around.
> 
> I saw something in SGI's notes on UDF implementation where they
> compromised on the UDF "rules" for some of XFS issues related
> to extended attributes especially large ones.

as in they placed them in space 4? The total space for extended attributes 
in space 3 is about 4 Gb per file (32 bit). In space 4 its effectively disc 
size. Could you give me a pointer to such info? Or any SGI documentation on 
extended attributes defined for XFS?

> > The only advantage of seeing both `file attributes' and `subfiles' as a
> > more generic `attribute' is that it doesn't care about what
> > implementation/model is behind them.
> 
> We never care about the implementation model behind the API model.
> We're not suppose to.

true, but i was refering to the flavours of attributes/subfiles definition 
before i read the NFSv4 attr's doc.

> The trouble -- confusing attributes and subfiles -- started when
> folks believe the superficial similarity would carry through APIs,
> representations, and so on. It does not.
> 
> > This approach however DOES have a
> > fundamental problem : it requires explicit and strict content
> > definition of
> > some named `attributes' to be defined and no definition for the rest!
> 
> And definition means a lot of things:
> 	- .h files for the API (think struct stat)
> 	- Canonical text representations for pax(1) encoding
> 	- RPC definition for NFS encoding
> 	- etc, etc, etc (see my presentation on nasconf)

yup.

> NFSv4 attributes are extensible. That was a design point.
> They are named by small integers. They are rigorously
> defined. They are not opaque. That's pretty much the
> duck test for attribute, IMO. Hence anything that
> is opaque (subfiles) should *NOT* be called attribute
> because it confuses the terminology unecessarily.

i agree. If attributes are allready defined for NFSv4 we could use them 
`natively' too making NFSv4 implementation a hell lot easier too and my 
guess is that they'll snugly map on UDF anyway too.

Reading the 275 page RFC is not very encouraging but a quick scan shows it 
has little if none compound attributes but only single value arguments 
being of a basic type. A quick scan of the NFSv4 attr :

    named_attr   : derived, basicly if it has subfiles or not
    ACLs/ACEs    : dont know how these map to posix but prolly compatible
    hidden       : metadata flag in UDF, bool flag to record in FFS
    mimetype     : small attribute with string
    rawdev       : one-to-one mapping on UDF attribute
    system       : optional bool flag to record in UDF and FFS
    time_backup  : part of times-attribute in UDF
    create_time  : (optional) part of times-attribute in UDF, present in 
                   ufs2
    owner, group : discussion point

Note that all such flags and new flags could be made one NFSv4 attributes 
attribute saving space.

Guessing at the sizes of all attributes together that can be set i'd say 
that apart from the ACLs i'd say it all fits snugly in one disc block so 
internal in-inode storage in FFS is most likely possible since a ffs2 
dinode struct is about 256 bytes and a sector size is normally at least 
512. In normal disc layouts, a basic disc block is 8kb addressable in max 8 
pieces so at 1kb intervals so that ought not to give problems.

In struct ufs2_dinode i read :
        int32_t         di_extsize;     /*  92: External attributes block. */
        int64_t         di_extb[NXADDR];/*  96: External attributes block. */

Does this mean two 8 kb blocks can be used for external attributes? i'm 
puzzled now. Could a FFS guru comment on these fields? Are they also 
consumed for in-node storage? Does our current UFS2 support in-node 
attributes/file/directory storage in the first place? and if not, why not?

> > Indeed. Maybe NetBSD can fullfill that reference implementation? AFAIK we
> > as NetBSD are not bound by `legacy' yet and we as NetBSD would
> > get best of
> > both worlds.
> 
> Yup. That's my hope.
> 
> That FreeBSD ACLs and EAs haven't been
> integrated yet is a good thing. They ultimately will be the
> wrong thing, and I'm quite sure FreeBSD will have a horrible
> time trying to resolve NFSv4 OPENATTR and the weenie extended
> attribute model that's there.

I dont think that'll be a problem for FreeBSD; having skimmed the NFSv4 
attributes and OPENATTR function they are distinct. The GETATTR/SETATTR can 
only modify the attributes i summed up above and the openattr (bad name 
indeed) is for subfiles ONLY.

> NetBSD is in an excellent position to pursue the likely
> course of the file model.

hope so too.

> Keep up the good work, Reinoud.

thanks,
Reinoud

--da4uJneut+ArUgXk
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (NetBSD)

iQEVAwUBQsSLkoKcNwBDyKpoAQJ55wf6AlDweOGq3F+IYhOmrfRNR1OKdrRSZIX1
z3Pg42bJCgjPCE5B+xjvbTspk/lX3RV12y3dnQOIGX/FvAMPcdf0TZGTLgnU4xCA
xQBOJqKalvXQoWel6jebQGVvxDxUQKdqOGq3KQoKzPiVdukKVZxpJTT7blj4Idxq
W+AdtezllzAd/+cEhb+BVsy1aWvL3xAbLUbrEJyYewi0kg7OQNjqaet6ryrhg4By
9rSUfT8ZH++78CUDp98Gm/qPQzckQMfRXjkb7tQtNtzXdIi3SiKR4SBqm2Qhr4P7
mrD898+JYkRUOuGB8Lx+4IYAwAse0JrW0lKP7xafMhk4QTMg+q3+jQ==
=i9fk
-----END PGP SIGNATURE-----

--da4uJneut+ArUgXk--