On Fri, 26 Jun 1998 10:24:54 -0400 (EDT) 
 "Nathan J. Williams" <nathanw@MIT.EDU> wrote:

 > >For the HSM we're building on NetBSD/alpha, I need a couple of additional
 > >mode bits to indicate two separate additional file states:
 > >
 > >	- File has been archived
 > 
 > 	Don't we already have this? From <sys/stat.h>
 > #define SF_ARCHIVED     0x00010000      /* file is archived */

To be honest, I'm not entirely certain what the semantics of this flag
are supposed to be.  Specifcally, the MSDOS file system sets it on a
stat if the DOS "archived" attribute is set... and I suppose dump(8)
could set this flag, but it doesn't look like it does...

 > and chflags(1) knows about it. I don't know if anything uses this
 > flag, so I've no idea if its intended semantics are different from
 > yours. 

Right... this is also my confusion... There's another problem... these
flags need to be protected from the user mucking with them (I guess I
should have mentioned that in the previous message).  They are to be
set only by the whatever special interface the HSM has with the kernel,
not via the standard chmod/chflags system calls.

 > >	- File is not resident on disk
 > 
 > 	It feels a little bit weird to use flags for these. I think
 > that's because they seem like part of the internal implementation of a
 > filesystem (the HSM), but the interface to them is exposed to
 > userspace. What happens to my HSM system if I start using chflags(2)
 > to frob these bits? If I turn off the "File is not resident" bit, what
 > do I get when I read the file? Or, if I'm writing a backup utility, I
 > might grab the stat structure and then the file - but reading the file
 > will cause the stat structure to become invalid. If I then write back
 > the same information, I will have a existing file with the
 > non-resident bit set, which is bizzare. 

See above; random user programs can not be allowed to change these bits,
otherwise, as you have noted, Very Bad Things can happen.

I've had two suggestions, BTW, to use the flags, not the mode.  Part of
my motivation for using mode is historical precedence for the HSM that
we are using (called NAStore)... a simple "ls -l" is capable of displaying
the HSM status of a file, e.g. on our current ConvexOS implementation:

chuck (thorpej) ~ 2% ls -l
total 40
   8 drwxr-xr-x  2 thorpej       512 Jun 25  1996 9/
   8 drwxr-xr-x  3 thorpej       512 Jul 25  1997 cvsroot/
   8 drwxr-x--- 10 thorpej       512 Jan  5  1995 etc/
   8 drwxr-xr-x  2 thorpej       512 Jun 25  1996 other/
   0 Arw-r--r--  1 thorpej  1000473600 Jun 10  1997 ralph-5000-200-rz0-rz57
   8 drwxr-x---  2 thorpej      1024 Jan  8  1995 scripts/
chuck (thorpej) ~ 3% 

Note the file `ralph-5000-200-rz0-rz57' is marked with a file type of `A'.
This means "archived, non-resident".  If I were to open the file, the HSM
locates the tape, loads the tape, and begins making the file resident.  Once
the file is resident, the `A' becomes `a'.  To make this work is only a
simple 4 line change to strmode(3).

As Erik Fair mentioned, there are other "common" APIs for this sort of
thing, but IIRC, they don't address this problem, in particular.  (The
Cray API [I forget the name] and DMAPI mostly deal with the interface
between the archive/restore agent and the kernel.  FWIW, while we are
currently using our home-grown HSM API right now, we're planning a DMAPI
version of our HSM for the next revision... unfortunately, DMAPI is
fairly complicated.)

Jason R. Thorpe                                       thorpej@nas.nasa.gov
NASA Ames Research Center                            Home: +1 408 866 1912
NAS: M/S 258-5                                       Work: +1 650 604 0935
Moffett Field, CA 94035                             Pager: +1 650 428 6939