Re: How to go on with ISO 9660 large file support ?

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

I don't know about CD9660 itself, but here are some general comments.

   Date: Tue, 27 May 2014 12:22:32 +0200
   From: "Thomas Schmitt" <scdbackup%gmx.net@localhost>

   The filesystem specific vnode.v_data struct iso_node needed a change
   to represent the 1:n relation between file and file section.
   This change caused adjustments all over the code of cd9660.
   It makes nearly full use of the 64 bits of NetBSD's ino_t and
   employs malloc(9) or kmem(9) memory for files with more than one
   section. (Currently 12 bytes per file section.)

New code should use kmem(9) for variable-sized or one-time
allocations, pool(9) for infrequent fixed-size allocations, or
pool_cache(9) for frequent fixed-size allocations.  malloc(9) is
supposed to be going away.

   ABI compatibility of the changed struct iso_node is guaranteed for
   systems with up to 96 bit pointer types.
   The API of cd9660_node.h is not compatible, in my current
   implementation. An alternative implementation is possible with
   100 % API/ABI compatibility for single-section files.
   It would cause 4 to 8 bytes size increase of struct iso_node.

   fstat(1) and pmap(1) include <isofs/cd9660/cd9660_node.h>.

For the most part, struct iso_node is private to the kernel
implementation of cd9660.  Since fstat(1) and pmap(1) use it, you
should avoid changing the offsets within the structure of the members
that fstat(1) and pmap(1) use, but I don't think you need to worry
about preserving the whole API or the structure size.

Some day, we ought to find a better way to do what fstat(1) and
pmap(1) are doing with kernel guts.

   A script is available for creating, mounting, and testing this
   filesystem. It is not trivially portable to other computers because
   there are several individual adaptions to be made.
   ....
   I will publish the script and be ready to support its conversion
   into an automatic test of what can be tested in general.

It would be nice if you could integrate your tests into the atf file
system tests under src/tests/fs so that they get run automatically.

   There is also an ISO image emerging which (by hex editor) exposes
   exotic or even illegal situations. 6.1.3 and the host operating system
   show interesting effects when mounting it.

This sounds bad (but not at all surprising).  If the `interesting
effects' are anything more than harmless messages printed to the
console and unreadable images -- e.g., programs crash, kernel panics
or discloses private memory to userland, &c. -- it would be nice if
you could identify little fixes or at least early checks that can be
easily back-ported.

     The hardest reason why this information has to be encoded in ino_t
     is the desire to implement method VFS_VGET(9). If VOP_LOOKUP(9) would
     be the only method which leads to creation of a vnode, then the address
     and count could be stored in some other members of struct iso_node.
     A simple EOPNOTSUPP would open this path.

NFS uses VFS_VGET, so it would be good to continue to support that.

This doesn't require that you embed an ino_t in struct iso_node, only
that the ino_t values that VOP_READDIR returns correspond with the
nodes that VFS_VGET returns for those ino_t values.

But embedding an ino_t in struct iso_node is probably the easiest way
to do it, and will avoid breaking fstat(1) and pmap(1).