Re: Text file type hint proposal for filexfer

[Jacob Nevins <jacobn+secsh%chiark.greenend.org.uk@localhost>]

denis bider <ietf-ssh%denisbider.com@localhost> writes:
> I would like to note that checking whether the file is textual or not
> is a cheap operation only on VMS.
> 
> Defining the text/binary hint as a file attribute makes it less likely
> that Unix and Windows servers will implement this.

My intention in defining the attribute extension was to support those
servers which have a reasonably authoritative text/binary distinction,
for example in the filesystem, rather than those which have to use a
heuristic approach, such as Unix/Windows, where I would much prefer that
UNKNOWN is returned rather than a guess made.

> I think we would benefit if it were possible for a client to query the
> likely textual or binary state of a file independently of anything
> else. In other words, without obliging the server to either show this
> possibly expensive information as part of regular easily-obtainable
> attrib-bits, or not support this feature at all.
> 
> I think the attribute hints are a nice idea and will work for VMS.
> However for Unix and Windows servers, we might also benefit from a way
> for the client to prompt the server to determine on the fly whether
> the file is textual or binary, if the server has not indicated the
> file type in attrib-bits. This query would be done right before
> downloading the file.

I'd prefer any such extension to be optional, and to have defined
semantics of being heuristic rather than authoritative if that's how
it's implemented; as a client (or rather client user), I would like to
be able to distinguish heuristic from authoritative information so that
I can choose not to use the former while still getting the benefit of
the latter. (Perhaps the extension could have a flag indicating whether
it contains "authoritative" information?)

I should come out and say that my personal view is that I'd rather not
trust the integrity of my files to text/binary distinction heuristics,
which is why I'm pushing for this distinction. I realise this isn't a
universal view, but I don't think I'm alone in this; it's a popular
feature but I'd rather it was possible for the end user to choose
whether to use it.

> However if there was widespread server-side support for a query to
> determine file type, the detection and conversion algorithms could be
> entirely removed from the client.

Except that the client would need to convert from canonical form.
I agree that if you're going to do file type detection on server->client
transfers, it's better to do it on the server.

(Clients will also need to have some means of file type detection for
client->server transfers if they don't have that information to hand.)

> My thinking is, either we require Unix and Windows servers to
> implement both (a) conversion - SSH_FXF_TEXT flag, AND (b) file type
> detection; or we should require them to do neither.
  [...]
> The idea is, let's be consistent, and let's not specify things halfway
> through.

I don't think there's anything inconsistent here.

The server may not know _which_ files are text files, but it could well
be in a position to know what the likely line ending convention is if
the user asserts that given file _is_ a text file, which is a better
position than the client is in; in such a situation the server should
convert to canonical form iff requested. This is the position I see
simple (non-heuristic) Unix/Windows servers as being in.

Given the which, having no text-hint-attrib-bits support and yet having
SSH_FXF_TEXT support does make sense; it leaves you in the similar
situation to plain FTP (or SFTP before this extension) of the user
having to decide which are text files, which is IMO where you should be
(or at least have the option of being) with such servers.

That said, I wouldn't object to making SSH_FXF_TEXT support optional in
general.