[ On , June 2, 2003 at 17:47:33 (-0700), Ian Lance Taylor wrote: ]
> Subject: Re: Adding TTL information to gethostbyname() and friends
>
> I have no quarrel with this.  But it does not actually fix the
> problem, because it doesn't change the browsers.

I think you've made the "fatal" assumption that changing the browsers is
both possible and "likely", as well as being necessary.  I'm saying
neither assumption is now, or ever will be, true.

The better, and pre-existing, solution requires only the assurance that
client platforms will, by default, either provide their own caching
"full resover", or alternatively ensure that their administrators will
provide low-latency access to a full (i.e. caching) DNS resolver.

Either way the applications need only use the existing DNS resolver API
since local low-latency caching will always be available and the latter
is already a well known, decade plus old, requirement.  If as you are
saying that this is not a well/universally adhered to requirement then
the best solution is not to ask all application authors to learn to
implement private DNS caches, but rather to ask all OS vendors to
implement full RFC 1123 compliant DNS resolvers.

Free OS platforms already provide multiple choices for full DNS resolver
implementations and are really only lacking in sane default
configuration options for those clients running as leaf nodes on
long-latency connections.  In fact this last piece to the puzzle is
something that I think is actually very near to being solved already, at
least for free OS platforms.  More and more systems provide easier ways
to configure caching resolvers and more and more people are aware of the
need for a local DNS cache.

If enough of the other popular non-free OS platforms were to ensure that
the proper RFC 1123 DNS resolver implementations are in place then
application authors would come to realize that they don't need to try to
implement their own caching DNS stub resolvers and your whole quest for
an easier way for them to do that will be moot.

Ideally of course the DNS stub resolvers in all platforms would also
implement a transparent in-core cache, but this would be a luxury, not a
necessity.  Furthermore although it would eliminate the worse case
problems for users on poorly managed leaf networks, it wouldn't really
eliminate the need for full compliance with RFC 1123 DNS caching
resolver requirements (i.e. availability of a local full caching
resolver).

Also in an ideal world all DNS using applications will be fixed to use
the name to address translations they acquire through the resolver for
reasonable lengths of time but no longer, i.e. without actually trying
to cache those mappings for extended use but rather only keeping them
for "single" use (e.g. for a web browser the limit is the time it takes
to retrieve and render a single top-level object and all its directly
referenced sub-objects).  There's a subtle but very critically important
difference between "caching" as defined by RFC 1123 et al, vs. caching
for use through a single phase of execution in an application.

> Not so.  If my patch, or something like it, is accepted, my next step
> is to patch Mozilla to detect whether the TTL information exists
> (easily done in configure) and to use it when available.  I will then
> look at other free software browsers.
> 
> Then I can use that as leverage to encourage other operating systems
> to adopt the same or similar patch.
> 
> As a practical matter, I have to start somewhere.

I think you've chosen the longest and hardest path possible to try to
fix a problem that would never exist in the first place if DNS caching
were simply done properly as the "original" design calls for.

Instead of campaigning for an API extension to be implemented in the
majority of OS platforms and for it's use in many applications it would
be sufficient to campaign the far fewer OS vendors (free and
proprietary) to provide and enable full caching resolvers on client
hosts by default and to configure their stub resolver libraries to use
the local recursive cache by default.

If indeed you care less about non-free environments then I would guess
that youcan see that attempts to make non-standard changes to an API
used across free and non-free environmetns are even more futile since
doing so can only complicate things for both OS vendors as well as
application authors and maintainers.  Only strong standards will likely
be implemented in non-free environments, but I'm sure you know already
that standards which try to invent new APIs from whole cloth, or even
just API extensions, are often prone to failure.

> As noted above, this problem does affect NetBSD, whether or not it
> ships with a configured caching resolver, because the browsers will
> cache the information anyhow.

I think it's irrelevant whether any application currently attempts to do
its own private "long-term" caching, though it would be sad if it were
not eventually fixed (by elimination of the private caching, of course).

The best long-term solution to the problem is to prove to application
authors that the platforms they support have fully RFC 1123 compliant
DNS caching resolvers already and that their applications don't need to
try to implement any private DNS caching.  I.e. make the application
level caching not only redundant but indeed visibly extra overhead that
is actually a detriment to application performance instead of a benefit.
Eventually this will cause browser authors to stop trying to cache DNS
information for any longer than it is needed to complete the immediate
task at hand.

> Frankly, this approach strikes me as being both more difficult to
> implement and less likely to succeed.

Of course it's more difficult -- I'm simply suggesting here that the
stub resolver caching be optionally implemented one-time only in a
common and transparent way for all applications and this is usually more
difficult to do than writing a private cache that might only meet the
needs of one specific application.  However if private caching is worth
doing at all then it is worth doing in this way by maintaining
application transparency instead of asking every application author to
use a different/changed resolver API and then to also ask them all to
implement their own caching.

Of course it's also less likely to succeed on its own -- caching in the
stub resolver is no replacement for a good full (caching) resolver
implementation that can be shared amonst multiple processes and multiple
clients and as such anyone with the foresight to ensure a local full
resolver is always available will be less inclined to want to bother
implementing any stub resolver caching.  This is evident by the fact
that the mostly widely used existing stub resolver implementation still
implements no caching of its own.

> Besides the problems noted above (how can the browser know there is a
> local cache?

The browser is not supposed to know if there is a local cache being used
or not.  If local caching is not available then performance suffers, but
the browser just browses and doesn't care either way how long it takes
to do what it does.  If the user complains about performance loading
pages with too many URLs then the user is told to try using a more local
and proper DNS cache.  All of us "experts" always knew that recursive
DNS caching was critical to the successful deployment and use of the
DNS, but now excessively poor HTML design is showing this need to
ordinary users as well, just not in such concrete and obvious ways as we
might have hoped for.

>  how can the browser distinguish DNS information from NIS
> information?),

The browser is not supposed to know the difference between DNS names and
NIS (or whatever) names.  The whole idea of integrating NIS (and even
/etc/hosts) into gethostbyname() is to make local naming conventions
vs. Internet naming transparent to the users.  The browser makes use of
this transparency through a common API and just asks for the name to
address translation without knowing where the name comes from.

> any stub resolver must start with a known list of the
> IP address of global name servers, and that list sometimes changes.

This list of addresses for global root servers is private information
that's effectively internal to the resolver implementation and certainly
not something every application needs to be aware of.

> if every stub resolver is a cache, then we need a new global
> mechanism for updating this list on a running system.

No, not really -- this information is extremely static already, and is
trivially updated at runtime after just one query to even just one well
known address.  The whole list doesn't always have to be correct any
more than it is necessary for all root servers to be equally reachable
from all clients world-wide at all times.

> Right now the administrator is responsible for tracking the relevant
> list;

I can't remember the last time a root server address was ever changed or
removed, and for certain they change or disappear far less often than
even ordinary people manage to upgrade their OS software and I don't
think the whole list has ever been turned over.  I.e. for all practical
purposes the OS vendor is the one reponsible for tracking the current
list of global root servers.

-- 
								Greg A. Woods

+1 416 218-0098;            <g.a.woods@ieee.org>;           <woods@robohack.ca>
Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>