Re: common configuration format (was Re: sysinst split project - The Configuration File)

[Matthew Mondor <mm_lists%pulsar-zone.net@localhost>]

On Wed, 16 Dec 2009 16:07:18 -0600
David Young <dyoung%pobox.com@localhost> wrote:

> On Fri, Dec 11, 2009 at 02:16:44PM -0600, David Young wrote:
> > Use XML for the configuration.  That ensures that the configuration is
> > queryable/transformable with NetBSD's forthcoming xmltools.  I know that
> > writing XML by hand is objectionable, so establish a one-to-one mapping
> > from your user-friendly format to an XML format.  Provide a tool to/from
> > the XML format.
> 
> I think that this point is important, but it seems to have caused a lot
> of misunderstanding and FUD, so I am going to try again.
> 
> If NetBSD puts structured[1] configuration files into a common
> format[2], then it begins to make sense to create the tools for the
> base system that can process that format in pipelines and in scripts.
> If we keep adding an ad hoc configuration format for every new
> application, like the bad old ways of UNIX, then it is not useful
> to write even a single tool to process anything.
> 
> It does not matter as much whether the common format is JSON, YAML,
> XML, or something else, as it matters that there is one format to
> process.  However, I recommend making XML the common format because so
> many programs and services already generate so much useful XML, because
> memory- and CPU-efficient tools for processing it in the UNIX idiom are
> in the works for NetBSD, and because XML, in the form of property lists,
> already has a lead over other formats in NetBSD.

Please note that this reply is not particularily argueing about the
sysinst configuration file format but is to continue the discussion
under the new topic "common configuration format".

XML and plists only entered recently in the picture on NetBSD, and
there have been older commonly used formats, i.e. the (admitedly ugly)
capabilities database text format along with its optimized binary db
format (cap_mkdb(1)), so I wouldn't be ready to say that XML has the
lead on NetBSD yet...  But I don't think that we had a commonly used,
decent, hierarchical format yet until plists :)

That said, if the binary plist format is efficiently designed and is
eventually supported, then it would be nice to indeed be able to
support it everywhere it's sane to do so, especially if a tool allows
to export/import XML (and potentially other format(s)).  Until then,
even though XML plists might suit a few scenarios, capabilities database
+ db still have a performance advantage in other situations.

So there's still some work to do for plists to become the leading
configuration format on NetBSD in my opinion, but it certainly could
get there.  Possibly that a common configuration tool could even
eventually exist again (I say again as there once were sushi in base,
which noone ever used and required too much work to be useful; maybe
another reason was that the NetBSD user base was more comfortable with
editing a few files using vi, often having sysadmin or developer
background).

Depending on how much integration is desired, if/when that happens,
other ad-hoc formats we use (such as the ini-style login.conf) could
also migrate to the new format.  Than perhaps that we even could
provide a wrapper in the style of vipw, exporting to the user's
favorite format and invoking the user's favorite editor, taking care of
locking, saving by importing back to binary (or whichever supported
storage format the admin configured the system to use, as long as the
plist loader library can differenciate the formats, otherwise a
binary format would most probably be the most efficient for performance
and space)...

And if XML was the storage format of most configuration files
in the long term, it wouldn't matter much, if the tools were available
to export/import/edit in human-friendly format, and that an efficient
binary file could be generated for circumstances where performance
matters (and that file have loading precedence over the XML file when
present, just like in the case of capability databases).
-- 
Matt