Subject: simplifying htdocs build procedure (long term)
To: None <>
From: Jan Schaumann <>
List: netbsd-docs
Date: 01/29/2006 11:58:32
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable


As you are probably well aware, the way that htdocs works at the moment
is rather complex and requires significant resources.  This directly
affects how users and developers contribute to the online documentation
-- or rather, how often they don't!  We need to make it easier for
anybody to contribute without having to install a large number of
complex applications from pkgsrc or without having to learn the
intricacies of the build process.

I'd like to start a detailed discussion about how we can make this
happen.  I think we should first figure out what exactly our goals are
in the documentation management framework.  Here's what I came up with:

- we want to have a consistent layout, look and feel for all pages
- we want to be able to make global changes without editing a lot of
- all files should be well structured and easy to edit for people who do
  only casual work in htdocs or who just want to make small corrections
- we want to automate the creation of new items, new port additions,
  hyerlinks to packages, ftp links, manual page references etc
- the tools used should have as few prerequisites as possible

I believe that we all agree that this is probably best done by
generating the resulting .html files from some input file(s) as we have
done previously with .list and are doing now with .xml.  If .html files
are generated from "source" files, then they should be treated as
"object" files and not be kept in CVS.  Instead, people should just make
changes to the source file, commit them and the webserver should
generate them automatically.

This stresses the fact that all tools required should not be
resource-hogs and in fact be as few and as as simple as possible.
(Otherwise developers and users won't install them, and the web server
won't be able to handle the load.)

So... let's take a look at our current tools and see if they meet our
criteria.  On the plus side:

- using XML and stylesheets, we do provide a fairly consistent layout
- making global changes works for some areas (footer, TOC design,
  certain entities such as 'last major release')
- we can easily create man page links, package references etc.

On the negative side:

- lots of automation is still missing (the last major release is still
  not put automatically on all port pages, for example)
- the tools used are numerous and huge resource hogs, fickle in the
  output generated (based on version numbers of prerequisite packages)

Now I don't know much about XML->HTML conversion tools, but are we sure
we are using the right tools for this job?  I feel like XML is the right
approach, but the current tools are making it overly complicated.  Does
anybody here know other, better tools to work in this area?  Or are the
tools just fine and we just use them in a brain-dead manner?

Would it be possible (or possibly even easier) to implement this in
straight .xhtml with style sheets, central <div> templates and includes
plus a handful of homegrown tools for things like developer names,
manual pages and package references?  Maybe even something that doesn't
require _any_ add-on packages (think awk)?

Would the results be cross-browser compatible?

What other ideas do people have?


"Life," said Marvin, "don't talk to me about life."

Content-Type: application/pgp-signature
Content-Disposition: inline

Version: GnuPG v1.4.1 (NetBSD)