Subject: Plans for importing ATF
To: None <tech-userlevel@NetBSD.org>
From: Julio M. Merino Vidal <email@example.com>
Date: 11/04/2007 20:53:24
I will try to release ATF 0.3 within this month and, when that
happens, I feel that the code will be in pretty good shape to be
imported into the tree. (Skip to the last part of the mail if you
are in the mood for bikesheds... but let's hope not ;-)
Here is what I plan to do:
- Import a verbatim copy of the final release into dist/atf.
- Apply the patches that you can find in the following address, where
each of them represents an individual commit:
Click on "Browse Files" later on, and then on "patches".
I can't post a direct URL because it'd become obsolete quickly, as
there seems to be no way to link to the head revision's contents
from ViewMTN. Sorry.
The 'series' file in that directory specifies in which order the
patches will be applied.
- Update doc/CHANGES. No patch for this as it was a PITA to keep
up to date.
What the patches do, basically, is the following:
- Add reachover makefiles to build and install the ATF libraries,
tools and tests.
- Change sysinst to deal with the new 'tests.tgz' test.
- Update some documentation to refer to the way the user can run
the tests. (Basically afterboot(8).)
- Convert the regress/bin tree to the new tests infrastructure.
The original files disappear and the new ones are put in
- Like previous one, but for the regress/sys/fs/tmpfs tree.
The new tests end up in tests/fs/tmpfs.
Now... there are some changes that I know will be controversial:
- ATF can -- OPTIONALLY and NON-EXCLUSIVELY -- generate XML reports
with the results of test suites. These are ideal to later generate
pretty HTML pages that can be served to other developers (hmmm,
now with our built-in server!).
These generated XML reports need a DTD that describes their
structure, and this DTD is required for tools such as xsltproc.
Hence, I am installing the DTD into /usr/share/xml/atf/ and I am
putting an XML catalog in /usr/share/xml/catalog.
(NO /etc/xml/catalog yet.)
Why oh why am I doing this if we don't have tools to deal with XML
in the base tree? Because the end user won't know how to set up
catalogs himself, and he need not know. Having the files in place,
it is trivial to make xsltproc find them by doing a minor edit to
/usr/pkg/etc/xml/catalog and redirecting the lookups to the
/usr/share/xml/catalog file. (We can and will automate this later
on in pkgsrc.)
So, having the DTD installed means one can simply do:
$ atf-run | atf-report -o xml:mylog.xml
$ xsltproc -o /var/www/tests.html .../path/to/xsl mylog.xml
$ cp .../path/to/css /var/www
And have a nice tests.html page to be served to users.
- Mimicking the previous point, I install a XSLT stylesheet to
easily convert from XML to XHTML, and I put it in
/usr/share/xsl/atf/. This file needn't be there because the
user has to give its path to xsltproc as shown above, but it
seems standard practice to keep xml and xsl hierarchies in
parallel. At least that's what we do in pkgsrc.
If we add the xml subtree, I see no harm in adding the xsl one
- Just for completion, the .../path/to/css above resolves to
/usr/share/examples/atf/tests-results.css. This really is an
example file, as opposed to the XSLT, because they user has to
copy it to htdocs and most likely edit it. OTOH, the XSLT is
or should be generic enough to not require hand-tuning, hence
not being appropriate for the examples directory.
- The structure of the src/tests tree. This was already explained
And there were no answers. So I went with my preferred approach,
which is the second one.
Nothing else for now, I think.
Please raise your concerns quickly. I probably won't be able to do
all this for the following two weeks, but who knows, maybe I'll find
Julio M. Merino Vidal <firstname.lastname@example.org>