[ Reposting because the previous mail was cut, presumably because of  
a '.' that appeared on a line on its own. ]

On 08/07/2007, at 21:12, David Laight wrote:

> On Sun, Jul 08, 2007 at 05:34:54PM +0200, Julio M. Merino Vidal wrote:
>> Hello,
>>
>> [ Please CC me any replies ]
>> [ CCing Marting because he is my SoC mentor ]
> ...
>> It seems to me that keeping
>> each test program in its own directory will be overkill, given that
>> test programs should be fine-grained and thus there will be many of
>> them.
>
> Are you sure that multiple test programs is the way to go?
> You'll almost certainly end up with a lot of replicated and/or
> red tape code that makes the actual test difficult to see.
> Any changes to the infrastucture will then need making to every clone
> of the test program.
>
> Certainly you want to be also to run each test separately, but that
> doesn't require one program per test.

But it makes sense to keep them separate, in some cases.   
Furthermore, it's up to the person developing the tests to write them  
as a single program or multiple ones, so he can choose what better  
suits his needs.  (For an example see src/regress/sys/fs/tmpfs; I  
think those different test programs make sense, and collapsing them  
all into a single one could be confusing.)

Anyway, that doesn't mean that code is duplicated among the tests.   
I'm keeping all the infrastructure centralized so that the test  
program's code is minimal.  Test programs written in C/C++ rely on a  
shared library, and test programs written in sh include a file that  
lives in share.

> ...
>> Following the current principle of one program per source directory
>> will make our source tree explode in the number of required inodes to
>> host it -- an extra Makefile plus a directory for each of these tiny
>> programs!
>
> Given the extra directories (etc) would be subdirectories of a single
> program why is this an issue?

I'm not sure I understand you here.  The current layout I have looks  
like this:

calypso:~/Projects/NetBSD/src/tests> find .
   ./Atffile
   ./Makefile
   ./atf
   ./atf/Atffile
   ./atf/Makefile
   ./atf/tools
   ./atf/tools/Makefile
   ./atf/units
   ./atf/units/Makefile

'tools' builds and installs the test programs that test the atf tools  
themselves.  Similarly 'units' has test programs for the different  
modules that form the atf's library.

Sure, I could mimic the current usr.bin directory and keep those in  
tests/usr.bin/atf-* or tests/lib/libatfprivate/...  I'm unsure about  
this part.  Hmm yeah, why haven't I done it that way?  (The only  
reason I can think about now is that keeping everything "spread"  
doesn't let you easily test a given component, whereas the current  
structure does.)  If I went this route, then these changes to  
bsd.prog.mk wouldn't be needed that much.

> You'll end up with more inodes used for the objects and programs  
> anyway.
>
>> My bsd.test.mk file is built on top of bsd.prog.mk file for
>> simplicity (and because I see no reason to do it otherwise).
>
> Did you look at the existing regression tests ?

At some of them, yes.  Will look at some more, but they'll have to be  
rewritten anyway.

>> In order to get the above to work, I have adapted our bsd.prog.mk  
>> file
>> to support building several different programs in the same directory.
>
> My view is that these makefiles are too complicated already (yes I  
> have
> modified them myself) and that this adds complexity that isn't needed.

Back when I wrote the tmpfs tests, I was annoyed about not being able  
to build different helper utilities straight from a single Makefile,  
so I had to resort to an ugly workaround.  It seems to me that this  
extra functionality could be welcome in some places/by somebody else.

I'm also trying to simplify some of the current pieces of that file  
(by defining some aux variables, for example), so actually I think  
this won't be that bad and may even make things more understandable.

-- 
Julio M. Merino Vidal <jmmv84@gmail.com>