Subject: Re: Plans for importing ATF
To: Julio M. Merino Vidal <jmerino@ac.upc.edu>
From: David Holland <dholland+netbsd@eecs.harvard.edu>
List: tech-userlevel
Date: 11/05/2007 21:47:53
On Mon, Nov 05, 2007 at 02:46:15PM +0100, Julio M. Merino Vidal wrote:
 > As a matter of fact, there are a couple of sh(1) regression tests  
 > that fail at the moment in current, and I bet they remain unfixed  
 > because no one actually executed the test suite to discover them  
 > (which is understandable because it's not trivial to do).  

Enh, what's so hard about "cd /usr/src/regress/bin/sh && make regress"?

I think the problem is that most of the sh tests check things
sufficiently obscure that most people aren't willing to venture an
opinion regarding whether it's sh or a failing test that's wrong.
Then, the set of people willing to fiddle inside sh probably isn't
that large either.

I remember taking a look at the set_e.sh failures a while back,
actually, and I think the conclusion was that at least some of those
test cases only fail inside the complex eval and quoting regime found
in the test script... which means that what's wrong must be something
pretty subtle inside sh and there's likely no easy fix.

Admittedly it would help a good deal if the test failures were put in
front of people, because then you only have to take the intersection
of the above two groups of people, instead of the intersection of
those with the small set of people who happen to run the sh tests.

-- 
   - David A. Holland / dholland+netbsd@eecs.harvard.edu