Current-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Automated email notifications of NetBSD-current build failures

Hi all,

As you may have noticed, periodic automated builds and tests of
NetBSD-current are now being run on a dedicated TNF server,  Reports from these builds and tests are
published at the following URLs:

The scripts generating those web pages also have the capability to
generate email notifications of build failures.  I have been testing
this facility for some two years now by having the notifications sent
only to myself, and I think it is now mature enough for production

This would mean sending the notifications to a public mailing list.
I propose using the current-users list, because there is already an
established practice of manually reporting -current build breakage
there, and the automated notifications are not that different from the
reports that are already being sent by hand, just hopefully more
timely, reliable, and complete.

During the testing phase, the automated reports have typically arrived
in my mailbox many hours before any corresponding manual report on
current-users, and they have had a smaller proportion of false reports
because automated builds eliminate many common human errors that can
cause a build to fail, such as stale files in the build tree.

Nevertheless, there are some limitations to keep in mind:

 - Although false positives have been rare compared to manual reports,
   they do sometimes occur.  In particular, there have been a couple
   of cases where spurious build failures were reported because
   mirroring the CVS repository using rsync can result in a repository
   where only some of the files affected by a multi-file commit have
   been updated.

 - Notifications are generated for build failures only, not for
   failures at the install, boot, or ATF test stage.  The ATF
   tests in particular still have too many non-deterministic
   failures to allow automated reporting.

 - Notifications are currently only generated for a single NetBSD
   port (i386).

 - Notifications are only generated when the build transitions from working
   to broken; additional breakage committed while the build is already
   broken is not automatically reported.

Here is an example of what an automatically generated build failure
report might look like (this one is for the most recent failure at the
time of writing):

  From: NetBSD Test Fixture <>
  Subject: NetBSD-current/i386 build failure
  Date: Wed,  2 Nov 2011 00:56:11 +0000 (UTC)

  This is an automatically generated notice of a NetBSD-current/i386
  build failure.

  The failure occurred on, a NetBSD/amd64 host,
  using sources from CVS date 2011.

  An extract from the output follows:

      --- dependall-mtree ---
      --- mtree.lo ---
      #   compile  mtree/mtree.lo
      cc -O -DNO_MD5 -DNO_RMD160 -DNO_SHA1 -DNO_SHA2 
-I/bracket/i386/work/2011. -c -o mtree.lo.o    
      --- dependall-uudecode ---
      #    create  uudecode/uudecode.d
      /bracket/i386/work/2011. -f 
uudecode.d --     -DNO_BASE64 
      --- dependall-nbperf ---
      --- nbperf ---
      #      link  nbperf/nbperf
      cc -O -I/bracket/i386/work/2011. 
-DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64   -o nbperf nbperf.lo 
nbperf-bdz.lo nbperf-chm.lo nbperf-chm3.lo graph2.lo graph3.lo 
-L/bracket/i386/work/2011. -lnbcompat -lz
      nbperf.lo: In function `main':
      nbperf.c:(.text+0x543): undefined reference to `getline'
      --- dependall-mtree ---
      --- spec.lo ---
      #   compile  mtree/spec.lo
      cc -O -DNO_MD5 -DNO_RMD160 -DNO_SHA1 -DNO_SHA2 
-I/bracket/i386/work/2011. -c -o spec.lo.o    
      --- dependall-nbperf ---

  The following commits were made between the last successful build and the 
failed build:

     2011. joerg src/tools/Makefile,v 1.150
     2011. joerg src/share/mk/,v 1.690
     2011. joerg src/tools/nbperf/Makefile,v 1.3
     2011. joerg src/sys/arch/x86/include/cpu.h,v 1.40
     2011. uwe src/external/gpl3/gdb/dist/gdb/shnbsd-tdep.c,v 1.9
     2011. uwe src/external/gpl3/gdb/dist/gdb/sh-tdep.c,v 1.2
     2011. pgoyette src/usr.sbin/i2cscan/i2cscan.c,v 1.3
     2011. pgoyette src/tests/lib/libc/net/Makefile,v 1.6
     2011. pgoyette src/tests/lib/libc/net/gen_ether_subr,v 1.1
     2011. pgoyette src/tests/lib/libc/net/t_ether_aton.c,v 1.1
     2011. pgoyette src/distrib/sets/lists/tests/mi,v 1.413
     2011. pgoyette src/regress/sys/net/Makefile,v 1.3
     2011. pgoyette 
src/regress/sys/net/ether_aton_r/Attic/Makefile,v 1.3
     2011. pgoyette 
src/regress/sys/net/ether_aton_r/Attic/autogen,v 1.3
     2011. pgoyette 
src/regress/sys/net/ether_aton_r/Attic/main.c,v 1.4
     2011. joerg src/tools/Makefile,v 1.151
     2011. wiz src/share/man/man9/skpc.9,v 1.2
     2011. wiz src/share/man/man9/scanc.9,v 1.2
     2011. uwe src/share/mk/,v 1.691

  More log files can be found at:

Any comments, suggestions, support for, or objections to turning this on?
Andreas Gustafsson,

Home | Main Index | Thread Index | Old Index