Subject: Re: Tracking Prevalance of Bugs
To: Curt Sampson <cjs@cynic.net>
From: Joseph Sarkes <joe@js.ne.mediaone.net>
List: current-users
Date: 05/03/1999 07:08:31
Curt Sampson writes:
> One of the things I've realised during the release process for 1.4
> is that we don't really have a way of tracking how many people are
> affected or likely to be affected by a particular bug. 
> 
...

> Unfortuantely, I don't have a solution for this at the moment. I'm
> just opening up this topic to see if anyone here has any thoughts
> on better ways to approach this problem.
> 
> cjs
> --
> Curt Sampson  <cjs@cynic.net>   604 801 5335   De gustibus, aut bene aut nihil.

How about a survey included with the distribution/snapshot/whatever. The
thing could include short descriptions of what changes have been made in
a particular area and then a section that can be grep'd for the results.
ie. 
[ ]UVM101 worked no problems 
[ ]UVM102 minor installation problems 
[X]UVM103 died a horrible death

[ ]bash301 worked no problems ...

and then have the survey emailed back to the central decoding/statistical
analysis/results posting/comment/additional data grabbing, super duper site.

anyways, one could ask specific questions about the system aspects that have
changed, and more general catchall questions to pick up interactions that
may have statistical significance but no direct relation to changes.

A stored state section could keep the system information (arch, peripherals,
memory, version, etc.) along with the past history of survey data in some 
sort of compressed form. This way, followups could be done to check whether
fixes take care of the problem reported, and don't cause other lossage.

At least on systems running -current or _ALPHA or _BETA this would make
sense, and the facility could exist on standard distributions for additional
feedback, automatically sending the version, hardware setup, etc. to 
help troubleshooting where the actual problem is. 

Perhaps /kern/msgbuf would be useful info to optionally include, unless 
there is a better system configuration description. 

One important thing would be to ensure that whatever was done wouldn't
grow to monster proportions, as my text above managed to do. One thing
that might work would be an initial system registration with configuration
info, etc. that returned a "cookie" to index into the database afterwards.
Then when updates were sent in, only the "cookie" is sent as the system
description, and local checks could be made to determine if a new cookie
is needed due to hardware changes, and a second "version cookie" could
be manufactured locally to describe the os version/sup date and enabled
options.  This could even be usefull to show what was enabled during the
build of the current kernel, as I have not seen this info available in
my searching around the system.

Obviously there is much possibility here, and it would need to be done
in a non-intrusive manner considering current privacy views. Anyways, 
these are my initial thoughts on the matter. Whether a web based html
form with cookies, or a separate protocol/utility is better, etc. is
better left for others to decide, but a small built in protocol for
the specific task might be better than requiring web browsers, etc.

I notice that the wind is picking up here rather than dissipating, so
it is better I shut up now.

-- 
Joseph Sarkes                   mailto:joe@mediaone.net