>Number:         32445
>Category:       pkg
>Synopsis:       x11/Xaw3d has an overflow error in geometry computation code, causing xfig to hang under KDE
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jan 03 14:05:00 +0000 2006
>Originator:     Stephan Thesing
>Release:        NetBSD 3.99.11
>Organization:
=  Tel.: +49-681-302-5571      = Universitaet des Saarlandes =
=  Fax.: +49-681-302-3065      = Postfach 15 11 50           =
=  Compiler Research Group     = 66041 Saarbruecken          =
=  FR 6.2 - Informatik         = GERMANY                     =
>Environment:
	
	
System: NetBSD gargoyle.cs.uni-sb.de 3.99.11 NetBSD 3.99.11 (Gargoyle) #5: Mon Nov 7 08:53:22 CET 2005 thesing@gargoyle.cs.uni-sb.de:/local/thesing/netbsd/current/obj/sys/arch/i386/compile.i386/Gargoyle i386
Architecture: i386
Machine: i386
>Description:
 The x11/Xaw3d library has an integer overflow error in the computation of the
  geometry for a Box Layout (file Box.c).  There, the box tries to extend its width until its
   height fits within the constraint height (influenced by the window manager).
 Unfortunately, widths are 16bit  unsigned and in the error case (occuring under KDE, windowmaker, twm
   work fine here:-), the constraint width is 65535, i.e. maximal.
 The code loops until either the computed height is smaller than the constraint height or
  the width exceeds the constraint width.  In each loop iteration, the width of the box layout is
  doubled.  This loop does not terminate, if one chooses unfortunate initial width, as the width
   wraps around if it overflows 16 bits and if the maximal constraint width is SHORT_INT_MAX.
>How-To-Repeat:
 Try to use xfig under KDE.
>Fix:
The following patch for X11/Xaw3d tries to capture the overflow before it wraps around the
 `width' variable, setting the width to the maximal one.
It makes xfig work under KDE.

--- Box.c.orig	1996-10-15 14:41:18.000000000 +0000
+++ Box.c	2006-01-02 15:13:32.000000000 +0000
@@ -352,8 +352,12 @@
 	}
 	else {
 	    width = preferred_width;
+           if (0==width) width=1;
 	    do { /* find some width big enough to stay within this height */
-		width *= 2;
+               if (width>=32768) /* overflow */
+                 width=constraint->width;
+               else
+                 width*=2;
 		if (width > constraint->width) width = constraint->width;
 		DoLayout(w, width, 0, &preferred_width, &preferred_height, FALSE);
 	    } while (preferred_height > constraint->height &&

>Unformatted: