greetings-

    Chad Mynhier and i have been looking into the X slowdown he
has been seeing under bsd, and i thought i'd post a status report:


to recap the problem, if you take the following Tk/Tcl program:

foreach level { a b c d e f } {
        foreach sublevel { a b c d e f } {
                toplevel .$level$sublevel
                               
                foreach name { a b c d e f } {
                        button .$level$sublevel.$name -text "$name"
                        pack .$level$sublevel.$name -side left
                }              
        }                      
}
update
exit

and save it to a file called "tktest" and start doing "time wish8.0 tktest" 
you may find that its run time gets longer and longer each time you run it.


here is what we found:
1. the problem only occurs when running an "fvwm" window manager.
   if you are not running a window manager at all, or using "twm"
   then you will not see any slowdown.

2. when running under "fvwm2" you will find that both "fvwm2" and
   the Xserver's memory usage grow the more you run the program.

3. the problem with "fvwm2"'s memory growth has been tracked down
   to the I18N patches that are applied to it by the BSD pkgsrc system.
   (I18N == internationization support for fvwm2...)

   let me draw your attention to this section of the XmbTextPropertyToTextList
   man page:

       To free the storage for the list and its contents returned
       by XmbTextPropertyToTextList, use XFreeStringList.  To
       free the storage for the list and its contents returned by
       XwcTextPropertyToTextList, use XwcFreeStringList.

   if you look at the fvwm2 sources (after the I18N patches have been 
   applied) you will find many calls to XmbTextPropertyToTextList, but
   no calls to either XFreeStringList or XwcFreeStringList anywhere in 
   the main fvwm2 code.  [i found two in the helper FvwmSave programs, 
   but not in fvwm itself].

   we contacted the guy who maintains the FreeBSD fvwm2 port (he is
   one of the sources for the I18N patches) and he says it is a known
   problem that he is currently too busy to fix.


   my advice is that if you want to use fvwm2 from the pkgsrc, turn
   off I18N in Fvwm.tmpl unless you really need it (look for 
   OPTION_DEFINES -- make sure you get both I18N's).

   with I18N switched off, the fvwm2 process' memory usage no longer
   grows when running the tktest program.

4. the problem with the Xserver's memory growth under fvwm2 has
   been partly tracked down.

   background: XF86 has an "internal malloc" that uses the following
   memory allocation policy:
    - small allocations come from a private pool of fixed sized
      buffers that are never freed (this pool is allocated with
      the system malloc)
    - medium allocations are allocated with the standard system malloc
    - large allocations (>11KB) are allocated using mmap(2) MAP_ANON.

   an interesting quirk of the internal malloc is that when you call
   realloc [Xrealloc], the internal malloc always copies the data
   (most realloc's only copy the data if it can't be grown in place).

   please read the comments in the top of xsrc/xc/programs/Xserver/os/xalloc.c
   for detail.


   using DDB's "ps/a" and "show map" commands, the X server's memory
   usage pattern was observed this way:

     [a] run "tktest" 3 times to prime things
     [b] take measurements of X server's memory usage
     [c] run "tktest" 200 times
     [d] take measurements of X server's memory usage again

   comparing the measurements in [b] and [d] the following was discovered:

     [1] the BSS shrunk by 2 pages (8KB)
     [2] the zero-fill mmap'd area grew by 39 pages (156KB)

   so, clearly the internal malloc's mmap(2)'d areas grew by 39 pages
   for the 200 runs of tktest.


5. we put printf's in the internal malloc to note whenever it mmap'd
   or munmap'd memory.   the result was the following:

   [a] there is no memory leak... all mmap allocations are later munmap'd
   [b] the first 15 times "tktest" runs, there is one mmap/munmap pair:

        mmap 20480 0x40778000 0x1228b2          # format: size, ptr, frameptr
        munmap 20480 0x40778000                 # format: size, ptr

       by translating the frame pointer (0x1228b2) back to a function, i can
       see that that alloc is taking place in _BufFilePushZIP().   this is
       fine. 
   [c] on the 16th run of "tktest" there are suddenly a truckload of 
       mmap/munmaps per run.    by the time "tktest" has been run 200 
       times there have been 34460 calls to mmap (and the same number 
       of calls to munmap).   if you are not running "fvwm2" then there
       would have only been ~200 calls to mmap.
   [d] of the 34460 calls about ~200 of the are from _BufFilePushZIP,
       and the rest are from Xrealloc().   an interesting thing to note
       is that the calls mmap from Xrealloc() are growing.   at the start
       of the 16th run we get calls from Xrealloc like this:

	    mmap 12288 0x40778000 0x1b1e	# run 16: allocate 12KB

       but by the 200th run of tktest they are like this:

	    mmap 151552 0x4079e000 0x1b1e	# run 200: allocate 148KB

       this is the source of the X server's memory growth.

6. we printed out the frame pointer in Xrealloc() to see what was
   doing all these allocations.   the offending function?   it is
   FindColor() in xc/programs/Xserver/dix/colormap.c.

       ppix = (Pixel *) xrealloc (pixp[client], (npix + 1) * sizeof(Pixel));

        (from xc/programs/Xserver/dix/colormap.c)

   clearly the value of "npix + 1" is growing without bound when
   you run fvwm2.   i'm not sure why.


7. looking at the log file, on the 200th run of tktest, realloc
   gets called 185 times (each call realloc's a size of 151552 -- 148KB).
   185 * 148KB = 27.38MB ... so the X server is bcopy'ing 27.38MB
   of RAM on that final run of "tktest" ... this seems like a possible
   source of a slowdown.



i'm not an X expert, but from looking at the fvwm2 source code the only
thing that looks suspect is the calls to XAllocColor() in add_window.c.
the man page for XAllocColor claims there is an XFreeColors() function.
I only see one call to that function and it is in builtin's FreeButtonFace().

when is it appropriate to call XFreeColors?   if you XAllocColor when
you add a new window, should you XFreeColors when the window goes away?
does anyone know?


chuck