> Unfortunately, even when I installed a big disk and made a huge swap
> partition, I got the same UVM out of memory messages

Was the swap space configured to be on?  Sysinst should enable it, but
it may not.  Can you shell out of sysinst (or suspend it or something)
and run 'swapctl -lk'?  If your swap device isn't listed, you can run 
'swapctl -a /dev/sd0b' (or whatever your swap device is), once you've
labelled the disk.

-allen