I have experienced this problem before, but I cannot recall what I
concluded the problem was.  I think I need to go drink some coffee. ;)

Intuitively, here's what I think is going on:

The data stream between your software, the kernel, and your hardware
is becoming desynchronized.  The azalia driver is especially prone to
this because DMA access (see bus_dmamem_mmap(9)) to the buffer is not
implemented.  If you're interested in the reasons for this, I can
expound more on that in another email.  More specifically, my guess is
you are experiencing buffer underruns.  I can't hear the sound myself,
so it's impossible for me to say if this isn't an IRQ conflict.  An
IRQ issue, however, would not manifest itself over time as you
describe, and so my money's on a DMA issue.

The solution is for you to learn the HDA CORB/RIRB protocols, perhaps
necessarily extending our audio API in the process. ;)  In the
interim, I suggest you try fiddling with the low and high watermarks,
buffer size, blocksize, etc using audioctl(1).  You can probably find
some information on this by searching mail-index.netbsd.org.  Please
do read audio(4) as well.

If you make no progress, then file a PR and (optionally) contact
kent@netbsd.org for assistance.

Is your CPU under load when this is happening?   If so, another thing
you might want to try (though I suspect it won't help) is setting

options HZ=1024

in the kernel.  This will give you ~10x as many context switches in
exchange for a mild performance penalty for computationally-intensive
tasks.

Regards,

--Blair