[[setitimer() vs setttimeofday()]]

One plausible answer is to never, ever use settimefoday(), except
immediatley after booting.  I use NTP, which usually steps the system
clock with settimeofday *once*, at boot time, and  slews it via
ntp_adjtime() thereafter.

Why do you need to call settimeofday()?    
Is it, perhaps, because  you're running timed?