There was a problem with Zone Alarm and AVG interaction a while back which, IIRC, had this sort of impact. AVG claimed ZA had admitted it was their fault. Possibly still to be fixed.
Though if you're not using AVG then that's unlikely to be the issue.
Just a thought.