Signal problems can affect latency on a packet-by-packet basis.
Minimum latency (ie for every packet) is increased when interleaving is applied.
However, any tiny burst of noise (ie signal problems) can cause packet loss - particularly if FEC is turned off (and FEC+interleaving usually go hand-in-hand in the downstream direction). More noise bursts lead to higher amounts of packet loss.
This packet loss amounts to, for some packets, an infinite latency.
Packet loss has an impact to downloads - when you will see some amount of "pause" while the two sides work out that a packet has been lost, and resend the missing packet. If there are enough pauses, download speeds are impacted.
If you want to monitor your packet loss rate (and have a static IP), take a look at the TBB BQM at http://www.thinkbroadband.com/ping
Edit: There's some interesting information relating to packet loss at this tutorial. They set acceptable packet loss levels to 2.5%, while a linked paper on VoIP suggests that 1.5% gives a poor result for telephony.
Edited by WWWombat (Sat 04-Jan-14 11:17:00)