It's extremely unlikely Talktalk are having issues supplying your 2Mb ADSL line.
What you describe is nothing to do with the ISP at all, but your upstream being completely saturated hugely impacting latency.
Try setting up QoS if your kit has such settings.
I get the exact same on my 45Mb FTTC. If I'm uploading at full speed the downstream will drop considerably and latency increases from 16ms to +100ms.
Edited by j0hn83 (Sun 12-Aug-18 15:41:57)