Thanks for the additional feedback from everyone.
Good to find someone on the same exchange with the same issue!
It does seem that ADSL via Sky preforms OK, just VDSL doesn't - I suspect the backhaul for VDSL is awful while ADSL is fine. VDSL is pretty new to this exchange I think.
Here's 10 days of SmokePing
https://www.dropbox.com/s/z6uan3ubhvm98ds/10%20Days.png - it is the case that last night was the worst in the last 10 days.
It does appear normal from what I've seen on a number of online forums - that it takes Sky around 3 months to resolve these issues. It's not really good enough, they need to better manage their networks.
The fact they push hard on advertising they don't do any traffic management - I think that's wrong, I'd prefer to be traffic managed and have critical things like VoIP and streaming media to work and non time critical things to take longer.
The switch to anther provider will be happening in 3 days, shall be interesting to see how that works out!