Advice Req'd: Loss of PPP session (long post)

Hi,

My local exchange (Woolston, Southampton) went over to 21CN on 4th Jan this year and I migrated to the upto 20mbit/s service with my ISP (Zen through BTw). Since then, I have been having problems with loss of PPP session for up to 1 hour late afternoon/early evening (with other shorter disconnects as well) most days.

I am currently running from a microfilter in the test socket with a corded handset also attached. I have tried 4 different microfilters and three different modem routers (2 585v6 and a Netgear DGN2200) and get drops with all combinations. Today is a typical fairly typical:

My Broadband Ping

I managed to capture router stats before, during and after the drop 18:10-19:00:

Before (10:45):
Uptime:0 days, 0:29:22
Modulation:G.992.5 annex A
Bandwidth (Up/Down) [kbps/kbps]:660 / 8,280
Data Transferred (Sent/Received) [MB/MB]:1.97 / 32.80
Output Power (Up/Down) [dBm]:12.5 / 21.0
Line Attenuation (Up/Down) [dB]:24.0 / 43.5
SN Margin (Up/Down) [dB]:6.5 / 9.5
Vendor ID (Local/Remote):TMMB / TSTC
Loss of Framing (Local/Remote):0 / 0
Loss of Signal (Local/Remote):0 / 0
Loss of Power (Local/Remote):0 / 0
Loss of Link (Remote):0
Error Seconds (Local/Remote):10 / 0
FEC Errors (Up/Down):328,090 / 11,930
CRC Errors (Up/Down):1,677 / 22
HEC Errors (Up/Down):252 / 21

During:
Uptime:0 days, 8:05:34
Modulation:G.992.5 annex A
Bandwidth (Up/Down) [kbps/kbps]:660 / 8,280
Data Transferred (Sent/Received) [MB/MB]:52.99 / 172.49
Output Power (Up/Down) [dBm]:12.5 / 21.0
Line Attenuation (Up/Down) [dB]:24.0 / 43.5
SN Margin (Up/Down) [dB]:6.5 / 2.0
Vendor ID (Local/Remote):TMMB / TSTC
Loss of Framing (Local/Remote):0 / 0
Loss of Signal (Local/Remote):0 / 0
Loss of Power (Local/Remote):0 / 0
Loss of Link (Remote):0
Error Seconds (Local/Remote):2,350 / 0
FEC Errors (Up/Down):328,090 / 3,043,748
CRC Errors (Up/Down):1,677 / 145,592
HEC Errors (Up/Down):252 / 139,066

After (21:05):
Uptime:0 days, 10:23:48
Modulation:G.992.5 annex A
Bandwidth (Up/Down) [kbps/kbps]:660 / 8,280
Data Transferred (Sent/Received) [MB/MB]:53.45 / 173.06
Output Power (Up/Down) [dBm]:12.5 / 21.0
Line Attenuation (Up/Down) [dB]:24.0 / 43.5
SN Margin (Up/Down) [dB]:6.5 / 7.5
Vendor ID (Local/Remote):TMMB / TSTC
Loss of Framing (Local/Remote):0 / 0
Loss of Signal (Local/Remote):0 / 0
Loss of Power (Local/Remote):0 / 0
Loss of Link (Remote):0
Error Seconds (Local/Remote):3,092 / 0
FEC Errors (Up/Down):328,090 / 6,346,640
CRC Errors (Up/Down):1,677 / 191,619
HEC Errors (Up/Down):252 / 185,157

Which, from the changes in the downstream SN margin, suggest to me that there is variable noise on the line which is disrupting the PPP session. There is nothing electrical in the house that is operating solely at the times that the drops have been happening. In between the drops the BT Speedtester gives consistent download speeds for the IP6500 profile that I am currently on.

I have been discussing this problem with my ISP (and the Zen Tech Support Team have been very responsive and knowledgable). So far we have arranged a Openreach engineer visit for this coming week as I think that I have exhausted all the things that I can try in order to pin-point the problem.

I would appreciate a sanity check on what I have done so far and the conclusion that I am coming to regarding noise on the line. I also had a couple of specific questions:
- Have I missed anything obvious?
- Is the fact that only the PPP session drops and not DSL sync odd (my assumption would be that noise that causes the PPP session to drop would also give a DSL drop as the sync speed is higher than the PPP speed)?
- Is there anything that I need to mindful of when the Openreach Engineer visits?

Thanks and apologies for the length of the post!

Too many errors. Get your ISP to have Openreach take a look.

I had brought the errors with my ISP. My ISP have escalated it to BT/Openreach due to intermittancy on the line and said that they were testing the line. I enquired about the testing that BT/Openreach were doing on the line and got this response:

they would have looked at the variation in Noise maring, and loop loss on the circuit, and at any errors seen.

There conclusion was

Right when tested remote diagnostic;All BT tests completed ok, diagnostics show no fault.;N/A

So require an engineer visit to investigate further.

Would this cover it or do I need to ask them to do anything else?

Thanks.

the PPP session may be failing due to data corruption or timeouts when you get hit with all those errors. Odd that the modem doesn't decide to retrain.

I would be tempted to find a Netgear 834v3 (NB the v3 or earlier reqd) or another TI AR7 based router (earlier Zyxels - the black ones in the 660 range I think AAISP sell them) as your exchange chipset is TSTC and you may find you hit a sweet spot if both ends are the same.

The other thing to try would be forcing G992.3 or G992.1 (ADSL2 / ADSL) to see if they are more robust than ADSL2+ as they potentially use less frequency spectrum.

Thanks for your response Phil.

The modem that I'm using currently is a brand new 585v6 which is AOL locked (I've managed to unlock the web interface), but I've checked the TBB line monitor from when I was using my original 585v6 and a brand new DGN2200 and they both show the intermittant 1-2% packet loss spikes as well (which I had figured were indicative of the downstream errors), so I think the downstream errors have been present all the time. Neither of the other two modems were retrainng to lower the errors. This suggests to me that either there's a generic problem with Broadcom chipsets (at least on my line) or something's not working correctly at the exchange.

I had looked at the Zyxel 661, which I think uses an AR7 chipset, as this had been recommended by an Apple reseller (I use an iMac), but had gone for Broadcom chipsets due to their reputation for stability - I have been toying with getting a Billion 7800N as a main modem and keeping the 585v6 as a back-up.

I had also thought about forcing ADSL1 or 2, but I'm pretty sure that it can't be done on the AOL 585 I'm using (another reason I was considering the 7800N).

I'll have to discuss this with the OpenReach Engineer on Tuesday as I won't be able to get another modem before this.

Thanks again.

This sounds like one of my line at the moment.

Good sync and margin but frequent *random* drops in PPP.

The last lot of error figures from the line where

Line characteristic: D/U
Speed: 20632 / 1163
SNR Margin: 5.6 dB / 6 dB
Loop Loss: 25.8 dB / 12.3 dB
Errored Seconds: 44 / 4
HEC Errors: / 13

RRT Downstream trends (min / avg / max)
MTBE (seconds) : 1 / 343 / 3600

RRT Upstream trends (min / avg / max)
MTBE (seconds) : 2 / 2440 / 3600

This is an ongoing case - so far since Nov

Edited by IamQ (Sat 12-Feb-11 22:40:20)

Just thought I'd post an update with how things are progressing.

An OpenReach Engineer (very pleasant and chatty guy) visited last Tuesday and replaced the NTE5 backplate with a new one, fitted a NTE2000 filtered front plate and fitted longer tails for the main cable and the internal extension wires. He explained that BT Wholesale had 'flexed' the line twice and his portable modem showed a downstream sync speed of 7.2Mb/s (SN margin 9.5dB) - my 585v6 gave 7.9Mb/s with the same SN margin. Things were stable for the rest of the evening. He also said that BTW could see my router trying to make multiple attempts to log-in when the PPP dropped until it got accepted and they thought the problem was with the ISP system.

However, since then I have still been getting the PPP drops for 45min-1hr in the late afternoon/early evening. My ISP have been continuing their investigations and all my kit is back in the test socket.

A couple of days ago my router's downstream sync speed has dropped to about 6.1Mb/s and the SN margin has gone to 15dB. Since this has happened, the PPP session doesn't drop, but I get 10-20% packet loss. e.g. Tonight between 19:45 and 21:00: My Broadband Ping

Router stats during packet loss episode:
Uptime:0 days, 6:13:35
Modulation:G.992.5 annex A
Bandwidth (Up/Down) [kbps/kbps]:684 / 6,135
Data Transferred (Sent/Received) [MB/MB]:14.28 / 32.20
Output Power (Up/Down) [dBm]:12.0 / 20.5
Line Attenuation (Up/Down) [dB]:24.0 / 43.5
SN Margin (Up/Down) [dB]:7.5 / 6.5
Vendor ID (Local/Remote):TMMB / TSTC
Loss of Framing (Local/Remote):0 / 0
Loss of Signal (Local/Remote):1 / 0
Loss of Power (Local/Remote):0 / 0
Loss of Link (Remote):0
Error Seconds (Local/Remote):1,770 / 0
FEC Errors (Up/Down):107,138,407 / 2,757,118
CRC Errors (Up/Down):10,379 / 101,901
HEC Errors (Up/Down):5,255 / 97,211

After packet loss episode:
Uptime:0 days, 7:29:59
Modulation:G.992.5 annex A
Bandwidth (Up/Down) [kbps/kbps]:684 / 6,135
Data Transferred (Sent/Received) [MB/MB]:15.88 / 42.31
Output Power (Up/Down) [dBm]:12.0 / 20.5
Line Attenuation (Up/Down) [dB]:24.0 / 43.5
SN Margin (Up/Down) [dB]:7.5 / 14.0
Vendor ID (Local/Remote):TMMB / TSTC
Loss of Framing (Local/Remote):0 / 0
Loss of Signal (Local/Remote):1 / 0
Loss of Power (Local/Remote):0 / 0
Loss of Link (Remote):0
Error Seconds (Local/Remote):5,036 / 0
FEC Errors (Up/Down):107,138,407 / 6,386,900
CRC Errors (Up/Down):10,379 / 304,304
HEC Errors (Up/Down):5,255 / 299,185

I managed to catch one of these episodes on Friday and spoke to the Tech Helpline of my ISP. One of the tests they had me do was to reboot the router and the downstream sync speed dropped to 3Mb/s, again with a 15dB SN margin.

So it looks to me as though there is some sort of intermittent fault/noise on the line causing about 9dB change in the SN margin which leads to packet loss and errors. It seems that the sync speed affects the amount of packet loss during these noisy periods as the router holds the DSL connection through all this. My IP profile is currently 2500 - as a result of the reboot that I did during the noisy period I think.

I'm glad that I managed to talk to my ISP whilst one of these events was happening as they have now seen it and have arranged another OpenReach Engineer visit - I just hope the problem occurs when he is around to see it!

Any ideas on what might be causing the noise (I've considered REIN, but my neighbours on BTW lines aren't seeing any problems at the same time as me)? Oxidised cable join reacting badly to drops in temperature (it always happens late afternoon/early evening? Exchange equipment?

The OR guy who visited mentioned the possibility of doing a lift and shift - is anything else I need to make sure he does before resorting to this?

Make sure the engineer performs a successfully pair quality test using their JDSU/EXFO or Hawk. Try and see if you can take a peek over their should whilst the results come in. There should be no more 10 ohms difference between the the two results for A to earth and B to earth, preferably the difference should even less. A big difference might indicate an HR fault.

Thank you for the advice Zarjaz.

A second BT OpenReach engineer visited last Friday - he carried out a pair quality test which came back okay. However, he did a lift and shift to another pair back to the exchange - the pair was in a different cable run. As a result, the line attenuation has gone down by about 8dB, sync rate has gone up 10.5-11.2Mb/s and the IP profile has settled to 9.0Mb/s:



Still getting 1 drop/day and some bits of packet loss, but I'm not seeing the extended drops/packet loss that would knock the connection for up to 1.5hours at a time: My Broadband Ping

I'm thinking that the main problem was a temperature sensitive fault in the original cable pair. My ISP are monitoring the line still due to the drops. Aside from the drops, does the level of packet loss look fairly standard? Could there be something a bit flaky at the exchange that is causing the remaining drops?