User comments on ISPs
>> Zen Internet

Pages in this thread: 1 | 2 | 3 | 4 | (show all)

Print Thread

deleted
(deleted) Tue 30-Jun-15 07:50:37

Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[link to this post]

Good Morning,

I've recently migrated to Zen, from BT, and am facing an annoying problem - head banging against a brick-wall has started. For the last 3 years my current setup has functioned fantastically on BT, since the migration to Zen things seem to have gone a bit wonky - the Zen aspect may or may not be related.

I have an OpenBSD 5.7 router connected to either an HG612/ECI/TG589 (bridge) modem, via a switch, I encounter the same problem with both. The problem? PPPoE drops frequently after between 1 - 15 minutes of connected time and reconnects, then repeats, the modem sync is not dropping. The router has an OpenVPN (UDP) VPN connection that routes all traffic to the OpenVPN server in the DC. I should add, I still have another line still with BT with the exact same setup and this does not encounter the problem and has been up for some 70 days.

Between migrating from BT -> Zen, the only thing that changed on the OpenBSD router was the PPPoE username/password. From the moment the migration occurred, this problem started occurring.

Thing's I have ruled out:

- Cabling, no errors on switch ports but all cables have been replaced
- Not HG612 or ECI related, problem happens with both (also the Zen TG589 in bridge mode). Initially thought it could be the HG612 bug with UDP/VPNs, however the modem is unlocked and running the latest release. The trick of unplugging and reseating the eth cable doesn't make any difference.
- OpenBSD config, there is minimal kernel PPPoE config same setup works with BT and continues to work
- OpenBSD OS versions (tried 3 different images)
- Rolled back RFC4638 patches, i.e for MTU 1500. The Max Payload is negotiated successfully during the connection process, so I don't believe this is the issue but have tried without anyway.
- LCP echo/replies are all being sent and responded to in a timely manner, there are no ignore/dropped echos/replies before the 'term-req' is received'
- Line itself is stable when using the TG589 in full router mode, i.e PPP does not disconnect as it does in bridge

Enabled debug on the OpenBSD pppoe interface Zen are sending 'term-req' - verified by checking the MAC address sending the term-req - from some forum searches, here and on Kitz, this seems to have come up a number of times with Zen but with no one really getting to the bottom of it?

Jun 28 21:15:56 rtr00 /bsd: pppoe0 (8864) state=3, session=0x2eb output -> 84:26:2b:a2:3c:da, len=139
Jun 28 21:15:56 rtr00 /bsd: pppoe0 (8864) state=3, session=0x2eb output -> 84:26:2b:a2:3c:da, len=139
Jun 28 21:15:56 rtr00 /bsd: pppoe0: lcp input(opened): <term-req id=0x0 len=4 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-3f-75-80-e0-83-59-1b-c3>
Jun 28 21:15:56 rtr00 /bsd: pppoe0: lcp opened->stopping
Jun 28 21:15:56 rtr00 /bsd: pppoe0: phase terminate
Jun 28 21:15:56 rtr00 /bsd: pppoe0: ipcp down(opened)
Jun 28 21:15:56 rtr00 /bsd: pppoe0: ipcp opened->starting
Jun 28 21:15:56 rtr00 /bsd: pppoe0: ipcp close(starting)
Jun 28 21:15:56 rtr00 /bsd: pppoe0: ipcp starting->initial
Jun 28 21:15:56 rtr00 /bsd: pppoe0: lcp send terminate-ack
Jun 28 21:15:56 rtr00 /bsd: pppoe0: lcp output <term-ack id=0x0 len=4>
Jun 28 21:15:56 rtr00 /bsd: pppoe0 (8864) state=3, session=0x2eb output -> 84:26:2b:a2:3c:da, len=12
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp TO(stopping) rst_counter = 0
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp stopping->stopped
Jun 28 21:16:06 rtr00 /bsd: pppoe0: phase dead
Jun 28 21:16:06 rtr00 /bsd: pppoe0: timeout
Jun 28 21:16:06 rtr00 /bsd: pppoe0: disconnecting
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp down(stopped)
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp stopped->starting
Jun 28 21:16:06 rtr00 /bsd: pppoe0: phase establish
Jun 28 21:16:06 rtr00 /bsd: pppoe0 (8863) state=1, session=0x0 output -> ff:ff:ff:ff:ff:ff, len=18
Jun 28 21:16:06 rtr00 /bsd: pppoe0: Down event (carrier loss), taking interface down.<7>pppoe0: lcp close(starting)
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp starting->initial
Jun 28 21:16:06 rtr00 /bsd: pppoe0: phase dead
Jun 28 21:16:06 rtr00 /bsd: pppoe0 (8863) state=2, session=0x0 output -> 84:26:2b:a2:3c:da, len=38
Jun 28 21:16:06 rtr00 /bsd: pppoe0: session 0x2ee connected
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp open(initial)
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp initial->starting
Jun 28 21:16:06 rtr00 /bsd: pppoe0: phase establish
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp up(starting)
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp starting->req-sent
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp output <conf-req id=0x5 len=14 05-06-d2-4f-8e-af-01-04-05-d4>
Jun 28 21:16:06 rtr00 /bsd: pppoe0 (8864) state=3, session=0x2ee output -> 84:26:2b:a2:3c:da, len=22
Jun 28 21:16:06 rtr00 /bsd: pppoe0: lcp input(req-sent): <conf-req id=0xa7 len=19 01-04-05-d4-03-05-c2-23-05-05-06-1a-89-24-a4-00-00-00-00-00-00-00-00-00-00-00-20-85-80-10-ff-1c-cb-c5>

LCP echo request/reply going back and forth, no sign of any timeouts on my end, 4 seconds after the last request/reply the term-req is received from the ISP/BT/Zen.

Jun 29 13:20:42 rtr00 /bsd: pppoe0: lcp input(opened): <echo-req id=0x16 len=8 55-fc-dc-3b-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-b1-d0-1d-2b-00-00-00-00>
Jun 29 13:20:42 rtr00 /bsd: pppoe0: got lcp echo req, sending echo rep
Jun 29 13:20:42 rtr00 /bsd: pppoe0: lcp output <echo-reply id=0x16 len=8 9b-cc-13-a7>
Jun 29 13:20:45 rtr00 /bsd: pppoe0: lcp input(opened): <echo-req id=0x17 len=8 55-fc-dc-3b-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-31-92-e8-9b-00-00-00-00>
Jun 29 13:20:45 rtr00 /bsd: pppoe0: got lcp echo req, sending echo rep
Jun 29 13:20:45 rtr00 /bsd: pppoe0: lcp output <echo-reply id=0x17 len=8 9b-cc-13-a7>
Jun 29 13:20:55 rtr00 /bsd: pppoe0: lcp input(opened): <echo-req id=0x18 len=8 55-fc-dc-3b-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-54-f8-2b-14-00-00-00-00>
Jun 29 13:20:55 rtr00 /bsd: pppoe0: got lcp echo req, sending echo rep
Jun 29 13:20:55 rtr00 /bsd: pppoe0: lcp output <echo-reply id=0x18 len=8 9b-cc-13-a7>
Jun 29 13:20:58 rtr00 /bsd: pppoe0: lcp input(opened): <echo-req id=0x19 len=8 55-fc-dc-3b-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-44-34-28-75-00-00-00-00>
Jun 29 13:20:58 rtr00 /bsd: pppoe0: got lcp echo req, sending echo rep
Jun 29 13:20:58 rtr00 /bsd: pppoe0: lcp output <echo-reply id=0x19 len=8 9b-cc-13-a7>
Jun 29 13:21:01 rtr00 /bsd: pppoe0: lcp input(opened): <echo-req id=0x1a len=8 55-fc-dc-3b-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-fc-ff-39-26-00-00-00-00>
Jun 29 13:21:01 rtr00 /bsd: pppoe0: got lcp echo req, sending echo rep
Jun 29 13:21:01 rtr00 /bsd: pppoe0: lcp output <echo-reply id=0x1a len=8 9b-cc-13-a7>
Jun 29 13:21:05 rtr00 /bsd: pppoe0: lcp input(opened): <term-req id=0x12 len=4 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-fc-08-46-fa-00-00-00-00>
Jun 29 13:21:05 rtr00 /bsd: pppoe0: lcp opened->stopping
Jun 29 13:21:05 rtr00 /bsd: pppoe0: lcp send terminate-ack
Jun 29 13:21:05 rtr00 /bsd: pppoe0: lcp output <term-ack id=0x12 len=4>

.. If anyone has any suggestions, or seen anything similar previously, I'm all ears smile

Gut feeling is that there is a general incompatibility somewhere, one forum post I found (chap on Zen again) tried 4-5 different routers/modems and still saw the problem. Although, I accept there are many users without this problem using non-Zen-supplied hardware.

I currently have my second line in a migration state from BT, which I think I'm going to have to put on hold until I get to the bottom of this.

Edited by deleted (Tue 30-Jun-15 08:21:07)

rippedcotton
(experienced) Tue 30-Jun-15 12:28:16

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

Quick suggestion - try reducing MTU to 1492 and see if it makes any difference.

I have an entirely stable setup with Zen using HG612, Draytek 2830 and Linux/Windows network all using MTU 1492. I have seen problems in the dim and distant past with MTU 1500.

--

Brian

Zen Fibre 2 - 80/20 sync

deleted
(deleted) Tue 30-Jun-15 12:51:55

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: rippedcotton] [link to this post]

Hi there,

Thanks for the reply - going back to 1492 was one of the first things I tried, including going back to a version of OpenBSD that had no support for RFC4638 at all. Same problem occurred.

At the moment I'm testing using a TCP OpenVPN tunnel on the router, rather than UDP, so far PPP has stayed up for over an hour - it never did this with UDP. But need to complete more tests to see if this is a fluke and/or it reoccurs when going back to UDP - or if someone/somewhere at BT/Zen noticed/fixed something in the background smile

Thanks,

Edited by deleted (Tue 30-Jun-15 13:28:26)

deleted
(deleted) Wed 01-Jul-15 05:27:43

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

Let me know how you get on. I will be interested to see if using UDP was a simple cause.

deleted
(deleted) Wed 01-Jul-15 10:54:55

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

Sounds like you may have possibly been hit by this bug:

http://www.revk.uk/2013/11/bt-huawei-fttc-modem-bug-...

deleted
(deleted) Wed 01-Jul-15 12:06:48

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

In reply to a post by mixt:
Sounds like you may have possibly been hit by this bug:

http://www.revk.uk/2013/11/bt-huawei-fttc-modem-bug-...

Hi there,

Thanks for the reply - however, I ruled this one out fairly early on, it happens/happened with the ECI modem and the TG589 smile

Also, when you used to disconnect/reseat the eth cable on an impacted HG612 it worked around it, until PPP reset (i.e should be more than a few minutes). Not to mention my HG612 is unlocked and running the latest and greatest firmware.

The actual 'term-req' is/was coming from the remote MAC address, not local, confirmed with tcpdump.

Cheers,

Edited by deleted (Wed 01-Jul-15 12:08:28)

deleted
(deleted) Sun 05-Jul-15 18:23:24

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

When I had (potentially still have) this problem it just randomly went away, whilst I was trying a number of things - my setup went back to exactly what it was to start with, i.e router/modem/etc, problems disappeared. What was 100% clear from my diagnostics, is that it was Zen that was sending the 'term-req' and pulling the PPP session down - the problem being, no one from Zen could tell me why.

I did notice that at times when I was getting disconnected I was going through a gateway that was suffering high latency, dsl4 at the time seemed to be the worst for me, there are other threads relating to this on TBB. Shortly after the PPP session died after Zen killed it from their end, most of the time, I ended up on a different gateway and it would stay up for a bit longer. Then it would drop again, but dsl4 seemed to be consistently bad for me for the first 3-4 days of my line transferring over to Zen.

Current PPP session is just under 3 days, with 4.5 days modem sync. Am I happy since migrating to Zen? Not overly to be honest, I wasted considerable time looking into this and going through the support-script but with no clear outcome - if I'd wanted that, I could have stayed with BT! With hindsight, I wish I hadn't bothered and just stayed with BT. Having had an AA FTTC line previously I would class that as a premium service, Zen? Slightly better than BT, but not justifiably so with the cost - especially as BT would have given a good deal to retain my custom.

Time will tell if the service Zen provide changes my mind...

deleted
(deleted) Sun 05-Jul-15 22:30:17

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

That is a far cry from the comments we used to get from people about our service and I am sorry you feel this way.

Is DSL4 still one of the main culprits? Would you mind sending me some more info about it? I appreciate you have been through all this before, however working where I am, I can liaise directly with Networks to see if we can find the issue.

deleted
(deleted) Wed 08-Jul-15 21:54:05

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

Today has been absolutely terrible for me - PPP has been bouncing all over the place since 14:00, every 1 - 20 minutes - I spoke to a chap at Zen earlier who seems to know his stuff, they did some work on DSL4 and apparently moved 6,000 lines to rebalance things. I have to say, I'm still not overly happy - still periods of latency etc, I think their rudimentary load balancing is no longer fit for purpose (if it ever was)

https://f8lure.mouselike.org/index.asp?DoAction=View...

However, still no resolution on the PPP reseting. Calmed down for 4-5 days, then gradually got worse, now I'm back to where I was. My 2nd BT FTTC line continues to be rock solid PPP up time of around 15 days (last reset was me).

deleted
(deleted) Wed 08-Jul-15 23:02:32

Re: Just migrated to Zen, from BT, PPPoE dropping on OpenBSD

[re: deleted] [link to this post]

For completeness sake, as I haven't posted proof of the MAC sending the term-req yet - tcpdump from the pppoedev interface, LCP echo-request/reply goes back and forth, nothing dropped, then the remote end sends the 'term-req' @21:49:22.681482.

94:de:80:8f:a4:23 is the local MAC of the pppoedev interface, 84:26:2b:a2:3c:da is/was the remote for this session

Text

21:49:12.583483 84:26:2b:a2:3c:da 94:de:80:8f:a4:23 8864 60: PPPoE-Session
        code Session, version 1, type 1, id 0x0264, length 10        LCP: Echo-Request, Magic-Number=1419402018
21:49:12.583505 94:de:80:8f:a4:23 84:26:2b:a2:3c:da 8864 30: PPPoE-Session        code Session, version 1, type 1, id 0x0264, length 10
        LCP: Echo-Reply, Magic-Number=211993925721:49:15.624809 84:26:2b:a2:3c:da 94:de:80:8f:a4:23 8864 60: PPPoE-Session
        code Session, version 1, type 1, id 0x0264, length 10        LCP: Echo-Request, Magic-Number=1419402018
21:49:15.624828 94:de:80:8f:a4:23 84:26:2b:a2:3c:da 8864 30: PPPoE-Session        code Session, version 1, type 1, id 0x0264, length 10
        LCP: Echo-Reply, Magic-Number=211993925721:49:18.604635 84:26:2b:a2:3c:da 94:de:80:8f:a4:23 8864 60: PPPoE-Session
        code Session, version 1, type 1, id 0x0264, length 10        LCP: Echo-Request, Magic-Number=1419402018
21:49:18.604655 94:de:80:8f:a4:23 84:26:2b:a2:3c:da 8864 30: PPPoE-Session        code Session, version 1, type 1, id 0x0264, length 10
        LCP: Echo-Reply, Magic-Number=211993925721:49:22.681482 84:26:2b:a2:3c:da 94:de:80:8f:a4:23 8864 60: PPPoE-Session
        code Session, version 1, type 1, id 0x0264, length 6        LCP: Terminate-Request
21:49:22.681509 94:de:80:8f:a4:23 84:26:2b:a2:3c:da 8864 26: PPPoE-Session        code Session, version 1, type 1, id 0x0264, length 6
        LCP: Terminate-Ack21:49:32.742741 94:de:80:8f:a4:23 84:26:2b:a2:3c:da 8864 32: PPPoE-Session
        code Session, version 1, type 1, id 0x0265, length 12        LCP: Configure-Request, Magic-Number=-82992445[|lcp]
21:49:32.787984 84:26:2b:a2:3c:da 94:de:80:8f:a4:23 8864 60: PPPoE-Session        code Session, version 1, type 1, id 0x0265, length 21
        LCP: Configure-Request, Max-Rx-Unit=1500, Auth-Prot CHAP/MD5, Magic-Number=127734709, Vendor-Ext21:49:32.787987 84:26:2b:a2:3c:da 94:de:80:8f:a4:23 8864 60: PPPoE-Session
        code Session, version 1, type 1, id 0x0265, length 12        LCP: Configure-Ack, Magic-Number=-82992445, Vendor-Ext
21:49:32.788012 94:de:80:8f:a4:23 84:26:2b:a2:3c:da 8864 41: PPPoE-Session        code Session, version 1, type 1, id 0x0265, length 21
        LCP: Configure-Ack, Max-Rx-Unit=1500, Auth-Prot CHAP/MD5, Magic-Number=127734709[|lcp]21:49:33.367008 84:26:2b:a2:3c:da 94:de:80:8f:a4:23 8864 60: PPPoE-Session
        code Session, version 1, type 1, id 0x0265, length 21        LCP: Configure-Request, Max-Rx-Unit=1500, Auth-Prot CHAP/MD5, Magic-Number=127734709, Vendor-Ext
21:49:33.367041 94:de:80:8f:a4:23 84:26:2b:a2:3c:da 8864 41: PPPoE-Session        code Session, version 1, type 1, id 0x0265, length 21
        LCP: Configure-Ack, Max-Rx-Unit=1500, Auth-Prot CHAP/MD5, Magic-Number=127734709[|lcp]

Resetting on average of every 5 minutes at the moment.

Edited by deleted (Wed 08-Jul-15 23:05:59)

Pages in this thread: 1 | 2 | 3 | 4 | (show all)

Print Thread

Jump to