What is going on with Zen tonight ?????????

https://www.thinkbroadband.com/_assets/speedtest/but...

https://www.thinkbroadband.com/broadband/monitoring/...

Edited by tommy45 (Wed 04-Dec-19 21:00:47)

Whatever it was, your BQM suggests it may have been fixed.


eta- the packet loss, anyway.

Edited by billford (Wed 04-Dec-19 22:40:30)

I didn't notice any change in my service with Zen.
Just a thought. By the times on your graph and speed test it seemed to coincide with Amazon streaming tonight's Premier League games.

In reply to a post by funkydan:

it seemed to coincide with Amazon streaming tonight's Premier League games.

I wondered if it might be some event being streamed, but didn't think to check what Amazon were up to

My backhaul is Zen, nothing noticeable here.

there was some planned maintenance https://servicealerts.zen.co.uk/active/5/5427 but this was before that window, things have largely returned to normal service, but i'm left with a slower than normal ramp up on the single thread which i didn't have prior to this packet loss and next to no throughput, the slow single thread isn't apparent on speedtest.net or on zen's speedtester,

https://www.thinkbroadband.com/_assets/speedtest/but...
Older speed test result https://www.thinkbroadband.com/_assets/speedtest/but...
And i'm stuill trying to get zen to get my BT IP profile rebuilt as DLM has had me on =Ginp retx high whilst it reduced the target snr now it's back to a RETX low setting but my IP profile is still set to 91-92% of my sync which is 79999kbps so should be 77.35mbps not whatever it is ( cannot get IP profile from BTW tester, and zen's system hasn't updated,/changed

In reply to a post by billford:

My backhaul is Zen, nothing noticeable here.

Contracted bandwidth perhaps.

Are you on the exchange that the planned work was on? it could explain some things.

In reply to a post by MoM:

Are you on the exchange that the planned work was on? it could explain some things.

Nope, no where near it, and my circuit is not on Zen's GEA network it uses BT WMBC

In reply to a post by tommy45:

https://www.thinkbroadband.com/_assets/speedtest/but...

https://www.thinkbroadband.com/broadband/monitoring/...

That's interesting. I'm with IDNet but I thought in the early days of my contract they moved me off Zen backhaul onto BT. But my TBBQM shows much the same thing:
https://www.thinkbroadband.com/broadband/monitoring/...
I also noticed poor throughput last night.

In reply to a post by tommy45:

In reply to a post by MoM:

Are you on the exchange that the planned work was on? it could explain some things.

Nope, no where near it, and my circuit is not on Zen's GEA network it uses BT WMBC

Ah, my guess then is it/was a BT issue. Are you anywhere near Brackley (01280)?

In reply to a post by Andrue:

In reply to a post by tommy45:

In reply to a post by MoM:

Are you on the exchange that the planned work was on? it could explain some things.

Nope, no where near it, and my circuit is not on Zen's GEA network it uses BT WMBC

Ah, my guess then is it/was a BT issue. Are you anywhere near Brackley (01280)?

No, I'm up north So join zen's data centre in Manchester but more and more is being routed via London even though they do have some peering in Manchester

Seems to be a few more issues this afternoon - I experienced a minor blip overnight but since about 14:00 it's been fully down or i've only been able to reach some sites. Seems they're working on it though - https://twitter.com/zeninternet/status/1202592114422...

In reference to twitter.com/zeninternet/status/1202592114422489089:

Our engineers are starting work to fix the connection issues being seen today. You may notice disconnections in service between now and 3pm. Updates will be available on our service alerts page https://bit.ly/38d9joO. Thanks for your patience 🤝 Chris T

In reply to a post by KingJ:

Seems to be a few more issues this afternoon - I experienced a minor blip overnight but since about 14:00 it's been fully down or i've only been able to reach some sites. Seems they're working on it though - https://twitter.com/zeninternet/status/1202592114422...

In reference to twitter.com/zeninternet/status/1202592114422489089:

Our engineers are starting work to fix the connection issues being seen today. You may notice disconnections in service between now and 3pm. Updates will be available on our service alerts page https://bit.ly/38d9joO. Thanks for your patience 🤝 Chris T

My TBBQM looks quite awful today and I've had at least one disconnection. Still don't know if the common factor is Zen or BT.

https://www.thinkbroadband.com/broadband/monitoring/...

May be a Zen internal issue. A friend of mine was showing packet loss on his BQM over the last 24 hours, mine was fine.

He and I both had an outage at identical times at around 3pm this afternoon. We are on completely different and geographically separate exchanges in different counties, so looks like Zen were doing some re configuring this afternoon.

In reply to a post by Andrue:

That's interesting. I'm with IDNet but I thought in the early days of my contract they moved me off Zen backhaul onto BT. But my TBBQM shows much the same thing:
https://www.thinkbroadband.com/broadband/monitoring/...
I also noticed poor throughput last night.

I am also on IDNet with a BTw backhaul (having been moved off Zen) and seeing some unusual packet loss spikes:
https://www.thinkbroadband.com/broadband/monitoring/...

Speedtest graphs looking fairly normal regardless.

In reply to a post by funkydan:

I didn't notice any change in my service with Zen.
Just a thought. By the times on your graph and speed test it seemed to coincide with Amazon streaming tonight's Premier League games.

We have a winner. That produced record traffic levels on ISPs' networks and, hence, record levels on BT Wholesale's.

I'm not going into sources but I understand that Sky, TalkTalk and Virgin Media were okay, however BT Retail had some severe regional performance issues due to capacity issues with Wholesale.

Seems a super-duper, telemetry-driven, cloud-powered, software-defined *insert buzzwords here* network still needs adequate capacity overhead to manage exceptional events.

In reply to a post by CarlTSpeak:

In reply to a post by funkydan:

I didn't notice any change in my service with Zen.
Just a thought. By the times on your graph and speed test it seemed to coincide with Amazon streaming tonight's Premier League games.

We have a winner. That produced record traffic levels on ISPs' networks and, hence, record levels on BT Wholesale's.

I'm not going into sources but I understand that Sky, TalkTalk and Virgin Media were okay, however BT Retail had some severe regional performance issues due to capacity issues with Wholesale.

Seems a super-duper, telemetry-driven, cloud-powered, software-defined *insert buzzwords here* network still needs adequate capacity overhead to manage exceptional events.

This doesn't really bode well for the future, either. Last night's Premier League coverage also triggered capacity issues.

The cheaper Openreach 550 and 1000 Mb products are coming online in March.

That'll be an experience for the Wholesale network when the next 150 GB+ triple A game is released and gamers, those folks likely to want the latest and greatest broadband to go with their latest and greatest rigs, start downloading en masse.

I wonder if we're seeing Wholesale's historic periodic capacity issues again or if they are gone for good?

ISPs and wholesale providers have a talent for monitoring individual ports but seem to have a blind spot for the lead times for installing new chassis and backhaul to feed them when they run out of ports on line cards then run out of line card slots for chassis.

Smart networks that can move traffic around are great but they only work if there's capacity available to move traffic on to. Past 2 night on BT Wholesale it looks as though there has not been - the Amazon streams are killing them.

Be interesting to see if it's a direct regional correlation between the area where the footy is likely to have been most popular, London-ish yesterday, Liverpool day before.

Manchester United v City coming up. Saturday 5:30pm.

My TBBQM is looking bad at the moment (empty house) so that's unusual.

https://www.thinkbroadband.com/broadband/monitoring/...

Thought at first it was my server (the IPv6 test) but this IPv4 is bouncing off the router. Seems odd.

In reply to a post by RobertoS:

Manchester United v City coming up. Saturday 5:30pm.

But on Sky Sports so a lot of people will be watching through satellite TV rather than the exclusively internet offering from Amazon Prime.

Next big Amazon Prime live football event is over Christmas when they are streaming one match at 12:30 on Boxing Day, 7 matches simultaneously at 15:00, one match at 17:30, one match at 20:00 and one match at 19:45 on Friday 27th.

Be interesting to see if the reduced business traffic on Boxing Day allows for a better streaming outcome.

NowTV?

In reply to a post by RobertoS:

NowTV?

Yes indeed, but given the number of people with dishes on their wall I would suspect that the internet streaming load would be less than for the totally internet based Amazon offering.

Can you imagine what it will be like if Corbyn gets in and gives free bb to everyone we will be lucky to see a couple of kbps never mind mbps and only allowed to visits sites that they want you to see of course

In reply to a post by RobertoS:

NowTV?

Not making a dent compared to Amazon.

In reply to a post by tommy45:

Can you imagine what it will be like if Corbyn gets in and gives free bb to everyone we will be lucky to see a couple of kbps never mind mbps and only allowed to visits sites that they want you to see of course

I'm kinda suspecting he might backtrack on that "single ISP" idea as its rather dumb, but possibly was the only way to get across the idea to the public who barely understand what an ISP is.

Nationalised Openreach, fair enough, though arguably the worst time to do it now they finally are investing in FTTP.

But a single ISP is going to cripple the country. If it has a fault, the whole country comes to a standstill (rather like demanding we only have a single route to any location over our roads), very dumb idea, not counting the privacy issues and site blocking potential.

As for the topic, I've definitely been seeing issues on both my Zen and PlusNet lines the last few days. Its played absolute havoc with my VPN, it had 50% packet loss and 2000ms ping at one point when not even heavily loaded.

In reply to a post by alexatkin:

As for the topic, I've definitely been seeing issues on both my Zen and PlusNet lines the last few days. Its played absolute havoc with my VPN, it had 50% packet loss and 2000ms ping at one point when not even heavily loaded.

Yeah my line continues to look poor. I'm also getting failures trying to launch the TBB speed test. Speedtest.net runs and gives a reasonable speed.

In my case a couple of days prior to 'the big one' DLM gave me a rare boost. This came after an engineer had finally fixed a noisy voice line (it had never seemed to bother my broadband as it was still connecting at 70Mb/s). So some of my issues could be down to the DLM having gone too far. Except that the issues weren't present for the first couple of days and appear to have been triggered by 'the big one' unlikely though that seems.

Edited by Andrue (Sat 07-Dec-19 09:01:40)

In reply to a post by alexatkin:

In reply to a post by tommy45:

Can you imagine what it will be like if Corbyn gets in and gives free bb to everyone we will be lucky to see a couple of kbps never mind mbps and only allowed to visits sites that they want you to see of course

I'm kinda suspecting he might backtrack on that "single ISP" idea as its rather dumb, but possibly was the only way to get across the idea to the public who barely understand what an ISP is.

Nationalised Openreach, fair enough, though arguably the worst time to do it now they finally are investing in FTTP.

But a single ISP is going to cripple the country. If it has a fault, the whole country comes to a standstill (rather like demanding we only have a single route to any location over our roads), very dumb idea, not counting the privacy issues and site blocking potential.

As for the topic, I've definitely been seeing issues on both my Zen and PlusNet lines the last few days. Its played absolute havoc with my VPN, it had 50% packet loss and 2000ms ping at one point when not even heavily loaded.

There is no such thing as FREE either Someone has to pay for everything, Could you imagine how long you would need to wait for a repair? if it was nationalized investment would slump, sftaff pensions would be raided it would not work, he also talks of nationalization of the utilities , which he can't because they are no longer British owned any more , then he wants to tax the motorist fuel hikes , so the food prices also go up so his pipe dream of a high min wage will not mean anything

Not seeing any issues here in MK this evening (using a 39 Mbps BTW FTTP connection).

In reply to a post by techguy:

Not seeing any issues here in MK this evening (using a 39 Mbps BTW FTTP connection).

Me neither so far tonight, it seems to have been an isolated incident that is thought to be due to demand and lack of supply generated by AMAZON PRIME's live coverage of football though no official confirmation or explanation as to why throughput dropped to sub 3mbps at one point

In reply to a post by techguy:

Not seeing any issues here in MK this evening (using a 39 Mbps BTW FTTP connection).

It seems to have settled down here in Brackley as well.

https://www.thinkbroadband.com/broadband/monitoring/...

What a strange week. I don't see how a surge in demand one evening can disrupt the network for days after but my TBBQM looks like that. Got to be something else in play.

Looks like either the router is randomly dropping packets or not responding,to the pingbox, weird, fault maybe?

In reply to a post by tommy45:

Looks like either the router is randomly dropping packets or not responding,to the pingbox, weird, fault maybe?

It's the same router I've had for a couple of years now (a Billion BiPAC 8800AXL R2). The only changes on my end are finally getting the noisy line fixed. Definitely a bit odd but at least it's working

In reply to a post by Andrue:

In reply to a post by tommy45:

Looks like either the router is randomly dropping packets or not responding,to the pingbox, weird, fault maybe?

It's the same router I've had for a couple of years now (a Billion BiPAC 8800AXL R2). The only changes on my end are finally getting the noisy line fixed. Definitely a bit odd but at least it's working

It's not your kit Andrue. If you check my IDNet BTw graph with yours, I am mostly seeing spikes at exactly the same time as you.

https://www.thinkbroadband.com/broadband/monitoring/...

I've posted a real time link as your's seems to be real time too.

Seeing the same spikes on an IDNet line in rural Northumberland.

In reply to a post by Andrue:

In reply to a post by tommy45:

Looks like either the router is randomly dropping packets or not responding,to the pingbox, weird, fault maybe?

It's the same router I've had for a couple of years now (a Billion BiPAC 8800AXL R2). The only changes on my end are finally getting the noisy line fixed. Definitely a bit odd but at least it's working

I wonder if its some kind of session drop, on the PPPoE side, or load balancing ? i would favour the latter if you aren't seeing a break in connectivity they could be peering changes with tbb's network , i have 1 today on the tbbqm, but it isn't on the f8lure graphs,

Edited by tommy45 (Sun 08-Dec-19 01:19:36)

what would help diagnose this stuff is if TBB publicised individual data/speedtests so people could view and categorise by isp etc. but for some reason after many years of doing this they wont do it.

In reply to a post by CarlTSpeak:

In reply to a post by CarlTSpeak:

In reply to a post by funkydan:

I didn't notice any change in my service with Zen.
Just a thought. By the times on your graph and speed test it seemed to coincide with Amazon streaming tonight's Premier League games.

We have a winner. That produced record traffic levels on ISPs' networks and, hence, record levels on BT Wholesale's.

I'm not going into sources but I understand that Sky, TalkTalk and Virgin Media were okay, however BT Retail had some severe regional performance issues due to capacity issues with Wholesale.

Seems a super-duper, telemetry-driven, cloud-powered, software-defined *insert buzzwords here* network still needs adequate capacity overhead to manage exceptional events.

This doesn't really bode well for the future, either. Last night's Premier League coverage also triggered capacity issues.

The cheaper Openreach 550 and 1000 Mb products are coming online in March.

That'll be an experience for the Wholesale network when the next 150 GB+ triple A game is released and gamers, those folks likely to want the latest and greatest broadband to go with their latest and greatest rigs, start downloading en masse.

I wonder if we're seeing Wholesale's historic periodic capacity issues again or if they are gone for good?

ISPs and wholesale providers have a talent for monitoring individual ports but seem to have a blind spot for the lead times for installing new chassis and backhaul to feed them when they run out of ports on line cards then run out of line card slots for chassis.

Smart networks that can move traffic around are great but they only work if there's capacity available to move traffic on to. Past 2 night on BT Wholesale it looks as though there has not been - the Amazon streams are killing them.

Be interesting to see if it's a direct regional correlation between the area where the footy is likely to have been most popular, London-ish yesterday, Liverpool day before.

Looks like my choice of aaisp over zen was a good one, one of the reasons was I didnt want to go back to BTw as it always seemed to have some kind of capacity issue (no zen LLU on my exchange). AAISP also auto monitor for congestion related events, and there seems to be nearly always something on their status page for some part of the BTw network.

The fact that these amazon prime streams caused this is concerning, it suggests maybe not much excess capacity available. Did viewing numbers get released by amazon?

In reply to a post by Chrysalis:

In reply to a post by CarlTSpeak:

In reply to a post by CarlTSpeak:

... nested quotes trimmed ...

We have a winner. That produced record traffic levels on ISPs' networks and, hence, record levels on BT Wholesale's.

I'm not going into sources but I understand that Sky, TalkTalk and Virgin Media were okay, however BT Retail had some severe regional performance issues due to capacity issues with Wholesale.

Seems a super-duper, telemetry-driven, cloud-powered, software-defined *insert buzzwords here* network still needs adequate capacity overhead to manage exceptional events.

This doesn't really bode well for the future, either. Last night's Premier League coverage also triggered capacity issues.

The cheaper Openreach 550 and 1000 Mb products are coming online in March.

That'll be an experience for the Wholesale network when the next 150 GB+ triple A game is released and gamers, those folks likely to want the latest and greatest broadband to go with their latest and greatest rigs, start downloading en masse.

I wonder if we're seeing Wholesale's historic periodic capacity issues again or if they are gone for good?

ISPs and wholesale providers have a talent for monitoring individual ports but seem to have a blind spot for the lead times for installing new chassis and backhaul to feed them when they run out of ports on line cards then run out of line card slots for chassis.

Smart networks that can move traffic around are great but they only work if there's capacity available to move traffic on to. Past 2 night on BT Wholesale it looks as though there has not been - the Amazon streams are killing them.

Be interesting to see if it's a direct regional correlation between the area where the footy is likely to have been most popular, London-ish yesterday, Liverpool day before.

Looks like my choice of aaisp over zen was a good one, one of the reasons was I didnt want to go back to BTw as it always seemed to have some kind of capacity issue (no zen LLU on my exchange). AAISP also auto monitor for congestion related events, and there seems to be nearly always something on their status page for some part of the BTw network.

The fact that these amazon prime streams caused this is concerning, it suggests maybe not much excess capacity available. Did viewing numbers get released by amazon?

They seem to let svlans run at near capacity,with little or no redundancy no wonder why they are in no rush to roll out FTTP

Hi all im with idnet too and been getting packet loss for 4 days now, idnet did say there was issues with network 2 days ago and now it says its been fixed yet im still getting red spikes

My Broadband Ping

I just found this though not much help now https://www.ispreview.co.uk/index.php/2019/12/custom...

IDnet status page said everything is back to normal but i still have packet loss here in the north of uk.

Mine has been free of packet loss since that blip, Are you on Zen's GEA or BTW WMBC?

IDnet FTTC 80/20

Edited by beefcakechipz (Mon 09-Dec-19 17:47:44)

In reply to a post by beefcakechipz:

IDnet status page said everything is back to normal but i still have packet loss here in the north of uk.

As do we, with the problem looking as if it has got worse over the last 10 hours.

Should this bit of the topic be hived off to the IDNet page?

Just switch ISP. I did as Zen was poor for me and their network is flakey

In reply to a post by beefcakechipz:

IDnet FTTC 80/20

Do you have an BT wholesale IP profile or not? if not then you will use Zen's GEA cable links (On net)

It used to show up on btwholesale but now it only gives me my current speed

Zen's GEA cable links is that a good thing or bad ?

Edited by beefcakechipz (Mon 09-Dec-19 20:56:18)

It�s for situations like these that it would be awesome if BQMs could have metadata associated with them like ISP, cabinet etc. With that type of info it would be possible to spot trends and hold under performing parties to account.

I think i might give plusnet a try

glad i didnt go with idnet.

and corbyn didnt have a chance. they could have steamrolled it up to 3 months before the election if they'd dumped the lefty left terrorist symathiser but they chose the option for themselves not the people.


hard luck.

voted tory got plaid cymru (what have you done hywel williams, nothing, i did see you posting in a pic with a well known drug dealing family up here tho, grow more and export is that what you thinks good for wales?). new boundaries comming in tho. from what i see.


oh and before i forget lets get a vote on getting rid of the welsh assembly every five years till we win.......................

Edited by deleted (Tue 24-Dec-19 15:34:17)

In reply to a post by beefcakechipz:

It used to show up on btwholesale but now it only gives me my current speed

Zen's GEA cable links is that a good thing or bad ?

Can be both good or bad, depending on what the issue is, You won't have a BT bras IP profile on GEA so should get a slightly higher ds throughput, compared to BT WBMC

This **** is back again tonight BT doesn't have the capacity WBMC is not managed properly from broken IP misconfigured profiles to capacity issues Twice in the same month is taking the P https://www.thinkbroadband.com/_assets/speedtest/but... And the evident packet loss associated with capacity issues https://www.thinkbroadband.com/broadband/monitoring/... bang on 8pm lecs v lpool started

Edited by tommy45 (Thu 26-Dec-19 21:06:45)

In reply to a post by tommy45:

This **** is back again tonight BT doesn't have the capacity WBMC is not managed properly from broken IP misconfigured profiles to capacity issues Twice in the same month is taking the P https://www.thinkbroadband.com/_assets/speedtest/but... And the evident packet loss associated with capacity issues https://www.thinkbroadband.com/broadband/monitoring/... bang on 8pm lecs v lpool started

No problems here on BT Business (also streaming Premier league footy on Amazon Prime)
https://www.thinkbroadband.com/broadband/monitoring/...

Maybe its a problem at Zen's end, such as congestion?

Edited by deleted (Thu 26-Dec-19 21:23:12)

I'm with idnet and my speeds have dropped since 8pm

Edited by beefcakechipz (Thu 26-Dec-19 21:22:53)

In reply to a post by baby_frogmella:

In reply to a post by tommy45:

This **** is back again tonight BT doesn't have the capacity WBMC is not managed properly from broken IP misconfigured profiles to capacity issues Twice in the same month is taking the P https://www.thinkbroadband.com/_assets/speedtest/but... And the evident packet loss associated with capacity issues https://www.thinkbroadband.com/broadband/monitoring/... bang on 8pm lecs v lpool started

No problems here on BT Business (also streaming Premier league footy on Amazon Prime)
https://www.thinkbroadband.com/broadband/monitoring/...

Maybe its a problem at Zen's end, such as congestion?

My back haul isn't reliant on zen it's BT WBMC not Zen GEA, and WHY ONLY WHEN AMAZON PRIME HAS POPULAR LIVE SPORTING EVENTS? And I suspect this could be regional

Edited by tommy45 (Thu 26-Dec-19 21:29:37)

Perhaps Zen haven't purchased sufficient bandwidth from BT Wholesale? If it was a nationwide BT Wholesale issue then it would also be affecting all BT lines but its clearly not:

https://www.speedtest.net/result/8890107625

https://www.thinkbroadband.com/broadband/monitoring/...

Btw I'm in Scotland.

In reply to a post by baby_frogmella:

Perhaps Zen haven't purchased sufficient bandwidth from BT Wholesale? If it was a nationwide BT Wholesale issue then it would also be affecting all BT lines but its clearly not:

https://www.speedtest.net/result/8890107625

https://www.thinkbroadband.com/broadband/monitoring/...

Btw I'm in Scotland.

N' West here so Liverpool closer which may have a bearing , i don't see this **** at any other time only when Amazon stream certain football games live, packet loss now ceased as the after match waffle draws to an end, surprise , not

Edited by tommy45 (Thu 26-Dec-19 22:20:14)

In reply to a post by tommy45:

In reply to a post by baby_frogmella:

Perhaps Zen haven't purchased sufficient bandwidth from BT Wholesale? If it was a nationwide BT Wholesale issue then it would also be affecting all BT lines but its clearly not:

https://www.speedtest.net/result/8890107625

https://www.thinkbroadband.com/broadband/monitoring/...

Btw I'm in Scotland.

N' West here so Liverpool closer which may have a bearing , i don't see this **** at any other time only when Amazon stream certain football games live, packet loss now ceased as the after match waffle draws to an end, surprise , not

Manchester based so also North West and on a BTw connection (albeit with IDNet). No sign of any packet loss here.

In reply to a post by jaydub:

Manchester based so also North West and on a BTw connection (albeit with IDNet). No sign of any packet loss here.

Lack of packet loss doesn't mean there's no problem. I think there definitely is a problem, because I noticed a slow down as well. If you check my automated speed tests (link in sig), you can definitely see there's something wrong as the average speed test result has plummeted since mid-day yesterday (26th Dec).

locking this thread due to size and age/posting profile.
Feel free to create a new one if you want to post on this topic.