User comments on ISPs
  >> Entanet International Ltd.


Register (or login) on our website and you will not see this ad.


Pages in this thread: 1 | 2 | 3 | 4 | 5 | (show all)   Print Thread
Standard User therioman
(knowledge is power) Thu 21-Mar-13 19:39:11
Print Post

Oops!


[link to this post]
 
I'm surprised nobody has mentioned the "whoops" moment on Enta's network for the past hour or so (and counting)...

..or does nobody have a backup/3g/other way to get online?!?
Standard User Adam3k
(knowledge is power) Thu 21-Mar-13 19:45:08
Print Post

Re: Oops!


[re: therioman] [link to this post]
 
Yes its very broken smile

Our 2Gbp private transit links are online. but DSL, EFM and Leased are all dead.

-------------------------------------------------------------------------------
StructuredCommunications Bonded 2 x 80/20 FTTC 74.5Mbps Download & 22.4Mbps Upload. Transit provided by Level3
Standard User therioman
(knowledge is power) Thu 21-Mar-13 19:47:43
Print Post

Re: Oops!


[re: Adam3k] [link to this post]
 
Indeed and the noc.enta.net 21cn interconnects look a bit quiet smile

(accessible on IPv6 still).


Register (or login) on our website and you will not see this ad.

Standard User elshiftos
(member) Thu 21-Mar-13 19:50:26
Print Post

Re: Oops!


[re: therioman] [link to this post]
 
I can't see the noc page or any other come to that. Currently tethering on my phone. What's the problem?

Ta
Standard User Adam3k
(knowledge is power) Thu 21-Mar-13 19:53:29
Print Post

Re: Oops!


[re: therioman] [link to this post]
 
We can hit things via our private transit peer, but thats all.

Just seen the Interxion Core Peer go down and lns17.inx.dsl.enta.net - lns22.inx.dsl.enta.net stop respond for just over 5 mins.

-------------------------------------------------------------------------------
StructuredCommunications Bonded 2 x 80/20 FTTC 74.5Mbps Download & 22.4Mbps Upload. Transit provided by Level3
Administrator seb
(founder) Thu 21-Mar-13 19:55:01
Print Post

Re: Oops!


[re: therioman] [link to this post]
 
In reply to a post by therioman:
Indeed and the noc.enta.net 21cn interconnects look a bit quiet smile
(accessible on IPv6 still).


I knew there was something useful about v6 smile

seb

Sebastien Lahtinen
Co-Founder,
thinkbroadband.com
seb@thinkbroadband.com

personal blog - blog.seb.me.uk
twitter - @sebtweet
The author of the above post is a thinkbroadband staff member. It may not constitute an official statement on behalf of thinkbroadband.
Standard User therioman
(knowledge is power) Thu 21-Mar-13 19:55:50
Print Post

Re: Oops!


[re: elshiftos] [link to this post]
 
Enta haven't made any statement about "what" is wrong, but my guess is some sort of router issue since routes just disappeared - perhaps BGP issues or something.

It seems to be intermittently starting to work now. Not stable, but progress...
Standard User therioman
(knowledge is power) Thu 21-Mar-13 19:57:31
Print Post

Re: Oops!


[re: seb] [link to this post]
 
"let's you know when the real internet is broken"

Shame ipv6 isn't "standard" yet, but seems we're all miles off that.
Standard User Remidizer
(member) Thu 21-Mar-13 20:00:24
Print Post

Re: Oops!


[re: therioman] [link to this post]
 
well, offline now in the highlands, is this a country wide outage...methinks so.

Andy G

Inverness
www.britishsergeantsmess.com

Vivaciti ISP user
Standard User cedge
(member) Thu 21-Mar-13 20:05:52
Print Post

Re: Oops!


[re: Remidizer] [link to this post]
 
Glad its not something wrong my end.

Migrated from Plus to UKFSN 16 Feb 07.
Upgraded to 21CN in Jun 12.
Standard User FootLongHero
(member) Thu 21-Mar-13 20:05:53
Print Post

Re: Oops!


[re: Remidizer] [link to this post]
 
Been down for a while now.
Change my dns server to open dns and am now back up and running.
Standard User Remidizer
(member) Thu 21-Mar-13 20:09:54
Print Post

Re: Oops!


[re: FootLongHero] [link to this post]
 
good idea mate but I cant connect now lol

Andy G

Inverness
www.britishsergeantsmess.com

Vivaciti ISP user
Standard User FootLongHero
(member) Thu 21-Mar-13 20:16:26
Print Post

Re: Oops!


[re: Remidizer] [link to this post]
 
Enta are aware and have sent engineers to sort it.
Noc.enta.net is unavailable for me, but the enta twitter feed has updates.
https://twitter.com/entanet
Standard User Lethe
(fountain of knowledge) Thu 21-Mar-13 21:11:43
Print Post

Re: Oops!


[re: Remidizer] [link to this post]
 
Yep, in Pompey here... deep south. [censored] me up as I was measuring different configurations on my router and thought I caused it!

Nick
Standard User Adam3k
(knowledge is power) Thu 21-Mar-13 21:12:56
Print Post

Re: Oops!


[re: therioman] [link to this post]
 
Makes for a good read!

Seems Enta may have been to blame..

http://status.coreix.net/services/network---ip-transit

-------------------------------------------------------------------------------
StructuredCommunications Bonded 2 x 80/20 FTTC 74.5Mbps Download & 22.4Mbps Upload. Transit provided by Level3
Standard User professor973
(committed) Thu 21-Mar-13 21:21:38
Print Post

Re: Oops!


[re: FootLongHero] [link to this post]
 
I also switched back to open DNS, was not aware that my Fritzbox had switched away from it. Getting too many outages and lunchtime slowdowns with Enta lately.

The difference between genius and stupidity is; genius has its limits.
http://www.pingtest.net/result/68380009.png
Standard User Bystander333
(newbie) Thu 21-Mar-13 21:45:37
Print Post

Re: Oops!


[re: Adam3k] [link to this post]
 
Nice thanks Andy, for posterity then, reposting the important part (at this time),

http://status.coreix.net/services/network---ip-transit

"We have completed our initial analysis of the network issue and it is as follows: 1. At approximately 5:45pm we noticed an inability to contact certain locations and started to investigate. 2. We disconnected LINX, LONAP and ENTA within 5 minutes as it appeared that there was an issue with one of these providers. 3. This did not solve this issue as anticipated and issue became worse. 4. At approximately 6:30pm the ENTA edge router to which we connect went down and full connectivity was reinstated (we where not connected to it at this point per point 2). It is our belief at this point in time that ENTA suffered from a serious route leak on their systems and was falsely advertising other companies routes. This caused the internet in general to send traffic to ENTA rather than Coreix. We are awaiting a response from ENTA to confirm this was the case. We will report back further as soon as we receive further information and clarification. Our apologies for the inconvenience caused by this issue."

I'm a non-"business" user but run my business from home on an LLU line (and pay for it in topups). Am going to be shifting providers tomorrow to some unlimited company, sorry UKFSN I know it's not your fault.
I particularly didn't like the bit where they said "portion of our network".

I'm in the Midlands, somebody earlier was in Scotland. Exactly how big was this "portion"? and how much damage did they do to other companies ?

Edited by Bystander333 (Thu 21-Mar-13 22:08:00)

Standard User RobertoS
(sensei) Thu 21-Mar-13 22:13:58
Print Post

Re: Oops!


[re: Bystander333] [link to this post]
 
I'm a non-"business" user but run my business from home on an LLU line (and pay for it in topups). Am going to be shifting providers tomorrow to some unlimited company, sorry UKFSN I know it's not your fault.
An Enta LLU line via UKFSN? Would that be O2 Wholesale then, or Enta extremely rare business LLU? Which I didn't think UKFSN offered.

Or are you confused and think all Enta connections are LLU?

My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet UnLim Fibre (FTTC). Sync ~ 54.2/15.2Mbps @ 600m. - BQM

"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.

Edited by RobertoS (Thu 21-Mar-13 22:14:13)

Standard User Bystander333
(newbie) Thu 21-Mar-13 22:26:57
Print Post

Re: Oops!


[re: RobertoS] [link to this post]
 
No. I'm afraid you're confused RobertoS.

My ISP is UKFSN, I'm on domestic and I pay Enta each month. When I log into my UKFSN account to check how much allowance I have left they tell me to check billing.enta.net for the most up to date information.

Edit: May be on ADSL2+ checking the exchange.

Edited by Bystander333 (Thu 21-Mar-13 22:32:44)

Standard User RobertoS
(sensei) Thu 21-Mar-13 22:36:03
Print Post

Re: Oops!


[re: Bystander333] [link to this post]
 
smile
I meant to include a question, but forgot. Your reference to running your businees on an LLU line was meaning another line?

My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet UnLim Fibre (FTTC). Sync ~ 54.2/15.2Mbps @ 600m. - BQM

"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.
Standard User sbrads
(member) Thu 21-Mar-13 22:44:39
Print Post

Re: Oops!


[re: cedge] [link to this post]
 
After swapping routers to no avail and deciding it was time to ring Enta, they confirmed it was country wide and the support guy said he didn't even have any internet access. All fine now.

Steve
Standard User Bystander333
(newbie) Thu 21-Mar-13 22:48:34
Print Post

Re: Oops!


[re: RobertoS] [link to this post]
 
Not on another line, just honest confusion on my part.

Two years ago I was with ADSL24 on Entanet, they shifted to Murphx and soon after my exchange was unbundled by TalkTalk I think - or some deal was struck. I ended up shifting as ADSL24 or the LLU weren't coping but have had it stuck in my mind ever since I was still on LLU.

Nonetheless my point still stands, my carrier is Enta - I switched back to them two years ago via UKFSN quite deliberately. Am now going to be shifting not due to the retailer/ISP, but fundamentally because Enta pricing is now too high and because I've lost faith in them due to this.
Standard User RobertoS
(sensei) Thu 21-Mar-13 22:57:36
Print Post

Re: Oops!


[re: Bystander333] [link to this post]
 
smile
You said it was me that was confused. But that was because you mentioned LLU.

Anyway, for "unlimited" have a look at Plusnet. If it worries you, there is a monthly contract option and still far cheaper than you pay now I expect.

My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet UnLim Fibre (FTTC). Sync ~ 54.2/15.2Mbps @ 600m. - BQM

"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.
Standard User Kathphlox
(regular) Thu 21-Mar-13 23:22:28
Print Post

Re: Oops!


[re: RobertoS] [link to this post]
 
Putting up the BT Wholesale page a lot sooner would have saved a lot of grief, less phone calls, saved more of us shuffling through old modems or new ones for a solution to our loss.

Just glad it's back online.. a rare blip with Enta.. solid as a rock normally smile

~Kath~
Always judge a person by the way they treat someone who can be of no possible use to them
Standard User aquiss
(fountain of knowledge) Fri 22-Mar-13 07:27:41
Print Post

Re: Oops!


[re: Kathphlox] [link to this post]
 
The BT wholesale page has nothing to do with Entanet, nor can be requested by Entanet.

Martin Pitt
Aquiss Limited
http://www.aquiss.net

Unlimited Family Broadband Packages : http://www.aquiss.net/broadband-unlimited.php
Standard User b4dger
(knowledge is power) Fri 22-Mar-13 09:49:55
Print Post

Re: Oops!


[re: Kathphlox] [link to this post]
 
Not good when these things happen - especially as my 20CN rural line is pretty 'delicate' without any of this! smile

FWIW
My tip for outages such as this (router still in sync and PPP connected) is to use Twitter to see what's going on. Entanet use @entanetnoc and @entanet but unfortunately were slow in posting on this occasion. But searching for Entanet showed lots of others in the same situation which puts your mind at ease that it's not your line or kit at fault and saves a lot of stress.

Standard User Lethe
(fountain of knowledge) Fri 22-Mar-13 10:00:43
Print Post

Re: Oops!


[re: b4dger] [link to this post]
 
In Portsmouth, I had no route at all last night - and also the 0333 101 0600 number wasn't working (just a quiet line, no ringing tone).

So I just had to wait.

Nick
Standard User b4dger
(knowledge is power) Fri 22-Mar-13 11:12:24
Print Post

Re: Oops!


[re: b4dger] [link to this post]
 
As an example of using Twitter...

There was a WBC 'issue' around 10 this morning.
Searching for entanet shows people were affected quite a while before Enta were able to confirm...

I have a 'saved' Twitter search to make things easy smile

Standard User Lethe
(fountain of knowledge) Fri 22-Mar-13 11:29:37
Print Post

Typical!


[re: b4dger] [link to this post]
 
http://noc.enta.net/

I configured my router earlier (I am testing certain settings to see which give least CRC errors), and scheduled a reload at... you guessed it, 09:59! The router came up OK, but sortly after, I lost throughput again - the VTi went down and up, but no throughput. I thought it was what I done so reverted the change laugh

Oh well. Welcome to 1995 again.

Nick
Standard User denize
(experienced) Fri 22-Mar-13 14:29:11
Print Post

Re: Oops!


[re: Kathphlox] [link to this post]
 
I also got the BT Wholesale page, as you say I wished it would have come up sooner
after lots of messing about with my router,, even getting my old one out of the attic

BT Wholesale SERVICE INFORMATION


You have been connected to this page due to one of the following reasons. You must now shut down your browser and Internet applications before attempting to reconnect.

The BTWholesale access circuit to your Service Provider is currently down.
Your service should be resumed soon.
Or
You are testing your local connection to BT using
'bt_test_user@startup_domain'
Or
You are testing your connection to your Service Provider using 'bt_test_user@domain' where domain is your Service Provider domain name.
Or
Your Service Provider is currently unable to accept your connection request please wait until service is restored or contact your Service Provider for further information.
Or
You have attempted to access an invalid Service Provider dom

Edited by denize (Fri 22-Mar-13 14:35:54)

Standard User Lethe
(fountain of knowledge) Fri 22-Mar-13 16:28:02
Print Post

Laurel & Hardy


[re: therioman] [link to this post]
 
http://noc.enta.net/2013/03/virgin-media-leased-lines/

This appears to have been the result of a Virgin Media engineer accidentally disconnecting part of the Virgin Media equipment. Connectivity has already been restored.


For our younger viewers - this is Laurel and Hardy:

http://www.youtube.com/watch?v=DZ8D-bnwBK0

Nick
Standard User professor973
(committed) Fri 22-Mar-13 20:29:34
Print Post

Re: Laurel & Hardy


[re: Lethe] [link to this post]
 
That outage is a new one as of 15:29 this afternoon which was quickly corrected. Who are you, Laurel or Hardy? smile

"We have just experienced a brief outage on several Virgin Media provided leased line circuits provided into Telehouse East.
This appears to have been the result of a Virgin Media engineer accidentally disconnecting part of the Virgin Media equipment. Connectivity has already been restored
."

Yesterdays outage as of 17:15 this evening.

"The line card has remained stable, however the underlying cause which triggered this incident in the first place warrants further investigation with Cisco. As a result of this, the line card will remain running on a reduced workload and we will continue monitoring it's status for the time being.

Once the case is fully investigated and concluded an incident report will be published. We expect to be able to provide the next update on Monday 25th March
. "
... So now we know!

The difference between genius and stupidity is; genius has its limits.
http://www.pingtest.net/result/68380009.png

Edited by professor973 (Fri 22-Mar-13 20:53:18)

Standard User Lethe
(fountain of knowledge) Fri 22-Mar-13 20:58:43
Print Post

Re: Laurel & Hardy


[re: professor973] [link to this post]
 
Did I say it was related? No. So please read the post and don't put words in my mouth.

I was just jesting what a situation this is.

At least Enta report the L&H carry-ons, anyhow.

Nick
Standard User professor973
(committed) Fri 22-Mar-13 21:06:19
Print Post

Re: Laurel & Hardy


[re: Lethe] [link to this post]
 
Nor did you say it was a seperate incident, just hung it on this thread relating to yesterdays outage, but I expect like most you will want the last word!

The difference between genius and stupidity is; genius has its limits.
http://www.pingtest.net/result/68380009.png
Standard User therioman
(knowledge is power) Fri 22-Mar-13 21:18:10
Print Post

Re: Laurel & Hardy


[re: Lethe] [link to this post]
 
Yeah that's 4 different outages in 24 hours at least.

A bad couple of days over at Enta...
Standard User therioman
(knowledge is power) Fri 22-Mar-13 21:21:00
Print Post

Re: Laurel & Hardy


[re: Lethe] [link to this post]
 
To be honest, what worries me most from this Enta outage is that any "lab environment" could in any way be connected to a live, production service.

by my definition, any system that is "lab" should not touch "live. Period.

Poor decisions somewhere at enta.
Standard User Adam3k
(knowledge is power) Sat 23-Mar-13 21:45:22
Print Post

Re: Laurel & Hardy


[re: therioman] [link to this post]
 
I had the same thought.. But its fine they have BGP filtering wink

Ive yet to bring back up our peer with them.

-------------------------------------------------------------------------------
StructuredCommunications Bonded 2 x 80/20 FTTC 74.5Mbps Download & 22.4Mbps Upload. Transit provided by Level3
Standard User Lethe
(fountain of knowledge) Fri 29-Mar-13 11:04:58
Print Post

Re: Oops!


[re: therioman] [link to this post]
 
I think we all may owe Enta an apology here - the date/time corresponds with:

http://blog.cloudflare.com/the-ddos-that-almost-brok...

Nick
Standard User Adam3k
(knowledge is power) Fri 29-Mar-13 21:29:49
Print Post

Re: Oops!


[re: Lethe] [link to this post]
 
Why?

Enta have already advised that the issue relating to the orignal post here was due to a BGP filter and was also released here: http://noc.enta.net/wp-content/uploads/2013/03/Incid...

-------------------------------------------------------------------------------
StructuredCommunications Bonded FTTC 74.5Mbps Download & 22.4Mbps Upload.
Standard User Lethe
(fountain of knowledge) Fri 29-Mar-13 21:52:39
Print Post

Re: Oops!


[re: Adam3k] [link to this post]
 
Yes, I read that... but if the card was flaky, then all of a sudden the huge increase in traffic could have made it die pretty quick.

I do think this is related, anyhow.

Nick
Standard User professor973
(committed) Fri 29-Mar-13 23:33:19
Print Post

Re: Oops!


[re: Lethe] [link to this post]
 
The Attack:-
The attack fluctuated between 90Gbps and 30Gbps until 01:15 UTC on on March 21.

The attackers were quiet for a day. Then, on March 22 at 18:00 UTC, the attack resumed,

The incident:-
The report below is a breakdown of the incident that was experienced across our network from approximately 18.30hrs on Thursday 21st March 2013.

Do the maths!

The difference between genius and stupidity is; genius has its limits.
http://www.pingtest.net/result/68380009.png
Standard User Adam3k
(knowledge is power) Sat 30-Mar-13 02:11:13
Print Post

Re: Oops!


[re: Lethe] [link to this post]
 
This post was to do with the network outage on the 21st at 6pm odd. Nothing to do with the line card, that was the following morning.

The report i posted which was linked to this OP:

This indicated that the core routers were seeing excessive BGP routes , hence the overrun. This was then rapidly traced to our lab environment within our Telford HQ where a stress test of new hardware and equipment was taking place.

Think you getting your dates mixed up wink

-------------------------------------------------------------------------------
StructuredCommunications Bonded FTTC 74.5Mbps Download & 22.4Mbps Upload.
Pages in this thread: 1 | 2 | 3 | 4 | 5 | (show all)   Print Thread

Jump to