P2P & Newsgroup Traffic Management

I'm thinking of signing up for the 50Mbit XXL service, as I understand it P2P & Newsgroup traffic will be regulated during peak hours for all users no matter which VM package you are on. So how does this work in the real world. Can anyone on VM's 50Mbit XXL package who uses P2P or Newsgroups during peak hours detail their experiences, at what point were your speeds cut and how much were they cut by. Overall are you happy with the service you receive from VM ? Thanks in advance for any replies.

Edited by deleted (Tue 15-Feb-11 20:36:24)

don't really use p2p but on usenet from giganews, simply use port 443 and this bypass's the throttling. if not and you use 119 or 563 you get stuck on about 1.75MB/s compared to 6MB/s

good info thanks for that

In reply to a post by nvidamour:

don't really use p2p but on usenet from giganews, simply use port 443 and this bypass's the throttling. if not and you use 119 or 563 you get stuck on about 1.75MB/s compared to 6MB/s

That is not true on my father's 10 meg virgin connection i run port 443 on my newsserver and it's throttled.

In reply to a post by aj2703:

That is not true on my father's 10 meg virgin connection i run port 443 on my newsserver and it's throttled.

Good job he was asking about 50Mb then.

The poster i quoted said port 443 gets past throtteling. I said it does not on my fathers 10 meg. If the port is throttled i wouldn't have thought it would matter what package you are on, moreso if 563 is throttled too on all packages.

Correct. The shaping is on all products. It is only declared to be active part of the time and I'm not sure if they've finished experimenting with it as comments I've seen suggest it isn't active all the time they've said it will be.

If the VM kit can be fooled simply by running NNTP on the standard SSL port for HTTPS I'll be very surprised and VM will be looking for a refund from their DPI equipment vendor.

Edited by kwikbreaks (Mon 21-Feb-11 21:47:33)

I might not have been technically correct in that port 443 gets past throttling but what i've seen (and other people) is using it doesn't affect my download speed like the other ports do. This is at any time of the day. Generally i'd always see anywhere between 5.5 - 6MB/s

I'd also agree that it seems a very simple way to bypass the hardware they have put in place.

Remember that they're essentially on 2 different networks. Virgin's 10/20 meg offerings run over DOCSIS 2, whereas 30/50/100 are on their DOCSIS 3 network. Maybe that has something to do with it?

I've found that with torrents, during the day it does seem harder to get higher speeds, but if there are enough seeds (or some fast seeds), I can still max out this 30 meg line and get roughly 3.6 MB/s. 10am, 2pm, 5pm, 8pm etc.

In reply to a post by kwikbreaks:

Correct. The shaping is on all products. It is only declared to be active part of the time and I'm not sure if they've finished experimenting with it as comments I've seen suggest it isn't active all the time they've said it will be.

If the VM kit can be fooled simply by running NNTP on the standard SSL port for HTTPS I'll be very surprised and VM will be looking for a refund from their DPI equipment vendor.

Except it's not NNTP on port 443 it's NNTP inside TLS. The only ways to spot that would be via non-protocol based traffic analysis which is quite effective and the equipment is quite capable of if configured to do so which anecdotally it appears that it isn't.

If it's encrypted it's obviously harder to detect but that is what the hardware was designed for I thought. If they haven't got their kit configured to detect NNTP regardless they are either being very cynical or very naive. I suspect it just isn't operating all the time so they can do some real world evaluation of its impact on network loading.

In reply to a post by Roph:

Remember that they're essentially on 2 different networks. Virgin's 10/20 meg offerings run over DOCSIS 2, whereas 30/50/100 are on their DOCSIS 3 network. Maybe that has something to do with it?

I've found that with torrents, during the day it does seem harder to get higher speeds, but if there are enough seeds (or some fast seeds), I can still max out this 30 meg line and get roughly 3.6 MB/s. 10am, 2pm, 5pm, 8pm etc.

10mbit typically on docsis1 up and down
20mbit will either be on docsis 1 up and down or docsis3 down, maybe on docsis2 up if uplifted area.
30, 50mbit will all be on docsis3 down and either docsis1 or docsis 2 up depending if uplifted area.
100mbit should always be on docsis3 down and docsis2 up as its a uplifted area only service.

also as far as I know 443 does evade the traffic shaping for encrypted traffic.

In reply to a post by Chrysalis:

also as far as I know 443 does evade the traffic shaping for encrypted traffic.

That's interesting. I wonder what they are playing at.

I know, I did wonder that myself once news got out that traffic shaping was purely by port and not using DPI, seems odd.

In reply to a post by kwikbreaks:

In reply to a post by Chrysalis:

also as far as I know 443 does evade the traffic shaping for encrypted traffic.

That's interesting. I wonder what they are playing at.

Same here. My dad's connection gets throttled on the 10meg. Why would they not do it on the other packages regardless wether they are docsis 1,2 or 3.

Hmmm....

I don't know much at all about the DPI kit or how VM organise their data centre so the only guess I can make is that the DOCSIS1/2 and DOCSIS3 parts of the network are handled separately and only the DOSCIS1/2 portion actually has the DPI kit which I guess is very expensive and they are using something simpler/cheaper for the DOCSIS3 portion? That would deter downsizing by heavy downloaders because all products get similar speeds on p2p and nntp.

Whatever the reason I can't se it staying that way as the news will spread fast amongst those it impacts (it isn't only files that get shared) and the shaping will quickly become totally ineffective. It will only get worse (from the VM viewpoint) as more and more move to DOCSIS3.

In reply to a post by kwikbreaks:

Hmmm....

I don't know much at all about the DPI kit or how VM organise their data centre so the only guess I can make is that the DOCSIS1/2 and DOCSIS3 parts of the network are handled separately and only the DOSCIS1/2 portion actually has the DPI kit which I guess is very expensive and they are using something simpler/cheaper for the DOCSIS3 portion? That would deter downsizing by heavy downloaders because all products get similar speeds on p2p and nntp.

Whatever the reason I can't se it staying that way as the news will spread fast amongst those it impacts (it isn't only files that get shared) and the shaping will quickly become totally ineffective. It will only get worse (from the VM viewpoint) as more and more move to DOCSIS3.

the traffic shaping is same for everyone, regardless of docsis version, however it will vary I think depending on loal congestion levels so I think people in congested areas will possibly see lower shaped speeds.

It is STM that differs per package. There is tons of confusion as people often think STM and shaping are one and the same thing.

It boils down to one thing really. Hard core serial torrent users will always find ways to evade shaping, but the casual torrent users wont bother messing with ports, VPNs proxies etc. There is tons of casual torrent use out there. So this will be having an affect I suspect but I dont think the effect is significant overall as now days p2p is only a small % of download traffic, its only still a majority on upload traffic. So VM have been shaping the wrong direction Especially when considering 90% or so of congestion on VM is on upstream capacity not downstream.

Edited by Chrysalis (Thu 24-Feb-11 11:49:22)

I thought I read that the STM was a built in feature of the CMTS so was reasonably local. Being just volume driven it can't differentiate between congested and non-congested areas.

The shaping kit is used by ADSL ISPs too so must be designed as high capacity for use at the ISP data centre and I would have thought by then it would all be aggregated so there would still be no different action depending on local congestion although I suppose it could be done based on IP or something. What I was wondering was whether or not the DOCSIS1/2 and DOCSIS3 networks were handled differently back at the data center or if they just get combined at a local level anyway.

Given the low STM thresholds on the old products maybe is i just that kicking in and not shaping which makes the 10Mbps poster above think using 443 doesn't help him.

Traffic is marked centrally and shaping policy enforced locally.

In reply to a post by kwikbreaks:

If it's encrypted it's obviously harder to detect but that is what the hardware was designed for I thought. If they haven't got their kit configured to detect NNTP regardless they are either being very cynical or very naive. I suspect it just isn't operating all the time so they can do some real world evaluation of its impact on network loading.

Zero point in doing that, the hardware can sit in monitor mode to assess traffic distribution.

It detects NNTP fine but cannot detect NNTP encapsulated in TLS, that's the point of TLS.

I imagine there's a performance reason why other ways to detect, such as by source IP, aren't being used.

In reply to a post by Ignitionnet:

It detects NNTP fine but cannot detect NNTP encapsulated in TLS, that's the point of TLS.
I imagine there's a performance reason why other ways to detect, such as by source IP, aren't being used.

Or destination IP.

Pretty easy to work out whats being accessed if its to a host in the ".giganews.com" domain and the data part of the IP packet is encrypted.

In reply to a post by Ignitionnet:

Traffic is marked centrally and shaping policy enforced locally.

Thanks that's interesting. Seems to be fairer than just applying a blanket throttle.

So the DPI kit can't in any way detect protocol when encryption is used? I assumed it must be able to in some way. Seems to me to be an expensive waste of money to me in that case as the heaviest users will almost certainly be the most knowledgeable and just bypass it.

In reply to a post by kwikbreaks:

So the DPI kit can't in any way detect protocol when encryption is used? I assumed it must be able to in some way. Seems to me to be an expensive waste of money to me in that case as the heaviest users will almost certainly be the most knowledgeable and just bypass it.

Cost-benefit analysis. I've zero doubt the hardware can identify it but it would require using more hardware, the more work you make it do to identify a stream the less streams it can handle concurrently.

The very heaviest will fall foul of the DUP anyway and get nailed that way

Got it. Thanks.

In reply to a post by nvidamour:

don't really use p2p but on usenet from giganews, simply use port 443 and this bypass's the throttling. if not and you use 119 or 563 you get stuck on about 1.75MB/s compared to 6MB/s

To try using 443 out do I go into Preferences > connection > change port used for incoming connections from nnnnn to 443?

Assistance appreciated?

You need to set the port in your chosen usenet program. I had problems with grabit crashing along with my brother so used newsleecher instead. There are probably others that you can use as well.

you may need to explicitly set it to use ssl as well. in newsbin pro i check a checkbox to use ssl which then automatically uses 443, though i can set an alternate port such as 563.

Anyone found a workaround? My connection was turned on today, just found out I have a 80kbs torrent connection from a 30mbs line, lovely flat line at that 80k point over time, fluctuates by under 2kbs. vpn (hotspotshield) doesn't affect that cap

Turn on Forced encryption in your torrent client.

Rarely used torrents before but the packet loss/latency on my VM connection makes it no so good for streaming or gaming so downloading is the only thing the connection is good for.

Never had slow torrents speed at times when they throttle due to the ssl/forced encryption setting. Stupid really as I would grab a few things from VM's own newsgroups if they didn't throttle, which would mean 2 connections instead of the torrent type of saturation.

Got a feeling VM will throttle SSL at some point, upsetting VPN users, SSL newsgroups subscribers and torrent users.

with encryption forced I'm still hitting a wall at 80kbs, that's on openoffice which is both legal hand heavily seeded. upload hits 220kbs though