Stopping the hogging...

Picture the scene - you drop a reasonably large file into cloud storage app (One Drive in this case) and it starts to sync. However, because it's maxing out the upload link, everything else grinds to a halt.
Back in the day I used to run a linux based router, and I used to prioritize ACK packets so that the rest of the connection had a better chance of working as normal.
I'm running a BiPac 7800N now, and whilst it appears do do QOS, it's a little more basic than being able to prioritize specific types of packet.
Is it just me who has this problem? And is there a way to control it at a network level? Are newer routers better at this? Does is disappear with fibre (I'm on ADSL 2+ at the moment)?
Many thanks!

I'm of the opinion that clients which have a lot of uploading to do should always have the ability to cap the upload speed (e.g. Filezilla).

That said, I have an HG533 router (which is relatively sophisticated for an ISP-supplied router), and when clients are copying large files across the LAN via Robocopy, internet access for clients is practically lost. I suppose the router should handle this situation more gracefully.

That said, the router does have QoS, but I haven't got around to figuring out what these settings actually do:

http://s15.postimg.org/ypgwrofbv/qos1.png
http://s18.postimg.org/u5paanzyx/qos2.png

Completely agree. It's why uploading via a webclient is so annoying (my other bugbear!). Dropbox has a throttle, so I don't see why Onedrive shouldn't. Especially for a piece of software which is supposed to work in the background!
I've just been reading up on DSCP, but I've left more confused than when I started. The concept is pretty simple, but I don't understand why it's left for the user to set up when a default configuration should serve 99% of people.

In reply to a post by Oliver341:

I'm of the opinion that clients which have a lot of uploading to do should always have the ability to cap the upload speed (e.g. Filezilla).

That said, I have an HG533 router (which is relatively sophisticated for an ISP-supplied router), and when clients are copying large files across the LAN via Robocopy, internet access for clients is practically lost. I suppose the router should handle this situation more gracefully.

with robocopy you can specify an inter packet gap, which will give other programs a chance to use some of the bandwidth

I suppose most people will not want the hassle of setting their client's upload speed to 75% sync, and want it to just work. But there's no reason not to put the option in there, for those that want to throttle it.

Differentiated services does seem like a potential solution though, perhaps in the long term if router manufacturers decide to adopt it.

In reply to a post by ggremlin:

with robocopy you can specify an inter packet gap, which will give other programs a chance to use some of the bandwidth

That sounds useful. I saw the definition "Specifies the inter-packet gap to free bandwidth on slow lines" which I thought would not be applicable to 100 mbit LAN, but I will certainly give it a go.

Make sure that your TCP/IP settings are optimised, especially the MTU size. You can do it manually or you can use a utility such as TCP Optimizer. Packet fragmentation will be more noticeable on uploads because of the lower bandwidth available.

Edited by micksharpe (Tue 14-Apr-15 16:46:07)

Good to see the Help page gives a good insight to the settings. Also, I note all those users with an unlocked HG612 have a QOS-enabled modem

In reply to a post by BatBoy:

Good to see the Help page gives a good insight to the settings. Also, I note all those users with an unlocked HG612 have a QOS-enabled modem

Have you looked into configuring QoS on Huawei's? I've had a read of help tabs, but I'm unclear if these settings can control traffic priority per protocol per device, or are simply intended to manage traffic per device (i.e. de-prioritising all traffic from a specific device on the LAN).

Let the dog see the rabbit

QoS

When the bandwidth is limited, through the QoS function, the gateway can allocate all the bandwidth to a certain application, increase the priority of the application.
Global Settings

On this page, you need to set certain global parameters of the QoS. The parameters are described as follows:

Enable: It is used to specify whether to enable the QoS function.

Bandwidth: It is used to specify the bandwidth used for the QoS.

Queue type: It is used to select a policy used for the QoS. The available types are described as follows:

PQ: If you set Queue type to PQ, four queues are listed based on the priorities. The priority is classified into the highest priority, high priority, medium priority, and low priority. The gateway first schedules the queue with a higher priority. If the queue with a higher priority has a packet cache, the queue with a lower priority cannot be scheduled.
WFQ: If you set Queue type to WFQ, four queues are listed. You can set the weight of the bandwidth occupied by each queue. The bandwidth is allocated to each queue based on the corresponding weight.
WFQ,PQ: If you set Queue type to WFQ,PQ, fore queues are listed. The first queue has the highest priority, The other queues scheduled by WFQ.

Note: The templates have certain preset classification rules. After you select a template, you can modify the parameter settings of the template.

Queue: The parameters of the queue vary with queue types. You need to set the queue type in the global settings, and then set the corresponding queue parameters. For each queue type, all the queues can be enabled or disabled separately.

Policer: You can limit the rate of stream though configing the policer parameters.

Traffic Classification

On this page, you can manage the traffic classification rules on the basis of the QoS settings. The parameters are described as follows:

Rule name: It is used to specify the name of the current rule.

Enable: It is used to specify whether to enable the current rule.

Source MAC : They are used to specify the source MAC addresses to be classified.

Dest MAC : They are used to specify the destination MAC addresses to be classified.

Source IP and Source IP Mask: They are used to specify the source IP addresses and mask to be classified.

Dest IP and Dest IP Mask : They are used to specify the destination IP addresses and mask to be classified.

Source Port and Source Port Max: They are used to specify the range of the source ports to be classified.

Dest Port and Dest Port Max: They are used to specify the range of the destination ports to be classified.

Lan Interface: It is used to select the lan interface on which the packets are classified.

Wan Interface: It is used to select the wan interface on which the packets are classified.

DSCP : They are used to specify the DSCP to be classified.

Protocol: It is used to select the protocol used by the packets to be classified.

VLAN ID : They are used to specify the VLAN ID to be classified.

802.1p : They are used to specify the 802.1p values to be classified.

Note: After setting the preceding matching conditions, you can perform the following operations on the matched streams:

DSCP mark: It is used to add a DSCP mark to the matched stream.

802.1p mark: It is used to add a 802.1p mark to the matched stream.

Policer: It is used to select a Policer that the configured rule is to be applied. For details about how to configure the Policer, see the description about the QoS page.

Queue: It is used to select a queue that the configured rule is to be applied. For details about how to configure the queue, see the description about the QoS page.

Note: After setting the parameters, click Submit so that the rule can take effect. In addition, you can modify, delete, or disable the rules.

Edited by deleted (Tue 14-Apr-15 17:38:08)

I suppose we need a real world example. E.g. how would one configure http to be prioritised over ftp using this interface?

I think you could just specify the ports and assign them to 2 priority queues.

In reply to a post by BatBoy:

I think you could just specify the ports and assign them to 2 priority queues.

Yeah, Source/Dest Port is configurable, so I suppose where that is a known value it can be used in QoS. In the case of Robocopy, I could experiment with Dest Port 445 (the PCs on the LAN seem to be communicating via link-local IPv6).

In reply to a post by Oliver341:

That sounds useful. I saw the definition "Specifies the inter-packet gap to free bandwidth on slow lines" which I thought would not be applicable to 100 mbit LAN, but I will certainly give it a go.

any link running at 100% can be considered a slow link

What you're all describing are the effects of bufferbloat. There are too many, unnecessarily large buffers in far too many network devices between you and the destination. TCP by its very definition, given sufficient CPU power, will fill up any buffer given enough time. Once the buffer is full then your packet, whatever size, whatever priority will have to wait its turn in the queue as data drains away down the slow pipe.

The trick is to manage the queue/s and not quite fill the pipe, thus latency (which is the measure you are using to measure how 'sticky' your network connection is) can be controlled. Once latency is managed, interactive/time critical packets can be interwoven amongst the longer term 'but hog all the bandwidth' data flows. The huge 'elephant' flows as they are known can be managed by dropping packets to tell the other end to slow down.

This allows space for the 'ants' (DNS, ICMP, VOIP, TCP/SYN/ACK - small usually single packet but desperately important) and 'mice' (Short, small transfers) to do their work and give the interactive (ie low latency) experience you actually required.

Google for cerowrt, Dave Taht, Jim Gettys, bufferbloat, 'tcp ants mice elephants', fq_codel.

https://gettys.wordpress.com/bufferbloat-faq/

The cerowrt bufferbloat project fed a lot of information back into the mainline Linux kernel, and latest versions of openwrt with 'SQM Scripts' (for fq_codel queue management) do extremely well in managing latency and perceived performance of bloated edge of internet routers.

If that doesn't scare you then the talk http://cyber.law.harvard.edu/events/luncheon/2014/06... certainly should!

Friends don't let friends run factory firmware.

Ignore the red bits and anything pre 12:00 but at 1830ish I started a 2GB 'one drive' upload along with a 1GB Ubuntu image download (that's the 6min duration slightly higher latency spike) Interactive performance during this time has been pretty much unaffected. This is on a 40/10 Sky IPv4 link with a HG612 modem with TP-Link Archer C7 AC1750 running Openwrt CC latest trunk (as of 12:00 today ish) and SQM scripts - SQM limits are set to 38000 down & 9500 up respectively.

Oh and then one of the machines upstairs started doing some sort of automatic download...all pretty much unnoticeable.

My Broadband Ping

Edited by deleted (Tue 14-Apr-15 20:05:28)

If the only tool you have is a hammer, then every problem becomes a nail.

In reply to a post by Oliver341:

That sounds useful. I saw the definition "Specifies the inter-packet gap to free bandwidth on slow lines" which I thought would not be applicable to 100 mbit LAN, but I will certainly give it a go.

I wonder if your router is using the CPU to copy packets to/from the LAN ports and running out of CPU ?

Perhaps try putting a cheap 5port separate switch in front of the router (ie, all clients on router into this switch, and one link to the router) as that should ensure LAN traffic stays on the switch and WAN traffic goes to the router.

I've been using OpenWrt with fq_codel for the past couple of years now with great success (see here for full details of my config including the multiple wireless access points question i noticed you asked about recently on the CeroWrt list).

I don't tend to do big uploads to services like Dropbox because i host everything on my FreeNAS server but because of this there's almost always uploads and downloads happening on my connection. Thanks to fq_codel no matter what i do my connection remains extremely responsive (even with the netperf-wrapper tests the additional latency stays within 10ms) usually within 3-5ms of additional latency.

People just don't seem to realise how bad a problem bufferbloat is. It's absolutely everywhere and the fix is simple; use a router that supports fq_codel.

In reply to a post by jchamier:

Perhaps try putting a cheap 5port separate switch in front of the router (ie, all clients on router into this switch, and one link to the router) as that should ensure LAN traffic stays on the switch and WAN traffic goes to the router.

Yeah, could do.

I've had a play with some of the Huawei QoS settings. Certainly it's very easy to see the "Policer" in action, since it's a straight speed throttle. Creating a Policer with CIR and PIR set to 100 limits the speed to 100 Kbps. Assigning port 22 (SFTP) to it throttles it to 10 KBps and it's verifiable in Filezilla.

However, putting port 445 into the Policer and running Robocopy had no effect on the slowdown, which makes me think the QoS on this router is purely for WAN traffic. But even so, it's a nice feature and seems to work well for WAN traffic.

The IPG option in Robocopy is a workable solution. It obviously slows down the transfer considerably when a sizeable gap is specified, but since it's generally being run as a background operation, it's not time critical.

When I enabled QoS on my router (using tomatousb) to prioritize ack's my max speed dropped to under 50mbps due to the fact the router cpu could not handle scanning all the packets for my line speed, so the problem QoS has with faster connections is you need quite a beefy cpu. Also it tends to add bufferbloat and for it to be effective you generally need to have some bandwidth in reserve.

funny enough the hg612's first firmware had QoS on by default which took 10% of the upload speed.

Edited by Chrysalis (Thu 16-Apr-15 11:13:44)

This is something I miss. being on ADSL 2+ also for now I notice this but I never did before when I had fibre. Oh the pain!.