Pages in this thread: 
Print Thread
Without buffering an overbooked network will simply replace latency with packet loss
And that, if you knew anything about TCP/IP would explain everything. What does a TCP/IP implementation do when packets are delayed? Nothing. What about if they're dropped? It throttles the stream, reducing its internal estimate of the available bandwidth. So the way to signal that the link is full is to drop packets. Something like RED does this rather crudely but still effectively, while newer strategies like CoDel are smarter about it. But just having larger and larger queues makes it worse.
and without queuing, which requires buffers, it's impossible to apply QoS to a network, which is essential in enterprise environments.
Only very tiny buffers are needed for this purpose. We need keep only enough packets to fill the pipe for our anticipated maximum latency. Any more packets (yes even if you think they're super-important urgent double-plus high-priority) are just bloat, our refusal to drop them makes things worse because it removes the feedback for the TCP/IP stack.
Surely having enough bandwidth out of the buffer to ensure it doesn't start to fill is the way to go
All else being equal having infinite bandwidth would be great. But we can't afford infinite bandwidth. So, we will have less bandwidth than some upstreams and that download of the entire world map at 1 pixel per 100 metres will have to take a little longer than zero seconds. The download will fill any available buffer. The larger the queue the worse for everyone except the world map downloader, for them everything is very, very slightly better if they can have a huge queue. Fitting the whole map in the queue would mean that they get it a tiny bit faster, and the "only" downside is that the entire system is completely unusable for everyone else sharing the link meanwhile.
, or if that's not feasible the approach that I've seen more commonly which is not to use FIFO but to use slightly smarter queuing which favours smaller packets over larger ones?
This hack is a slight improvement but it relies on a heuristic (guessing that "important" packets will be smaller) which is not reliable. Some things will work better, others will not. Still, it does have one nice feature which is that it doesn't require configuration.
You're aware of weighted fair queuing, for example, which works well to avoid single high bandwidth flows hogging bandwidth, or class-based H-FSC which is effective as a scheduling algorithm across TCP and UDP traffic?
We're actually talking (in case you've forgotten) about residential broadband, which means anything that requires setup doesn't get done. When step 1 is "read this manual about how to configure your broadband for best VoIP performance" virtually nobody will get to step 2. A lot of WiFI APs with bad buffer bloat can be fixed by replacing their firmware with images from the Internet. Guess how many people do that? Virtually nobody.
Experienced network engineers tend to see their own bit but tend to ignore what's actually running on the network at layer 4 and above. A VoIP call can cope with a little delay, what it can't cope with is loss.
Many years ago now I did a presentation in which the audience included some people from big cell phone companies. We were demoing a VoIP system as part of a larger concept, and when it was apparent that our methodology had introduced 200ms latency (which we'd been pretty impressed by considering what we were doing) the cell phone engineers laughed their heads off. Nobody wants that, it's garbage, their customers would leave in droves.
Loss is much more acceptable than latency. Codecs like GSM designed for raw radio with mobile endpoints are tolerant of bit loss, but even the Internet codecs like Opus are tolerant of packet loss and perform "concealment" when they detect a lost packet. Obviously it's not ideal to drop anything, but dropping one packet now is definitely better than delivering the next dozen packets late.
My employer's products are built both to apply QoS and to mitigate packet loss on the WAN, not to introduce loss by dropping VoIP traffic because another flow is trying to eat through the bandwidth.
You will glad to know that even older techniques like RED will tend to drop mostly the "other flow" in these circumstances. Not because of any clever (and thus never to be configured by real end users) QoS but because statistically the larger flow has more packets in it and thus will get hit more often.
The simplest way to prevent serialisation is to throw more bandwidth at a problem. The best way is to apply QoS, with maximum delay parameters set on multiple leaky-bucket classes of traffic. A packet spends too long in a queue it gets dropped, the VoIP / real-time / interactive queue gets emptied first.
Leaky buckets are cool, we had leaky bucket based traffic management in a shared house I lived in about 10-15 years ago. But (a) they need configuring by the end user which won't happen and (b) they can't work around some bloated buffer inside a modem or similar. For that you need active management.
Incidentally I can assure you that 20Mbit/s doesn't start to feel so nice when you have an unmanaged link and one person hammering at it with downloads. It is precisely to avoid such issues that I have 2 load balanced FTTC lines.
I suspect that most people would rather spend half as much (just one line) and have equipment that didn't encourage Buffer Bloat, if only they knew the difference. The good news is that fixing this doesn't really cost anything, we just need to dribble the relevant best practices into Linux and other code used to build consumer network gear and it'll "magically" make everything better.
This is quickly going off topic, please Google "Buffer Bloat" if you care to know more.
