Broadband Quality Monitor

Hi

Can anybody advise me as to what the BQM graph is telling me?

I would like to understand:

1. Does the red packet loss mean the connection is not being used at that time?

2. If so, why does this earlier example of the graph show activity during the early hours when nothing was knowingly being used?

My Broadband Ping

For the first couple of days the graphs were similar to this.

3. What has changed to now give this example of the graph over the last couple of days?

My Broadband Ping

We have a couple of iPhones in the house but they seem to disconnect when not actively viewing anything.

4. What could be causing the latency spikes and rapid average latency increases?

5. Why do speed testers 9 times out of 10 show 0ms for ping? I have ping tested in the CMD menu and 7-8ms seems about average but it almost never shows when speed testing.

6. Or is this all entirely normal?

Would just be nice to know that it is, or if the router set up is an issue. The graph is shared below.

Thanks.

Neil

.

Edited by mrnelster (Mon 25-Apr-11 11:17:34)

That's chaos!

Is your IP address static or dynamic?

Click the live link in my sig for mine. That's normal.

Yours is what I thought it should look like

I am using a netgear wndr3300 router and have set it to respond to pings as stated in the TB FAQ's.

I have been connected since 19th and was told that training would be at least 1 month, but wouldn't think that could cause this?

All ideas greatfully received! I think I have shared it correctly below?

Ta.

.

Is your IP address static or dynamic?

I think this means static?


IP Address Assignments

IP Type Single NAT (ipv4)


.

One obvious question- do you turn the router off at night or when you leave the computer for extended periods?

That will give the long red blocks of 100% packet loss... the BQM pings the router, not the BT modem.

(Live BQM link in my sig too)

In reply to a post by billford:

One obvious question- do you turn the router off at night or when you leave the computer for extended periods?

That will give the long red blocks of 100% packet loss... the BQM pings the router, not the BT modem.

(Live BQM link in my sig too)

No. It is left on permanently. I sometimes turn the computer to hibernation and sometimes just leave it on.

But the router is always on. I have this computer connected via an ethernet cable but everything else connects wirelessly.


.

.

Before I start messing about with the router, does this post sound feasible?

http://forum1.netgear.com/showthread.php?t=26360
.

Have you tried a different router, just to rule out that your Netgear router is at fault?

Yes, it certainly does- as the second post says, your idle timeout should be set to zero. I don't know where it is on a Netgear, but it shouldn't be difficult to find and check.

In reply to a post by maniac886:

Have you tried a different router, just to rule out that your Netgear router is at fault?

Not yet, but I think I may have found the issue.

The engineer very kindly installed my router (he isn't obliged to do so) but had only ever fitted Home Hubs previously.

I have just found that the connection status was set to "On Demand" with a 5 minute idle time out setting. That also answered a question I had with Aquiss about multiple Radius Logions of 5 minute durations (give or take a few seconds) registering on their control panel.

That would appear to be a rather big coincidenece, so hopefully that will sort it out.

Fingers crossed.

.

Thanks for you help guys.

It seems to have settled the packet loss issue by changing the router to "always on" mode.

Now to find out what is causing the latency spikes.

.

Can carryimg out speed tests (speedtest.net/Aquis speedtest etc.) cause the BQM graph to show latency spikes?

.

Yes, a "good" (ie fast) speedtest usually will. Same with any high-speed down- or up-load.

Your router is too busy to bother immediately with trivial matters like pings, they're low priority and have to wait.

Thanks Bill, I thought I was starting to notice a pattern.

I must resist the urge to test for a day or two then, to see how smooth thte BQM can get!

Do you have any advice as to how much bandwidth browsing uses? I ask because this morning in a 44 minute window my forum posting and speedtesting, combined with my daughter playing on the "build a bear" website (dont ask!) managed to chalk up 800MB download and 130MB upload.

Seemed very high to me? I may have to re think my usage allowance at this rate!

25 Apr 2011 10:00:17 44 mins 46 secs 181.15 MB 868.28 MB

25 Apr 2011 15:27:11 3 hrs 20 mins 48 secs 88.61 MB 360.79 MB

.

Edited by mrnelster (Mon 25-Apr-11 23:46:49)

A speedtest using TBB will use around 60 to 70 MB down and 5 to 10MB up load ... others will be similar.

It sounds rather a lot, though I'm not too familiar with building bears

Speedtesting can eat a lot of bandwidth- a tbb test is around 40-50Mb (if you check the "Detailed output" box it will tell you), speedtest.net is about the same and from memory I think a BT one is around 120MB. It can soon add up!

Thanks to you both guys.

My own post pretty much answered my question then!

I was wondering what the kids were doing or whether there was anything going on in the background, and there was......me kicking the a*** out of the speed tests!

Would still like to know why I keep getting 0ms ping on the speed tests though?

.

In reply to a post by mrnelster:

Would still like to know why I keep getting 0ms ping on the speed tests though?

.

Get PingPlotter and try running it to BBC.com, tbb and a couple of others and see what you get for the local hops - you MAY be right on top of a server.

Get PingPlotter and try running it to BBC.com, tbb and a couple of others and see what you get for the local hops - you MAY be right on top of a server.

Right. Just downloaded and tried pingplotter. Assuming hop 1 will be my router (192.168.1.1), then the average seems to be around 8ms.

The odd time that a speed tester has shown a ping result, it has only ever been 5ms or 10ms. I suppose eight splits the difference. Just 0ms 9 times out of ten doesnt seem right somehow.

.

Still not a very pretty BQM.


.

I've seen a lot worse, but I'm inclined to agree

I am extremely close to the cabinet and probably very close to a server as MHC posed.

The connection hasn't been used today between 9am and 12pm, but the latency seems worse than when not in use throughout the early hours.
In those early hours Bill, it resembles yourself and Bob's graphs which I would say look extremely stable.

Congestion?

.

Hard to say, but seems likely.

Interested to see your packet loss over the last few hours- quite a few IDNet customers have been seeing that too, including me to a small extent. But you're not with IDNet...

I've seen the odd mention that BT group the smaller FTTC ISPs together, maybe the capacity they provide somewhere is suffering now they've taken the cap off Infinity?

Hmmm....going from 6km from the exchange, to 150 odd metres from the cabinet led me to expect a much more stable latency.

Hopefully things will settle after the "training period" which the engineer told me would probably take a month or so?

Wishful thinking perhaps.

.

I hear a lot about this "training period"... apart from a (two week?) period in which the line speeds are recorded in order to define a fault level, I'm not aware that there is (or ever has been) one.

If my experience is typical, the DLM is quite capable of re-syncing you to a higher speed if it thinks the line will take it, and the corresponding profile change is almost immediate. Afaik it keeps this up indefinitely.

If someone can come up with different information (and a link to an authoritative source) I'd be very interested.

In reply to a post by billford:

I've seen the odd mention that BT group the smaller FTTC ISPs together, maybe the capacity they provide somewhere is suffering now they've taken the cap off Infinity?

We are the process of trying to gather information if this is the case (i've suspect something myself based on some recent evidence). If proved to be true, then a product being sold in 2 classifications would raise serious questions against BT Openreach.

Have you any links?

In reply to a post by aquiss:

Have you any links?

I haven't, I'm afraid.

The closest I've seen to anything definite was a recent outage of some sort in Slough (I think), where the comment was made that it would only affect customers of the smaller, independent ISPs.

As it happened, it didn't appear to affect me at all so I didn't pay as much attention as perhaps I should, hence I can't remember who made the comment or even the forum where I saw it

But your interest is noted... if I come across anything more concrete I'll pass it on to you.

I'll join the club, but mine is due to an ongoing REIN issue.

My Broadband Ping

Funny how mine changed dramatically at almost exactly 12am through till about 7am.

Will check the again tomorrow for a pattern.

.

Apart from the obvious packet loss mountain caused by a power cut today, I have to say my BQM still doesn't look too pretty.

I'm no expert, but the times and levels of increase look suspiciously uniform to me. It's been the same times each day.

I know congestion is to be expected at this time of day on a business package, but isn't that how you would expect a shaped connection to look?

.

In reply to a post by mrnelster:

but isn't that how you would expect a shaped connection to look?

As we don't shape our connections, then no.

There seems to be alot of trust placed on the TBB graphs to be correct. I presume you are graphing via another site for comparisons?

The 8/9am-5:30pm latency rise could very well be a local business starting up and finishing, causing a lot of cross-talk on the copper between you and the cabinet.

I certainly used to get that, (though before I had the benefit of the BQM to give evidence). I was pretty sure which house 50 metres away was responsible, and sure enough when the people moved away my stability improved.

Hmm very similar to what maniacs live graph looks like for today though.
I have been getting extremely low speed with Aquiss and the last few days I have had time off work and been getting as low as 6Mbps during the day. I have even bought a different modem to eliminate things.
I had a very similar graph to yours recently and found it was a lot of Bt work going on at exchanges around the country on which I seem to route through, so keep an eye open for BT work or outages aswell as local issues.

I would post my graph but the new fritzbox I have doesn't respond to a series of pings very well (security/firewall protection kicking in) it has been disabled as I have my line being monitored I don't want the BQM messing up the readings of the monitoring.

In reply to a post by aquiss:

In reply to a post by mrnelster:

but isn't that how you would expect a shaped connection to look?

As we don't shape our connections, then no.

There seems to be alot of trust placed on the TBB graphs to be correct. I presume you are graphing via another site for comparisons?

I wasn't trying to insinuate that you did. Perhaps a bad choice of wording on my part.
I was more inclined to think maybe it was BT prioritising capacity in some form or another.

But what seems obvious is the uniformity between certain hours. They may be peak hours during the day for business and peak in the evening for home use. But still very strangely uniform. I doubt that can be blamed on the tester.

For a connection so close to the cabinet with a 10ms ping on fastpath, to double during peak hours so nicely seems odd. What happens to people a bit further away with a ping of perhaps 35ms? Do we expect there's to be 70ms at peaktimes? That's achievable on ADSL at greater distances than FTTC. Like I said, seems odd.

Do you have any suggestions what other site to monitor with for comparison?

I presume you're asking about the distinct stepped-rise in the yellow section?

The obvious guess (made by someone else too) is that a local business has started work, but the actual effect you'd expect to see isn't quite so obvious.

Certainly a business turning up can add to local congestion - more traffic carried by the DSLAM - but you wouldn't expect to see a straight-line step throughout the day. Would a business be likely to have the same amount of extra traffic throughout the day?

Or they could turn on their ADSL/VDSL at 8AM (and off again at 5PM), and add to your interference - which could cause additional retries of packets. But again, you wouldn't expect to see linear behaviour, as noise doesn't behave linearly - and you'd probably expect to see a few more line drops (and more red packet-loss sections) too.

My guess is that the extra latency is introduced by some piece of kit (either intermediate router, or the endpoint) that queues ICMP Echo packets in a lower priority queue. When that router gets busy enough with "real" traffic, the ICMP queue gets left alone until sometime later - measured by an extra timeout value - and the ICMP packet is only then forwarded/replied-to.

The fact that the green line stays the same suggests that a regular amount of packets are getting through in the original, minimum times. However, the yellow suggests that an equally regular amount are being delayed by consistent amounts.

OK, so my guess for the cause might be totally out, but the consistency of the delay suggests a timer is at work here, not just random traffic.

Could it be caused by shaping? Well yes - of a kind. It certainly looks like some of the packets are being dropped into a lower priority queue, where the priority causes extra fixed latency. That isn't the usual kind of thing meant by "shaping", but it is one kind. And it could be in your ISP, or in TBB's ISP, or in some peering kit.

Here I have the BQM monitoring pings into my router, and I have some internal software that monitors ping times going out to "www.bbc.co.uk". It works in two ways - a plain ICMP ping, and a TCP packet on port 80. They don't always give the same results...

@wwwombat

Thank you. I don't pretend to be anything other than an amateur when it comes to broadband, but I know when something looks odd.

Heres some other snapshots. No businesses near here. Would have to be a home business on this cab, so doubt it's that.

My Broadband Ping

My Broadband Ping

.

Edited by mrnelster (Fri 29-Apr-11 02:21:41)

Here is a snapshot of my TBB monitor yesterday:

My Broadband Ping

No packet loss so far today, presumably because its a Bank Holiday and I have a feeling the REIN is coming from a local business in and around where the cab is located, so they will be closed today. At least one day I will have a perfect connection!

Edited by maniac886 (Fri 29-Apr-11 11:27:53)

I guess I am lucky only having houses close by...

Edited by D_an_W (Fri 29-Apr-11 13:16:28)

Hmmm...I understand the business thing but it sounds like an easy cop out.
I'm on a residential cab. Business connections are on the other side of the estate with their own cabs.

My line has been fine this morning, perhaps in line with what you would expect peope to do on this bank holiday. Get up later, go on line. Then perhaps go to a street party or watch the royal wedding on the telly?

That seems to fit in with today's graph perfectly, just as the work hour times do on my previous graphs.n it seems to me that every time you would expect people to log on, the increases start. And they seem far too symetrical to blame simply on contention and congestion.
One would surely expect greater variations in the spikes? Unless if course they are permanently oversubscribed.

.

Or they could turn on their ADSL/VDSL at 8AM (and off again at 5PM), and add to your interference - which could cause additional retries of packets. But again, you wouldn't expect to see linear behaviour, as noise doesn't behave linearly - and you'd probably expect to see a few more line drops (and more red packet-loss sections) too.

I would expect a linear effect, as the cross-talk is a constant noise. Don't forget that when an ADSL/VDSL connection is on there is a constant stream of packets at the sync speed - their data content is irrelevant.

I have another neighbour running off the same pole. When I was on ADSL I had RouterStats running all the time I was connected. I could easily see the times he switched his router on or off, and it had IIRC 0.4dB effect on my noise margin. During the day I always almost flat-lined, and that didn't change whether he was on or off. Just an immediate step up or down.

I could find the graphs/log but don't feel particularly motivated to do so at the moment.

I never looked to see what happened on BQM, sorry, and now I haven't got access to the history.

I note what you say about the minimum staying stable though. The average isn't fluctuating hugely either. So something interesting is happening, possibly the sort of thing you suggest.

In reply to a post by mrnelster:

No businesses near here. Would have to be a home business on this cab, so doubt it's that.

Are you sure nobody on the cab is running a business from home? There are a few around here, as well as the one I mentioned earlier. Just that their kit isn't doing the same to me as that one did.

Even a home worker VPN'ed into work?

No businesses near here. Would have to be a home business on this cab, so doubt it's that.

Aready thought of that, but that type of uniform increase would mean a constant stream of data. Can't really think of anybody round here that would, and moving data constantly like that wouldn't be the norm for a home business would it?

I am more inclined to believe what you and wwwombat have touched upon and that it would be a piece of equipment and/or a setting.

The reason for that is the speedtests. And before anyone mentions that they aren't accurate, I know. But they aren't uniformly innaccurate either. On different testers I still receive readings of 0ms latency. Many have commented on that anomaly, but nobody as yet has come up with a possible answer to it. But it is that reading that makes me think something isn't being recognised correctly.

I've avoided speedtests all week to help monitor the spikes on the graph, but these have shown the same result as usual:

[IMG]http://www.speedtest.net/result/1275407766.png[/IMG]

[img]http://speedtest.aquiss.net/result/29798/[/img]

[img]http://speedtest.aquiss.net/result/29799/[/img]

If I can eliminate that irregularity (or at least get a reasonable possible explanation for it) then other users on the cabinet/home businesses etc. would be worth consideration.

.

It doesn't have to be a single business, nor one on your cabinet. It could be something on the same exchange, or the same metro, or could be just the accumulation of regional effects.

There are plenty of indications that this isn't a mass congestion effect - in fact, it looks more like the occasional ICMP packet gets delayed by up to 10ms. The only odd thing is that the delays , when they happen, tend to peak at the same value - leaving a straightish yellow area.

Whatever the cause is, it has these properties:
1. - Some packets get through undelayed - as the "minimum" green section stays a straight line
2. - Some packets are delayed, and by variable amounts - as the "average" blue section varies
3. - There is a very definite maximum delay - as the top of the "maximum" yellow section is mostly a straight line
4. - The maximum delay doesn't happen very often - as the blue section is much smaller than the yellow. Maybe the split is 90%minimum, 10% maximum a lot of the time?
5. - The maximum delay happens at least every 100 seconds - as the yellow section is pretty much ever-present during daytime
6. - But packets are dropped - the red section is denser when the yellow section is present

My best guess remains the same - that some router is employing some queuing to the ICMP packets used for the ping. It could be a rate-limiting queue to try to prevent some DoS attack, or it could be something else (I only know a limited amount of stuff too).

I would have thought that most queuing rules would just drop excessive traffic, rather than queue it for an extra 10ms. But certainly you could just be seeing some QoS artifact appearing when the router gets to be "middling busy".

Yes, you're right that crosstalk is going to be constantly present while the modem is turned on. And that it will have a measurable effect on the noise margin that *appears* linear. I too have had the visible steps in my SNR graphs in the past...

However, it is the effects of noise that I really meant to be non-linear, rather than the crosstalk itself. The linear steps in the the SNRM graphs didn't end up having any noticeable effect on the error rates.

I was trying to say that I didn't think that I'd expect to see a straight-line to the yellow peaks if it were caused by the random effects of noise on the line. More effect on the timing, rather than on the height of the line.

But ... as I know more about the functioning of BQM, I can see that it would be possible. It only needs one delayed packet per 100s for the yellow line to appear, and maybe that's not too hard to achieve.

So lets do some different engineering...

ADSL runs at 4000 frames per second, where 69 consecutive frames are termed a superframe. The CRC for superframe N is carried in superframe N+1 - so a retry can only happen 17ms after the original frame was sent - suggesting a minimum delay of 17ms.

I haven't yet worked out what rate CRCs get applied to VDSL2 lines, so can't work out what the minimum delay would be there... More reading needed.

Yes, I accept what you are saying. I was just showing, mainly for the benefit of the OP, that cross-talk effects can be linear and don't depend on the volume of traffic. Latency effects are another matter, and you seem to be following a worthwhile path.

What I would ideally want to establish then, is if this could be a router setting my end?

.

Ok, so this one is late last night.

My Broadband Ping

Now I know there was definitely no usage other than myself, and the spikes fit the time I used it perfectly. I was generally browsing and youtubing video feeds.

So these spikes matched my usage.

.

Well that looks fine. But it isn't a working day.

You can completely eliminate any cause at your end by disconnecting your router from the modem for 30 minutes if you get that 8am step up on Tuesday morning.

In reply to a post by RobertoS:

disconnecting your router from the modem for 30 minutes if you get that 8am step up on Tuesday morning.

Ermm... that will just give a big red block on the BQM

Might be worth disconnecting all computers from the router and turning the wireless off in case it's being pirated.

'Doh!

What I was getting at, was why does the latency spike up when it is being used in that way?

Yours and Bill's are far flatter, so unless you aren't ever using your connections I would assume that it shouldn't spike when you use it unless there are bandwidth issues.

I wouldn't expect that to be a problem between 12.30 and 2.30 am, so why such big spikes when I used it?

Apologies if I am confusing you guys, but I am trying to establish how the graphed latency should behave under normal conditions, during normal functions such as browsing or streaming YouTube etc. I am assuming that your graphs are normal and aren't changing much when you use them?

The reason for the late night excercise was to see what happened when there was likely very little business traffic, and the graph was already flat. I didn't expect it to jump like that during my "normal" type usage.

If I can get my head around the normal behaviour of the monitor, then spotting the different patterns will be far easier. That's the theory anyway.

.

I see from an earlier post that you're using a Netgear WNDR3300 and according to a spec sheet (here) it's got 10/100Mbps Ethernet ports.

This tends to imply a slower processor inside it, and I know that some people have had problems with the slower Netgears not being able to fully keep up with FTTC speeds.

It's a long shot, but any chance you could borrow a router with gigabit ports, implying a faster processor?

Mine's a Buffalo with 100Mbps ethernet. No idea what the processor is sorry.

Any usage carried over the main bottleneck will start adding to latencies - and a fibre connection is still the main bottleneck overall. It *is* good to see that your own personal usage causes variable latencies - that's what I'd expect to see, and it probably mean that the "stepped-yellow" effect isn't caused as an artefact of your own router.

I don't know why you would see latency effects when you aren't fully using your fibre pipe, but I can start having a guess...

I recall seeing the Openreach customer spec for fibre products, and they point out a feature of the bandwidth: That 20Mbps (or line speed, if it is lower than this) is "prioritised".

I don't know what form this prioritisation takes, or how they choose which half of your data deserves priority, but it certainly implies that BT's network is capable of applying QoS measures over both your FTTC link and through their own network back to the ISP. A little more info here.

We can't know how they apply the rules, but if I had to guess... I'd say one type of traffic that wouldn't get the priority would be ICMP echo; and one type of traffic that would get the priority would be streaming audio & video.

If they are separated in this kind of way, then you'd expect some packets to get delayed by streaming traffic (explaining the yellow spikes), but because you haven't saturated your link, you'd expect most to go through unimpeded (so the blue "average" levels stay low, near the "green" minimum).

I suspect you might be getting too worried about the spikes you are seeing in yellow - which is the "maximum". It shouldn't be too much of a worry if the "average" isn't following... Remember only one delayed packet in 100 will cause a yellow spike.

Mine spikes about to your level when I do a tbb speed test, and far more when I do a BT one which uses 16 threads.

I don't often check it to be honest, and I don't use YouTube. Using this site just carries on with the tiny murmuring at the bottom. Occasionally it goes a bit higher for 30 minutes or so. Here's 29 April. I did a tiny bit of BBC streaming around 5pm, looking for the Aston Martin, and IIRC the big spike was a BT speed test. No idea what the 7:30pm one was.

Note the packet loss just after midnight. That is congestion at IDNet when the "heavy" downloaders kick in.

I may well do that Bill. If I remember correctly Bob paid very little for his Airstation, so for 30 quid it might be a worthwhile test to restore my sanity!

I greatly appreciate wombats help, but just cannot follow the logic at the moment.

If my latency is 10 ms, my assured throughput is 12Mbps, and I am browsing the net such as this site, I cannot see any reason for increased latency on the graph other than lack of capacity externally, a hardware problem or a settings issue.

And that is borne out by yours and Bob's graphs.

Tonight there will be no machines or iPhones left with connection, to rule out the hijacking part.


.