Bandwidth monitoring on DG834Gv4 (DGTeam)

I've worked out how to get the amount of traffic being sent or recieved on the ADSL port on this router. I use RouterStats stats tab. Basically you times the MTU by the number of packets.

Easy enough.

What I want to do though is have an automated script that will pull this from the router and throw it into something useful and meaningful like how much data was sent and received that day. I did some Googling and found a VBS script that suppsedly did it but it gave me an error in Win7 and a blank log file in XP (It uses something called W3Sockets dll or somesuch).

Please do not suggest TBB Meter or other software meters that run on the LAN side unless they:-
1. Monitor all PC's.
2. Monitor IPv6 tunnel traffic (TBB Meter currently doesn't without disabling it's internet monitoring mode. See caveat below).
3. Does not measure internal LAN file traffic (Sending files etc). If I want TBB Meter to measure internet traffic only this works but it does not measure IPv6 tunnel traffic. If I set it to measure all traffic it will work but that is no good since it measures file transfers etc. I just want to measure WAN traffic.

The only other option would be to buy a better router with dd-wrt but I don't see why I should have to when the data is there to be had. It's just a bit awkward making it into something useful. A script that can pull the packets sent and received from the Netgear IP, times it by the MTU and dump it into a log would be ideal as I could then set a schedule to run it.

Has anyone had experience with trying to do this?

Edited by deleted (Sun 10-Apr-11 08:01:32)

Why not grab the stats off the web or telnet inerface and graph them using Cacti for example?

I do something similar for the DG834Gv3, but on Linux.

Every half hour I run a cron script which enables debugs on the Netgear, telnet's to it, and pulls the bandwidth figures.

On the v3 the upstream and downstream byte figures are given so you don't have to muck about with MTU or packet sizes.

The cron entry is

15,45 *  *   *   *     /home/common/bin/get-router-stats

The basic script is

#!/bin/bash
#
T1=`mktemp`
T2=`mktemp`
U=/home/common/bin/router-stats/$HOSTNAME.log
#
curl --basic --user user_name:password --url http://router/setup.cgi?todo=debug \
   &>/dev/null
#
/home/common/bin/router-stats/telnet.script >$T1
date +%F\ %R:%S >$T2
grep 'Payload' $T1 >>$T2
awk -v v=1048576 \
    '{if (index($0, "Payload")) printf " %5.1fMB %6.1fMB\n",$4/v,$7/v; \
      else printf "%s", $0}' $T2 >>$U
rm $T1 $T2

The telnet script is

#!/usr/bin/expect -f
spawn telnet "router"
expect "# "
send "cat /proc/avalanche/avsar_modem_stats\r"
expect "# "
send "exit\r"

Of probably no use to you, but perhaps useful for someone with a Linux box.

Edited by deleted (Sun 10-Apr-11 09:11:26)

Thought DG Team had enabled SNMP. There must be something available from one of the interfaces.

Don't know about the DG Team firmware but SNMP with the standard firmware is a bit of a pain.

You have to enable it on a hidden page (router/snmp.htm) and alter the firewall rules to allow outbound snmp-traps and snmp services.

Then you can do a snmp walk (version 1 or 2c) to get the oids (total of 220), then I guess you'd have to sum all the various protocol traffic (tcp, udp, etc.). I used to collect stats that way but in the end decided it was more fuss than just telnet'ing to the device.

Edited by deleted (Sun 10-Apr-11 13:10:58)

There is SNMP on the DGTeam firmware and I looked into it but it on;y tells me what I can already get from the stats page anyhow and it's harder to dig down through the interfaces. That script would be nice if I ran Linux but I don't. Oh well. Thanks anyhow.

Hooray! PRTG does what I need and is free for 10 sensors. I am still trying to figure out how it how to change some of the readings though. Using kilobits and kilobytes kinda sucks as I want it to tell me how much in megabytes and gigabytes instead.

Select sensor, edit, sensor settings, advanced.

Am I missing something here, or have I mis-understood something?

Do RouterStats and PRTG distinguish between packets with and without data?

I'm not sure but aren't MRTG/PRTG daemons in the sense they happily run in the background and present an on-demand web interface, whereas RouterStats is a normal foreground application requiring a "window".

So MRTG/PRTG will run without anyone being logged in.

They both interrogate the router/modem to get the stats (RouterStats telnet, MRTG/PRTG snmp I think).

eta: my DG834Gv3 reports payload as well as packets.

Edited by deleted (Mon 11-Apr-11 20:10:20)

MRTG/PRTG run as a service and don't require any user intervention, yes they use snmp to gather the info. Though MRTG has been tweaked by various people to do many things since it's open source.

I think PRTG knows packets vs data and works it out for you. At least I hope that is the cvase. I had to select the root settings to change the units from kilobytes to megabytes. I've even set PRTG as a cluster node so if one PC is down the other carries on measuring.

EDIT: Hmm. Now I am not so sure of it's accuracy. I did the 10 megabyte download file and it came out as 11 megabytes in the PRTG readout. If I set it to kilobytes instead it shows as 11,151 which when converted to megabytes using base 2 is 10.889 megabytes or close to.

Computer calculations always do my head in.

EDIT2: I just did some internal testing sending a 100MB file across my LAN. I used an alternate tool from PRTG justt o compare. I am genuinely surprised to find that there is a lot of overhead when transfering data. That 100MB file actually sends as 108MB worth of data. When I downloaded that same 100MB from TBB and checked PRTG it gave a similar amount of around 109MB. Not sure why 10MB would give 12MB though. Perhaps it's all the other traffic like cacheing from the browser etc. I guess the only real way to test is do a clean startup (aart from PRTG) with MSCONFIG, make sure it's the only PC running and use an FTP client as opposed to a browser and then check PRTG's readings.

EDIT3: Hmm. Looking at eth0 instead of atm0 seems to be closer to what it should be 10MB out (Sent to me connected PC so out rather than in). If my ISP were monitoring though I am guessing they would measure total traffic including overhead.

Edited by deleted (Tue 12-Apr-11 08:13:30)

PRTG is pretty accurate, have it as a backup / dev system at work and it's fine but that's with Cisco, HP, Juniper etc. The issue I've found is the domestic hardware sometimes struggles, most likely lack of memory, processing power etc.

i have set up WAN only monitoring on my dg834gt running dgteam fw , in two different ways.
Easiest is to enable snmp on 834gt and then run an hourly .bat file such as ...


cd \MyDocs\SNMP
set MIBDIRS=.
snmpget -v2c -cpublic -Ovq 192.168.0.1 ifInOctets.9 ifOutOctets.9 sysUpTime.0>snmp.txt ( part of previous line)
dir snmp.txt>>snmp.txt
copy snmp.txt s.txt
rs6
copy /a /y snmp.log+snmp.txt

to ouput download, upload and system uptime data to a text file
You need to download Net-SNMP package to match the snmp in the dgteam fw, but you end up using just a few OIDs and the SNMPGET command. Ouput looks like...


28/05/11 20:00 3561587221 104887674 4:13:12:24.09
28/05/11 21:00 3673604471 107494517 4:14:12:27.06
28/05/11 22:00 3673847374 107936745 4:15:12:30.93
28/05/11 23:00 4034678501 114202544 4:16:12:31.03
29/05/11 0:00 357884479 125108556 4:17:12:34.09
29/05/11 1:00 796007118 131161612 4:18:12:36.64
29/05/11 2:00 846943450 131875039 4:19:12:39.13
29/05/11 3:00 847620508 131912958 4:20:12:42.18
29/05/11 4:00 847629768 131923827 4:21:12:43.84
29/05/11 5:00 847635092 131930023 4:22:12:46.82
29/05/11 6:00 847642184 131938435 4:23:12:49.31
29/05/11 7:00 847648291 131944932 5:0:12:52.09
29/05/11 8:00 847653512 131950252 5:1:12:54.11
29/05/11 9:00 872257702 136593085 5:2:12:58.46
29/05/11 10:00 877429816 137413252 5:3:13:01.79
29/05/11 11:00 877575726 137622314 5:4:13:01.99
29/05/11 12:00 877743816 137909460 5:5:13:04.82
29/05/11 13:00 919680523 158399048 5:6:13:08.59
29/05/11 14:00 927177756 166107775 5:7:13:11.86
29/05/11 15:00 927242926 166274636 5:8:13:13.91
29/05/11 16:00 929049847 166507762 5:9:13:16.07

NB this is bytes, not packets

Second lower tech solution is to create a linux command file and save it into the router NVRAM in the tmp directory. Easiest way is to use custom setup tab in gui. This file will happily survive reboots . Content of file is

while :
do
cd /tmp
rm tmpp
cat tmpp>tmpp
a=$(ls -lg tmpp|cut -c 35-47)
b=$(cat wan_ipaddr)
c=$(ifconfig ppp0|sed -n /bytes/p)
d=$(cat /proc/uptime)
echo \'$a\',\'$b\',$d>tmpp
echo $c>>tmpp
ftpput 192.168.0.2 tmpp.txt tmpp
sleep 3600
done


Output of command file is ...

May 22 15:29 78.146.76.166 1d:22h RX 1789198672 (1.6 GiB) TX 149081639 (142.1 MiB)
May 22 18:33 78.146.63.242 0d:00h RX 39243 (38.3 KiB) TX 53135 (51.8 KiB)
May 24 04:35 78.146.63.242 1d:10h RX 253694048 (241.9 MiB) TX 14987929 (14.2 MiB)
May 24 07:00 78.146.73.82 1d:12h RX 2160 (2.1 KiB) TX 1237 (1.2 KiB)
May 28 23:15 78.146.73.82 6d:04h RX 4057159244 (3.7 GiB) TX 114546845 (109.2 MiB)
May 29 00:14 78.146.73.82 6d:05h RX 368949740 (351.8 MiB) TX 125274544 (119.4 MiB)
May 29 08:16 78.146.73.82 6d:13h RX 847653864 (808.3 MiB) TX 131950480 (125.8 MiB)
May 31 23:20 78.146.73.82 9d:04h RX 1454655508 (1.3 GiB) TX 202665882 (193.2 MiB)
May 31 23:22 78.146.73.82 9d:04h RX 1454663104 (1.3 GiB) TX 202673655 (193.2 MiB)
Jun 1 00:20 78.146.73.82 9d:05h RX 1456623153 (1.3 GiB) TX 203063851 (193.6 MiB)
Jun 1 00:22 78.146.73.82 9d:05h RX 1456624330 (1.3 GiB) TX 203066116 (193.6 MiB)
You don't need anything as complicated as mrtg/prtg.

The WAN bytes measured by either of these methods match exactly what TalkTalk say I have downloaded each month, so I know I'm not exceeding my 40GB limit.
The router only has 32bit counters , meaning it can only count up to 4GB before it rolls over back to zero. I've written a routine to cope with the output having such a saw-tooth format , to give me an overall total per month/period

Thankfully Netgear saw sense and put a data traffic counter inside the GUI of the DGN2200.