|
|
|
I have occasionally mentioned that a new & enhanced version of the stats graphing system is to be released soon.
Programming this in 'C', Burakkucat has developed a Linux/Windows cross-platform stats harvesting program for the HG612 (a single version for use on ADSL & VDSL2 connections) & I have further developed it to munge the obtained data into a format that can be graphed (currently using a revised version of the graphpd.BAT graphing script for Windows users).
I have fully tested the .exe program resulting from compiling the C code with MingW over a couple of months & have found it to be very quick in operation & far more reliable than the scripted versions when coping with various flavours of virus checkers etc. hogging all the PC's resources & causing gaps in the harvested data.
Currently the program is controlled by a batch file called by a simple VBS script that is run every minute via Windows Task Scheduler (or via Cron for Linux users).
I would like to release at least the VDSL2 version very soon. However, I would firstly like it to use an .ini configuration file to make setting the defalut variables for different users' systems as easy as possible & avoid having to use a controlling batch file.
Unfortunately, as I am not a programmer and I have limited available time to research this, I have ground to a halt.
Initially, I imagine a basic ini file would look something like this:-
[login]
IPaddress=192.168.1.1
Username=admin
Password=admin
Shell=sh
[folder_locations]
Drive=C:
Main_folder=\HG612_Stats
Apps=\Apps
Program=\Program
Ongoing_Stats=\Ongoing_Stats
Current_Stats=\Current_Stats
The program would then read the ini file each time it is run from Task Scheduler/Cron & store the values as global variables.
The same ini file could then be used for obtaining & storing snapshot data and ongoing data, along with graphing programs already partially developed.
I/we would be happy to provide the C code "as is" if anyone was willing/able to add in the facility to use an ini configuration file for the default variables.
Initially I would be happy to manually edit the ini file, but at some stage, it would be nice to use a graphical front end where folder locations etc. could be chosen via a point & click method via a GUI rather than the "old fashioned" manual method.
Is anyone up for assisting with this?
It would have to be written in C, for inclusion into the currently developed program(s).
Intentionally, graphs will still only be generated on demand. However this is easily automated via Task Scheduler (if required).
|
|
|
inih (INI Not Invented Here) is a simple .INI file parser written in C. It's only a couple of pages of code, and it was designed to be small and simple, so it's good for embedded systems. It's also more or less compatible with Python's ConfigParser style of .INI files, including RFC 822-style multi-line syntax and name: value entries.
http://code.google.com/p/inih/
Edited by deleted (Wed 26-Dec-12 10:14:47)
|
|
|
Sorry I'm not really in a position to help with the programming of this at the moment, but you should not use the ini file to store program data and results. (Although technically you could, as you suggest). It would be completely non-standard, confusing, and potentially disastrous.
It is what it says, a repository for configuration startup values such as you list. Keep it that way  .
You imply that you are going to run the program every minute on the scheduler, as per your existing batch files. I suggest instead simply loading it at boot time, (which is easy), with an internal timer running the harvesting every 60 seconds.
Ideally under Windows it would be run as a service, but that's another question and not essential.
My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet Extra Fibre (FTTC). Sync ~ 54.0/14.9Mbps @ 600m. - BQM
"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.
|
|
Register (or login) on our website and you will not see this ad.
|
|
|
When I was coding (years ago) - we used WritePrivateProfileString and GetPrivateProfileString windows API calls to read and write the INI files. All the parsing is done for you by the OS.
Viz:
http://msdn.microsoft.com/en-gb/library/windows/desk...
http://msdn.microsoft.com/en-us/library/windows/desk...
See the comments from MS that they suggest now holding such data in the registry.
You could go all XML which a lot of .NET developers perfer. I don't know MingW.
James BT Infinity 2 19/09/2012 - Estimate 44.6/6.5 - Install 52/12 - Actual 46 / 8 Mbps
13 years of broadband - 1999 ntl:(512k/1M)/BTbusiness(2M)/Metronet(2M)/Bulldog(8M/16M)/BE(19M/16M)/BT FTTC(46M)
|
|
|
I reckon using the registry for this type of task is overkill. Also I think C has been wisely chosen for pure speed and portability. .NET would kill the Linux portability and bloat the whole thing for Windows.
My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet Extra Fibre (FTTC). Sync ~ 54.0/14.9Mbps @ 600m. - BQM
"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.
|
|
|
Thanks for that link Batboy.
I have downoladed a couple of versions & a quick glance suggests it is exactly what I have previously tried to find.
As soon as I can find some time, I'll have a dabble with it.
Edited by deleted (Wed 26-Dec-12 14:02:23)
|
|
|
I reckon using the registry for this type of task is overkill. Also I think C has been wisely chosen for pure speed and portability. .NET would kill the Linux portability and bloat the whole thing for Windows.
I happen to agree, but to write/read parameter files using the layout given previously is done for you in the Windows OS, but you have to write code (or use free libraries) for other OSes.
James BT Infinity 2 19/09/2012 - Estimate 44.6/6.5 - Install 52/12 - Actual 46 / 8 Mbps
13 years of broadband - 1999 ntl:(512k/1M)/BTbusiness(2M)/Metronet(2M)/Bulldog(8M/16M)/BE(19M/16M)/BT FTTC(46M)
|
|
|
Sorry I'm not really in a position to help with the programming of this at the moment, but you should not use the ini file to store program data and results. (Although technically you could, as you suggest). It would be completely non-standard, confusing, and potentially disastrous.
It is what it says, a repository for configuration startup values such as you list. Keep it that way .
I may have mislead you about my intentions in my original post.
The ini file will indeed only store very basic details, similar to those I mentioned.
At some stage it may also store details of which graphs to include on a yes or no or a true or false basis
The global variables will only be stored in memory for the duration of the harvesting (now usually taking less than 2 seconds every minute due to the more efficient/reliable logging in method).
All the harvested data is stored in modem_stats.log (or Plink.log for snapshot data), as per the ecripted version.
This ensures a level of compatibility with previously harvested data from the scripted versions for continuity of graphing.
The issue of incorrect error count data (CRC/FEC) being stored/plotted has also been resolved.
The exe version now ONLY obtains its data from xcdslcmd info --stats & no longer bothers to obtain any data from the code that provides the modem's GUI data.
You imply that you are going to run the program every minute on the scheduler, as per your existing batch files. I suggest instead simply loading it at boot time, (which is easy), with an internal timer running the harvesting every 60 seconds.
I did toy with a similar idea when I first put the scripted versions together, but I chose to use Windows Task Scheduler instead, mainly to avoid accidentally closing a running program, thus removeing the potential risk of losing continuity of data.
Ideally under Windows it would be run as a service, but that's another question and not essential.
Maybe an idea for a later date.
My current aim is to try to maintain one set of code that can simply be compiled for Windows or Linux, simply by using different compilers.
|
|
|
I reckon using the registry for this type of task is overkill. Also I think C has been wisely chosen for pure speed and portability. .NET would kill the Linux portability and bloat the whole thing for Windows.
I happen to agree, but to write/read parameter files using the layout given previously is done for you in the Windows OS, but you have to write code (or use free libraries) for other OSes.
For me, with an openly admitted very limited understanding of programming, simplicity is key.
I had/have no desire to write anything to the registry etc. as I have no intention of formally "installing" anything or writing uninstall programs & the like.
Simply ceasing to use the program or deleting the relevant folders and their contents is sufficient (in my mind) to restore a computer to its original state.
Edited by deleted (Wed 26-Dec-12 14:11:29)
|
|
|
For me, with an openly admitted very limited understanding of programming, simplicity is key.
I had/have no desire to write anything to the registry etc. as I have no intention of formally "installing" anything or writing uninstall programs & the like.
Simply ceasing to use the program or deleting the relevant folders and their contents is sufficient (in my mind) to restore a computer to its original state.
Exactly why my colleagues programs still use the INI files and PrivateProfileString APIs. However those won't work on any other OS - so you'll have to either write your own file handling code and parse the attributes or investigate BatBoy's suggestion.
James BT Infinity 2 19/09/2012 - Estimate 44.6/6.5 - Install 52/12 - Actual 46 / 8 Mbps
13 years of broadband - 1999 ntl:(512k/1M)/BTbusiness(2M)/Metronet(2M)/Bulldog(8M/16M)/BE(19M/16M)/BT FTTC(46M)
|
|
|
why is it non standard, his idea is what I was going to have as a feature request.
It is actually standard for progams to use an include line to a central config file.
Whether or not that can be done on windows batch files I dont know tho, it certianly can be done on linux bash scripts.
I prefer ini files to his current scripts (where have to edit many scripts) or using registry storage, this is probably the best way he can do it.
BT Infinity 2 Since Dec 2012 - Estimate 65.9/20 - Attainable peak 110/36 - Current Sync 71/20
Edited by Chrysalis (Wed 26-Dec-12 15:28:41)
|
|
|
why is it non standard, his idea is what I was going to have as a feature request. That part of my post was referring to "The same ini file could then be used for obtaining & storing snapshot data and ongoing data". By definition such are not initialisation values and have no place in the file. He has since clarified his intentions, viz "The ini file will indeed only store very basic details, similar to those I mentioned. At some stage it may also store details of which graphs to include on a yes or no or a true or false basis", which is fine. It is actually standard for progams to use an include line to a central config file. Assuming I understand your word "include" correctly, a "#include" file is included during the early passes of a compilation. It's contents at that time are in effect hard-coded into the exe. That is not the purpose of an ini file nor is it what Bald_Eagle1 envisages. Whether or not that can be done on windows batch files I dont know tho, it certianly can be done on linux bash scripts. Yes, batch files can be executed from within a Windows exe. I prefer ini files to his current scripts (where have to edit many scripts) or using registry storage, this is probably the best way he can do it. There is no disagreement from anyone in the thread about that for this particular application. I was merely unhappy with the thought of transient data, (as opposed to possible system status indicators), being included in an ini file.
My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet Extra Fibre (FTTC). Sync ~ 54.0/14.9Mbps @ 600m. - BQM
"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.
|
|
|
I should be able to help with the parsing side of this (much as I hate C).
For simplicity I'd leave out the use of sections (as in [login]), I don't think they's add anything useful.
I'd make sure # comment lines were allowed, these could be used for informal sections if anyone wanted to.
I wouldn't go near trying to write a GUI in C though.
~~~~~~~~~~
On the graphing side, users of up to date browsers may like to try my more interactive graphing that works from the current log file:
http://programbits.co.uk/modemGraph/plotstats.html
--
Moved (with trepidation) to BT Infinity 2 for upload speed. Happy BE user for several years.
Edited by StephenTodd (Wed 26-Dec-12 17:20:22)
|
|
|
I agree about not using the INI file to store persisted data.
I would however strongly suggest that care is taken in choosing the location of any data file. It should not be written to the application folder. Microsoft has tried over recent years to introduce security into its OS and one of the better things they have done is make it harder for applications to write to anywhere under c:\program files\. of course the user may put this application anywhere they want but as a general rule I think EXEs should never share a folder with data.
Unfortunately working out where to write the file isn't obvious because it's changed so much. Even the API call used to determine the location has been superceded twice. The latest incarnation is SHGetKnownFolderPath(). I would choose FOLDERID_LocalAppData as the base path with the application name appended to complete it.
More choices here: http://blogs.msdn.com/b/patricka/archive/2010/03/18/...
Personally I'd have a wrapper function that forks underneath into separate Linux and Windows modules. The Windows module can then store the data where it wants. The registry is the simplest way frankly and wouldn't be difficult. For the registry just use the bits in []s as the key name and then the name/value pairs are key name and key value. I actually have a C++ class somewhere that does just that. To the outside it looks and acts like an INI file wrapper but it uses the Registry instead.
Edited by Andrue (Wed 26-Dec-12 17:27:39)
|
|
|
ideally it should be in the app folder in my view and it will then make this app easily portable.
I would like this app to not use an installer, stay out of program files completely, rather you just unzip to a dir and run it as is, and edit ini if needs be.
BT Infinity 2 Since Dec 2012 - Estimate 65.9/20 - Attainable peak 110/36 - Current Sync 71/20
|
|
|
The problem being he wishes and needs to cater for different settings for the subnet, and the user needs to be able to set and modify this at will to cater for router changes. The rest of it I see as being relatively fixed so no reason not to be hard-coded.
Also, not being up-to-date with the why's and wherefore's of what you are saying, is there any problem with using the Program Data folder? That's assuming he intends to use a Windows installer to put the exe in a c:\Program Files directory.
I have the feeling he is thinking of dropping the whole lot into a folder in C:\, as per his existing scripts. Which should work fine.
My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet Extra Fibre (FTTC). Sync ~ 54.0/14.9Mbps @ 600m. - BQM
"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.
|
|
|
I have the feeling he is thinking of dropping the whole lot into a folder in C:\, as per his existing scripts. Which should work fine.
Yes, that's about the jist of it.
The default folders (unless users wish to change them) will still be:-
C:\HG612_Modem_Stats (main folder)
C:\HG612_Modem_Stats\Scripts (that's where I also intend to store the compiled exe program)
C:\HG612_Modem_Stats\Apps (where all the graphing programs & other utilities are stored)
C:\HG612_Modem_Stats\Current_Stats
C:\HG612_Modem_Stats\Ongoing_Stats
|
|
|
Also, not being up-to-date with the why's and wherefore's of what you are saying, is there any problem with using the Program Data folder? That's assuming he intends to use a Windows installer to put the exe in a c:\Program Files directory.
It assumes everyone has Administrator rights. If only targeting home users that is fine, but for those of us who run Windows networks in corporates will hate you. Using your own c:\app directory (yuck its not a folder) is the best solution.
%progdir% has system default Access Control Lists on all operating systems, XP's is the weakest, Vista, 7, and 8 are all very good. Even home edition.
James BT Infinity 2 19/09/2012 - Estimate 44.6/6.5 - Install 52/12 - Actual 46 / 8 Mbps
13 years of broadband - 1999 ntl:(512k/1M)/BTbusiness(2M)/Metronet(2M)/Bulldog(8M/16M)/BE(19M/16M)/BT FTTC(46M)
|
|
|
I have the feeling he is thinking of dropping the whole lot into a folder in C:\, as per his existing scripts. Which should work fine.
Yes, that's about the jist of it.
Meh. Well it's up to you and it's not a deal breaker
It's just that as a professional developer I've seen Windows security evolve from 'none' to 'very good if only people used it'. Your solution is simpler but it's a solution from a by-gone era when home computers were just that. These days they are corporate tools and attached to public networks. My instinct is to do the job properly if I can
Edited by Andrue (Wed 26-Dec-12 18:31:46)
|
|
|
Well, I now have the (still undergoing testing) new compiled.exe programs working with an ini file on Windows systems.
It is a very basic file so far, just for testing purposes:-
| Text | 1
23
45
67
89
1011
1213
1415
1617
1819
| [Login]
IP_address = 192.168.1.1Username = admin
Password = adminShell = sh
[Locations]
Drive = C:Main_folder = C:\HG612_Modem_Stats
Scripts_and_Programs = C:\HG612_Modem_Stats\ScriptsApps_folder = C:\HG612_Modem_Stats\Apps
Current_Stats_folder = C:\HG612_Modem_Stats\Current_StatsOngoing_Stats_folder = C:\HG612_Modem_Stats\Ongoing_Stats
[Mode]
Mode=extensive
[Graphing]Auto_graph_Snapshot_data = YES |
More configurable options will be added in due course.
Could anyone who is interested please let me know the details of how you would like to be able to configure the settings?
e.g. Harvest & store the data on a server yet plotting the output graphs on another PC, graph resolution, which graphs to actually plot, colours etc. etc.
EDIT:
Also, how does one get to show a backslash instead of the 'W' character in the code section?
Edited by deleted (Sat 05-Jan-13 16:48:48)
|
|
|
I can't see a "W" only "\" 
Is it much faster that the current one?
|
|
|
Looks promising.
It would be best if the Locations could all work as relative locations.
You may well find they do anyway without any extra work, as long as the exe is launched with the appropriate current directory.
I'm not sure why the Drive value is necessary; unless it is used to set appropriate defaults for the rest.
[Locations]
Main_folder = .
Scripts_and_Programs = Scripts
Apps_folder = Apps
Current_Stats_folder = Current_Stats
Ongoing_Stats_folder = Ongoing_Stats
(some people might prefer ".\Scripts" etc for clarity; either should work)
Personally, I'm not worried about automatic graphing.
Robuster automatic scripting will be great; but I'd rather do graphing on demand.
--
Moved (with trepidation) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
|
I disagree. The explicit paths are fine.
|
|
|
I can't see a "W" only "\"
Ah well, it must be some setting or other at my end.
I also see it on some other websites.
Is it much faster that the current one?
About 7 seconds to gather the stats & produce the snapshot graphs as opposed to the 30 seconds via the scripted version (using a half reasonable spec Windows 7 machine).
Edited by deleted (Sat 05-Jan-13 17:48:07)
|
|
|
Note I said ''It would be best if the Locations could all work as relative locations."
I certainly agree explicit paths should also be allowed.
Relative paths make it much easier to install the application where you like, and to relocate it later if required;
no need to change a default .ini file in either case.
--
Moved (with trepidation) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
I would rather have it working now before it gets bogged-down with nice-to-haves 
|
|
|
Looks promising.
It would be best if the Locations could all work as relative locations.
You may well find they do anyway without any extra work, as long as the exe is launched with the appropriate current directory.
I'm not sure why the Drive value is necessary; unless it is used to set appropriate defaults for the rest.
[Locations]
Main_folder = .
Scripts_and_Programs = Scripts
Apps_folder = Apps
Current_Stats_folder = Current_Stats
Ongoing_Stats_folder = Ongoing_Stats
(some people might prefer ".\Scripts" etc for clarity; either should work)
I agree about the Drive (at the moment).
That value isn't actually used. It may be at some other time though, so I included it just in case.
I had to include the full paths for users that may wish to split up the various folders across various drives/other folders elsewhere (maybe some on a server & some on a local PC or a memory stick/SSD drive for whatever reasons.
Personally, I'm not worried about automatic graphing.
Robuster automatic scripting will be great; but I'd rather do graphing on demand.
Perhaps I could/should change its title.
I only ever intended to graph on demand.
It can be completely automated by running a simple batch file if required.
What it is used for is to turn graphing on or off when obtaining the snapshot Plink log data.
By default, the scripted version automatically went on to generate the graphs.
That option could be edited directly in the script (TestStats2.BAT), but it is easier/safer to do in the ini file.
e.g. I remotely monitor a couple of remote connections where there is no need to graph the Plink logs each time they are harvested on a schedule.
I have the logs emailed to me (again on a schedule) & I graph them at my end by dragging & dropping the log onto the older script or new program icon.
|
|
|
I would rather have it working now before it gets bogged-down with nice-to-haves
Try it out if you like.
. Test Snapshot graphing programs
All 3 files have to reside in the same folder (for now).
The output Plink logs & graphs should end up in time stamped subfolders (as per usual) of whatever folder you specify in the HG612_stats.ini file for the Current_Stats.
You have to create that new Current_Stas folder first though.
I might get round to automating that at some time.
Note: NO SPACES ALLOWED IN FOLDER OR FILE NAMES & do NOT change the name of the program
Bear in mind that this is a test version, so retain your scrited versions for now
Edited by deleted (Sat 05-Jan-13 18:42:15)
|
|
|
|
The HG612_current_stats worked well, but the Graph6 just seemed to loop. Impressive though, thanks.
Is the stats collector also going to be a .exe?
|
|
|
Works for me, including Graph6.exe
after I had copied the Apps directory from my more permament installation
and used absolute (not relative) paths in the ini file for Apps
Looking promising.
--
Moved (with trepidation) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
|
So how did you start Graph6 and what folder was it in?
|
|
|
The HG612_current_stats worked well, but the Graph6 just seemed to loop.
I realised that & have now added this message for when anyone tries to run GRAPH6.exe directly:-
| Text | 1
23
45
67
89
1011
1213
1415
1617
1819
2021
22 | *********************************************************
* ** *
* GRAPH6.exe should not be run directly. ** *
* A valid log file name needs to be specified. ** *
* This is done automatically when running ** HG612_current_stats.exe with the option to *
* go on to generate the graphs switched ON ** *
* Alternatively, it can be specified on ** the command line as a parameter *
* or by dragging and dropping a valid log file ** onto the GRAPH6.exe icon *
* ** *
* ** Now press a key to exit the program.... *
* ** *
********************************************************* |
I haven't uploaded it to the link yet though as I am still tweaking here & there.
Is the stats collector also going to be a .exe?
Yes, it is currently being tested as we write.
Edited by deleted (Sun 06-Jan-13 01:20:09)
|
|
|
Works for me, including Graph6.exe
after I had copied the Apps directory from my more permament installation
and used absolute (not relative) paths in the ini file for Apps
Hmmm.
You should have just been able to point to the existing Apps folder by editing the ini file to state where the apps were stored.
Did you try that?
|
|
|
Some people have the operating system installed as E: i.e. someone with dual boot operating systems - so something to be aware of.
|
|
|
You should have just been able to point to the existing Apps folder by editing the ini file to state where the apps were stored.
I thought of that after doing the copying. Just tested it out, and yes it does work that way too.
I have an oddity that all the files create by graph6.exe sometimes have wrong timestamps.
The ones I just did on 6 Jan at 10:11 GMT are reporting 5 Jan 22:28.
Some seem to come up with 'real' log creation time, some with a timestamp to match the .log file (sensible), and some with some other apparantly arbitrary time.
So how did you start Graph6 and what folder was it in?
BatBoy: I hope Bald_Eagle's other reply has sorted your looping issue.
But for info: I copied into folder "D:\Temp\modem"
I did have to create the folder D:\Temp\modem\Current_Stats explicitly;
otherwise HG612_current_stats.exe failed to operate correctly (without error message).
HG612_current_stats.exe did create appropriate subdirectories when run.
--
Moved (with trepidation) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
You should have just been able to point to the existing Apps folder by editing the ini file to state where the apps were stored.
I thought of that after doing the copying. Just tested it out, and yes it does work that way too.
Good That's how it is intended to work i.e. the folders don't all have to be subfolders of the main folder.
I have an oddity that all the files create by graph6.exe sometimes have wrong timestamps.
The ones I just did on 6 Jan at 10:11 GMT are reporting 5 Jan 22:28.
Some seem to come up with 'real' log creation time, some with a timestamp to match the .log file (sensible), and some with some other apparantly arbitrary time.
Are you 100% sure you are looking in the new subfolder after generating the graphs?
The timestamp should just come from the PC's system time.
It works correctly every time for me.
Daft question:-
Is the PC where you get incorrect timestamps actually running at the correct system time?
BatBoy: I hope Bald_Eagle's other reply has sorted your looping issue.
But for info: I copied into folder "D:\Temp\modem"
I did have to create the folder D:\Temp\modem\Current_Stats explicitly;
otherwise HG612_current_stats.exe failed to operate correctly (without error message).
HG612_current_stats.exe did create appropriate subdirectories when run.
There is very little error checking built in so far in these test files.
I'll be adding some as time goes on, especially one to check that obtaining snapshot data doesn't clash with the harvesting of ongoing data.
When that happens, one of the programs fails to harvest the data for that instance.
It works straightaway afterward though.
|
|
|
Note I said ''It would be best if the Locations could all work as relative locations."
I certainly agree explicit paths should also be allowed.
Relative paths make it much easier to install the application where you like, and to relocate it later if required;
no need to change a default .ini file in either case. Yeah. However I'd make them relative to something fixed. Either the location of the EXE or better yet somewhere in the official application data or user data folders. Absolute paths can be a bit of a pain sometimes.
Edited by Andrue (Sun 06-Jan-13 10:53:03)
|
|
|
BatBoy: I hope Bald_Eagle's other reply has sorted your looping issue.
But for info: I copied into folder "D:\Temp\modem"
I did have to create the folder D:\Temp\modem\Current_Stats explicitly;
otherwise HG612_current_stats.exe failed to operate correctly (without error message).
HG612_current_stats.exe did create appropriate subdirectories when run. Unfortunately not. I don't have a problem with HG612_current_stats.exe, that works fine.
It's Graph6 that loops. I left it running overnight, but it was still running just now.
|
|
|
Unfortunately not. I don't have a problem with HG612_current_stats.exe, that works fine.
It's Graph6 that loops. I left it running overnight, but it was still running just now.
I wonder if you have a few incomplete instances of GRAPG6.exe and/or CMD.exe still running.
Task Manager should show what's running (ensure you look at all users).
If you have any spurious ones, end the tasks manually, or try a PC reboot to clear everything out.
BTW, what Windows version are you running?
I have tested on Windows 7 Home Premium & XP so far.
|
|
|
Yeah. However I'd make them relative to something fixed. Either the location of the EXE or better yet somewhere in the official application data or user data folders. Absolute paths can be a bit of a pain sometimes.
It would be a lot easier if someone could write a point & click front end for confiiguring & writing the settings to the ini file as a standalone application (preferably in plain old 'C' so I could have half a chance of understanding it & developing it for future configuration options).
|
|
|
FWIW, I have uploaded some slightly amended test programs, here:-
Test programs 06/01/2013
They are still test versions & ALL constructive feedback is much welcomed.
If using the ongoing harvesting program (HG612_stats.exe), it may be wiser to save your existing modem_stats.log & allow the program to start off a new one.
It still needs to be run via Task Scheduler & if it is visible when run, edit a copy of the original getstats.VBS to point straight to HG612_stats.exe & edit Task Schedulker to point to the new .VBS file.
It would also be wise to have the new .VBS file in the same folder as the new programs.
P.S. DON'T ever change the program names as everything hinges on them remaining as they start off.
|
|
|
Thanks. I won't have a chance to look at those today, but look forward to trying them tomorrow.
--
Moved (with trepidation) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
Hi Paul, I've not had chance to take a look yet either, it's the wife's birthday weekend, off out for dinner shortly.
|
|
|
Hi Paul, I've not had chance to take a look yet either, it's the wife's birthday weekend, off out for dinner shortly.
& she comes first?
You obviously haven't spent as much time in the doghouse as I have then
|
|
|
You obviously haven't spent as much time in the doghouse as I have then
Beleive me, plenty of years in the dog house I was just reading your post around the mother-in-laws on my phone and the wife said 'what you grinning at!'
On the subject of the stats collection did you manage to implement my idea for losing the first line (erroneous line) of stats that were recorded after a break?
|
|
|
Should HG612_current_stats.exe generate current stats graphs as teststats2.bat does?
I have the following set, all I get is the Plink_20130106-1531.log, no graphs
[Graphing]
Auto_graph_Snapshot_data = YES
|
|
|
On the subject of the stats collection did you manage to implement my idea for losing the first line (erroneous line) of stats that were recorded after a break?
Not yet.
Thanks for reminding me.
You may have to remind me to look at that again as I have a lot going on just at the moment.
|
|
|
Should HG612_current_stats.exe generate current stats graphs as teststats2.bat does?
I have the following set, all I get is the Plink_20130106-1531.log, no graphs
[Graphing]
Auto_graph_Snapshot_data = YES
Yes, it should.
It works at this end.
Assuming you are using the latest versions that I uploaded this morning, I can only assume that a folder setting in the HG612_stats.ini file isn't 100% right.
Are you doing this over a network, with the drives/folders mapped as drive letters accordingly?
|
|
|
I've switch over to the new exe's, but I've left TestStats2.bat to graph the current logs pending your reply to this post.
Should HG612_current_stats.exe generate current stats graphs as teststats2.bat does?
I have the following set, all I get is the Plink_20130106-1531.log, no graphs
[Graphing]
Auto_graph_Snapshot_data = YES
I'm still using my original modem log, I use NotePad++ to edit or open it, so much quicker than note pad.
|
|
|
We posted at the same time.
I'm doing this all on the server, so I have the following ini file - using the defualt folders. Yes I downloaded the latest versions this afternoon.
[Login]
IP_address = 192.168.0.3
Username = admin
Password = admin
Shell = sh
[Locations]
Drive = C:
Main_folder = C:\HG612_Modem_Stats
Scripts_and_Programs = C:\HG612_Modem_Stats\Scripts
Apps_folder = C:\HG612_Modem_Stats\Apps
Current_Stats_folder = C:\HG612_Modem_Stats\Current_Stats
Ongoing_Stats_folder = C:\HG612_Modem_Stats\Ongoing_Stats
[Mode]
Mode=extensive
[Graphing]
Auto_graph_Snapshot_data = YES
I also seem to be getting isrunning files for every minute
I've put the new files (the uploaded ones) in the C:\HG612_Modem_Stats directory/folder
Edited by R0NSKI (Sun 06-Jan-13 16:06:42)
|
|
|
... it's the wife's birthday weekend, off out for dinner shortly. Who with, and what's your wife going to say?
My broadband basic info/help site - www.robertos.me.uk | Domains,website and mail hosting - Tsohost.
Connection - Plusnet Extra Fibre (FTTC). Sync ~ 54.0/14.9Mbps @ 600m. - BQM
"Where talent is a dwarf, self-esteem is a giant." - Jean-Antoine Petit-Senn.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allergy information: This post was manufactured in an environment where nuts are present. It may include traces of understatement, litotes and humour.
|
|
|
Who with, and what's your wife going to say?
shhhhhh, don't tell her!
|
|
|
Thank goodness that I only write C-code for Unix / Linux kernel using OSs and not BGW! 
100% Linux and, previously, Unix.
|
|
|
We posted at the same time.
I'm doing this all on the server, so I have the following ini file - using the defualt folders. Yes I downloaded the latest versions this afternoon.
[Login]
IP_address = 192.168.0.3
Username = admin
Password = admin
Shell = sh
[Locations]
Drive = C:
Main_folder = C:\HG612_Modem_Stats
Scripts_and_Programs = C:\HG612_Modem_Stats\Scripts
Apps_folder = C:\HG612_Modem_Stats\Apps
Current_Stats_folder = C:\HG612_Modem_Stats\Current_Stats
Ongoing_Stats_folder = C:\HG612_Modem_Stats\Ongoing_Stats
[Mode]
Mode=extensive
[Graphing]
Auto_graph_Snapshot_data = YES
I also seem to be getting isrunning files for every minute
That's the start of an attempt (not yet implemented) to determine if HG612_stats.exe is still running at the same time as say, HG612_current_stats.exe is started.
The modem doesn't seem to like 2 simultaneous login attempts.
I've put the new files (the uploaded ones) in the C:\HG612_Modem_Stats directory/folder
I wonder if that could be the problem?
You have the scripts & programs set as "Scripts_and_Programs = C:\HG612_Modem_Stats\Scripts" in the ini file, yet the programs are currently stored one level up at "Scripts_and_Programs = C:\HG612_Modem_Stats"
|
|
|
Thank goodness that I only write C-code for Unix / Linux kernel using OSs and not BGW!
Get used to it!
Some poor soul (who shall remain nameless until he exposes himself - not in the biblical sense, of course) might have to read my very, very messy code some day & add a load of extra code to ensure a level of compatibility for that strange band of penguin huggers out there
|
|
|
|
isrunning sounds like a potential nightmare.
|
|
|
I've shifted the programs including the ini file to the scripts folder, but it's still leaving the isrunning file and it doesn't generate graphs when I run current_stats.
Just thinking the only time you really need to to run current_stats is when the modem re-synced as thats the only time they will change iirc. Would it not be possible for the get stats exe to run current_stats when it see's the up time has reset, thus automatically logging those stats after every re sync?
Edited by R0NSKI (Sun 06-Jan-13 19:32:12)
|
|
|
Get used to it! Hiss, spit and swipe (with claws exposed). Some poor soul (who shall remain nameless until he exposes himself - not in the biblical sense, of course) might have to read my very, very messy code some day & add a load of extra code to ensure a level of compatibility for that strange band of penguin huggers out there Meow!
100% Linux and, previously, Unix.
|
|
|
isrunning sounds like a potential nightmare.
Why's that?
If you have any other suggestions (in 'C') for checking if a given program is still running, please speak up as I don't know how to do that.
|
|
|
|
I wouldn't do it, it just looks too hard for little benefit. For instance, what if the isrunning file doesn't get deleted?
|
|
|
I've shifted the programs including the ini file to the scripts folder, but it's still leaving the isrunning file and it doesn't generate graphs when I run current_stats.
Do you mean there are multiple instances of ISRUNNING clocking up?
Are there any clues in the error log?
It should be found in the same folder as modem_stats.log.
It should look something like this, with the ISRUNNING file being deleted at the end of HG612_stats.exe:-
| Text | 1
23
45
67
89
1011
1213
1415
1617
1819
2021
2223
2425
2627
2829
3031
3233
3435
3637
3839
4041
4243
4445
46 | 06/01/2013 21:06:00.040 - [ IN HG612_stats.EXE ] - Start of 1 minute sampling
06/01/2013 21:06:00.040 - Temp File ISRUNNING-210600-040 was created06/01/2013 21:06:00.041 - malloc() O.K.
06/01/2013 21:06:00.041 - Initializing Winsock06/01/2013 21:06:00.043 - WSAStartup() SUCCESSFUL!!!
06/01/2013 21:06:00.043 - Starting getaddrinfo()06/01/2013 21:06:00.043 - getaddrinfo() O.K.
06/01/2013 21:06:00.043 - Starting socket()06/01/2013 21:06:00.047 - socket() O.K.
06/01/2013 21:06:00.047 - Starting connect()06/01/2013 21:06:00.048 - connect() O.K.
06/01/2013 21:06:00.850 - About to reply(xdslcmd info --stats)06/01/2013 21:06:00.850 - reply(xdslcmd info --stats) O.K.
06/01/2013 21:06:01.062 - get_data() O.K.06/01/2013 21:06:01.062 - About to determine the xDSL mode
06/01/2013 21:06:01.062 - About to parse_stats()06/01/2013 21:06:01.063 - parse_stats completed
06/01/2013 21:06:01.063 - VDSL2 mode detected so about to reply(xdslcmd info --pbParams)06/01/2013 21:06:01.063 - reply(xdslcmd info --pbParams) O.K.
06/01/2013 21:06:01.271 - get_data() O.K.06/01/2013 21:06:01.271 - About to parse_pbParams_data()
06/01/2013 21:06:01.271 - parse_pbParams_data() completed06/01/2013 21:06:01.271 - About to exit Busybox
06/01/2013 21:06:01.271 - Busybox exited06/01/2013 21:06:01.471 - About to exit ATP
06/01/2013 21:06:01.472 - ATP exited06/01/2013 21:06:01.477 - About to get_log_data()
06/01/2013 21:06:01.933 - About to calc_ES_delta()06/01/2013 21:06:01.934 - About to calc_RSCorr_delta()
06/01/2013 21:06:01.935 - About to calc_RSUnCorr_delta()06/01/2013 21:06:01.935 - About to calc_RS_delta()
06/01/2013 21:06:01.936 - About to calc_OHF_delta()06/01/2013 21:06:01.936 - About to calc_OHFErr_delta()
06/01/2013 21:06:01.937 - About to calc_HEC_delta()06/01/2013 21:06:01.937 - About to calc_CRC_delta()
06/01/2013 21:06:01.938 - About to calc_FEC_delta()06/01/2013 21:06:01.938 - Appending data to MSLOGFILE.log
06/01/2013 21:06:01.938 - Closing MSLOGFILE.log06/01/2013 21:06:01.939 - END of 1 minute sampling
06/01/2013 21:06:01.939 - About to close(sockfd)06/01/2013 21:06:01.939 - About to freeadddrinfo(res)
06/01/2013 21:06:01.939 - About to WSACleanup()06/01/2013 21:06:01.939 - About to free(txbuf)
06/01/2013 21:06:01.939 - About to free(rxbuf)06/01/2013 21:06:01.939 - End of HG612_stats.exe program, closing ERROR.LOG
06/01/2013 21:06:01.939 - ISRUNNING-210600-040 DELETED |
Just thinking the only time you really need to to run current_stats is when the modem re-synced as thats the only time they will change iirc.
Not quite.
Hlog & QLN remain as static data until the next resync.
Bitloading & SNR are dynamic throughout the duration of a connection.
I'll send you a version with progress output echoed to screen that might give us a clue as to where things are getting stuck.
I can't recall now, but are you using an OS something like Windows Server 2012?
Would it not be possible for the get stats exe to run current_stats when it see's the up time has reset, thus automatically logging those stats after every re sync?
That's a thought that I might try to implement.
|
|
|
I wouldn't do it, it just looks too hard for little benefit. For instance, what if the isrunning file doesn't get deleted?
This is from the error log:-
06/01/2013 21:06:00.040 - Temp File ISRUNNING-210600-040 was created
and another entry when it is deleted (assuming the program completes properl:-
06/01/2013 21:06:01.939 - ISRUNNING-210600-040 DELETED
Each time ISRUNNING is created it adds a time stamp to its filename.
That COULD be useful for if the program crashed BEFORE it had chance to build up a complete error log & as a novice in this programming malarky, I thought it to be one way of determining things.
I used "tasklist" in the later scripted versions, but I don't know its equivalent in 'C'
|
|
|
|
Well, I guess you could interact with Windows using some win32 calls but that wouldn't be portable and, selfishly I'm interested in lean and mean code which is quick to execute.
|
|
|
If you have any other suggestions (in 'C') for checking if a given program is still running, please speak up as I don't know how to do that.
I'd create a file but then use file locking to check if the program is already running. I.e. create a file if it doesn't exist yet, then try to lock it.
The advantage is that if the program crashes the OS will remove the lock automatically.
In Unix/Linux/BSD/MacosX this would mean calling flock(). Figuring out what the Windows equivalent is will be left as an exercise to the reader because I have never done any Windows programming and I have no plan to start in the next 1000 years.
|
|
|
http://en.wikipedia.org/wiki/File_locking
Microsoft Windows uses three distinct mechanisms to manage access to shared files:
using share access controls that allow applications to specify whole-file access-sharing for read, write, or delete
using byte-range locks to arbitrate read and write access to regions within a single file
by Windows file systems disallowing executing files from being opened for write or delete access
etc..
Edited by deleted (Sun 06-Jan-13 22:12:14)
|
|
|
bitloading can change during a sync (bitswapping).
are we sure everything else never updates?
BT Infinity 2 Since Dec 2012 - Estimate 65.9/20 - Attainable peak 110/36 - Current Sync 71/20
|
|
|
please try not to overcomplicate the app.
I was expecting you to just make a master ini file to set paths and then distribute the app portable.
the current older app batch file used to enable the task schedule was neat, and I dont see a reason for you to change the batch script system for that.
BT Infinity 2 Since Dec 2012 - Estimate 65.9/20 - Attainable peak 110/36 - Current Sync 71/20
|
|
|
Yes, multiple copies of isrunning stacking up - I added a batch file to delete them on the hour, well 30 seconds past so it doesn't delete the current one
Doesn't seem to be a log file, could be just me having just got out of bed.
I'll have to leave testing the file you emailed me until tonight, data should be uploaded to dropbox just after 7am
Edit. Forgot to add it's Windows Home Server 2011, which is based on Server 2008 IIRC
Edit again.Also forgot to change the path for the modem log from the exe folder, so correct graphs have not been generated - must leave for work now.
Edited by R0NSKI (Mon 07-Jan-13 07:16:25)
|
|
|
please try not to overcomplicate the app.
I was expecting you to just make a master ini file to set paths and then distribute the app portable.
I'm trying hard to keep it as simple as possible for all users (including me!).
Some users did need to manually edit the batch file scripts to suit their personal preferences such as removing the "Press a key to continue" type stuff in order to automatically schedule the generation of logs and graphs or just the logs etc. etc.
The new programs operate quicker, but any individual configuration requirements now have to be set in the ini file (unless manually typing in potentially very long lines at a command prompt etc.).
For many users, the default settings will suffice, but I have had to start allowing for users to configure things somehow or other.
This is a work in progress, so it may take a while for it to suit everyone as further configurable options may/will need adding accordingly & other tweaks may be needed to suit the majority of the different Windows systems being used.
the current older app batch file used to enable the task schedule was neat, and I dont see a reason for you to change the batch script system for that.
I intend to keep that, or at least something very similar.
Edited by deleted (Mon 07-Jan-13 10:59:48)
|
|
|
All looking good for me. Much quicker and cleaner than the old code.
I had to change the .ini file around a bit to get things to fit with my experimental setup.
I assume the Ongoing_Stats\modem_stats.log file is exactly compatible with the previous version?
It appears to be from my quick tests.
That's the start of an attempt (not yet implemented) to determine if HG612_stats.exe is still running at the same time as say, HG612_current_stats.exe is started.
The modem doesn't seem to like 2 simultaneous login attempts.
By convention, such a file should be called '.lock'.
The 'isrunning' files are being cleaned up fine for me.
--
Moved (with trepidation) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
I didn't get round to using HG612_stats.exe for my regular monitoring.
Even though it was working before, just checking it again yesterday and it crashes. (Version from 6 Jan).
I'm not aware of any changes I have made that would affect it.
Has anyone else seen issues?
Maybe one of the router numbers reports has become larger and is overflowing some int or long value?
Is there an update on the say?
--
Moved (with trepidation turned relief) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
I've been testing some updated versions and I did have one that started crashing, the actual program was identical to the one I was running on my server, even checked the file hash, which was the same.
I couldn't work out what was causing it, so in the end I backed up my log file deleted the directories, and created new ones with a complete copy from my server. It then worked without a problem, so may be worth you refeshing you're installation.
|
|
|
Just to let you know, as loosely mentioned by RONSKI, a new & updated version of the whole package is on its way - possibly within the next 2-3 weeks.
It is all now a lot quicker, more robust & easier to configure for individual user requirements & scheduled logging/graphing via the use of an ini file editor with its own GUI (many thanks to RONSKI for its creation/development - far too modest to have mentioned it himself)
|
|
|
Excellent, thank you both.
--
Moved (with trepidation turned relief) to BT Infinity 2 for upload speed. Happy BE user for several years.
|
|
|
Thanks Bald_ Eagle, as he says things are progressing well, and for those that still like to do things themselves and not rely on or use the GUI the programs can still be setup and run without using it.
|
Pages in this thread: 
Print Thread
