"Parsing" a web page for useful information

Hi Guys,

Not sure if anyone can help with this, but will give it a shot

At the place I work we have a "building management system" with all kinds of sensors dotted around the place, which measure temperature, gas/electric consumption, humidity etc. You can login to the thing and get a webpage up with all of these readings listed, but I am only interested in a few of these.

I would like to display some of these readings onto a custom webpage - lets assume my webpage is hosted internally and has access to the building management system.

So, what I assume I need to do is somehow "read in" the contents of the html output page of the building management system, keep the useful bits (ie current temperature) and get rid of everything else, and then put this onto my custom webpage, all automatically! Perhaps only updated whenever the page is refreshed (ie doesnt have to be real time or anything!)

As the system that displays my webpage can not run PHP/IIS etc I would guess any processing would have to be done client side? Java perhaps?

Anyone ever done anything like this before!?

Thanks!

Edited by ste__ (Fri 01-Jan-10 23:42:59)

There isn't one answer to this and the exact situation might dictate the best one. However, if you want to go the route you mentioned, the sourceforge project will help with building an HTML parser.

As this is for business, you could also shell out £20 on a ready made alternative such as this data extractor. Will still need a little tweaking but save a fair amount of time. Believe this example has a free trial if you want to play with it.

your first issue will be logging in and collecting the pages - you might be able to do it with empty frames, and using Javascript to log in and set each frame with desired page details - remember to handle login failures

when writing "automatic" tests you often end up parsing tests using the browser object model to parse for information - if you are lucky they will given the key fields an ID - so things like: alink = document.getElementById(linkid); will allow acess to html entity, which should allow access to values

if you can't run server side script, then client side script is the only option - you can use javascript to make http requests but it's all a bit long-winded without a framework like jquery (which is what i'd use)