Kitz Forum
Chat => Tech Chat => Topic started by: Weaver on May 03, 2018, 03:02:47 AM
-
I suspect that if I can find the energy then I can now remotely monitor and remote control my three DLink modems using my ipad or some other machine using http.
I have managed to write a heap of tools for the ipad that can monitor the state of the lines by talking to my router (a Firebrick) over http. It has been very fiddly because I have had problems because the data returned has a variable or at least hard-to-parse format after it is output by the library routines I am stuck using - it changes format (in a sense) according various line or modem related error conditions. And basically screen-scraping the web ui has not been that easy partly because of the limitations of the tools I have used and partly because of the general fact that the web tends to be only fit for human consumption and is not usually designed for maximum machine-readable friendliness. Using the WorkFlow programming language engine for the iPad I can check the status of modems and dsl lines, find modem vs port vs line interconnection mistakes, send new config xml to the router, and remote reboot it.
I suppose I could do something similar by talking to the modems’ http admin interfaces, check error rates and force them to sync now if there are problems. G.DMT has done a lot of excellent work in talking to this model of modem, much of it using telnet, some of it using http, I forget.
If my raspberry pi were working then I could run it on that. But I would have a lot of choices to make about tools and libraries. One good thing then would be that a monitoring server process could more easily run continuously. Mind you, I could have something running continuously on my older, spare iPad, provided that it does not decide to go to sleep for some reason.
One problem would be that of gathering stats and spotting resynch events. Perhaps I just say that if an error count goes down, then there has just been a resync and I have lost some data in the interval between polls when I query a modem for stats, so I just difference values and ignore differences that are negative.
If I send the modem an http request, it just immediately generates a page prompting me to login with username and password. I tried sending http ‘basic authentication’ with the original request to see if I could sweet talk it that way, but no luck, assuming I got the format right (used the example of code that I have used successfully for this before in talking to a Firebrick by http) then perhaps it just doesn't support this.
What kind of stuff should I send it to login ? I've never done exactly this before.
[Moderator edited to merge two successive posts.]
-
Actually, a thought, our hero who wrote MDWS might know how to http log in to a router’s web admin interface? Unless that is entirely telnet-based alone though.
-
MDWS depended on uploads from monitoring programs using telnet to access the stats.
-
In a previous job I wrote some software to configure some old D-Link DWP access points by faking http web page requests. The devices didn't support telnet config, so I had to script all the form content and auth stuff for each request.
Your best friend is Wireshark, and the Chrome or Safari Web Inspector. Just as a general tip, keep an eye out for cookies and hidden form values. Often, visiting the login page will issue a session cookie, which you'll need to store and include with future calls.
Use Web Inspector to see what's going back and forth during a 'real' login, and use Wireshark to cross-reference.
Good luck -- I'm sure it's do-able!
-
The web proxy ’Charles’ running on a Mac is very useful too. Slightly easier interface than wireshark which can be a bit intimidating.
-
I used a tool to look at the html source - results are very weird with lots and lots of javascript crap involved in calculating hashes. I don't seem to have got things right, although I don't see any error message, which is just the way it chooses to behave if you have got the password wrong I suspect.
The html shows user name and password entry fields as you would expect, but then there are also two other fields with strange long names, of type hidden, and it looks as if there is javascript code that is supplying md5 hashes of the username and password values as well, which I just don't understand at all. I could certainly understand ‘instead’. It seems I have failed to replicate this setup in my own code.
-
I have attached the modem html to this post - perhaps someone will tell me where I am being so stupid, it's just confusing me.
-
I don't know if it is significant but shouldn't lines 3 & 4 just be one contiguous line? (I.e. no new line character between the pair of them?)
-
I did a copy-paste from the view-source tool that I used on my iPad, I didn't download the html properly, so perhaps that is just a mistake due to the formatting introduced by the tool.
-
Sanity check: I send back the long-ish textual secret as the value of the Login_Pwd field and the md5 hash of the actual password as the value of the hidden password field whatever that is called. The username just goes back straight. Anything else I need to be doing?
I am going mad.
I am not sure what the Workflow programming language for iOS is doing with all of this. I have told it to do a http post and that the data is a form, so I am hoping that it will url-encode the field values and concatenate them sticking ”&” characters between each field, is that right? But I can't see what's going on underneath and if the modem is unhappy I can't see what I got wrong.
-
Looks like the Charles proxy is available for iOS now too. You could install this on your iPad and see exactly what it is sending over WiFi
https://itunes.apple.com/gb/app/charles-proxy/id1134218562?mt=8
-
Andy you're a star. That tool is incredibly useful. Thank you so very much.
I spotted the mistake, I had got the url wrong, everything else I had guessed correctly.
I am now in.
-
I can now poll all known modems for stats, have an automated monitoring server watch them like a hawk continuously for evil. I could say watch out for too many errors or snrm going bad or who knows what. I should then also be able to remotely force a resynch with a little more digging around.
I would need to get telnet going perhaps to get more detail. Kitizen hero G.DMT has already written a ton of code to talk to these modems, some of it using telnet.
-
Andy you're a star. That tool is incredibly useful. Thank you so very much.
No problem, glad to help. Been an iOS developer for 9 years and it’s been a lifesaver for me many times both for reverse engineering and checking implementation.
-
Duh. I have now noticed that kitizen G.Dmt wrote all the code (https://forum.kitz.co.uk/index.php/topic,17065.msg314539.html#msg314539) to do exactly this and posted it up two years ago in an earlier thread (https://forum.kitz.co.uk/index.php/topic,17065.0.html), and I never noticed, anyway completely forgot. God what a fool I am. :-[
-
God what a fool I am. :-[
I wouldn't say that. I think we have all been in a similar situation, in finding that details of what we have required has been posted in the past. It's just a case of finding the relevant link.
-
The problem with the wretched drugs is that whole things get erased completely. I didn't even know there was anything worth doing a search for amongst the old kitz posts that are relevant.
Don't do drugs kids, opiates are bad. Never thought I would ever get to learn such a thing during my life, never even having tried cannabis. :-) God what a state to get into. I do beg the kitizen community’s forbearance anyway.
-
The monitoring program on iPad (written in Workflow) is proceeding slowly. Have done relevant maths on all the stats that are exposed in the web ui.
I don't have access to telnet protocol though annoyingly, not using the Workflow language. I would have to learn some Python perhaps, because there is a Python system for iPad and in fact Workflow can talk to Python. I would hope that someone has written the relevant code in Python as a library already.
It's a shame that that Worfklow does not just handle urls with telnet: scheme in them. I know they would have to write the code to handle a new protocol and the code to parse the results and post process them would ne different.
For normal web stuff Workflow can fetch and display a web page in a mini browser, or get rich text and show you that, or it can strip everything out and give you the gist of a page as plain text only but then the resulting text it is not ascii-formatted prettily like Lynx, mind you that is not necessarily what you want, not at all, so that would need to be an option, because no pretty formatting can make it easier for machines to understand and parse. Also there is no option to fetch the html text from a webpage nor to give you the raw http response as a whole, which is amazingly bad.
I have already obtained some useful results, I noticed that another one of my lines had developed a really high HEC error rate, 0.6% per cell downstream. Far far higher error rate than the other two lines, which were about ten times and a hundred times better respectively.
I have written some alarm code that rings a bell if certain values are outside various thresholds. I really need to think about having another version though that takes multiple timesnaps and differences error counts for example as that would be much more usable and relevant.
I'm thinking about linking the performance alarm code to a routine that will force a resynch of a line that is misbehaving.
-
Hooray, thanks to the http sniffer ‘Charles Proxy’ tool recommended to me earlier ?I have worked out what is going on a bit more.
I can now force a modem to resynch remotely !
-
Btw, anyone who has one of these modems could help me here
When you log in, the html for the login form page has a hard coded fixed text string value which is passed to the server as part of the login request along with a hash of the username and a hash of the password, let us call this text string ‘the fixed secret’.
My modem specifies the value Ha2S+eOKqmzA6nrlmTeh7w== for the fixed secret.
Is your modem exactly the same?
G.Dmt’s modem is the same as mine. He was a good boy though and in his code he got the fixed secret by extracting it from the html text sent by the modem. I was lazy and just hard-coded it. I have no idea why they are doing this. It’s hardly a security feature.
Can anyone spot anything meaningful in it? If you decode it by algorithm x does it then become something significant?
-
Can't help much, but that string looks to be base-64 encoded. But decoding it doesn't really give anything meaningful - it's the following 16 byte (128-bit) binary sequence:
1d ad 92 f9 e3 8a aa 6c c0 ea 7a e5 99 37 a1 ef
It's perhaps some kind of CSRF protection (in which case this is what is known as the "CSRF token"). However, this should be random, and not fixed. Here is some info on CSRF if you're not familiar:
https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet
-
Agreed. I don't see how that would protect anything. Anyone can get hold of the fixed secret, G.DMT did. As you say they would have to vary it and make it valid only for particular requesters at least.
It's interesting that if you just google that string value you get a load of hits. I'm wondering if this html or javascript code was doing the rounds and was a copy-paste job by someone or other because one google hit result mentions “TP-Link” in some code.
(A grrr too. I don't like sites that refuse to work with javascript turned off. When I did a small amount of web stuff I always ensured that I did the right thing.)
-
I've now got routines finished that show status info and give status reports with derived stats and display configurable performance related alarms if certain number go out of permitted ranges. What I haven't yet written is something to run continuously, nor something to time-difference the numbers and report and alarm on changes or deltas.
I have also written something to force a resynch remotely. I had written a second improved, much faster and more responsive version of that, but I accidentally deleted it, clicked on the wrong file, so I need to write that code again now. Grrr. It turns out that logging in and authenticating is annoyingly slow so it's best to avoid doing that unless you really have to. And having three modems is all the worse as I don't have access to an asynchronous routine to initiate that so I can kickstart three operations in parallel. So I had made a mod to first attempt to command it to resynch in hope and if it works fine, if not login and then do it again. That was very successful and that was the code I lost. I also wrote some ui code to allow you to pick n modems out of the three and choose which ones you wanted to reset, I lost that too, so yet another thing to reinvent. I wish the Workflow app on the iPad had a bin so you could recover / undelete routines.
-
If anyone would find these useful, then do let me know. Need an iPhone or iPad and a DLink DSL-320B-Z1 modems, plus need to download a copy of the WorkFlow app (author is now owned by Apple) from the Apple App Store.
-
I keep making improvements all the time. I have an analysis and health/wellness program that gives me an extremely concise report on my four modems ranging from all good vs not good with details and the implications of any errors found to a full report of all the basics. The first one also detects certain diseases that don’t show up in the basic stats. In fact I’m fixing some algorithmic bugs today.
The previous post was a long time ago, and "Workflow", is now called "Shortcuts". It was bought by Apple and is preinstalled.