Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: 1 [2] 3 4 5

Author Topic: RaspberryPi - gui-less stats logger  (Read 20114 times)

roswellgrey

  • Member
  • **
  • Posts: 73
Re: RaspberryPi - gui-less stats logger
« Reply #15 on: October 27, 2014, 08:16:45 AM »

HG612 + Pi here, what you need? :)

Boost : can I trouble you for a telnet log for connecting to the HG612 and dumping the stats? I need to check exactly what the prompts and sequence is ....

i.e. I am not sure as to the exact login prompt, password prompt and command line/shell prompt , and a copy of the stats output would be handy ....

Also, you don't happen to know what the telnet inactivity timeout is on the HG612 ?
i.e. you telnet in, login and then don't do anything - at some point it will disconnect the client due to inactivity, and will send something like "bye" - how long this takes to happen and the message itself would be very useful info as well ...

thanks :)
« Last Edit: October 27, 2014, 08:24:55 AM by roswellgrey »
Logged
Billion 8800NL

Dray

  • Kitizen
  • ****
  • Posts: 2362
Re: RaspberryPi - gui-less stats logger
« Reply #16 on: October 27, 2014, 08:54:05 AM »

Isn't this where we started? HG612 Modem Stats was GUI-less in the beginning. It's deja-vu all over again  ;)
Logged

loonylion

  • Reg Member
  • ***
  • Posts: 723
Re: RaspberryPi - gui-less stats logger
« Reply #17 on: October 27, 2014, 04:19:17 PM »

Now, loony :)

I have had a play with smoothwall in a virtual machine, and there is a way forward, but it is probably more complex that you would like ideally, for the following reasons. I recompiled the daemon for what I believe is your host system, and it runs just fine on a virtual machine running smoothwall (i386). It's a special build as I needed to statically link in the mysql libraries as, unlike a standard distro, one cannot just add required libraries to smoothwall (understandably - it's a security device after all)

The special case is the fact we need a mysql/webserver somewhere (as it's normally on the Pi) , and running a mysql server on smoothwall would appear to be a no-no (ignoring the technical aspects, it's probably a bad idea to even try)

So, you need a mysql server to talk to.

Now, the stats daemon can talk to any mysql server that it can connect to, whether local or on the web

3 options spring to mind to overcome this problem.

1. I could, with the devs approval and help, make the stats daemon update the MyDslWebStats solution, removing the requirement for you to supply a mysql server/webserver (any of the MyDslWebStats team care to chip in with a comment about this in principle ?)
2. You could, for approx $15 US (approx 8 per annum) rent a virtual private server (VPS) with a good enough spec to host the mysql server/webserver yourself. (sry if this is teaching you to suck eggs, but you would need to look after it though - keep it patched and keep it secure - you cannot ignore an internet-facing device/server)
3. You could host a mysql server/nginx server on your lan on a 24/7 machine (buy a Pi ?  >:D), but this is probably a non-realistic option as if you had a linux machine on 24/7, you wouldn't have asked about the smoothwall option - lol

However, being technically curious, I would like to prove it would actually work in the real world on a proper smoothwall device, rather than just in my virtualization.
If you wanted to test this out, I would be more than happy to temporarily host a database/site for your data just to see if it really works as anticipated.
I don't ever want to get into hosting data storage for this solution - the whole point was to utilise the users Pi as it fits the requirements so well, but I would be intrigued to see if actually works in your case, hence the offer.

Let me know if you want to take this forward by any of the above means, and if so, we will see what we can work out...

host architecture is i586/i686 (it's a not quite 64bit HT capable single core atom (z530) with an SSD and 1gb soldered ram).

MySQL likely wouldn't pose too much of a security risk if it was limited to using named pipe/local socket only, and networking disabled, but it would likely be a task and a half to get it actually running on the stripped OS, and even after climbing that mountain, there's no php support in the onboard apache.

I have a pi, but the whole point is I already have one machine on 24/7, my dad wouldn't appreciate another one, even an ultra low power unit like the pi, and the smoothwall box HAS to be on 24/7 to maintain the internet connection, so he puts up with it  :P. Full specs here: http://www.fit-pc.com/web/products/fit-pc2/fit-pc2-2i-specifications/fit-pc2i-specifications/ under the heading 'fit-pc 2i diskless'

I do have a dedicated server that could be configured to run MySQL, it already runs nginx and php. I'm a bit paranoid with my server so I'd want some form of security/encryption on the connection between the daemon and the mysql server, either an ssh tunnel or TLS. Alternatively mydslwebstats would be an option.

For modem I'm running the ubiquitous HG612 :)
Logged

roswellgrey

  • Member
  • **
  • Posts: 73
Re: RaspberryPi - gui-less stats logger
« Reply #18 on: October 27, 2014, 07:51:39 PM »

I had a look at the hardware specs of your hw the other day when you posted it . I hadn't come across it before, and it does look like a very nice bit of hw for a smoothwall device (and other tasks of course) :)
I will drop onto the HG612 next when I have finished a few other things, and will consider the best way forward for the smoothwall situation.  A couple of thoughts did spring to mind though.
I have no issue with implementing a mysql SSL connection (I have never done it, but it's part of the api  &  would be interesting to do)
However, as the non-secure mysql client connection is TCP (not UDP) and hence cannot be easily (or at all) spoofed, if one firewalled your server to only allow connections from known hosts (either your fixed ip if you have it, or a dyndns lookup if you don't, as the smoothwall appears to have a nice dyndns client built in), the only real risk is a man-in-the-middle / sniff attack.
As the client (the daemon) only has select and insert privileges, the worst case scenario is someone could read your dsl stats or insert dummy data. Whilst not ideal, it would not be catastrophic, even if it is possible.   
However, you can never be too paranoid about security (I am very - lol)

Anyways, I need to get the HG612 gathering sorted afore we can take this any further. Will return to this topic when that bit is done :)

Logged
Billion 8800NL

roswellgrey

  • Member
  • **
  • Posts: 73
Re: RaspberryPi - gui-less stats logger
« Reply #19 on: October 28, 2014, 08:30:08 PM »

Another question if I may .....

Tomorrow the 1st beta release will be available. It will just support the Billion 8800NL - the HG612 is next on the todo list.

So, a question to the potential beta-testers  .... I don't know what you are expecting in terms of how you actually install it, and I don't know (a) if you just want a really easy install option, or (b) if you are interested in doing a manual install , so ....

Given this www.xdaemonstats.mooo.com/Docs  are you happy to follow those instructions, or are you going to look at that and think "nah - too much hassle ..." ?

If the answer is the latter, I will offer a download of an image of a raspbian install with my package already installed, and you would only have to edit one config file :)

Would appreciate any thoughts on this ...
Logged
Billion 8800NL

G.DMT

  • Member
  • **
  • Posts: 74
Re: RaspberryPi - gui-less stats logger
« Reply #20 on: October 28, 2014, 11:00:51 PM »

Hi Roswellgrey.

I had a quick look through your install document and one thing did strike me immediately.

For those of us whose Pis already have nginx, php-fpm and mysql in use, it would be much better if the nginx instructions demonstrated the (straightforward) procedure to install your service into a virtual host container, rather than the webroot -which may (in my case most definately is) already be otherwise engaged.

Other than that it occurs to me that you could avoid having to use mysqld altogether if you wrote your data to a database file instead e.g. sqlite3?
Indeed, if you linked your executeable to libdbi you could write generic code that accesses mysql & sqlite plus many other data storage options, without being wedded to one particular implementation choice.

Just a thought.  :)

« Last Edit: October 29, 2014, 10:13:22 AM by G.DMT »
Logged

roswellgrey

  • Member
  • **
  • Posts: 73
Re: RaspberryPi - gui-less stats logger
« Reply #21 on: October 29, 2014, 05:58:51 PM »

Many thanks for reading it, and for the comments :)

Good point about nginx, but as there are no domain names involved (unless you start hacking hosts files on all the clients), to me this means 2 default-server blocks, which you cannot do ?  (I may well be missing a trick here - it happens a lot - lol)
However, I have added a "install in the existing default-server block" guide as an alternative method to achieve the same result (well sort of, anyway).

You are right about mysqld - it is total overkill for this task, and there are better and more code portable options available.
To be honest, I had written a C based linux console app for a completely different requirement a few months ago, and being a lazy ***, wanted to reuse as much code as I could to make this development as rapid as possible. ;D
However, in my defense, I am seriously impressed how well mysqld runs on the Pi  >:D

Anyway, despite me stating that the HG612 would be next, as my poor old Billion 7800N was sulking in the cupboard since I got fibre a few weeks ago, I broke it out last night, and whilst it doesn't offer many stats for me to test it properly (as I haven't got an ADSL connection anymore), it does however appear to fundamentally work .... Do you fancy being the guinea pig for that one ?
« Last Edit: October 29, 2014, 06:00:55 PM by roswellgrey »
Logged
Billion 8800NL

roswellgrey

  • Member
  • **
  • Posts: 73
Re: RaspberryPi - gui-less stats logger
« Reply #22 on: October 29, 2014, 07:57:57 PM »

If anyone is interested in trying this, the first beta release is out

- Tested on Billion 8800NL in VDSL mode
- Untested on Billion 8800NL in ADSL mode (I could do with some logs for this if anyone tries it - not sure how the ADSL stats output of the router is different to the VDSL stats output )
- 1/2 tested on a Billion 7800N (1/2 tested as I don't have a adsl connection anymore to test it fully)

Any other routers are unsupported at the moment.

Instructions and download link are here www.xdaemonstats.mooo.com/Docs

It has simple functionality at the moment, but I want it tested by more than me afore it progresses to the more complicated areas.

On a related note, I could really do with a telnet dump of the HG612's stats output if anyone would be so kind (including initial login/password prompts and the output of xdslcmd info --stats) . This would be sorely appreciated :)

Any questions, problems, comments, please ask ......
Logged
Billion 8800NL

burakkucat

  • Global Moderator
  • Senior Kitizen
  • *
  • Posts: 27904
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: RaspberryPi - gui-less stats logger
« Reply #23 on: October 29, 2014, 08:52:31 PM »

On a related note, I could really do with a telnet dump of the HG612's stats output if anyone would be so kind (including initial login/password prompts and the output of xdslcmd info --stats) .

You need to make an offering to the Eagle's Nest . . .  :D
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

NewtronStar

  • Kitizen
  • ****
  • Posts: 4844
Re: RaspberryPi - gui-less stats logger
« Reply #24 on: October 29, 2014, 09:56:05 PM »

If you're having problems with the display on a Pi, there are some things you can change in the config.txt file that can help.

If you're already remotely logged into the pi, you can find the file at /boot/config.txt

OR

unplug the SD card from the PI, pop it into a PC, and you will see a small selection of files.  config.txt is one of them, so you can edit on a PC, then plug it back into the pi.

The lines of interest in config.txt are :

Code: [Select]
# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1

# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4

# uncomment for composite PAL
#sdtv_mode=2

I had to remove the # from config_hdmi_boost=4 to make a pi work with a particular DVI monitor using an HDMI to DVI cable, for example.

Ian

Thanks sheddy  :) I am using an HDMI to VGA adapter the maxium output seems to be 800*600 using hdmi_group=2 hdmi_mode=9 if i select mode 16 for 1024*768 the rasperry pi screen is blank and use the PC to ssh the boot/config.txt to get the RPi back into mode 9

So only 640*480 and 800*600 seems to work on both TV monitor and PC monitor it is 1920*1080 native.
Logged

G.DMT

  • Member
  • **
  • Posts: 74
Re: RaspberryPi - gui-less stats logger
« Reply #25 on: October 29, 2014, 11:19:36 PM »

Many thanks for reading it, and for the comments :)
However, in my defense, I am seriously impressed how well mysqld runs on the Pi  >:D

If you are going to use mysqld then it might perhaps be a good idea to include a mention of that common Pi pitfall - potential memory card corruption.
 Many, if not most, Pi owners may not be aware that continuous database writes uses up the finite write cycles of the memory card and can make it unuseable.

 At least if they know to plug a memory stick into one of the USB ports and accumulate the database writes on to that, then only their data would be lost, and the Pi at least will remain bootable.  :'(

What I can do that might be of some immediate use to you is to attach a file of the text from my Billion 7800N connected to ADSL1 (on Beattie)
-obviously the pbParams information does not exist for G.DMT   ;)
« Last Edit: October 29, 2014, 11:52:33 PM by G.DMT »
Logged

roswellgrey

  • Member
  • **
  • Posts: 73
Re: RaspberryPi - gui-less stats logger
« Reply #26 on: October 30, 2014, 08:29:39 AM »

Thanks for the 7800N log :)

That is a good point about the SD card write cycles and mysqld.
To be honest, whether it uses a different embedded db, or even wrote to a flat text file, whilst the scale of the write cycles might differ, the underlying potential problem will remain  - the only way to be sure of the longevity of the card in this situation is to leave it running for many months and see if it kills the card.

The last thing I would like to be known as is a "Pi Card killer" - it's probably prudent that I pull this project from public access and let it run here for many months and see what happens, so that is what I will do.

However, if anyone wants to try out the daemon, and will be using a non-Pi hosted mysql database(s), let me know ....

Thanks to those who have commented on this concept :)
« Last Edit: October 30, 2014, 10:59:40 AM by roswellgrey »
Logged
Billion 8800NL

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 39533
  • Penguins CAN fly
    • DSLstats
Re: RaspberryPi - gui-less stats logger
« Reply #27 on: October 30, 2014, 11:18:17 AM »

I must say that DSLstats does a fair bit of writing to the SD card as well. It doesn't use a database, but it does store data in its own formats. So maybe DSLstats will prove to be a Pi Card killer too. :(
Logged
  Eric

tbailey2

  • Kitizen
  • ****
  • Posts: 1245
Re: RaspberryPi - gui-less stats logger
« Reply #28 on: October 30, 2014, 11:26:09 AM »

I must say that DSLstats does a fair bit of writing to the SD card as well. It doesn't use a database, but it does store data in its own formats. So maybe DSLstats will prove to be a Pi Card killer too. :(

What if you aren't storing any graphical data locally though on the pi, just uploading (which is what I am doing now)?

I was going to query anyway whether the DSLStats install could be totally to an external USB drive if wanted so no data is written to the board SD card?
Logged
Tony
My Books!
Plusnet 80/20 - DSLstats - HG612/TG582n - ECI

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 39533
  • Penguins CAN fly
    • DSLstats
Re: RaspberryPi - gui-less stats logger
« Reply #29 on: October 30, 2014, 12:02:05 PM »

I don't want to hijack this thread, but on thinking about it more, DSLstats doesn't save any graphical data to disk (unless you ask it to) so perhaps it doesn't hit the card so hard.
Logged
  Eric
Pages: 1 [2] 3 4 5