Kitz Forum
Broadband Related => Router Monitoring Software => Topic started by: roswellgrey on October 25, 2014, 08:25:16 AM
-
Just a poke if to see if anyone would be interested in this ....
I run my Pi with no graphical environment i.e. as a gui-less server.
Now, DSLStats is a great bit of software (major kudos to the developer), and it does run really well on the Pi, but I am having to run up a vnc session just for this purpose, which for me personally is not ideal ( have been using ssh for far too long to change my ways - lol). Hence, I decided to develop a gui-less stats gathering daemon. It stores all data captured in a mysql database (obviously hosted on the pi itself), and the stats view is html (nginx server) .
The performance hit on the Pi running this is minimal - mysql runs like a dream, and offloading the graph rendering to the client by using Google Charts means even when browsing the data the additional load on the Pi is, on average, very low.
Now, if anyone else likes to run their Pi without a graphical environment, and hence would be interested in this, post here and please let me know what router you are using. I might need you to give me a telnet dump of a command/responses if I cannot find that info by other means. At the moment, it only knows about a Billion8800NL. It's probably a week off being fit for release of basic level functionality (needs a few more days soak test)
I should point out that it will never be as comprehensive as DSLStats ; it's solely a solution to fit a particular niche need of mine :) and I was just wondering if anyone else would find it useful....
-
I should point out that it will never be as comprehensive as DSLStats ; it's solely a solution to fit a particular niche need of mine :) and I was just wondering if anyone else would find it useful....
I am interested and am already running a script to access the stats on my Billion 8800. My log format is as follows:
Time Status Sync kbps----- SNR (dB) ----- Atten (dB) --- Power (dBm)--- FEC (15 min)-- CRC (15 min)-- ES (15 min)--
Down Up Down Up Down Up Down Up Down Up Down Up Down Up
00:00:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 3 6 0 4 0
00:05:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 6 3 1 2 1
00:10:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 6 3 1 2 1
00:15:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 6 3 1 2 1
00:20:02 Showtime 56614 14361 6.1 5.6 19.1 0.0 13.1 7.4 0 18 1 4 1 3
00:25:02 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 18 1 4 1 3
00:30:02 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 18 1 4 1 3
00:35:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 0 5 0 4 0
00:40:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 0 5 0 4 0
00:45:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 0 5 0 4 0
Happy to share my script for constructive criticism ;D
-
I'd be interested in something like this for an x86 processor. I run a headless firewall on a low power atom device and it has no gui (except a web based one) If there was a version of dslstats that could be configured with a conf file and ran without a gui it would be perfect.
-
I'm afraid that a GUI-free version of DSLstats would need a substantial rewrite, and there just aren't enough hours in the day...
-
HG612 + Pi here, what you need? :)
-
( have been using ssh for far too long to change my ways - lol).
Thats something i'll need to understand as got myself into a predicament with the HDMI to VGA converter on the PI selected hdmi_mode=4 720p 60 Hz and got a blank screen on VGA monitor and aslo with HDMI TV so I was blind could not even use the composite video, if I new how to use ssh could have got myself out of this instead of going down the re-install os root with Noobs :-[
-
Martin: - thanks for the interest :)
In a few days I will post some install instructions and a download link for you to give it a try
Loony : what are you running exactly ? and more importantly is there a freebee version available that I could play with ? Would be interested to explore what options are available, if any, for what you are running .....
boost : think there is enough info here and elsewhere for the HG612 for me to make a stab at it. Will let you know when you can play guinea pig ;D
newtron : post away with any ssh questions - will certainly help if I can ....
-
I'm running smoothwall express 3.1 on a fit-pc 2i. It's an opensource firewall distro.
-
thanks - will do a little research on what might be feasible ...
-
newtron : post away with any ssh questions - will certainly help if I can ....
Cheers roswellgrey at the moment ssh is disabled in the config will do my best to research into the ssh before taking the plunge into uncertain waters, this Raspbian OS has a simulatory to the Amiga boot configs which had no Bios either the PI has defo rekindled my enthusiasm for computing :)
-
Sounds interesting to me too. :)
I have a Pi connected to a Billion 7800 and a Netgear DGND3700
with scripts collecting the modem data from them.
Its all modular so you would be welcome to pick up any pieces that might be of use.
At The moment:
cron calls a script which interrogates the modem via telnet > modem_out_file
A script parses modem_out_file and to extract the data of interest.
optionally script functions can split out into individual files and create graphs
optionally script functions can archive selected data files
optionally script functions can archive the collected data
optionally script functions can extract data from the archives
PHP classes written & running under Eclipse remote debugger / nginx / php-fpm to access the data.
;-)
-
It occurred to me that some of you that are using RasPis might be interested in a simple technique that I used to sidestep a range of issues related to connectivity and file storage on a Pi, which makes the whole process significantly simpler by using two brilliant features of SSH. ;)
1) by adding the following line to ~/.ssh/config on your PC
LocalForward 127.0.0.1:10123 192.168.3.1:23
you can telnet directly to the modem locally from your PC by using the sshd on the Pi as a proxy
In this example the modem at 192.168.3.1 is firewalled on a separate subnet from the LAN and can not be directly be contacted from the LAN.
{The same can be accomplished on a M$ winblows machine using putty.}
2) On the linux client ( Pi or a Linux PC) use SSH_File_System to locally mount storage that is physically located elsewhere on the netwiork.
http://fuse.sourceforge.net/sshfs.html
On Raspbian 'aptitude show sshfs'
On Pidora 'yum info fuse-sshfs'
Thus I can either
a) Access data files stored on a Pi as if it were local to my PC.
or
b) Not have to save data onto memory card/usb stick on the Pi, because the locally visible file system can actually be fuse-mount to a directory on my PC.
;D
-
Yes got the PC to ssh commands to RPi using two small programs this is great to get me out of impossible situation on the RPi as you can see and interact with RPi's desktop GUI from the PC or get down an dirty with commands in the terminal ;D
-
Thanks for the responses,and the offers for the work you have done ....
Now, the architecture of this is as follows:
A stats gathering daemon, which is a linux console app written in C, telnets to the router, requests the a/vdsl stats and parses the response. It then populates a mysql database (schema/table definition will be available to you), sleeps and repeats at a configurable rate.
With the exception of Loony's particular requirement (more on that in a bit), the mysql database and nginx server are hosted on your Pi on your local Lan. (although, with proper security measures taken, you could obviously open up the Pi to the wider internet world should you so wish, and hence access it from anywhere)
Now, using the mysql databse, an nginx server (although it could be apache/lighttpd etc) processes php scripts and uses Google Charts javascript to pass the data to the webclient to produce charts and the like.
Now, as of now , and for the first beta release, the charts are pretty crude/basic.
What would be great however is if those with interest in php/google charts could develop charts of their own and feed them back so that all could inherit their efforts. (you obviouly get the current php code with the install for use as a basis/template)
Now, doing this might mean that the daemon would need to capture more/different data, and obviously I would incorporate any such mods into the stats daemon.
Now, loony :)
I have had a play with smoothwall in a virtual machine, and there is a way forward, but it is probably more complex that you would like ideally, for the following reasons. I recompiled the daemon for what I believe is your host system, and it runs just fine on a virtual machine running smoothwall (i386). It's a special build as I needed to statically link in the mysql libraries as, unlike a standard distro, one cannot just add required libraries to smoothwall (understandably - it's a security device after all)
The special case is the fact we need a mysql/webserver somewhere (as it's normally on the Pi) , and running a mysql server on smoothwall would appear to be a no-no (ignoring the technical aspects, it's probably a bad idea to even try)
So, you need a mysql server to talk to.
Now, the stats daemon can talk to any mysql server that it can connect to, whether local or on the web
3 options spring to mind to overcome this problem.
1. I could, with the devs approval and help, make the stats daemon update the MyDslWebStats solution, removing the requirement for you to supply a mysql server/webserver (any of the MyDslWebStats team care to chip in with a comment about this in principle ?)
2. You could, for approx $15 US (approx £8 per annum) rent a virtual private server (VPS) with a good enough spec to host the mysql server/webserver yourself. (sry if this is teaching you to suck eggs, but you would need to look after it though - keep it patched and keep it secure - you cannot ignore an internet-facing device/server)
3. You could host a mysql server/nginx server on your lan on a 24/7 machine (buy a Pi ? >:D), but this is probably a non-realistic option as if you had a linux machine on 24/7, you wouldn't have asked about the smoothwall option - lol
However, being technically curious, I would like to prove it would actually work in the real world on a proper smoothwall device, rather than just in my virtualization.
If you wanted to test this out, I would be more than happy to temporarily host a database/site for your data just to see if it really works as anticipated.
I don't ever want to get into hosting data storage for this solution - the whole point was to utilise the users Pi as it fits the requirements so well, but I would be intrigued to see if actually works in your case, hence the offer.
Let me know if you want to take this forward by any of the above means, and if so, we will see what we can work out...
-
( have been using ssh for far too long to change my ways - lol).
Thats something i'll need to understand as got myself into a predicament with the HDMI to VGA converter on the PI selected hdmi_mode=4 720p 60 Hz and got a blank screen on VGA monitor and aslo with HDMI TV so I was blind could not even use the composite video, if I new how to use ssh could have got myself out of this instead of going down the re-install os root with Noobs :-[
If you're having problems with the display on a Pi, there are some things you can change in the config.txt file that can help.
If you're already remotely logged into the pi, you can find the file at /boot/config.txt
OR
unplug the SD card from the PI, pop it into a PC, and you will see a small selection of files. config.txt is one of them, so you can edit on a PC, then plug it back into the pi.
The lines of interest in config.txt are :
# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1
# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1
# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2
# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4
# uncomment for composite PAL
#sdtv_mode=2
I had to remove the # from config_hdmi_boost=4 to make a pi work with a particular DVI monitor using an HDMI to DVI cable, for example.
Ian
-
HG612 + Pi here, what you need? :)
Boost : can I trouble you for a telnet log for connecting to the HG612 and dumping the stats? I need to check exactly what the prompts and sequence is ....
i.e. I am not sure as to the exact login prompt, password prompt and command line/shell prompt , and a copy of the stats output would be handy ....
Also, you don't happen to know what the telnet inactivity timeout is on the HG612 ?
i.e. you telnet in, login and then don't do anything - at some point it will disconnect the client due to inactivity, and will send something like "bye" - how long this takes to happen and the message itself would be very useful info as well ...
thanks :)
-
Isn't this where we started? HG612 Modem Stats was GUI-less in the beginning. It's deja-vu all over again ;)
-
Now, loony :)
I have had a play with smoothwall in a virtual machine, and there is a way forward, but it is probably more complex that you would like ideally, for the following reasons. I recompiled the daemon for what I believe is your host system, and it runs just fine on a virtual machine running smoothwall (i386). It's a special build as I needed to statically link in the mysql libraries as, unlike a standard distro, one cannot just add required libraries to smoothwall (understandably - it's a security device after all)
The special case is the fact we need a mysql/webserver somewhere (as it's normally on the Pi) , and running a mysql server on smoothwall would appear to be a no-no (ignoring the technical aspects, it's probably a bad idea to even try)
So, you need a mysql server to talk to.
Now, the stats daemon can talk to any mysql server that it can connect to, whether local or on the web
3 options spring to mind to overcome this problem.
1. I could, with the devs approval and help, make the stats daemon update the MyDslWebStats solution, removing the requirement for you to supply a mysql server/webserver (any of the MyDslWebStats team care to chip in with a comment about this in principle ?)
2. You could, for approx $15 US (approx £8 per annum) rent a virtual private server (VPS) with a good enough spec to host the mysql server/webserver yourself. (sry if this is teaching you to suck eggs, but you would need to look after it though - keep it patched and keep it secure - you cannot ignore an internet-facing device/server)
3. You could host a mysql server/nginx server on your lan on a 24/7 machine (buy a Pi ? >:D), but this is probably a non-realistic option as if you had a linux machine on 24/7, you wouldn't have asked about the smoothwall option - lol
However, being technically curious, I would like to prove it would actually work in the real world on a proper smoothwall device, rather than just in my virtualization.
If you wanted to test this out, I would be more than happy to temporarily host a database/site for your data just to see if it really works as anticipated.
I don't ever want to get into hosting data storage for this solution - the whole point was to utilise the users Pi as it fits the requirements so well, but I would be intrigued to see if actually works in your case, hence the offer.
Let me know if you want to take this forward by any of the above means, and if so, we will see what we can work out...
host architecture is i586/i686 (it's a not quite 64bit HT capable single core atom (z530) with an SSD and 1gb soldered ram).
MySQL likely wouldn't pose too much of a security risk if it was limited to using named pipe/local socket only, and networking disabled, but it would likely be a task and a half to get it actually running on the stripped OS, and even after climbing that mountain, there's no php support in the onboard apache.
I have a pi, but the whole point is I already have one machine on 24/7, my dad wouldn't appreciate another one, even an ultra low power unit like the pi, and the smoothwall box HAS to be on 24/7 to maintain the internet connection, so he puts up with it :P. Full specs here: http://www.fit-pc.com/web/products/fit-pc2/fit-pc2-2i-specifications/fit-pc2i-specifications/ under the heading 'fit-pc 2i diskless'
I do have a dedicated server that could be configured to run MySQL, it already runs nginx and php. I'm a bit paranoid with my server so I'd want some form of security/encryption on the connection between the daemon and the mysql server, either an ssh tunnel or TLS. Alternatively mydslwebstats would be an option.
For modem I'm running the ubiquitous HG612 :)
-
I had a look at the hardware specs of your hw the other day when you posted it . I hadn't come across it before, and it does look like a very nice bit of hw for a smoothwall device (and other tasks of course) :)
I will drop onto the HG612 next when I have finished a few other things, and will consider the best way forward for the smoothwall situation. A couple of thoughts did spring to mind though.
I have no issue with implementing a mysql SSL connection (I have never done it, but it's part of the api & would be interesting to do)
However, as the non-secure mysql client connection is TCP (not UDP) and hence cannot be easily (or at all) spoofed, if one firewalled your server to only allow connections from known hosts (either your fixed ip if you have it, or a dyndns lookup if you don't, as the smoothwall appears to have a nice dyndns client built in), the only real risk is a man-in-the-middle / sniff attack.
As the client (the daemon) only has select and insert privileges, the worst case scenario is someone could read your dsl stats or insert dummy data. Whilst not ideal, it would not be catastrophic, even if it is possible.
However, you can never be too paranoid about security (I am very - lol)
Anyways, I need to get the HG612 gathering sorted afore we can take this any further. Will return to this topic when that bit is done :)
-
Another question if I may .....
Tomorrow the 1st beta release will be available. It will just support the Billion 8800NL - the HG612 is next on the todo list.
So, a question to the potential beta-testers .... I don't know what you are expecting in terms of how you actually install it, and I don't know (a) if you just want a really easy install option, or (b) if you are interested in doing a manual install , so ....
Given this www.xdaemonstats.mooo.com/Docs (http://www.xdaemonstats.mooo.com/Docs) are you happy to follow those instructions, or are you going to look at that and think "nah - too much hassle ..." ?
If the answer is the latter, I will offer a download of an image of a raspbian install with my package already installed, and you would only have to edit one config file :)
Would appreciate any thoughts on this ...
-
Hi Roswellgrey.
I had a quick look through your install document and one thing did strike me immediately.
For those of us whose Pis already have nginx, php-fpm and mysql in use, it would be much better if the nginx instructions demonstrated the (straightforward) procedure to install your service into a virtual host container, rather than the webroot -which may (in my case most definately is) already be otherwise engaged.
Other than that it occurs to me that you could avoid having to use mysqld altogether if you wrote your data to a database file instead e.g. sqlite3?
Indeed, if you linked your executeable to libdbi you could write generic code that accesses mysql & sqlite plus many other data storage options, without being wedded to one particular implementation choice.
Just a thought. :)
-
Many thanks for reading it, and for the comments :)
Good point about nginx, but as there are no domain names involved (unless you start hacking hosts files on all the clients), to me this means 2 default-server blocks, which you cannot do ? (I may well be missing a trick here - it happens a lot - lol)
However, I have added a "install in the existing default-server block" guide as an alternative method to achieve the same result (well sort of, anyway).
You are right about mysqld - it is total overkill for this task, and there are better and more code portable options available.
To be honest, I had written a C based linux console app for a completely different requirement a few months ago, and being a lazy ***, wanted to reuse as much code as I could to make this development as rapid as possible. ;D
However, in my defense, I am seriously impressed how well mysqld runs on the Pi >:D
Anyway, despite me stating that the HG612 would be next, as my poor old Billion 7800N was sulking in the cupboard since I got fibre a few weeks ago, I broke it out last night, and whilst it doesn't offer many stats for me to test it properly (as I haven't got an ADSL connection anymore), it does however appear to fundamentally work .... Do you fancy being the guinea pig for that one ?
-
If anyone is interested in trying this, the first beta release is out
- Tested on Billion 8800NL in VDSL mode
- Untested on Billion 8800NL in ADSL mode (I could do with some logs for this if anyone tries it - not sure how the ADSL stats output of the router is different to the VDSL stats output )
- 1/2 tested on a Billion 7800N (1/2 tested as I don't have a adsl connection anymore to test it fully)
Any other routers are unsupported at the moment.
Instructions and download link are here www.xdaemonstats.mooo.com/Docs (http://www.xdaemonstats.mooo.com/Docs)
It has simple functionality at the moment, but I want it tested by more than me afore it progresses to the more complicated areas.
On a related note, I could really do with a telnet dump of the HG612's stats output if anyone would be so kind (including initial login/password prompts and the output of xdslcmd info --stats) . This would be sorely appreciated :)
Any questions, problems, comments, please ask ......
-
On a related note, I could really do with a telnet dump of the HG612's stats output if anyone would be so kind (including initial login/password prompts and the output of xdslcmd info --stats) .
You need to make an offering to the Eagle's Nest . . . :D
-
If you're having problems with the display on a Pi, there are some things you can change in the config.txt file that can help.
If you're already remotely logged into the pi, you can find the file at /boot/config.txt
OR
unplug the SD card from the PI, pop it into a PC, and you will see a small selection of files. config.txt is one of them, so you can edit on a PC, then plug it back into the pi.
The lines of interest in config.txt are :
# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1
# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1
# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2
# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4
# uncomment for composite PAL
#sdtv_mode=2
I had to remove the # from config_hdmi_boost=4 to make a pi work with a particular DVI monitor using an HDMI to DVI cable, for example.
Ian
Thanks sheddy :) I am using an HDMI to VGA adapter the maxium output seems to be 800*600 using hdmi_group=2 hdmi_mode=9 if i select mode 16 for 1024*768 the rasperry pi screen is blank and use the PC to ssh the boot/config.txt to get the RPi back into mode 9
So only 640*480 and 800*600 seems to work on both TV monitor and PC monitor it is 1920*1080 native.
-
Many thanks for reading it, and for the comments :)
However, in my defense, I am seriously impressed how well mysqld runs on the Pi >:D
If you are going to use mysqld then it might perhaps be a good idea to include a mention of that common Pi pitfall - potential memory card corruption.
Many, if not most, Pi owners may not be aware that continuous database writes uses up the finite write cycles of the memory card and can make it unuseable.
At least if they know to plug a memory stick into one of the USB ports and accumulate the database writes on to that, then only their data would be lost, and the Pi at least will remain bootable. :'(
What I can do that might be of some immediate use to you is to attach a file of the text from my Billion 7800N connected to ADSL1 (on Beattie)
-obviously the pbParams information does not exist for G.DMT ;)
-
Thanks for the 7800N log :)
That is a good point about the SD card write cycles and mysqld.
To be honest, whether it uses a different embedded db, or even wrote to a flat text file, whilst the scale of the write cycles might differ, the underlying potential problem will remain - the only way to be sure of the longevity of the card in this situation is to leave it running for many months and see if it kills the card.
The last thing I would like to be known as is a "Pi Card killer" - it's probably prudent that I pull this project from public access and let it run here for many months and see what happens, so that is what I will do.
However, if anyone wants to try out the daemon, and will be using a non-Pi hosted mysql database(s), let me know ....
Thanks to those who have commented on this concept :)
-
I must say that DSLstats does a fair bit of writing to the SD card as well. It doesn't use a database, but it does store data in its own formats. So maybe DSLstats will prove to be a Pi Card killer too. :(
-
I must say that DSLstats does a fair bit of writing to the SD card as well. It doesn't use a database, but it does store data in its own formats. So maybe DSLstats will prove to be a Pi Card killer too. :(
What if you aren't storing any graphical data locally though on the pi, just uploading (which is what I am doing now)?
I was going to query anyway whether the DSLStats install could be totally to an external USB drive if wanted so no data is written to the board SD card?
-
I don't want to hijack this thread, but on thinking about it more, DSLstats doesn't save any graphical data to disk (unless you ask it to) so perhaps it doesn't hit the card so hard.
-
Thanks for the 7800N log :)
You are welcome! I hope it helps you. :)
That is a good point about the SD card write cycles and mysqld.
To be honest, whether it uses a different embedded db, or even wrote to a flat text file, whilst the scale of the write cycles might differ, the underlying potential problem will remain - the only way to be sure of the longevity of the card in this situation is to leave it running for many months and see if it kills the card.
The last thing I would like to be known as is a "Pi Card killer" - it's probably prudent that I pull this project from public access and let it run here for many months and see what happens, so that is what I will do.
Eeek! I didn't expect to cause a flap! :-[
I was only suggesting that if users were advised of the possiblities, then they could make an informed choice.
There exist multiple mitigation options for the SD card write issue.
Easiest for a non technical user is to save to a separate device, as I indicated above.
That device _could_ be fuse-ssh mounted over the network as I described in a previous post.
A local mount can be to a memory backed file system e.g. tmps or loop.
A traditional disk-based database engine makes many,many distributed updates (writes) due to its hash-table or b+ tree based file content indexing.
If the goal is to minimise the number of writes-to-flash then appending a single data block to a data store is obviously far less write intensive.
e.g. Creating any intermediary files on a RAM backed filesystem and then appending only the data of interest to your data store sounds like it is getting close to a 'good enough' solution.
A default Pi install takes some account of this already
i.e.
root@raspberrypi:~# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 7.4G 2.0G 5.1G 28% /
/dev/root 7.4G 2.0G 5.1G 28% /
devtmpfs 215M 0 215M 0% /dev
tmpfs 44M 268K 44M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 88M 0 88M 0% /run/shm
/dev/mmcblk0p1 56M 9.6M 47M 18% /boot
Maybe it would be enough to just allow the user to specify where they want their temporary working directory to be (in /run/shm, /tmp, -o loop, whatever)...
and you would be half way there already. ;D
-
Eeek! I didn't expect to cause a flap! :-[
I am glad you brought it up :)
There are options to alleviate this potential problem, but it's getting complicated for the potential end-user :-[
What I was trying to achieve was a very easy install of a self-contained, gui-less logger with a very low power footprint.
Now, it looks like some form of (a) more write tolerant external storage or (b) a potentially sacrificial external storage device (such as a usb stick) is required to counter the fears. And unfortunately this is going against what I was trying to achieve (i.e. the easy, self-contained bit - lol)
I know people can make up their own minds about this issue, but I don't want anyone pointing fingers at me saying
"I cooked their SD card" ("cooked their Pi" is too corny ...) .... life is too short >:D
In the meanwhile, I will continue to abuse my card - it has 9K entries in the db at the moment - will see how many it can get to before it fails :)
-
Being curious about the actual number of writes that mysqld causes, I conducted a little experiment (results attached)
The conclusion is that my daemon install on a Pi causes approx. 3x the number of writes/second (0. 19 w/s) that a vanilla Rasbian Pi install does (0.06 w/s) over a 10 minute test period .
My daemon is polling the modem every 1 minute, hence 10 stats db inserts are created during the test.
My test was slightly biased against my dameon install as I am running vnstatd on that install and that process's writes are included in the daemon install write stats. However, it does give a good general idea.
Due to the real-world uncertainties of wear-leveling (as a singular example, I not sure if my card actually has it) and as I cannot determine how many writes to the same physical locations on the card a new row insert into the database actually causes, I am unable to draw any firm conclusions as to how much my daemon will decrease the life of the SD card.
Long term soak test seems the only realistic way forward to see if a very early card failure happens :)
-
@roswellgrey Thank you for contributing some concrete information to the discussion. ;D
Your test file has prompted me to revisit this issue and I spent a few hours reading up on the current situation and checking it out on a long running Pi here.
I now have two large text files filled with commands and data, but I suspect this would be far too technical to post here.
So ... for those of you who are not Linux experts, I will try to formulate a short version.
On a RasPi the Operating System and all the user files and log files are stored on a small, cheap SD Flash memory card plugged into the mmc controller.
Flash memory is written in blocks of 512 bytes
SSD Flash drives are fast, expensive and use 'dynamic wear levelling' to spread the writes out evenly and maximise the useful life of the flash cells.
SD cards are cheap, slow and some have no wear Levelling at all, most implement 'static wear levelling' via proprietary algorithms embedded in the controller on the card.
This Flash Translation Layer is opaque and cannot be directly observed or interacted with.
They have a 'write pool' of writable blocks and _if_ the OS can tell the controller that a block is no longer needed, it can be added back to the pool of available blocks, and thus be recirculated to help share out writes among flash blocks.
The Linux mmc driver can issue a TRIM command which tells the flash controller to erase a block, and thus add it back to the free pool.
Unfortunately this functionality of the mmc controller on the RasPi has been broken for years. :(
see: 'FStrim does not work on RPI'
http://www.raspberrypi.org/forums/viewtopic.php?f=71&t=47484
root@raspberrypi:~# sudo fstrim -v /
fstrim: /: FITRIM ioctl failed: Operation not supported
TL;DR
1) Buy a (genuine) SanDisk Ultra SDHC card much bigger than you need.
2) DO NOT write a huge image file that fills up the whole partition, because every block that is written to is removed from the wear levelling pool.
INSTEAD copy the minimal OS image onto it, THEN expand the filesystem.
e.g. DD the <4 Gig image onto the card, then expand it to 8Gig.
If you bought a 16 Gig card that 'unused' space is not 'wasted' as it should be contributing to the wear levelling pool.
3) update to the latest kernel ('sudo aptitude update kernel')
root@raspberrypi:~# uname -a
Linux raspberrypi 3.12.28+ #709 PREEMPT Mon Sep 8 15:28:00 BST 2014 armv6l GNU/Linux
And Broadcom binary blob. ('sudo aptitude update firmware-brcm80211')
root@raspberrypi:~# /opt/vc/bin/vcgencmd version
Sep 8 2014 19:02:48
Copyright (c) 2012 Broadcom
version 3f2f2607186be72e4945cfa8edc77872dfc73195 (clean) (release)
Enable the Newly Fixed TRIM ;D
'The latest rpi-update kernel now includes a new MMC driver. This driver is completely rewritten'
http://www.raspberrypi.org/forums/viewtopic.php?f=63&t=85061#
root@raspberrypi:~# sudo fstrim -v /
/: 4681015296 bytes were trimmed
Automate the running of trim occasionally.
e.g.
Add this to your /etc/crontab
00 00 * * * root ionice -c 3 fstrim -v / | logger -t fstrim
For bonus points you could also reduce the wear from services which continually append to their log files -such as web server by e.g. installing ramlog:
http://www.tremende.com/ramlog/
I also came across this handy service called backup2mail where you can have your Pi email a backup of your database:
http://www.backup2mail.com/
So due to the magic of the Open Source development model, everything now should 'just work' as it is supposed to, and you didn't have to pay for the upgrade. ;)
-
This so helpfull G.DMT :) had moved from 8GB MSD card with Noobs to 32GB kingston MSD card with just the Raspbian OS installed and extended the files system.
-
That's very useful information, G.DMT, thank you.
-
Just did the Kernal update tonight on the RPi using sudo rpi-update a newer version was downloaded.
pi@raspberrypi ~ $ sudo uname -a
Linux raspberrypi 3.12.32+ #721 PREEMPT Fri Nov 7 16:50:31 GMT 2014 armv6l GNU/Linux
and sudo fstrim -v / trimmed 5000000000 bytes
Cheers
-
Just curious as to what router stats sample/poll rate people typically set up dslstats (or similar) with ?
1 min, 10 mins, 1 hour ?
Was just trying to get a guesstimate as to how many samples / month is a typical figure ....
-
I have Dslstats set to sample each minute
-
Every minute here
-
Every 5 minutes here.
-
Every minute.
-
Thank you :)
I thought you would be in the ballpark of 1 min samples, and it seem that most of you are (I am as well) :)
This makes for 525K rows in the database per annum. It will be interesting to see how the Pi behaves as it's database grows (just from a performance point of view), assuming the SD card doesn't nuke itself :lol:
I do have an (almost) "no writes to the SD card option" available if anyone wants to have a play .... ("almost" as it only adds to an error log if something unforeseen happens). Shout if interested, and I will make it publicly available.
-
I do have an (almost) "no writes to the SD card option" available if anyone wants to have a play .... ("almost" as it only adds to an error log if something unforeseen happens). Shout if interested, and I will make it publicly available.
This is a great idea, I hope you will release it publicly. :)
I have no problem with sdcard, since I have a NAS, with a central mysql tha hosts the DBs of all the software I use. I was also thinking if the software could actually be ported so I could run it directly on the NAS, which is the only computer always turned on in my house. If it's not possible I'll turn on the raspberry. :)
I use a Netgear DGN2200 router (as a pure ADSL modem/bridge), please find attached the log as requested, with telnet authenticaion and all the adsl commands required. Please take note the in the prompt string you will find the hostname, it's not the classic, plain-style prompt.
Thanks for your work.
-
Sounds interesting to me too. :)
I have a Pi connected to a Billion 7800 and a Netgear DGND3700
with scripts collecting the modem data from them.
Its all modular so you would be welcome to pick up any pieces that might be of use.
At The moment:
cron calls a script which interrogates the modem via telnet > modem_out_file
A script parses modem_out_file and to extract the data of interest.
optionally script functions can split out into individual files and create graphs
optionally script functions can archive selected data files
optionally script functions can archive the collected data
optionally script functions can extract data from the archives
PHP classes written & running under Eclipse remote debugger / nginx / php-fpm to access the data.
;-)
G.DMT, since I noticed this GUI-less project is not being brought ahead, actually I found your post very interesting.
What I hoped existed was a php/web/python solution, that queried the modem/router, gathered the output, parsed it and then send data to MySQL and to MDWS (it would be great if it also showed a GUI with graphs like MDWS does, in case a user does not want to send data externally).
PHP/Python solution could be easily run on a home nas, or raspberry, or another device that is always turned on and it would be accessible via any browser to analyze data, also remotely.
Reading your post, I thought what you developed could be a good start. :)
-
hi 'Axel'
thanks for the interest! :)
Reading your post, I thought what you developed could be a good start. :)
Since I posted that I have not had the spare free time to proceed further with the modem data logger.
It accomplishes what I needed - which was to log the modem data every 60 seconds.
When I have encountered problems with my line, I have been able to comb through the archived data to find out what happened.
28 days uptime :clap: ;D (since the last powercut) so I just live with the intermittent loud crackling on the line that BT refuse to sort out.
Your post has prompted me to take a quick look at how it has been performing.
In the 14 weeks since last I rotated the log - it has logged 138503 modem data sets.
Here is a quick look at the first and last summary stats. ;)
[user@K8 scripts]$ ./view_modem_stats.sh -n
138503
[user@K8 scripts]$ ./view_modem_stats.sh -l | head -n 4
-rw-r--r-- 1000/1000 637 2014-10-10 13:34 stats_2014-10-10_13:34:01.log.lzma
-rw-r--r-- 1000/1000 635 2014-10-10 13:35 stats_2014-10-10_13:35:01.log.lzma
-rw-r--r-- 1000/1000 635 2014-10-10 13:36 stats_2014-10-10_13:36:01.log.lzma
-rw-r--r-- 1000/1000 634 2014-10-10 13:37 stats_2014-10-10_13:37:01.log.lzma
[user@K8 scripts]$ ./view_modem_stats.sh -f stats_2014-10-10_13:34:01.log.lzma
DATE_STAMP: 2014-10-10 13:34:01
SESSION_START: 2014-10-06 17:04:22
LINK_TIME: 3 days 20 hours 29 min 39 sec
DATE_NOW: 2014-10-10 13:34:06
SNTP date: Tue Jan 4 22:06:34 2000
UPTIME: 3 days
LINE_STATUS: No Defect
RETRAIN_REASON: 0
RETRAIN_STRING: unknown
DATA_RATE_D: 4576, DATA_RATE_U: 448
MAX_ATTAIN_D: 5152, MAX_ATTAIN_U: 992
SNR_D: 5.8,SNR_UP: 18.0
LATN_D: 61.0,LATN_UP: 31.5
SATN_D: 61.0,SATN_UP: 0.0
TxPWR_D: 19.3,TxPWR_UP: 12.5
ADSL_TRAINING_MARGIN: 65526
K_D: 144(0),k_UP: 15
R_D: 16,R_UP: 16
S_D: 1,S_UP: 8
D_D: 32,D_UP: 4
SF_D: 19586937,SF_UP: 19587039
RS_D: 1331911746,RS_UP: 166488828
RSCorr_D: 1715762,RSCorr_UP: 1
RSUnCorr_D: 44997,RSUnCorr_UP: 0
HEC_D: 1499,HEC_UP: 0
OCD_D: 113,OCD_UP: 0
LCD_D: 0,LCD_UP: 0
Total_Cells_D: 3593495682,Total_Cells_UP: 0
Data_Cells_D: 28943909,Data_Cells_UP: 0
Drop_Cells_D: 44
Bit_Errors_D: 0,Bit_Errors_UP: 0
TOTAL_ES_D: 676,ES_UP: 0
TOTAL_SES_D: 13,SES_UP: 0
TOTAL_UAS_D: 206,UAS_UP: 206
AS_D: 332980
INP_D: 1.60,INP_UP: 1.88
PER_D: 1.75,PER_UP: 1.75
delay_D: 8.00,delay_UP: 8.00
OR_D: 32.00,OR_UP: 32.00
Bitswap_D: 27719,Bitswap_UP: 71
FEC_D: 1715762,FEC_UP: 1
CRC_D: 1670,CRC_UP: 1
ES_D: 665,ES_UP: 0
SES_D: 13,SES_UP: 0
UAS_D: 0,UAS_UP: 0
LOS_D: 0,LOS_UP: 0
LOF_D: 0,LOF_UP: 0
[user@K8 scripts]$ ./view_modem_stats.sh
DATE_STAMP: 2015-01-16 19:11:01
SESSION_START: 2014-12-19 08:22:57
LINK_TIME: 28 days 10 hours 48 min 4 sec
DATE_NOW: 2015-01-16 19:11:06
SNTP date: Sat Jan 29 10:48:28 2000
UPTIME: 28 days
LINE_STATUS: No Defect
RETRAIN_REASON: 0
DATA_RATE_D: 4576, DATA_RATE_U: 448
MAX_ATTAIN_D: 5120, MAX_ATTAIN_U: 1112
SNR_D: 5.6,SNR_UP: 22.0
ADSL_TRAINING_MARGIN: 65526
LATN_D: 60.0,LATN_UP: 31.5
SATN_D: 60.0,SATN_UP: 0.0
TxPWR_D: 18.8,TxPWR_UP: 12.3
K_D: 144(0),k_UP: 15
R_D: 16,R_UP: 16
S_D: 1,S_UP: 8
D_D: 32,D_UP: 4
SF_D: 144592892,SF_UP: 144593199
RS_D: 1242382094,RS_UP: 155300367
RSCorr_D: 16979195,RSCorr_UP: 4
RSUnCorr_D: 529483,RSUnCorr_UP: 0
HEC_D: 18018,HEC_UP: 4
OCD_D: 1186,OCD_UP: 3
LCD_D: 0,LCD_UP: 0
Total_Cells_D: 757475141,Total_Cells_UP: 0
Data_Cells_D: 431680065,Data_Cells_UP: 0
Drop_Cells_D: 0
Bit_Errors_D: 0,Bit_Errors_UP: 0
TOTAL_ES_D: 9554,ES_UP: 0
TOTAL_SES_D: 209,SES_UP: 0
TOTAL_UAS_D: 8,UAS_UP: 8
AS_D: 2458086
INP_D: 1.60,INP_UP: 1.88
PER_D: 1.75,PER_UP: 1.75
delay_D: 8.00,delay_UP: 8.00
OR_D: 32.00,OR_UP: 32.00
Bitswap_D: 99743,Bitswap_UP: 7
FEC_D: 16979195,FEC_UP: 4
CRC_D: 21130,CRC_UP: 4
ES_D: 9554,ES_UP: 0
SES_D: 209,SES_UP: 0
UAS_D: 0,UAS_UP: 0
LOS_D: 0,LOS_UP: 0
LOF_D: 0,LOF_UP: 0
phyR_D: On,phyR_UP: On
And a look at the log of the rest of the data split out by modem command.
[user@K8 scripts]$ ./get_modem_files.sh -l | head -n 4
-rw-r--r-- 1000/1000 1714 2014-10-10 13:34 files_2014-10-10_13:34:01.tar.lzma
-rw-r--r-- 1000/1000 1712 2014-10-10 13:35 files_2014-10-10_13:35:01.tar.lzma
-rw-r--r-- 1000/1000 1710 2014-10-10 13:36 files_2014-10-10_13:36:01.tar.lzma
-rw-r--r-- 1000/1000 1712 2014-10-10 13:37 files_2014-10-10_13:37:01.tar.lzma
[colin@K8 scripts]$ ./get_modem_files.sh -f files_2014-10-10_13:34:01.tar.lzma | tar --list
summary.txt
sysinfo.txt
adsl_info--stats.txt
counters.txt
link_errors.txt
adsl_info--vendor.txt
adsl_info--cfg.txt
adsl_profile--show.txt
adsl--version.txt
[user@K8 scripts]$ ./get_modem_files.sh -f files_2014-10-10_13:34:01.tar.lzma | tar xO adsl_info--stats.txt
adsl: ADSL driver and PHY status
Status: Showtime
Retrain Reason: 0
Max: Upstream rate = 992 Kbps, Downstream rate = 5152 Kbps
Channel: INTR, Upstream rate = 448 Kbps, Downstream rate = 4576 Kbps
Link Power State: L0
Mode: G.DMT
TPS-TC: ATM Mode
Trellis: ON
Line Status: No Defect
Training Status: Showtime
Down Up
SNR (dB): 5.8 18.0
Attn(dB): 61.0 31.5
Pwr(dBm): 19.3 12.5
G.dmt framing
K: 144(0) 15
R: 16 16
S: 1 8
D: 32 4
Counters
SF: 19586937 19587039
SFErr: 1670 1
RS: 1331911746 166488828
RSCorr: 1715762 1
RSUnCorr: 44997 0
HEC: 1499 0
OCD: 113 0
LCD: 0 0
Total Cells: 3593495682 0
Data Cells: 28943909 0
Drop Cells: 44
Bit Errors: 0 0
ES: 676 0
SES: 13 0
UAS: 206 206
AS: 332980
INP: 1.60 1.88
PER: 1.75 1.75
delay: 8.00 8.00
OR: 32.00 32.00
Bitswap: 27719 71
Total time = 1 days 22 hours 6 min 23 sec
FEC: 1735004 0
CRC: 1684 0
ES: 676 0
SES: 13 0
UAS: 206 206
LOS: 0 0
LOF: 0 0
Latest 15 minutes time = 6 min 23 sec
FEC: 1862 0
CRC: 2 0
ES: 2 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
Previous 15 minutes time = 15 min 0 sec
FEC: 3959 0
CRC: 2 0
ES: 2 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
Latest 1 day time = 22 hours 6 min 23 sec
FEC: 436716 0
CRC: 304 0
ES: 132 0
SES: 2 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
Previous 1 day time = 24 hours 0 sec
FEC: 406552 0
CRC: 565 0
ES: 265 0
SES: 3 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
Since Link time = 3 days 20 hours 29 min 39 sec
FEC: 1715762 1
CRC: 1670 1
ES: 665 0
SES: 13 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
The intention was for it to be as generic as possible hence it doesn't need to be compiled for a specific target platform and Obviously you would need to have a compatible modem (Broadcom), ATM the code that parses the raw text file from the modem has only been tested on the output from my Billion 7800 ;)
Do you have experience with unix shell scripting or PHP?
If so it would relatively straightforward to finish the graphing tools- I just haven't the time ATM
plus tbaileys GREAT MDWS project has that mostly covered now too.
-
The intention was for it to be as generic as possible hence it doesn't need to be compiled for a specific target platform and Obviously you would need to have a compatible modem (Broadcom), ATM the code that parses the raw text file from the modem has only been tested on the output from my Billion 7800 ;)
Do you have experience with unix shell scripting or PHP?
If so it would relatively straightforward to finish the graphing tools- I just haven't the time ATM
plus tbaileys GREAT MDWS project has that mostly covered now too.
Thanks for answering. :)
Let's say that it would be great to have a php/web solution that grabbed data every 60 secs and sent it to MDWS (don't know what format is required). It would cover the top priority (for me) that is having an history of stats, running on an always-on device like my NAS.
I have a Netgear DGN2200v4 (broadcom), that works perfectly with the latest DSLstats for windows. But I can't leave the notebook on all the time. If the DSLstats version for raspberry would have worked in "headless mode" it would have been a possible solution for me, but I read that's not planned for now.
Unfortunately I'm not a dev. I can work with shell, it depends on the level of complexity of the task though. :)
Thanks a lot for your support.
Unfortunately
-
I have a Netgear DGN2200v4 (broadcom), that works perfectly with the latest DSLstats for windows. But I can't leave the notebook on all the time. If the DSLstats version for raspberry would have worked in "headless mode" it would have been a possible solution for me, but I read that's not planned for now.
Unfortunately I'm not a dev. I can work with shell, it depends on the level of complexity of the task though. :)
Fortunately, using the information that you have supplied so far I can recommend solutions that should meet your requirements. ;)
Given
1)You have a RasPi, a NAS, a Netgear DGN2200v4 and a notebook.
2)You do not wish to run your main computer 24/7
3) You stated that you wish to run your Pi 'headless' which I shall assume to mean that you don't have a screen plugged in.
Given the above , I can see 2 'easy' choices for you (and 1 not so easy one)
1) If your 'main computer' is stuffed full of Linux gooodness, just use remote Xwindows for a transparent network enabled remote desktop experience from your PC to your Pi.
2) If your 'main computer' is cursed with WinBlows proprietary insecurity Disaster Area :P
install a VNC server on the Pi and a VNC Viewer on the PC.
3) (The tougher option) wrangle shell at the command line over SSH- it Works For Me! ;)
Since :
aptitude search vnc | wc -l
47
There would appear to be plenty of options to explore!
-
Thanks for the advices, I really appreciate it.
Will think about it, but who knows, this could be the chance to start playing with python or php and start a new project. :D
I know about VNC and XWindow for RPi, but it would just be a sort of "workaround", and that's not what I was thinking of, but as a temporary solution it's more than enough.
I'm running a few monitoring sw on the NAS because, together with the router and the switch, it's the only device always on. It's a pretty good Linux server (Synology DS1812+ with 32TB of raw space), able to run all sorts of software, it would be a pity to deserve a raspberry just for one function.
So I'll go for the NAS...let's see how...:)
Thanks again.
-
Will think about it, but who knows, this could be the chance to start playing with python or php and start a new project. :D
I'm running a few monitoring sw on the NAS because, together with the router and the switch, it's the only device always on. It's a pretty good Linux server (Synology DS1812+ with 32TB of raw space), able to run all sorts of software, it would be a pity to deserve a raspberry just for one function.
Synology DS1812+
Ah I see where you are coming from now.
http://www.anandtech.com/show/7071/synology-ds1812-8bay-smb-soho-nas-review
Nice piece of kit - I would probably try to get it working on there too ;-)
I am curious about how easy it will be to provide a basic 'development' environment on your NAS.
To me the greatest benefit of using a Pi (apart from low cost, and silent always-on, low power operation of course) is that the OS is based on Debian- for which there are 30,000+ open source software packages available to run on it- and most of them are available just by typing:
'aptitude install packagename'
You can expect most any 'normal' Linux distro to have shell tools like awk,sed,cut,head,tail,grep,tar,xv installed and
TBH if you can't easily install pcregrep and expect then attempting to do any more than fairly trivial ash/bash/dash scripting is probably going to be difficult.
Is there a similar facility available for the synology? If yes, can you post the link?
If not - you may be starting with a mountain to climb needlessly.
Best of luck to you - and have fun. ;-)
-
It's not a "closed" OS. DSM (DiskStationManager, Synology's OS), is Debian based. ;)
I checked all the tools you mentioned: I have all of them installed (either already installed in DSM, or installed via ipkg), except xv, that I don't even know what's for. ;)
There's an open community (SynoCommunity) which gathers a lot of native packages compiled specifically for Synology NAS's, using a cross compilation framework that you can find here: https://github.com/SynoCommunity/spksrc
So, apart all the great things already existing for DSM, if you really need a Debian env for packages, you can install one of the packages made for DSM, called Debian Chroot, here's an abstract of the help file (https://github.com/SynoCommunity/spksrc/blob/develop/spk/debian-chroot/src/app/help/enu/index.html):
Debian Chroot is a free operating system (OS) that comes with over 29000 packages, precompiled software bundled up in a nice format for easy installation on your DiskStation. Debian Chroot allows you to benefit from the Debian OS inside your DiskStation, alongside DSM. This package is intended for advanced users only.
So you can have lots of fun if you're a Linux dev/expert, which I'm not, but I'm having fun learning new things anyway. :)
-
Thanks for the Info and Links. :)
It sounds like you will have many of the facilities available for your NAS that you would have for a Pi.
Unfortunately https://synocommunity.com/ seem to be having issues with their server, so I wasn't able to check that out today.
Due to performance issues, the server will be down for maintenance until 2015-01-22.
I checked all the tools you mentioned: I have all of them installed (either already installed in DSM, or installed via ipkg), except xv, that I don't even know what's for. ;)
:blush: Oops, no wonder you couldn't find 'xv'- since xv is a mistake. It should say xz. :-[
Apologoies for the wild-goose chase there!
https://packages.debian.org/stable/utils/xz-utils
-
Yes unfortunately they're having some performance issues on the server, the community is getting too big. :)
We're covere also for xz-utils, probably not the latest since ipkg repository is not updated anymore (it's time to switch to debian chroot maybe):
[Sun Jan 18 18:47:57 root@nas:~]# ipkg list | grep xz
xz-utils - 5.0.5-1 - A free general-purpose data compression software with high compression ratio
[Sun Jan 18 18:48:03 root@nas:~]#
I was thinking, since DSLstats is also available for linux, if it would run under Debian chroot environment...didn't find prerequirements on the help page.
-
I was thinking, since DSLstats is also available for linux, if it would run under Debian chroot environment...didn't find prerequirements on the help page.
I expect so, although I have no experience of using chroot environments. DSLstats has few requirements - just the gtk2 libraries mainly. If you start it from the command line you should get an error message if any necessary libraries are missing.
-
I was thinking, since DSLstats is also available for linux, if it would run under Debian chroot environment...didn't find prerequirements on the help page.
I expect so, although I have no experience of using chroot environments. DSLstats has few requirements - just the gtk2 libraries mainly. If you start it from the command line you should get an error message if any necessary libraries are missing.
I will try it later and let you know. It would be good to have a solution running also on NAS, they're quite popular and, to my knowledge, chroot environment is available both on Synology and QNAP.
Eric, can I ask you if it would be a huge work to have a GUI-less version of DSLstats? I was thinking about passing a cmdline parameter (--nogui) and dslstats would just run the collection of data, creation of datafiles and upload to MDWS, skipping all the rest.
Thanks for the help, you guys are really great. :)
-
I'm afraid that a GUI-free version of DSLstats would need a substantial rewrite, and there just aren't enough hours in the day...
-
Thanks...that's what I suspected, unfortunately. :(
-
I will try it later and let you know. It would be good to have a solution running also on NAS, they're quite popular and, to my knowledge, chroot environment is available both on Synology and QNAP.
I may have inadvertently misled you with my previous reply. DSLstats is written from the ground up as a GUI program, so if you want to run it in a chroot environment, that environment will still need to have a GUI.
-
Ok, here's the status:
First try:
./dslstats: error while loading shared libraries: libgdk_pixbuf-2.0.so.0: cannot open shared object file: No such file or directory
solved with:
aptitude install libgdk-pixbuf2.0-0
Second try:
./dslstats: error while loading shared libraries: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
solved with:
aptitude install libgtk2.0-0
Third try seems way better:
(dslstats:22998): Gtk-WARNING **: cannot open display:
There's no display attached to the NAS. But I guess there's some way to redirect the output to VNC or, better, RDP.
Any advice on this would be greatly appreciated, in the meanwhile I'll google it. (I know, I'm a noob on linux, but I'm learning...:D)
-
There's no display attached to the NAS. But I guess there's some way to redirect the output to VNC or, better, RDP.
Any advice on this would be greatly appreciated, in the meanwhile I'll google it. (I know, I'm a noob on linux, but I'm learning...
Does the NAS have a monitor output? DSLstats can only run in a GUI environment - if there isn't one it won't start.
-
yes it has a VGA output, but I never used it. I'm trying to make it work via VNC or, better, RDP.
will let you know if I manage to make it work.
-
It's working. ;)
At the end of the post I put a screenshot of the Remote Desktop Connection (RDP) session from my Win8.1 notebook, you will see the terminal session from which I launch dslstats, a few lines are output to console (I have no idea of what those DUPLICATE RESOURCE messages are) and then the DSLstats GUI comes up, all running on my Synology NAS, with no display connected.
In order to make it work, after lots of tries, I had to install & configure the following:
- Debian Chroot package from SynoCommunity, once that is configured and working, from the chrooted domain, you install & configure the following packages:
aptitude install libgtk2.0-0
aptitude install libgdk-pixbuf2.0-0
aptitude install lxde-core
aptitude install xrdp
Hope this helps somebody else who has the same need. :)
Thanks for the help.
(http://i.imgur.com/jrfRNML.png)
-
Fantastic! Well done. :clap2:
-
Eric, can I safely ignore those DUPLICATE RESOURCE messages I see in the console when starting dslstats?
-
Yes, those messages are normal and don't indicate anything wrong.
-
Great. :)
Now I'm only trying to make the whole thing (chroot, xdg, and dslstats) autostart when the NAS is rebooted. ;)
I hope one day you decide to start a GUI-less version of DSLstats, or to add it inside the software through a command-line parameter. :D
Thanks for all the help.
-
I am interested and am already running a script to access the stats on my Billion 8800. My log format is as follows:
Time Status Sync kbps----- SNR (dB) ----- Atten (dB) --- Power (dBm)--- FEC (15 min)-- CRC (15 min)-- ES (15 min)--
Down Up Down Up Down Up Down Up Down Up Down Up Down Up
00:00:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 3 6 0 4 0
00:05:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 6 3 1 2 1
00:10:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 6 3 1 2 1
00:15:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 6 3 1 2 1
00:20:02 Showtime 56614 14361 6.1 5.6 19.1 0.0 13.1 7.4 0 18 1 4 1 3
00:25:02 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 18 1 4 1 3
00:30:02 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 18 1 4 1 3
00:35:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 0 5 0 4 0
00:40:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 0 5 0 4 0
00:45:01 Showtime 56614 14361 6.1 5.7 19.1 0.0 13.1 7.4 0 0 5 0 4 0
Happy to share my script for constructive criticism ;D
Just found this post again after losing it yesterday... Has there been any development on this? I'd like to see if I can get it to work with my HG533.
Thanks
-
Just found this post again after losing it yesterday... Has there been any development on this? I'd like to see if I can get it to work with my HG533.
Thanks
Hi cancunia.
As it happens I have an archived copy of the bash scripts that I wrote back then.
https://forum.kitz.co.uk/index.php/topic,14595.msg278113.html#msg278113
They provide separate commands to schedule, access, read, parse, graph, save, archive, retrieve and watch the broadcom chipset data from a (Billion) xDSL Modem.
It should be pretty simple to resurrect them, if you are not afraid of wrangling some bash script, can read PCRE regexes, and maybe a bit of gawk or expect.
;-)
-
Hi cancunia.
As it happens I have an archived copy of the bash scripts that I wrote back then.
https://forum.kitz.co.uk/index.php/topic,14595.msg278113.html#msg278113
They provide separate commands to schedule, access, read, parse, graph, save, archive, retrieve and watch the broadcom chipset data from a (Billion) xDSL Modem.
It should be pretty simple to resurrect them, if you are not afraid of wrangling some bash script, can read PCRE regexes, and maybe a bit of gawk or expect.
;-)
Hi, Yeah, I'll have a poke around and see what I can make of it. Please send me a link or an attachment or however is best for you.
Also, I wondered if this one is still available?
If anyone is interested in trying this, the first beta release is out
- Tested on Billion 8800NL in VDSL mode
- Untested on Billion 8800NL in ADSL mode (I could do with some logs for this if anyone tries it - not sure how the ADSL stats output of the router is different to the VDSL stats output )
- 1/2 tested on a Billion 7800N (1/2 tested as I don't have a adsl connection anymore to test it fully)
Any other routers are unsupported at the moment.
Instructions and download link are here www.xdaemonstats.mooo.com/Docs (http://www.xdaemonstats.mooo.com/Docs)
It has simple functionality at the moment, but I want it tested by more than me afore it progresses to the more complicated areas.
On a related note, I could really do with a telnet dump of the HG612's stats output if anyone would be so kind (including initial login/password prompts and the output of xdslcmd info --stats) . This would be sorely appreciated :)
Any questions, problems, comments, please ask ......
-
Hi, Yeah, I'll have a poke around and see what I can make of it. Please send me a link or an attachment or however is best for you.
I did not put any licensing on any of the source files that I wrote.
For fair public attribution and out of courtesy to and respect for he who originaly inspired me to do this, I shall reproduce here the comments from the single file that does bear licensing information.
file /Broadcom/generate_graphs.sh
#!/bin/bash
## This shell code in this file was inspired by and derived from GPL code
## which was Originally written by Burakkucat from forum.kitz.co.uk
## see: http://elrepo.org/people/ajb/HG6xx_stats/
##
## 2014: Separated into functions for ease of integration.
p.s. Hi to all - I had been away from this forum for a some time, since a VDSL cabinet finally arrived!
Great to see everyone still going strong. :-)
Please note this was never originally intended to be 'published' and the zip file is just a hastily cobbled together archive for posterity.
Disclaimer: I'm a bit embarrassed by some of it now as writing it was part of a learning curve and many of the implementation decisions that I made then, I would not make in the same way now.
Caveat emptor!
:-)
[Update1:]
I see now
Restrictions: 3 per post, maximum total size 900KB, maximum individual size 300KB
Zip file is 3.7M so I would have to cherry pick the important bits and dump archived data files.
[Update 2: slimmed down attachment added]
-
Thanks for posting, I'll take a look through there over the next few days.