Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[G.DMT]

hi 'Axel'

 thanks for the interest! 

Reading your post, I thought what you developed could be a good start.

Since I posted that I have not had the spare free time to proceed further with the modem data logger.

It accomplishes what I needed - which was to log the modem data every 60 seconds.
When I have encountered problems with my line, I have been able to comb through the archived data to find out what happened.

28 days uptime  (since the last powercut) so I just live with the intermittent loud crackling on the line that BT refuse to sort out.

Your post has prompted me to take a quick look at how it has been performing.

In the 14 weeks since last I rotated the log - it has logged 138503 modem data sets.

Here is a quick look at the first and last summary stats. 

Code: [Select]

[user@K8 scripts]$ ./view_modem_stats.sh -n
138503
[user@K8 scripts]$ ./view_modem_stats.sh -l | head -n 4
-rw-r--r-- 1000/1000       637 2014-10-10 13:34 stats_2014-10-10_13:34:01.log.lzma
-rw-r--r-- 1000/1000       635 2014-10-10 13:35 stats_2014-10-10_13:35:01.log.lzma
-rw-r--r-- 1000/1000       635 2014-10-10 13:36 stats_2014-10-10_13:36:01.log.lzma
-rw-r--r-- 1000/1000       634 2014-10-10 13:37 stats_2014-10-10_13:37:01.log.lzma

[user@K8 scripts]$ ./view_modem_stats.sh -f stats_2014-10-10_13:34:01.log.lzma
DATE_STAMP: 2014-10-10 13:34:01
SESSION_START: 2014-10-06 17:04:22
LINK_TIME: 3 days 20 hours 29 min 39 sec
DATE_NOW: 2014-10-10 13:34:06
SNTP date: Tue Jan  4 22:06:34 2000
UPTIME: 3 days

LINE_STATUS: No Defect
RETRAIN_REASON: 0
RETRAIN_STRING: unknown
DATA_RATE_D: 4576, DATA_RATE_U: 448
MAX_ATTAIN_D: 5152, MAX_ATTAIN_U: 992
SNR_D: 5.8,SNR_UP: 18.0
LATN_D: 61.0,LATN_UP: 31.5
SATN_D: 61.0,SATN_UP: 0.0
TxPWR_D: 19.3,TxPWR_UP: 12.5
ADSL_TRAINING_MARGIN: 65526
K_D: 144(0),k_UP: 15
R_D: 16,R_UP: 16
S_D: 1,S_UP: 8
D_D: 32,D_UP: 4
SF_D: 19586937,SF_UP: 19587039
RS_D: 1331911746,RS_UP: 166488828
RSCorr_D: 1715762,RSCorr_UP: 1
RSUnCorr_D: 44997,RSUnCorr_UP: 0
HEC_D: 1499,HEC_UP: 0
OCD_D: 113,OCD_UP: 0
LCD_D: 0,LCD_UP: 0
Total_Cells_D: 3593495682,Total_Cells_UP: 0
Data_Cells_D: 28943909,Data_Cells_UP: 0
Drop_Cells_D: 44
Bit_Errors_D: 0,Bit_Errors_UP: 0
TOTAL_ES_D: 676,ES_UP: 0
TOTAL_SES_D: 13,SES_UP: 0
TOTAL_UAS_D: 206,UAS_UP: 206
AS_D: 332980
INP_D: 1.60,INP_UP: 1.88
PER_D: 1.75,PER_UP: 1.75
delay_D: 8.00,delay_UP: 8.00
OR_D: 32.00,OR_UP: 32.00
Bitswap_D: 27719,Bitswap_UP: 71
FEC_D: 1715762,FEC_UP: 1
CRC_D: 1670,CRC_UP: 1
ES_D: 665,ES_UP: 0
SES_D: 13,SES_UP: 0
UAS_D: 0,UAS_UP: 0
LOS_D: 0,LOS_UP: 0
LOF_D: 0,LOF_UP: 0

[user@K8 scripts]$ ./view_modem_stats.sh
DATE_STAMP: 2015-01-16 19:11:01
SESSION_START: 2014-12-19 08:22:57
LINK_TIME: 28 days 10 hours 48 min 4 sec
DATE_NOW: 2015-01-16 19:11:06
SNTP date: Sat Jan 29 10:48:28 2000
UPTIME: 28 days

LINE_STATUS: No Defect
RETRAIN_REASON: 0
DATA_RATE_D: 4576, DATA_RATE_U: 448
MAX_ATTAIN_D: 5120, MAX_ATTAIN_U: 1112
SNR_D: 5.6,SNR_UP: 22.0
ADSL_TRAINING_MARGIN: 65526
LATN_D: 60.0,LATN_UP: 31.5
SATN_D: 60.0,SATN_UP: 0.0
TxPWR_D: 18.8,TxPWR_UP: 12.3
K_D: 144(0),k_UP: 15
R_D: 16,R_UP: 16
S_D: 1,S_UP: 8
D_D: 32,D_UP: 4
SF_D: 144592892,SF_UP: 144593199
RS_D: 1242382094,RS_UP: 155300367
RSCorr_D: 16979195,RSCorr_UP: 4
RSUnCorr_D: 529483,RSUnCorr_UP: 0
HEC_D: 18018,HEC_UP: 4
OCD_D: 1186,OCD_UP: 3
LCD_D: 0,LCD_UP: 0
Total_Cells_D: 757475141,Total_Cells_UP: 0
Data_Cells_D: 431680065,Data_Cells_UP: 0
Drop_Cells_D: 0
Bit_Errors_D: 0,Bit_Errors_UP: 0
TOTAL_ES_D: 9554,ES_UP: 0
TOTAL_SES_D: 209,SES_UP: 0
TOTAL_UAS_D: 8,UAS_UP: 8
AS_D: 2458086
INP_D: 1.60,INP_UP: 1.88
PER_D: 1.75,PER_UP: 1.75
delay_D: 8.00,delay_UP: 8.00
OR_D: 32.00,OR_UP: 32.00
Bitswap_D: 99743,Bitswap_UP: 7
FEC_D: 16979195,FEC_UP: 4
CRC_D: 21130,CRC_UP: 4
ES_D: 9554,ES_UP: 0
SES_D: 209,SES_UP: 0
UAS_D: 0,UAS_UP: 0
LOS_D: 0,LOS_UP: 0
LOF_D: 0,LOF_UP: 0
phyR_D: On,phyR_UP: On


 And a look at the log of the rest of the data split out by modem command.

Code: [Select]

[user@K8 scripts]$ ./get_modem_files.sh -l | head -n 4 
-rw-r--r-- 1000/1000      1714 2014-10-10 13:34 files_2014-10-10_13:34:01.tar.lzma
-rw-r--r-- 1000/1000      1712 2014-10-10 13:35 files_2014-10-10_13:35:01.tar.lzma
-rw-r--r-- 1000/1000      1710 2014-10-10 13:36 files_2014-10-10_13:36:01.tar.lzma
-rw-r--r-- 1000/1000      1712 2014-10-10 13:37 files_2014-10-10_13:37:01.tar.lzma
[colin@K8 scripts]$ ./get_modem_files.sh -f files_2014-10-10_13:34:01.tar.lzma | tar --list
summary.txt
sysinfo.txt
adsl_info--stats.txt
counters.txt
link_errors.txt
adsl_info--vendor.txt
adsl_info--cfg.txt
adsl_profile--show.txt
adsl--version.txt
[user@K8 scripts]$ ./get_modem_files.sh -f files_2014-10-10_13:34:01.tar.lzma |  tar xO adsl_info--stats.txt
adsl: ADSL driver and PHY status
Status: Showtime
Retrain Reason: 0
Max:    Upstream rate = 992 Kbps, Downstream rate = 5152 Kbps
Channel:        INTR, Upstream rate = 448 Kbps, Downstream rate = 4576 Kbps

Link Power State:       L0
Mode:                   G.DMT
TPS-TC:                 ATM Mode
Trellis:                ON
Line Status:            No Defect
Training Status:        Showtime
                Down            Up
SNR (dB):        5.8             18.0
Attn(dB):        61.0            31.5
Pwr(dBm):        19.3            12.5
                        G.dmt framing
K:              144(0)          15
R:              16              16
S:              1               8
D:              32              4
                        Counters
SF:             19586937                19587039
SFErr:          1670            1
RS:             1331911746              166488828
RSCorr:         1715762         1
RSUnCorr:       44997           0

HEC:            1499            0
OCD:            113             0
LCD:            0               0
Total Cells:    3593495682              0
Data Cells:     28943909                0
Drop Cells:     44
Bit Errors:     0               0

ES:             676             0
SES:            13              0
UAS:            206             206
AS:             332980

INP:            1.60            1.88
PER:            1.75            1.75
delay:          8.00            8.00
OR:             32.00           32.00

Bitswap:        27719           71

Total time = 1 days 22 hours 6 min 23 sec
FEC:            1735004         0
CRC:            1684            0
ES:             676             0
SES:            13              0
UAS:            206             206
LOS:            0               0
LOF:            0               0
Latest 15 minutes time = 6 min 23 sec
FEC:            1862            0
CRC:            2               0
ES:             2               0
SES:            0               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
Previous 15 minutes time = 15 min 0 sec
FEC:            3959            0
CRC:            2               0
ES:             2               0
SES:            0               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
Latest 1 day time = 22 hours 6 min 23 sec
FEC:            436716          0
CRC:            304             0
ES:             132             0
SES:            2               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
Previous 1 day time = 24 hours 0 sec
FEC:            406552          0
CRC:            565             0
ES:             265             0
SES:            3               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
Since Link time = 3 days 20 hours 29 min 39 sec
FEC:            1715762         1
CRC:            1670            1
ES:             665             0
SES:            13              0
UAS:            0               0
LOS:            0               0
LOF:            0               0

The intention was for it to be as generic as possible hence it doesn't need to be compiled for a specific target platform and Obviously you would need to have a compatible modem (Broadcom), ATM  the code that parses the raw text file from the modem has only been tested on the output from my Billion 7800 
 
Do you have experience with unix shell scripting or PHP?
If so it would relatively straightforward to finish the graphing tools- I just haven't the time ATM

plus tbaileys GREAT MDWS project has that mostly covered now too.

[Axel Foley]

The intention was for it to be as generic as possible hence it doesn't need to be compiled for a specific target platform and Obviously you would need to have a compatible modem (Broadcom), ATM  the code that parses the raw text file from the modem has only been tested on the output from my Billion 7800 
 
Do you have experience with unix shell scripting or PHP?
If so it would relatively straightforward to finish the graphing tools- I just haven't the time ATM

plus tbaileys GREAT MDWS project has that mostly covered now too.

Thanks for answering.

Let's say that it would be great to have a php/web solution that grabbed data every 60 secs and sent it to MDWS (don't know what format is required). It would cover the top priority (for me) that is having an history of stats, running on an always-on device like my NAS.

I have a Netgear DGN2200v4 (broadcom), that works perfectly with the latest DSLstats for windows. But I can't leave the notebook on all the time. If the DSLstats version for raspberry would have worked in "headless mode" it would have been a possible solution for me, but I read that's not planned for now.

Unfortunately I'm not a dev. I can work with shell, it depends on the level of complexity of the task though.

Thanks a lot for your support.



Unfortunately

[G.DMT]

I have a Netgear DGN2200v4 (broadcom), that works perfectly with the latest DSLstats for windows. But I can't leave the notebook on all the time. If the DSLstats version for raspberry would have worked in "headless mode" it would have been a possible solution for me, but I read that's not planned for now.

Unfortunately I'm not a dev. I can work with shell, it depends on the level of complexity of the task though.

Fortunately, using the information that you have supplied so far I can recommend solutions that should meet your requirements. 
Given
1)You have a RasPi, a NAS, a Netgear DGN2200v4 and a notebook.
2)You do not wish to run your main computer 24/7
3) You stated that you wish to run your Pi 'headless' which I shall assume to mean that you don't have a screen plugged in.

Given the above , I can see 2 'easy' choices for you (and 1 not so easy one)

1) If your 'main computer' is stuffed full of Linux gooodness, just use remote Xwindows for a transparent network enabled remote desktop experience from your PC to your Pi.

2) If your 'main computer' is cursed with WinBlows proprietary insecurity Disaster Area 
  install a VNC server on the Pi and a VNC Viewer on the PC.

3) (The tougher option) wrangle shell at the command line over SSH- it Works For Me! 

Since :
aptitude search vnc | wc -l
47

There would appear to be plenty of options to explore!
 

[Axel Foley]

Thanks for the advices, I really appreciate it.

Will think about it, but who knows, this could be the chance to start playing with python or php and start a new project.

I know about VNC and XWindow for RPi, but it would just be a sort of "workaround", and that's not what I was thinking of, but as a temporary solution it's more than enough.

I'm running a few monitoring sw on the NAS because, together with the router and the switch, it's the only device always on. It's a pretty good Linux server (Synology DS1812+ with 32TB of raw space), able to run all sorts of software, it would be a pity to deserve a raspberry just for one function.

So I'll go for the NAS...let's see how...

Thanks again.

[G.DMT]

Will think about it, but who knows, this could be the chance to start playing with python or php and start a new project.

I'm running a few monitoring sw on the NAS because, together with the router and the switch, it's the only device always on. It's a pretty good Linux server (Synology DS1812+ with 32TB of raw space), able to run all sorts of software, it would be a pity to deserve a raspberry just for one function.

Synology DS1812+
Ah I see where you are coming from now.
http://www.anandtech.com/show/7071/synology-ds1812-8bay-smb-soho-nas-review
Nice piece of kit - I would probably try to get it working on there too ;-)

I am curious about how easy it will be to provide a basic 'development' environment on your NAS.

To me the greatest benefit of using a Pi (apart from low cost, and silent always-on, low power operation of course) is that the OS is based on Debian- for which there are 30,000+ open source software packages available to run on it- and most of them are available just by typing:
 'aptitude install packagename'

You can expect most any 'normal' Linux distro to have shell tools like awk,sed,cut,head,tail,grep,tar,xv installed  and
TBH if you can't easily install pcregrep and expect then attempting to do any more than fairly trivial ash/bash/dash scripting is probably going to be difficult.

Is there a similar facility available for the synology? If yes, can you post the link?

If not - you may be starting with a mountain to climb needlessly.

Best of luck to you - and have fun. ;-)

[Axel Foley]

It's not a "closed" OS. DSM (DiskStationManager, Synology's OS), is Debian based.

I checked all the tools you mentioned: I have all of them installed (either already installed in DSM, or installed via ipkg), except xv, that I don't even know what's for.

There's an open community (SynoCommunity) which gathers a lot of native packages compiled specifically for Synology NAS's, using a cross compilation framework that you can find here: https://github.com/SynoCommunity/spksrc

So, apart all the great things already existing for DSM, if you really need a Debian env for packages, you can install one of the packages made for DSM, called Debian Chroot, here's an abstract of the help file (https://github.com/SynoCommunity/spksrc/blob/develop/spk/debian-chroot/src/app/help/enu/index.html):

Debian Chroot is a free operating system (OS) that comes with over 29000 packages, precompiled software bundled up in a nice format for easy installation on your DiskStation. Debian Chroot allows you to benefit from the Debian OS inside your DiskStation, alongside DSM. This package is intended for advanced users only.

So you can have lots of fun if you're a Linux dev/expert, which I'm not, but I'm having fun learning new things anyway.

[G.DMT]

Thanks for the Info and Links. 
It sounds like you will have many of the facilities available for your NAS that you would have for a Pi.

Unfortunately https://synocommunity.com/ seem to be having issues with their server, so I wasn't able to check that out today.

Code: [Select]

Due to performance issues, the server will be down for maintenance until 2015-01-22.


I checked all the tools you mentioned: I have all of them installed (either already installed in DSM, or installed via ipkg), except xv, that I don't even know what's for.

  Oops, no wonder you couldn't find 'xv'- since xv is a mistake. It should say xz. 
 
Apologoies for the wild-goose chase there!
https://packages.debian.org/stable/utils/xz-utils

[Axel Foley]

Yes unfortunately they're having some performance issues on the server, the community is getting too big.

We're covere also for xz-utils, probably not the latest since ipkg repository is not updated anymore (it's time to switch to debian chroot maybe):

Code: [Select]

[Sun Jan 18 18:47:57 root@nas:~]# ipkg list | grep xz
xz-utils - 5.0.5-1 - A free general-purpose data compression software with high compression ratio
[Sun Jan 18 18:48:03 root@nas:~]#
I was thinking, since DSLstats is also available for linux, if it would run under Debian chroot environment...didn't find prerequirements on the help page.

[roseway]

I was thinking, since DSLstats is also available for linux, if it would run under Debian chroot environment...didn't find prerequirements on the help page.

I expect so, although I have no experience of using chroot environments. DSLstats has few requirements - just the gtk2 libraries mainly. If you start it from the command line you should get an error message if any necessary libraries are missing.

[Axel Foley]

I was thinking, since DSLstats is also available for linux, if it would run under Debian chroot environment...didn't find prerequirements on the help page.

I expect so, although I have no experience of using chroot environments. DSLstats has few requirements - just the gtk2 libraries mainly. If you start it from the command line you should get an error message if any necessary libraries are missing.

I will try it later and let you know. It would be good to have a solution running also on NAS, they're quite popular and, to my knowledge, chroot environment is available both on Synology and QNAP.

Eric, can I ask you if it would be a huge work to have a GUI-less version of DSLstats? I was thinking about passing a cmdline parameter (--nogui) and dslstats would just run the collection of data, creation of datafiles and upload to MDWS, skipping all the rest.

Thanks for the help, you guys are really great.

[loonylion]

I'm afraid that a GUI-free version of DSLstats would need a substantial rewrite, and there just aren't enough hours in the day...

[Axel Foley]

Thanks...that's what I suspected, unfortunately.

[roseway]

I will try it later and let you know. It would be good to have a solution running also on NAS, they're quite popular and, to my knowledge, chroot environment is available both on Synology and QNAP.

I may have inadvertently misled you with my previous reply. DSLstats is written from the ground up as a GUI program, so if you want to run it in a chroot environment, that environment will still need to have a GUI.

[Axel Foley]

Ok, here's the status:

First try:

Code: [Select]

./dslstats: error while loading shared libraries: libgdk_pixbuf-2.0.so.0: cannot open shared object file: No such file or directory
solved with:

Code: [Select]

aptitude install libgdk-pixbuf2.0-0
Second try:

Code: [Select]

./dslstats: error while loading shared libraries: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
solved with:

Code: [Select]

aptitude install libgtk2.0-0
Third try seems way better:

Code: [Select]

(dslstats:22998): Gtk-WARNING **: cannot open display:
There's no display attached to the NAS. But I guess there's some way to redirect the output to VNC or, better, RDP.

Any advice on this would be greatly appreciated, in the meanwhile I'll google it. (I know, I'm a noob on linux, but I'm learning...)

[roseway]

There's no display attached to the NAS. But I guess there's some way to redirect the output to VNC or, better, RDP.

Any advice on this would be greatly appreciated, in the meanwhile I'll google it. (I know, I'm a noob on linux, but I'm learning...

Does the NAS have a monitor output? DSLstats can only run in a GUI environment - if there isn't one it won't start.