Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: 1 2 3 [4] 5 6 ... 8

Author Topic: Getting extra DSL stats from TP-Link modem/routers  (Read 162013 times)

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #45 on: May 20, 2015, 08:26:30 PM »

The way the config works, it often doesn't include parts that are unchanged from the defaults (although it does include some stuff, such as 200 empty URLs in the URL filter settings).

The config file structure is described by the reduced_data_model.xml file in the firmware. From the latest 9980 firmware: http://pastebin.com/g7iM2xix (some parts, like the voice telephony, are not supported)

The data model is probably based on TR-098 or some of the other https://www.broadband-forum.org/ "technical reports", other routers (like the Plusnet Sagem 2704N) are also based around the same standards, so have a partly similar xml config file, but with different manufacturer specific parts (things starting with X_TPLINK).

On Linux, the binwalk program can take apart the firmware, and it uses the unsquashfs program to extract the contents of the filesystem. Don't know how well it would work on Windows, but 7-zip can also read some squashfs filesystem images.

The StatPOSTer program can still be used for looking at the settings, although you do have to guess which of "Get value" or "Get list" is appropriate for the specified object.
Logged

Mooingall

  • Member
  • **
  • Posts: 45
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #46 on: May 22, 2015, 02:37:06 PM »

Thanks Ejs, I saw both the config template xml files while digging.

I had hoped to see that there would be some extra juicy bits in a decrypted + configured copy.

I had really hoped the FTP server config file was in there, but it seems to be hardcoded in libcmm.so



What did you make of the TFTP stuff within libcmm.so?

There seems to be at least two cases of it being willing to grab some files from a TFTP server and run them. Under what circumstances I don't know.


Code: [Select]
/proc/tplink/wlan_status    open wlan status error! mkdir /tmp/dut  mkdir /tmp/d    tftp -g %s -r dut_images.tar.gz -l /tmp/d/dut_images.tar.gz tftp -g %s -r progmodels.tar.gz -l /tmp/d/progmodels.tar.gz tar -zxf /tmp/d/dut_images.tar.gz -C /tmp/d/    tftp -g %s -r dutserver -l /tmp/dut/dutserver   tftp -g %s -r mtlk.ko -l /tmp/dut/mtlk.ko   chmod +x /tmp/dut/dutserver echo "/tmp/dut/dutserver /tmp/dut/mtlk.ko &" >/tmp/dut/run_dut.sh   chmod +x /tmp/dut/run_dut.sh    /bin/sh /tmp/dut/run_dut.sh tftp -g %s -r art.ko -l /var/tmp/art.ko tftp -g %s -r nart.out -l /var/tmp/nart.out insmod /var/tmp/art.ko  chmod 777 /var/tmp/nart.out /var/tmp/nart.out -console &    hostapd_wlan0
Code: [Select]
/dev/caldata    *********cal_data*******    *********end******* *****DUT has not been calculated! Auto start art!*****  echo 1 > /proc/tplink/eth_forward   arping -I br0 -c 1 192.168.1.100    tftp -g 192.168.1.100 -r art.ko -l /var/tmp/art.ko  tftp -g 192.168.1.100 -r nart.out -l /var/tmp/nart.out  tftp -g 192.168.1.100 -r ScanNart -l /var/tmp/ScanNart  chmod +x /var/tmp/nart.out  /var/tmp/nart.out -instance 1 -console &    chmod +x /var/tmp/ScanNart  /var/tmp/ScanNart & insmod /lib/modules/%s 
Related? http://sekurak.pl/more-information-about-tp-link-backdoor/ Did they just change what triggers this TFTP stuff and call it a day?
Logged

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #47 on: May 22, 2015, 07:21:54 PM »

That tftp stuff is related to that backdoor, but there's no obvious way to trigger it anymore. It's probably for testing or calibrating the wifi - dual band devices have two different ones, one for each wifi chip. The cli program contains a reference to a startMFG (and startMFG5G) command, but of course it won't let you run them. It looks like the cli program still contains the sh command too (and always has done previously), it's just been disabled somehow, I don't know if there's some secret way to enable it or if it's just permanently disabled somewhere.
Logged

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #48 on: August 24, 2015, 01:12:35 AM »

@ejs

Do you think there is any possibility of getting StatPOSTer to work with the Archer VR900.   

Ive been playing tonight but all I get is java.io.IOException: Server returned HTTP response code: 500 for URL: http://192.168.1.1/cgi?1&5

Ive just spent several unsuccessful hours attempting to get binwalk running on a windows system, the site says it may work but I just cant seem to get python to accept and install it to run as a script. 
There are no real set up instructions for Win other than the author saying this :/   
Searching seems to bring up the one same user asking for help getting it installed on Windows, but no answers.

I think you said you have binwalk and if so the f/w can be downloaded here.  If there's any more info you need from me, just yell.

tyvm
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #49 on: August 24, 2015, 10:18:15 AM »

It may be possible, parts of the TP-Link web interface seem fairly similar. I unpacked the firmware with binwalk, most of the TP-Link stuff is in the part built for the ARM cpu architecture. The /etc/default_config.xml and /etc/reduced_data_model.xml files are encrypted, it's probably possible to find the decryption key somewhere in the firmware, but I don't really know where. The key used on the saved config files was in the cli program in older models, and although I did find some bytes that looked similar in a similar place in the cli program in the VR900 fw, I have not yet succeeded in decrypting those files.

I think X_TPLINK_ has been shortened to X_TP_ in places but that shouldn't matter.

The StatPOSTer program mostly works by emulating the web interface. In previous models, you could see some extra information just be looking in wireshark or in Firefox Network Developer Tools (see screenshot).

When you login to the VR900 web interface, does it ask for a username and password, or just a password?
Logged

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #50 on: August 24, 2015, 05:57:26 PM »

Quote
In previous models, you could see some extra information just be looking in wireshark

Sorry, my fault I forgot to mention that.  I tried it several days ago and yes it is giving some additional info, which is why I wondered if it would work with StatPOSTer

This is part of the info that I managed to get about a week ago or so agao.   Im not sure if the ErrSec counter is really zero though as its now been on during some several heavy storms and when I tried again last night it was still showing as zero.   Anyways this shows there is a bit more there.
Code: [Select]
129
[1,0,0,0,0,0]0
status=Up
modulationType=VDSL2
X_TP_AdslModulationCfg=Multimode
upstreamCurrRate=20000
downstreamCurrRate=79987
X_TP_AnnexType=Annex A/L/M
upstreamMaxRate=30203
downstreamMaxRate=80125
upstreamNoiseMargin=127
downstreamNoiseMargin=63
upstreamAttenuation=0
downstreamAttenuation=177

a2
[2,0,0,0,0,0]1
ATUCCRCErrors=0
CRCErrors=166
ATUCFECErrors=15
FECErrors=0
severelyErroredSecs=0
X_TP_US_SeverelyErroredSecs=0
erroredSecs=0
X_TP_US_ErroredSecs=0

a2
[3,0,0,0,0,0]1
ATUCCRCErrors=0
CRCErrors=166
ATUCFECErrors=15
FECErrors=0
severelyErroredSecs=0
X_TP_US_SeverelyErroredSecs=0
erroredSecs=0
X_TP_US_ErroredSecs=0

8
[error]0
0

Quote
When you login to the VR900 web interface,

Just the password
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #51 on: August 24, 2015, 08:04:07 PM »

Hopefully it's as simple as the user:pass is incorrect, I looked at the source of /web/frame/login.htm and it appears to only use the password.

New version allows for an empty username field, which I think is what's needed:
StatPOSTer-20150824.jar or here

The error seconds items probably never get filled in.
Logged

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #52 on: August 24, 2015, 09:20:03 PM »

Excellent work that man!  :yay:

Code: [Select]
[1,0,0,0,0,0]0
enable=1
status=Up
linkEncapsulationRequested=
linkEncapsulationUsed=G.993.2_Annex_K_PTM
modulationType=VDSL2
lineEncoding=
allowedProfiles=8a,8b,8c,8d,12a,12b,17a,30a
currentProfile=17a
X_TP_VDSLProfiles_8a=1
X_TP_VDSLProfiles_8b=1
X_TP_VDSLProfiles_8c=1
X_TP_VDSLProfiles_8d=1
X_TP_VDSLProfiles_12a=1
X_TP_VDSLProfiles_12b=1
X_TP_VDSLProfiles_17a=1
X_TP_VDSLProfiles_30a=1
X_TP_VDSLProfiles_US0_8a=1
dataPath=Fast
interleaveDepth=0
lineNumber=0
upstreamCurrRate=20000
downstreamCurrRate=79987
upstreamMaxRate=30409
downstreamMaxRate=79986
upstreamNoiseMargin=129
downstreamNoiseMargin=63
upstreamAttenuation=0
downstreamAttenuation=177
upstreamPower=63
downstreamPower=143
ATURVendor=
ATURCountry=
ATUCVendor=
ATUCCountry=
totalStart=95873
showtimeStart=1390671
quarterHourStart=0
X_TP_Bitswap=On
X_TP_SRA=On
X_TP_AdslModulationCfg=Multimode
X_TP_AnnexType=Annex A/L/M
X_TP_SupportAdslMode=VDSL2:A;T1.413:A;G.dmt:A;ADSL2:A,A/L/M;ADSL2+:A,M,A/L/M;ADSL Auto Sync-up:A/L/M;Auto Sync-up:A/L/M
[2,0,0,0,0,0]0
enable=0
status=Disabled
linkEncapsulationRequested=
linkEncapsulationUsed=G.992.3_Annex_K_ATM
modulationType=
lineEncoding=
allowedProfiles=8a,8b,8c,8d,12a,12b,17a,30a
currentProfile=
X_TP_VDSLProfiles_8a=1
X_TP_VDSLProfiles_8b=1
X_TP_VDSLProfiles_8c=1
X_TP_VDSLProfiles_8d=1
X_TP_VDSLProfiles_12a=1
X_TP_VDSLProfiles_12b=1
X_TP_VDSLProfiles_17a=1
X_TP_VDSLProfiles_30a=1
X_TP_VDSLProfiles_US0_8a=1
dataPath=
interleaveDepth=0
lineNumber=0
upstreamCurrRate=0
downstreamCurrRate=0
upstreamMaxRate=0
downstreamMaxRate=0
upstreamNoiseMargin=0
downstreamNoiseMargin=0
upstreamAttenuation=0
downstreamAttenuation=0
upstreamPower=0
downstreamPower=0
ATURVendor=
ATURCountry=
ATUCVendor=
ATUCCountry=
totalStart=0
showtimeStart=0
quarterHourStart=0
X_TP_Bitswap=On
X_TP_SRA=On
X_TP_AdslModulationCfg=Multimode
X_TP_AnnexType=Annex A/L/M
X_TP_SupportAdslMode=VDSL2:A;T1.413:A;G.dmt:A;ADSL2:A,A/L/M;ADSL2+:A,M,A/L/M;ADSL Auto Sync-up:A/L/M;Auto Sync-up:A/L/M
[3,0,0,0,0,0]0
enable=0
status=Disabled
linkEncapsulationRequested=
linkEncapsulationUsed=G.992.3_Annex_K_ATM
modulationType=
lineEncoding=
allowedProfiles=8a,8b,8c,8d,12a,12b,17a,30a
currentProfile=
X_TP_VDSLProfiles_8a=1
X_TP_VDSLProfiles_8b=1
X_TP_VDSLProfiles_8c=1
X_TP_VDSLProfiles_8d=1
X_TP_VDSLProfiles_12a=1
X_TP_VDSLProfiles_12b=1
X_TP_VDSLProfiles_17a=1
X_TP_VDSLProfiles_30a=1
X_TP_VDSLProfiles_US0_8a=1
dataPath=
interleaveDepth=0
lineNumber=0
upstreamCurrRate=0
downstreamCurrRate=0
upstreamMaxRate=0
downstreamMaxRate=0
upstreamNoiseMargin=0
downstreamNoiseMargin=0
upstreamAttenuation=0
downstreamAttenuation=0
upstreamPower=0
downstreamPower=0
ATURVendor=
ATURCountry=
ATUCVendor=
ATUCCountry=
totalStart=0
showtimeStart=0
quarterHourStart=0
X_TP_Bitswap=On
X_TP_SRA=On
X_TP_AdslModulationCfg=Multimode
X_TP_AnnexType=Annex A/L/M
X_TP_SupportAdslMode=VDSL2:A;T1.413:A;G.dmt:A;ADSL2:A,A/L/M;ADSL2+:A,M,A/L/M;ADSL Auto Sync-up:A/L/M;Auto Sync-up:A/L/M
[1,0,0,0,0,0]1
receiveBlocks=0
transmitBlocks=0
cellDelin=0
linkRetrain=0
initErrors=0
initTimeouts=0
lossOfFraming=0
erroredSecs=0
X_TP_US_ErroredSecs=0
severelyErroredSecs=0
X_TP_US_SeverelyErroredSecs=0
FECErrors=0
ATUCFECErrors=308
HECErrors=2983
ATUCHECErrors=0
CRCErrors=2541
ATUCCRCErrors=40
[2,0,0,0,0,0]1
receiveBlocks=0
transmitBlocks=0
cellDelin=0
linkRetrain=0
initErrors=0
initTimeouts=0
lossOfFraming=0
erroredSecs=0
X_TP_US_ErroredSecs=0
severelyErroredSecs=0
X_TP_US_SeverelyErroredSecs=0
FECErrors=0
ATUCFECErrors=308
HECErrors=2983
ATUCHECErrors=0
CRCErrors=2541
ATUCCRCErrors=40
[3,0,0,0,0,0]1
receiveBlocks=0
transmitBlocks=0
cellDelin=0
linkRetrain=0
initErrors=0
initTimeouts=0
lossOfFraming=0
erroredSecs=0
X_TP_US_ErroredSecs=0
severelyErroredSecs=0
X_TP_US_SeverelyErroredSecs=0
FECErrors=0
ATUCFECErrors=308
HECErrors=2983
ATUCHECErrors=0
CRCErrors=2541
ATUCCRCErrors=40
[error]0


Quote
he error seconds items probably never get filled in.
I suspect so too, as per upstream attenuation.

Interesting the duplication of some data fields which are empty.  Possible different channels/bearers?
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #53 on: August 24, 2015, 09:37:48 PM »

Thank you. I'm not sure why there are 3 sets of each block of values. I assumed the blocks with CRC and FEC would be for different time intervals, such as all time, 15 mins or 1 day. But it's just 3 sets of duplicate information. I think the general tree structure, and the names (that don't begin with X_TP) are all based around the TR-098 standard.
Logged

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: Getting extra DSL stats from TP-Link TD-W8970
« Reply #54 on: August 31, 2015, 10:07:03 PM »

Pinning this because I can never find it when I want.

Ive also changed the topic title slightly because it now applies to other models.
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

frakitz

  • Just arrived
  • *
  • Posts: 4
Re: Getting extra DSL stats from TP-Link modem/routers
« Reply #55 on: October 19, 2015, 10:02:15 PM »

Hello everybody!

Did anyone manage to tweak the SNR value on a TD-W8970v1?

Thank you!
Logged

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Getting extra DSL stats from TP-Link modem/routers
« Reply #56 on: October 20, 2015, 05:55:40 AM »

I have not attempted to do it, but it should be possible with shell access. Getting shell access on the 8970v1 is easier than the 9980 because the saved config files are not compressed, so you can decrypt, edit, encrypt and upload.

The dsl_cpe_pipe command to try and adjust the target downstream noise margin is:

locs 0 x

where x is a value from -50 to 50 corresponding to -5.0 dB to 5.0 dB e.g.

locs 0 20

would increase the target SNRM by 2 dB

The comment in src/include/drv_dsl_cpe_api.h above DSL_OPT_NOISE_MARGIN_DELTA_DS says
Quote
If this parameter will be changed from it's default value it is
strictly recommended to use ONLY positive values. Negative values
are likely to result in link instabilities.

dsl_cpe_pipe being /firmware/dsl_cpe_pipe.sh in TP-Link firmware.
Logged

frakitz

  • Just arrived
  • *
  • Posts: 4
Re: Getting extra DSL stats from TP-Link modem/routers
« Reply #57 on: October 20, 2015, 08:49:24 AM »

Thanks for your reply.

I got shell access and run those commands without success.
I also forced ADSL resync with
 acs 2

but the SNR value is unaltered.

Since I got shell access is it possible to install another firmware (e.g. OpenWRT) which allows tweaking the SNR value, without opening the router and soldering?
Logged

ejs

  • Kitizen
  • ****
  • Posts: 2078
Re: Getting extra DSL stats from TP-Link modem/routers
« Reply #58 on: October 20, 2015, 01:08:14 PM »

It's probably possible to flash a new firmware from the shell, but I don't know how, plus you might want to build your own OpenWRT firmware with various updated lantiq components if applicable.

One part of the sources says this about the locs command:
Quote from: src/include/drv_dsl_cpe_api_ioctl.h
- Supported by ADSL only platforms (XWAY(TM) ARX100, XWAY(TM) ARX300)

But the ChangeLog contains:
Quote
V4.15.1 - 2013-07-18
...
- VRX (DSLCPE_SW-732) Include/rework implementation for NoiseMargingDelta
  configuration.

So I'm not sure if the locs command works on the VRX268 or not.

Another possibility would be to install the 9980 firmware, although this does make getting shell access slightly more difficult, it's not too bad because all the settings get wiped anyway when flashing the 9980 firmware.

8970v1 firmware contains Lantiq DSL driver 4.11.4
9980 firmware contains Lantiq DSL driver 4.16.6 and newer DSL firmware

The end of the crossflashing thread has 2 different methods for uploading a firmware image for a different model via the web interface.
Logged

frakitz

  • Just arrived
  • *
  • Posts: 4
Re: Getting extra DSL stats from TP-Link modem/routers
« Reply #59 on: October 20, 2015, 02:39:49 PM »

Very interesting.

I'll try to use the TD-W9980 firmware.
Is the latest version ok? (May 2015)

Thank you.
Logged
Pages: 1 2 3 [4] 5 6 ... 8