Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[finos]

Hello everyone ,
i was asked at my other thread at what software i use to make the line graphs i show.

Please do a detailed post about it. Software, hardware and all the tweaking.

and as i promised, here are the details about it .


To begin with the subjet , i will have to explain my needs my setup and my desisions behind this software (or rather set of ).

im runing a DGA4132 rooted and with ansuel installed on it , (previously when i had  50/5 service and 17a profile i had a TG789vac the scripts were developed for this modem but happeded to work for the DGA) a PC runing PFsense for my router and unifi for my Wifi .

i needed to have a way , when im away from home to recive notifications regarding line state (desync , re sync of the DSL line ) , IP changes from the pfsense , alert me if the ISP pulls a trick on me and "puts" me behind a CGnat when its short on IPv4  (yes the default here is public dynamic IPv4 , and when the ISP runs out , puts you in CGnat ) ruining VPNs and other services i have forwarded . These tools helped me perfom remote diagnosis and support for the family members that are not very tech savvy while i was away at univercity . thus the name TR-069

i was lucky to have 2 lines of internet terminating to my house (we needed 2 landline phone numbers , here we cannot get just a phoneline we had to have a internet conenction , the slowest package would solve the problem plus would give me a path , a ADSL line , that with some SNR tweeks i made very stable , gave me a way to connect my Rpi to the internet when my main connection would drop.

Since Discord is my primary messaging app , and its API Nextcord rockSolid it was obvious that i would use it.  there i created a "server" (more like gloryfied groups) that im allone with the bots , i recive notifications there or i can issue commands (to the modem Bot , PFsense API is ... at best limited)

Lets explain the parts of the project

1. ModemBot

we started development on the Technicolor TG789 , parsing data from it via SSH and broadcoms underdocumented  xdslctl command , this had a bug accros all 3 modems tested (tg788 tg789 DGA4132 ) it would return the attenuation as 0 no matter what . We dig on ansuels method for obtaining the values. There we found "Transformer-CLI" , i think this is used to ...transform openwrt`s way of displaying stats and handling configuration , to a XML style for regular tr069 isps use . Also we found some refrences deep in the CPE that this is the only software that technicolor has give out as open source. later we added support for some models of ZTE routers , but its best suited for technicolor

Transformer was returning a "correct" value , but it was slow , issuing a command via SSH waiting the response parsing with regex repete for over 10 parameters , it wasnt viable .
We decied that we would get almost all the values from xdslctl , and for any values that came back "funny" or the attenuation that we known had issue , we woud turn to the transformer .

2. PfsenseBot

This was the easyest part of the project (and most the disapointing for me , because the api accses was limited and i wouldnt want to go the webscraping path ...)
its job is easy , Report the WAN ip (sould the DDNS fails) , notify for CGnat , and give a quick look at RTT, packet loss etc , it uses a Flaux API implementation for PFsense.
But lets consider it "beta" since its rather unstable ..

3. InfluxDB Feeder
IMHO this is the bread and butter of the project , it uses the same base code as ModemBot , but instead of sending the data over to discord it logs it at a givven interval to a InfluxDB that then i read with Grafana that produces the graphs you saw. Actualy , this is what "saved" me at a recent adventure i had with my VDSL line ... they did not bileve that i had a stable line with 35b over that loop Length but the graphs i had and showed them made them to put effort in and fix the line!


======================
Installation .
You can find instalation instractions over at my Github : https://github.com/finos2/TR069-Bot

if you have any questions feel free to ask me .

Hope someone finds this usefull


in this stage i would like to thank the Developer , Spyros (https://github.com/thetonk)  that worked with me on this project his knowledge of python was critical for the complition of the bot

[burakkucat]

Thank you for providing those comprehensive details.

[Alex Atkin UK]

My pfSense monitoring is done by custom cgi/PHP scripts and SNMP.

I do find it hard to get an accurate average of current traffic though, I think my timing calculations are off somehow.  But its enough to get an idea of what's going on and have an overview of which WANs/VPNs/Clients traffic is going over.