Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: Displaying/Archiving Data on Raspberry Pi  (Read 3910 times)

tiffy

  • Kitizen
  • ****
  • Posts: 1326
Displaying/Archiving Data on Raspberry Pi
« on: September 16, 2021, 12:48:48 PM »

As a keen experimenter and big fan of everything "raspberry", my current project is running a data logger & graphical display on a RPI-3B+ (or RPi-4B).
With reference to many on line sources have managed to date to set up "Grafana" with "InfluxDB" as database and "Node-RED" as data bridge, currently just injecting numerical values from "Node-RED" to drive the system and prove operation.

I have a RPi-ZW configured with a BME-280 temperature, humidity & atmos pressure sensor collecting data currently just archieved to a spreadsheet, Wi-Fi connected to LAN.

The objective is to extract/collect the data from the RPi-ZW and input to the data logger for display/archieving.
The message broker of choice is "Mosquitto" MQTT broker, which should connect to the database (InfluxDB) via the data bridge (Node-RED).
Despite many hours of on line searching and experimenting can't establish a method to connect to/extract data from the RPi-ZW for application to the data logging system as described.

Have seen working examples of my requirements using Arduino's as a data source, however, I'am keen to utilise RPi-ZW's and feel that this should be possible, have other similar projects in mind such as domestic power monitoring if I can get the data link working, Grafana has fantastic display capabilities and is open source as are the other 3 packages.

Any help/suggestions appreciated as currently out of ideas!
Logged
Vodafone FTTP 200/28, VF THG3000 Hub

Alex Atkin UK

  • Addicted Kitizen
  • *****
  • Posts: 5272
    • Thinkbroadband Quality Monitors
Re: Displaying/Archiving Data on Raspberry Pi
« Reply #1 on: September 16, 2021, 11:34:35 PM »

I have a RPi-ZW configured with a BME-280 temperature, humidity & atmos pressure sensor collecting data currently just archieved to a spreadsheet, Wi-Fi connected to LAN.

Interesting, as I was wanting to do this myself, bought a load of sensors then forgot which model they were by the time I was looking into using them. lol
Logged
Broadband: Zen Full Fibre 900 + Three 5G Routers: pfSense (Intel N100) + Huawei CPE Pro 2 H122-373 WiFi: Zyxel NWA210AX
Switches: Netgear MS510TXUP, Netgear MS510TXPP, Netgear GS110EMX My Broadband History & Ping Monitors

tiffy

  • Kitizen
  • ****
  • Posts: 1326
Re: Displaying/Archiving Data on Raspberry Pi
« Reply #2 on: September 17, 2021, 11:42:04 AM »

Making some limited progress using "Mosquitto MQTT Broker" installed on both RPi's, at least managed to establish bi-directional Wi-Fi communication with transferring basic numeric values and messages.

Found these two web sites very helpful regarding understanding and basic configuration of Mosquitto/MQTT protocol:

MQTT with RPi & Arduino
MQTT Tutorial

Still a long way to go but at least keeping the brain cells active!
Logged
Vodafone FTTP 200/28, VF THG3000 Hub

tiffy

  • Kitizen
  • ****
  • Posts: 1326
Re: Displaying/Archiving Data on Raspberry Pi
« Reply #3 on: August 21, 2022, 04:52:53 PM »

A belated update on the project.

Using a RPi-3B running Raspberry Pi OS (Buster) on a 128Gb SSD via USB-SATA adapter and utilising Mosquitto, InfluxDB, Node-RED & Grafana free programs to configured a MQTT broker & data recorder, used Shelly EM hardware as the primary measuring source.
The RPi-3B is ethernet LAN connected to my Win 10 desktop PC.
The Shelly EM is Wi-Fi connected to the RPi-3B MQTT broker.
Over 3 quarterly electricity billing cycles, very happy with the results produced, overall consumption and cost are suprisingly accurate.

Total consumption and cost figures are manually reset on the quarterly meter reading day.
Tariff is entered/changed as required in the Node-RED formula.
Changing tariff within the quarterly measurement periods (very common occurance at present!) unfortunately results in incorrect total consumption/cost period readings for that quarter, probably a way around this other than manually resetting, recording and adding the two periods together, have not figured this out as yet.
Attached, a snapshot of the Grafana display showing a 7 day period.
The RPi-3B CPU temperature is also recorded.

Have also recently configured a hot water tank temperature monitor using a Shelly-1 relay with add-on temperature board & 2 probes, Wi-Fi connected to the MQTT broker, had reason to believe that the domestic HW temperature was inconsistant and thought the tank stat was on the blink.
One temp probe mounted directly beside the tank stat, the other attached to the tank top, HW discharge pipe.
Has actually proved that the HW tank stat is surprisingly accurate and repeatable!
Snapshot attached, again on 7 day display with boiler running morning and afternoon, tank stat set at 60 deg. C.
     
Logged
Vodafone FTTP 200/28, VF THG3000 Hub

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Displaying/Archiving Data on Raspberry Pi
« Reply #4 on: August 21, 2022, 06:53:28 PM »

I must admit that I had completely forgotten about this, your latest project. Thank you for the eleven months update.

Good results have been achieved, I see.  :)
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

tiffy

  • Kitizen
  • ****
  • Posts: 1326
Re: Displaying/Archiving Data on Raspberry Pi
« Reply #5 on: August 21, 2022, 08:26:42 PM »

@b*cat:
Quote
I must admit that I had completely forgotten about this, your latest project.

Yes, I have so much on the go at times that I forget about things I start and something else takes priority, likely an age thing. ;D
I'am just a sucker for anything RPi, cant resist.
Just as well I have a good stock as most models have been unobtainable, anywhere for some time, proverbial chip shortage, there's even a black market on RPi's now!
Have managed to get a recently released RPi Pico W which sold out very quickly at launch but has now come back into stock in a few outlets.
Logged
Vodafone FTTP 200/28, VF THG3000 Hub
 

anything