Kitz Forum
Computers & Hardware => Other Technologies & Hardware => Topic started by: 22over7 on November 11, 2018, 12:52:18 PM
-
Has anyone any experience with "pi-hole"? (https://pi-hole.net/).
I'm pretty intolerant of ad's, and for sometime have been using https://github.com/StevenBlack/hosts. This constructs
a gigantic /etc/hosts, in which thousands of dns names are "/dev/null"d. This has not been problem-free.
Then I came across pi-hole, and installed it on a (3B) pi that I was using as a cups server for some ancient non-wifi printers,
and doing little else. I put this top of the DNS servers for a machine or two, just out of curiosity. So far, I'm quite
impressed. For one thing, I can allow a handful of sites (like kitz.co.uk, sndbforums.com, ..) to show me ads, in the hope that
it thereby might bring them in a few pennies. There's a nice admin-page (it sets up a lighttpd server) where you can configure such things.
I can now see some unobtrusive ads on the kitz sites, and elsewhere.
I've only been using it a day or two. It's quite crisp, given that the pi involved has fallen down behind an immensely heavy metal filing cabinet,
and has rather dodgy wifi accessibility. I wondered about getting another lower-specced pi, ethernet connected to my router, and powered
via a router USB port. It doesn't seem to need a 3B.
Does anyone here use a pi-hole "in anger"? Any gotchas? Advice?
-
I’ve never used this but it certainly looks interesting. I have an old Pi Zero gathering dust somewhere and when I get time I’ll try it out on that.
Thank you for pointing it out.
Kevin
-
Yes I used it for about 3 weeks and it works well.
You do need control over the DNS settings for clients for it to work best.
I first ran it as the upstream DNS from the router but in this mode it does not record client usage
but shows all requests as from the router IP.
I then ran it as the DHCP server (instead of the router) as well as the DNS and this provided a great solution.
There is a lot of details and support for it on the Internet.
It then wet my appertite to go further and gain more control of my local network and I
now run pfSense with the pfBlockerNG plugin (which does the same thing as Pi-Hole) and so
now have everything in one box.
-
i used it for a while, but it is worse than just using an adblock browser extension.
it couldn't handle blocking youtube ads natively, so it resulted in having to run both an adblocker extension anyway.
i dumped it shortly after that as it became pointless.
-
Likewise, I tried it for a few weeks and I found it blocking more pages and content completely. There is a lot of tweaking involved to get it working. It has a good online community with lots of tips and advice, the Whitelist suggestions they have on there helped a lot.
I still found it sometimes not blocking ads, or just blocking the whole site altogether. It also played havoc with my Hive Heating. The app would take a minute to refresh data from my thermostats.
-
In my experience anti-adblocking techniques and ads built by JavaScript code are getting so common now that sophisticated ad-blocker software is needed, which inspects html and / or even JS maybe and interferes with it. My adblocker in my iPad (1Blocker-X) seems very competent on the whole, and works with half a dozen vast database of regexes that are tested against URLs by a special engin inside Safari. I don’t think a simply DNS-based thing is sophisticated enough, as blocking an entire domain name or not is not sufficiently specific.
I did the hosts file thing myself many years ago, under Windows, but for some common nuisance content rather than ad blocking.
-
I have been running Pi-Hole for nearly two years, one installed at my plaqce, one at my partners, the lack of ads on any device is wonderful.
I'm only running it on a Pi B+ as anything else is overkill, it runs quite happily on a Pi Zero or Pi A+ if you have a usb to ethernet adapter.
-
I am, quite shamelessly, going to request that all members please ensure kitz.co.uk is white-listed in your ad-blocker.
-
I have been running Pi-Hole for nearly two years, one installed at my plaqce, one at my partners, the lack of ads on any device is wonderful.
I'm only running it on a Pi B+ as anything else is overkill, it runs quite happily on a Pi Zero or Pi A+ if you have a usb to ethernet adapter.
You must not use YouTube much then as it can't block those ads without some major constant manual editing
-
I am, quite shamelessly, going to request that all members please ensure kitz.co.uk is white-listed in your ad-blocker.
Nope sorry, my adblock extension applies to all sites without prejudice
-
Blocking ads with a DNS server is useful but it wont replace a browser adblocker. Reason been is that DNS based solutions work on blocking an entire domain as that is all they can do.
This means if a domain hosts adverts but also serves proper content it either is blocked (which is a problem if you need it for the proper content) or left unfiltered. Also community based blacklists tend to have false positives and even have domains added for ideological reasons, e.g. a few lists block betting sites when they supposed to be just for malware/ads.
Since I first started using pfblockerNG on pfsense, the lists I am using have slowly dwindled down, I removed all lists that have no sane policy for false positive removals (As after all if there is no way to report a false positive then how can they be removed), as well as any lists that had excessive false positives in my usage of them. Some lists are defenitly useful but some also cause more trouble than they worth.
Where DNS based filtering has value more is for things like phones, where apps integrate ads into the app, generally the only way to filter those is via DNS based filtering.
The potential for browser quality of filtering at the network level is implementing a solution similar to how child porn is filtered by isp's. So a DNS lookup at the router, then if it hits, traffic is redirected to a proxy on the router, and then specific path url's can be filtered instead of entire domains, particularly for malware blacklists this would be far superior to just DNS based filtering. However this would be for http only, because otherwise on https the router would be a MITM on encrypted traffic which is a big no no in my opinion. I dont know how the uk child porn filters handle scanning of https links.
Generally the solution is as good as bad as the blacklists you choose to use, I am assuming pi-hole comes with preset lists configured, but I also expect they can be disabled as well as new ones added. So if you getting false positives, disable the bad blacklists.
-
I just had a go at installing pi-hole. However I didn’t get very far. My machine is hosted externally out on the internet, and I get the feeling that an installation is assumed to be internal, within some lan that it is ‘serving’. The second thing is that my machine is IPv6-only, but the installation was asking for IPv4 addresses of upstream DNS servers. Looking at the installation routine’s code, I could fix the latter by changing the chosen upstream DNS servers’ addresses to the corresponding IPv6 ones. I’m not sure how far it would get me. Given that the machine is not inside my main LAN, I had just thought about trying to use it as an intermediate DNS relay which would serve my router, a Firebrick, telling the Firebrick that the pi-hole server was to be the preferred upstream DNS server. That was the idea anyway.
-
pi-hole is meant to be a local DNS server on your LAN
-
Yes. As I thought. I had a second go but it just created a horrendous mess.
It asked some incomprehensible question about using IPv4 and or IPv6, I couldn’t tell whether it meant accessing DNS upstream over one or t’other or both, or filtering both types of queries, or providing services in both aspects as a DNS server, or a DHCP server or both, or what. I tried saying no to IPv4 and then it got a lot further, before finally wrecking the existing DNS on the system so it would not resolve anything at all any more. That meant that every thing was broken since no DNS lookups could be done, couldn’t use apt-get any more as lack of DNS resolver made it fall over, so it was time to admit defeat and wipe the system. A pretty disastrous attempt at writing an installer, with no global system transactional rollback facility. Maybe really really can’t cope with an IPv6-only system, difficult to know.
It may well not be possible to simply use it as a straight relay dns server independently. The designers perhaps never thought about such a usage.
-
well you are trying to use it out-with the scope of what it's designed to be, so you can't blame the installer for the mess you created.
-
An installer needs to back out its changes if it fails. Leaving a mess lying around is very poor design. That’s a simple enough rule. It detected some error condition and then just fell to pieces.
-
it's not an "installer" though, it's just a script running various linux commands to make sure all the required components exist.
each command runs and completes, they are all standalone parts.
-
I had a delicious experience upgrading a Windows10 laptop.
I needed to uninstall something, and it chuntered on for a few minutes,
and then the progress bar went *backwards*. I have never seen this
before. I fell into an inescapable mental loop on the nature of time.
As for pi-hole, it seems to be imperfect, at least for this average moron blundering around.
Maybe TLS has something to do with a rather laggy effect
on my machine using a pi-hole for DNS. And it doesn't kill youtube-adverts, so it is not sufficient.
But it could well be part of a plan to de-crap one's internet connection.
At the very least, it is a learning tool. It sheds a lot of light on network shenanigans that go on
on your computer.
Although kitz.co.uk & co (good-guy sites who use unobtrusive ads, and deserve whatever they get from advertising)
are whitelisted, it seems (with no systematic experiment), to show in some browsers and not others.
-
the good think about most adblocker extensions (certainly for ublock origin) is that they tend to also get rid of the frames the ads are placed in, so the website won't have big gaps where the ads would have been.
pi-hole can only block the ad request thus it leaves the place the ad would have been intact.
-
Weaver we a bit too early yet for single stacked ipv6 ;)
pfblockerNG is the same, it assumes you have ipv4 connectivity and is designed to work on ipv4.
Also yes these systems tend to be designed to be within a LAN as well.
-
One issue with using browser extensions is that they add to the CPU/memory usage and in
doing so can slow down the browser (worse on older CPUs). They also need to be installed on
every clients/browser configurations in use and don't work for some other apps (e.g. email clients).
They are also may not be as effective on mobile/ipad style devices.
Maybe a solution is to use both a DNS and a browser extension solution but this may be overkill and
will have the resource issues on client devices detailed above, but with the DNS catching most before
they reach the extension it may be so bad.
-
ublock origin is way way lighter than adblocker, and you save a ton of cpu by not loading trackers/ads, trackers can be especially high on cpu. e.g on skysports website with no filtering the page is constantly processing data without filtering and uses cpu resources, but with filtering once its loaded its idle. Trackers in my view are sometimes worse than ads, they all over the place and horrible.
-
To add to what VDSL2User was saying, although my 1Blocker-X is very good on the iPad, if you need to go cover the whole LAN that means going around installing software in every machine and even then you may find it impossible. On the iPad my adblocker only covers Safari as it uses the web-browser ad-blocker interface provided by Apple and works within the an individual app not low down at the system level. In order to stop ads appearing in other random apps, a different approach is required and success may well be impossible. Ad-blocking using DNS, if it works, can affect every app in the machine or even an entire group of machines that use one relay DNS server, so a huge benefit. Apps that do not use DNS, but go straight to fetch ads by iPad address are if course immune from this trickery. That’s a nuisance from the point of view of maintenance, but if they are going update the software with new lists of ads, then the app may almost just as well fetch an updated list of IP addresses from the internet. Perhaps designers will start to do this in order to defeat ad blockers? I could certainly do with blocking ads in some of my other apps besides than Safari.
-
You must not use YouTube much then as it can't block those ads without some major constant manual editing
I watch a lot of YouTube and Twitch, lots of YouTube adverts get blocked, the embedded ones remain, but it's not as bad as having them all.
-
I am, quite shamelessly, going to request that all members please ensure kitz.co.uk is white-listed in your ad-blocker.
No thank you, but I'll happily donate if Kitz wants to suggest a nominal annual donation.
-
I watch a lot of YouTube and Twitch, lots of YouTube adverts get blocked, the embedded ones remain, but it's not as bad as having them all.
pi-hole can't block them unless you spend hours manually updating files every day/week/month.
so either you've done that or something else is blocking the youtube ads that google insert.
trust me, and even pi-hole openly admit this, pi-hole is not a solution for blocking youtube ads.
-
Just a quick question. Does anyone else find that memory usage gradually increases over a number of days? I have Pi-hole, DSLStats and Unifi Controller on my Pi3. Just Pi-hole and DSLStats are normally running. If I restart the Pi, memory usage drops. Wondering if that's normal?
-
Just a quick question. Does anyone else find that memory usage gradually increases over a number of days? I have Pi-hole, DSLStats and Unifi Controller on my Pi3. Just Pi-hole and DSLStats are normally running. If I restart the Pi, memory usage drops. Wondering if that's normal?
I haven't noticed (haven't really been looking: it's at 19.1% ATM), and thanks for the alert.
My pi-hole Pi3 is shared with a (rather bloated) cups server (I think they share a lighthttpd).
It seems to me a very good question how fast they (individually/severally) leak memory.
-
Maybe set a regular system reboot cron job
-
Just a quick question. Does anyone else find that memory usage gradually increases over a number of days? I have Pi-hole, DSLStats and Unifi Controller on my Pi3. Just Pi-hole and DSLStats are normally running. If I restart the Pi, memory usage drops. Wondering if that's normal?
Pi-Hole 4.2, the most recent update, is hanging any Pi B+ I put it on.
Might get into debugging it at some point
-
If I use the “top” command, there doesn’t appear to be any obvious process that’s using up memory. Java is the highest but that’s for UniFi, but only installed that yesterday. The increasing memory usage was happening before that. I use Raspbian for the OS, don’t think that would have a memory leak but could be possible. If memory was released when needed, which it may do, it wouldn’t be so bad. The Pi just seems to become very sluggish eventually. Nothing that a reboot doesn’t sort though.
-
I've been using pihole for a bit now.
I'd say it's main value is educational. Just tail the log of DNS traffic.
yes, it blocks lots of stuff you don't want to see.
But in real life (meaning, not using it as a DHCP server, but sometimes for DNS, and in combination
with other things like javascript consent), it's quite hard to understand.
What translations are being cached where? Locally, router, pi, somewhere else?
I seem to hit "sweet spots", where everything is behaving like I want.
Then something happens, and some important website is giving me interminable GDPR screens.
Could be anything. Maybe something to do with quantserve, who knows... .
Or something else .. life is short. There might be better fish to fry.
-
If something is leaking memory, what about a script that kills certain offending processes selectively at certain times?
Does anyone know anything about containerisation in *nix? Could that be relevant here as a way of helping to keep this errant stuff under control? (Don’t know what on earth I’m talking about until I read up on this.)
-
Old post I know and somewhat late to the party...
Been having a play with Pi-Hole, initially installed on my very old, redundant RPi 1B where it worked surprisingly well and the Pi is really too old for most current applications.
Operating system, Raspberry Pi OS Lite using Win 10 SSH interface.
Also installed "log2ram" to relieve SD card over usage.
https://github.com/azlux/log2ram
Tried to install "unbound", makes the DNS server recursive:
https://www.youtube.com/watch?v=FnFtWsZ8IP0&t=77s
This seemed to be too much for the old RPi 1B and would not load/run properly.
Repeated the complete exercise on a RPi Zero W using a micro USB to ethernet adaptor for LAN connection, all 3 programs installed and appear to be working to expectation, RPi is powered from router USB port.
Chose not to apply Pi-Hole DNS routing on my router, only on my main desktop and reserve PC's.
Quite impressed with the program interface and displayed stat's..
As per comments earlier in this thread, add filtering its not perfect by any means, it does miss some of the add's that "AdBlocker" catches and will probably require quite a lot of manual filtering to refine.
Strangely, one long standing anomaly on my main Ryzen-5 desktop PC that it has cured:
On booting, network almost always displayed "no internet connection" (all modem & router LED's normal) for up to 100 sec's, sometimes would remain that way and a "network repair" had to be carried out, this anomaly has persisted with a full Win 10 clean re-load (upgraded from Home to Pro), could never find a solution.
Has never occured since routing PC network DNS through Pi-Hole!!
-
Interesting. :)
My only grumble is why do such video creators need to gabble? :-\
-
Also did I miss something or did he never actually start Unbound after setting the configuration? That could really confuse people.
-
Also did I miss something or did he never actually start Unbound after setting the configuration? That could really confuse people.
"Unbound" would not install without errors on my RPi-1B, that's after I had already installed Pi-Hole and "log2ram".
Repeating the same sequence on my RPi-ZW, ie, installing the 3 programs in same sequence was successful and "unbound" operation was confirmed from Pi-Hole query log.
-
"Unbound" would not install without errors on my RPi-1B, that's after I had already installed Pi-Hole and "log2ram".
Repeating the same sequence on my RPi-ZW, ie, installing the 3 programs in same sequence was successful and "unbound" operation was confirmed from Pi-Hole query log.
Just going on what I saw in the video, Unbound WILL fail before the config change due to the conflicting port with the Pihole DNS resolver. Once he made that config change he just babbled on about WHY it was needed but gave no indication of actually reloading Unbound so it would use the new config and start correctly.
-
Interesting. :)
My only grumble is why do such video creators need to gabble? :-\
Because that particular Youtuber's videos are just adverts for beer, and a video could take a few takes, so 6 or so beers later you are going to gabble? ::)