Kitz Forum
Computers & Hardware => Networking => Topic started by: Chrysalis on November 16, 2016, 10:36:57 PM
-
Following the post here http://forum.kitz.co.uk/index.php?topic=18942.msg337615#msg337615 which I do not want to hijack, I thought I would show my proposed new LAN which will be live when I built my PFSense device.
So The pfsense box is this baby http://amzn.eu/6Y5E48h
Ram and SSD are already here. The SSD is way overkill but is a 60gig mini sata SSD (30gig is same price so 60gig is no brainer).
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2F7ClEpXm.jpg&hash=3dc51a4f9d2b1d0ad3ad2e4df84cbb3a5a510809)
-
Hi chrysalis
We use pfsense on some commercial setups, usually on dell poweredge xion dual quad core, 16gb ram, with raid 60 drives, and 2 x 4 1000mb nic cards, as well as onboard 4 1000 nic cards. Works lovely on multi nodes
We also use other firewalls and setups, but for home use, your setup looks ok
I do not use Sam knows box for anything other then Sam knows, and our setup is dynamic for sam knows, so it does not interfere with our IP range
Many thanks and I hope you don't mind my post, but please delete it if you want
Many thanks
John
-
Very cool, have always wanted to fiddle with pfsense but have never been brave enough as a primary device.
As a (relevant) aside, personally I have had bad experiences of the passive Celeron microPC's, in my case the Asrock Beebox N3000 which has problems with heat dissipation under any kind of load and also very poor wireless and bluetooth performance - admittedly I was running Windows 10 on it and using it as a HTPC. I have since switched it to LibreElec and it is much better but I have had to remove the wireless cards and use an external dongle instead and also take out 2.5" disk to keep it cool.
Yours is quite likely a better design that the Beebox and i imagine will be running at a low load but worth having a snoop round perhaps:)
o7
Chunks
-
love my pfsense box.
I use pc engines apu2 with 4gb ram and 30gb mini ssd. Works a treat with my sky fibre pro too, supports sky IPV6 along with open vpn works really well as I got my static IP before sky stopped offering them.
Its 12volt as well so low power use.
Spec http://pcengines.ch/apu2c4.htm (http://pcengines.ch/apu2c4.htm)
CPU AMD Embedded G series GX-412TC, 1 GHz quad Jaguar core with 64 bit and AES-NI support, 32K data + 32K instruction cache per core, shared 2MB L2 cache.
Not seen it break into a sweat yet..
Purchased it from LinITX in the UK - came pre built and pre-installed with a full version of Pfsense.
if anyone wants sky settings PM me should you fancy having a go..
sorted :cool:
-
Tempted to build one of these, although would probably go for what skieci has, but purchase direct from PCEngines for just over half the price that LinITX want, although that would mean I have to work out how to install Pfsense
-
All the info you will ever need in this series of vids:
https://www.youtube.com/watch?v=agieD5uiwYY&list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk (https://www.youtube.com/watch?v=agieD5uiwYY&list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk)
;)
-
Tempted to build one of these, although would probably go for what skieci has, but purchase direct from PCEngines for just over half the price that LinITX want, although that would mean I have to work out how to install Pfsense
I have a usb install stick I made here with serial console support install if you want to borrow it :)
I was having so much packet loss with my old asus I just needed a next day solution to resolve the issues quickly ::) I did run it on my i7 with 2 lan cards pre purchae - worked just fine but the power consumption was a bit crazy as it was my gaming pc..
-
Tempted to build one of these, although would probably go for what skieci has, but purchase direct from PCEngines for just over half the price that LinITX want, although that would mean I have to work out how to install Pfsense
I read your post and I was thinking hmmmm.
Because I did check the price of the APU2 before (someone else recommended it to me) was over £200 hence what I got instead, but then thought some more about your post and you mentioned direct, and indeed I noticed the buy direct link and its significantly cheaper than the reseller and cheaper then what I have ordered so is the better choice. Sort of feel an idiot now but I will persist with what I ordered. What I ordered does come with the casing and is more compact so its not a loss all round.
-
Thanks Underzone for the link, and skyeci for the offer - I'll see how I get on.
Chrysalis, I only stumbled upon the buy direct option because I was reading this thread (https://hardforum.com/threads/anyone-tried-a-pc-engines-apu2c4-yet.1901041/) and someone asked how he got it so cheap. You can also buy a case from them as well, in a choice of colours. I prefer the looks of yours, but the APU2 has three NICS, so if I ever got a second line it could load balance as well. I'll probably go back to using the HG612 as the modem, my upload is now down to 4192kbps, whilst D/S is now at 50412kbps, and I know the HG612 will give me more upload.
-
The annoyance with mine is it has realtek nic's and the FreeBSD drivers for realtek are not great, I will probably run them with offloading disabled as my experience dictates. Whilst the APU2 nic's are all intel. The cpu on the box i am buying however will be fine without checksum offloading its easily powerful enough.
The APU2 is also more future proof as you said with the extra NIC but not just that but also the extra mini pci-e connectivity on board for future addon cards.
-
Thats also why I went for apu2, 3 Intel lan ports :)
-
All the info you will ever need in this series of vids:
https://www.youtube.com/watch?v=agieD5uiwYY&list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk (https://www.youtube.com/watch?v=agieD5uiwYY&list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk)
;)
These are great, thanks for the link. I am starting to get excited about the possibility of building my own router :)
-
Yeah me too. When you watch the vid and he says how most routers are 400MHz and 128Mb RAM it makes you realise how good a standalone box @2GHz with 8GB and an SSD could be!
-
Thanks Underzone for the link, and skyeci for the offer - I'll see how I get on.
Chrysalis, I only stumbled upon the buy direct option because I was reading this thread (https://hardforum.com/threads/anyone-tried-a-pc-engines-apu2c4-yet.1901041/) and someone asked how he got it so cheap.
I don't want to highjack this thread but have a couple of questions based on your comment and reading through the thread as I really like the idea of getting one, at £140 it seems like a good deal as the 3 NIC version could load balance my dual WAN's:
- Do you need a VAT number to buy this stuff direct?
- One of the posters in the thread talk about how the unit lacks power - would a Celeron unit like Chrysalis is buying be a more fkexible option?
Feel free to feedback if this needs to broken out as a separate thread, I am pretty sure I will be pricing up and getting some hardware for this myself
Chunks
-
My unit has more cpu grunt, but my opinion is the apu2 also has enough grunt comfortably to do its job, both have cpus way more powerful than current high end retail consumer routers.
If I have to I will add intel nic's via the mini pcie connector or even buy an apu2 at a later date, hopefully I am fine on the realtek's tho.
As a comparison my ac68 is running at 1200mhz (overclocked, 800 is stock), but it has no aesni acceleration and has worse performance per clock than both systems.
A big factor in my decision is that the dev of the firmware on my asuswrt has started to imply he is not willing to do bug fixes that only I have found (he is moving into only fixing for the masses mode) and that a lot of the software behind asuswrt is reliant on very old code due to a locked down closed source driver from broadcom. I had to manually apply some workarounds to get ipv6 fully stable on my sky connection, and the dev refuses to even acknowledge its broken so that was the trigger point for me. Is a shame the unit I have ordered is using realtek nic's as otherwise I would have considered it perfect (assuming one is ok using a separate WAP and gigabit switch).
Bear in mind regarding the number of network ports, pfsense has very good vlan support, it even has a wizard when you first boot it guiding you through vlan setup, this is all done in mind that alot of people will be running pfsense on devices with limited ethernet ports and can allow things like bonded connections to run via a single ethernet cable, its how I managed to as an example share my wan with lan access to my billion 8800nl stats over one cable.
-
Bear in mind regarding the number of network ports, pfsense has very good vlan support, it even has a wizard when you first boot it guiding you through vlan setup, this is all done in mind that alot of people will be running pfsense on devices with limited ethernet ports and can allow things like bonded connections to run via a single ethernet cable, its how I managed to as an example share my wan with lan access to my billion 8800nl stats over one cable.
According to the videos linked to earlier you will need a managed switch to use VLANs for the above.
-
Hi
I hope you don't mind my thoughts, but we are just talking home network, not commercial grade networks, so my best advice with all networking is to keep it as simple as possible.
This helps with setup and problem solving
If you would like to experiment with networking, I would advice you setup a test network for this purpose and leave your home network alone
Many thanks
John
-
Hi John, I don't mind your thoughts, but if that was the case we'd still all be using ISP supplied routers, and I think the majority of us posting on here are savy enough to have networks a bit above the most basic, and if we're not then we can learn. I have a server at home, and I'd prefer a much better firewall than my modem/router provides, I often get people from all over the world trying to login to my server, so once I have Pfsense set up most of the world will be blocked/dropped :-)
-
Hi ronski
Many thanks
Sorry, I was meaning more in terms of creating vlans and more in depth networking.
I apologise for not been very clear
Many thanks
John
-
Hi John, no need to apologise I was just thinking whilst stripping the wall paper that you probably meant vlans.
Basically your saying don't complicate it for the sake of it, only if you need to.
-
Looking at the network map, isn't all the wired traffic supposed to go via the Sam Knows box (as one of the requirements for using it?)
-
Looking at the network map, isn't all the wired traffic supposed to go via the Sam Knows box (as one of the requirements for using it?)
Hmm . . . I noticed that a few days ago but failed to mention it. :-\
-
Hi
I could be wrong, so would need to check back but my understanding was the reason for Sam knows to have traffic flowing through it, was so the tests that Sam knows runs, are done at a time of less traffic on your network
Our Sam knows runs under a different external IP to our range, so it runs whenever it needs
Many thanks
John
-
Our Sam knows runs under a different external IP to our range, so it runs whenever it needs
Ah, so that might be how Chrysalis has things set-up within his own domain.
-
Looking at the network map, isn't all the wired traffic supposed to go via the Sam Knows box (as one of the requirements for using it?)
it is, but the reason for that is so it always knows when the connection is active.
it doesnt stop the tests from been carried out, but I suppose runs the risk of a skewed result during a download.
-
LOL just checked the disc space on my ssd. 30GB disc installed & pfsense is using less than 1GB. ;D
Have implemented the mod to keep the sky ip6 stuff on reboot/patch upgrade - working just great after a few reboots to test
ipv4 & ipv6 pinger now working too with a few rules added to the firewall to get it setup properly
-
glad you happy with the mod as I am the one who provided the source files for it ;)
-
:thumbs: :thumbs:
-
looks like its finally at the uk depot
Hi Chris, you have a parcel coming tomorrow.
Scheduled Delivery Date: Thursday, 24/11/2016
Estimated Delivery Time: End Of Day
-
I have been using pfSense for some time with a 64 bit LinITX box and a HG612. No router. Works very well.
More recently elsewhere I have been using pfSense with a 32 bit LinITX box and a BT HH5. Also works well, but I am in the throws of upgrading to a 64 bit box, as pfSense are dropping 32 bit support. As I understand it, the 32 bit release will receive security patches but nothing else.
-
Well I decided to go for one of these in the end, the version which takes 2.5" SSD as I have an 80GB Intel SSD that's doing nothing.
https://www.amazon.co.uk/dp/B01GBHC62K/ref=pe_385721_37986871_TE_item
http://www.qotom.net/goods-129-QOTOM-Q190G4+4+LAN+Mini+PC.html
There's a thread on the PFsense forums (https://forum.pfsense.org/index.php?topic=114202.0), does it really need 8GB of memory if running quite a few packages?
The PC Engines one ended up looking like it'd cost about £180 including delivery if it got picked up for import duty as Switzerland is not in the EU.
-
just for the curious my mini keyboard works as well with this. Works in the bios etc.
Rii i8+ 2.4Ghz LED Backlit Mini Wireless Keyboard With Touch Pad Mouse UK Layout With Built-in Rechargeable Battery Black
https://www.amazon.co.uk/gp/product/B00T2SJUWA/ref=oh_aui_search_detailpage?ie=UTF8&psc=1 (https://www.amazon.co.uk/gp/product/B00T2SJUWA/ref=oh_aui_search_detailpage?ie=UTF8&psc=1)
Also inside there is a connector and power cable for a normal sata storage device although I recommend a msata ssd like I got and actually removing those sata cables.
-
Well I decided to go for one of these in the end, the version which takes 2.5" SSD as I have an 80GB Intel SSD that's doing nothing.
https://www.amazon.co.uk/dp/B01GBHC62K/ref=pe_385721_37986871_TE_item
http://www.qotom.net/goods-129-QOTOM-Q190G4+4+LAN+Mini+PC.html
There's a thread on the PFsense forums (https://forum.pfsense.org/index.php?topic=114202.0), does it really need 8GB of memory if running quite a few packages?
The PC Engines one ended up looking like it'd cost about £180 including delivery if it got picked up for import duty as Switzerland is not in the EU.
Your find made me realise I suck at searching for stuff :p , you guys all making me feel down with the realtek nics :/
Nice find tho, looks good. The cpu has no AES tho but at the same time the cpu has a decent amount of raw grunt so it should be able to handle vpn at FTTC speeds.
Also 4 gig should be plenty of ram.
-
The PC Engines one ended up looking like it'd cost about £180 including delivery if it got picked up for import duty as Switzerland is not in the EU.
I'll be able to verify this as I am currently trying to get one ordered direct at the moment, the price for a 4Gb board, case and UK PSU is EU 121 + P&P plus I will have to pay VAT, I think. So I am guessing price will be around £135 ish delivered.
I was not able to order without a VAT number so am ordering via family business
I've already git an mSATA SSD
Chunks
-
ok some information for those using pfsense with an ssd.
The pfsense devs are using an old way to configure partitions, in short they -
dont enable trim
dont align partitions to 4k alignment
and they also enable SUJ which I think is best disabled on an ssd.
to fix the alignment follow this guide https://forum.pfsense.org/index.php?topic=86139.0
if you try to do it all manually pfsense will rewrite the partitions, but I have confirmed the above guide does lead to a 4k aligned partition.
The commands to enable trim and disable soft updates journaling are. (assuming the ssd is on ada0, which it will be if its the only sata device)
tunefs -t enable /dev/ada0s1a
tunefs -j disable /dev/ada0s1a
Less important is to have the swap on a swapfile not partition as the partition wont utilise trim, but I think its unlikely a pfsense box with at least 4 gig of ram will even use the swap.
bottom of this page is a couple of commands to make the swapfile http://www.wonkity.com/~wblock/docs/html/ssd.html
so
mkdir /usr/swap (although can be put anywhere you like)
dd if=/dev/zero of=/usr/swap/swap bs=128k count=16384 (2 gig swap)
add these 2 lines to /etc/fstab
# Device Mountpoint FStype Options Dump Pass#
md99 none swap sw,file=/usr/swap/swap,late 0 0
then after run swapon -aL
-
@Chunkers, I allowed 40 Euro for shipping, if shipping was 30 Euro, then with import duty it would come to £153. There's a handy calculator here (https://www.dutycalculator.com/new-import-duty-and-tax-calculation/saved_calculations/view_details/214304922/) (one time use). There's also the possibility the courier will charge a handling fee.
@Chrysalis. Thanks for that very useful information, does this change the swap file from what's set up during installation. You'd have thought the dev's would have made the OS SSD friendly by now.
-
ok some information for those using pfsense with an ssd.
The pfsense devs are using an old way to configure partitions, in short they -
dont enable trim
dont align partitions to 4k alignment
and they also enable SUJ which I think is best disabled on an ssd.
to fix the alignment follow this guide https://forum.pfsense.org/index.php?topic=86139.0
if you try to do it all manually pfsense will rewrite the partitions, but I have confirmed the above guide does lead to a 4k aligned partition.
The commands to enable trime and disable soft updates journaling are. (assuming the ssd is on ada0, which it will be if its the only sata device)
tunefs -t enable /dev/adas0p1
tunefs -j disable /dev/adas0p1
Less important is to have the swap on a swapfile not partition as the partition wont utilise trim, but I think its unlikely a pfsense box with at least 4 gig of ram will even use the swap.
bottom of this page is a couple of commands to make the swapfile http://www.wonkity.com/~wblock/docs/html/ssd.html
so
mkdir /usr/swap (although can be put anywhere you like)
dd if=/dev/zero of=/usr/swap/swap bs=128k count=16384 (2 gig swap)
add these 2 lines to /etc/fstab
# Device Mountpoint FStype Options Dump Pass#
md99 none swap sw,file=/usr/swap/swap,late 0 0
then after run swapon -aL
This answers one of my (many) questions regarding pfSense config and also the setup of the APU2 box when it arrives, to avoid cluttering this thread with my questions I'll start a new thread - really appreciate the posts from you guys as they will help me a lot as I thrash around trying to make it work, lol.
C
-
@Chunkers, I allowed 40 Euro for shipping, if shipping was 30 Euro, then with import duty it would come to £153. There's a handy calculator here (https://www.dutycalculator.com/new-import-duty-and-tax-calculation/saved_calculations/view_details/214304922/) (one time use). There's also the possibility the courier will charge a handling fee.
@Chrysalis. Thanks for that very useful information, does this change the swap file from what's set up during installation. You'd have thought the dev's would have made the OS SSD friendly by now.
in my case I had configured a 2 gig swap file during install, if you wish to disable it then do this.
open /etc/fstab
comment out the line as follows.
#/dev/label/swap0 none swap sw 0 0
then run swapoff -a
or simply reboot
if you doing this after you had already added the swapfile then the above command would disable both so run swapon -aL again after or the reboot will take care of it.
Back to my box, I have sourced a dual intel nic mini pcie card, its pricy tho at 50 notes, but the option is there if the box is unstable using realtek. If purchased it would take the cost of this unit all in, including ram, ssd and the 2xintel nic to about £200 with it been a 4 nic unit (2xrealtek and 2xintel)
-
You may well have already found this information, but according to the FreeBSD documentation at https://www.freebsd.org/relnotes/CURRENT/hardware/article.html#ethernet :
The re(4) driver supports RealTek RTL8139C+, RTL8169, RTL816xS, RTL811xS, RTL8168, RTL810xE and RTL8111 based Fast Ethernet and Gigabit Ethernet adapters including:
Alloy Computer Products EtherGOLD 1439E 10/100 (8139C+)
Compaq Evo N1015v Integrated Ethernet (8139C+)
Corega CG-LAPCIGT Gigabit Ethernet (8169S)
D-Link DGE-528(T) Gigabit Ethernet (8169S)
Gigabyte 7N400 Pro2 Integrated Gigabit Ethernet (8110S)
LevelOne GNC-0105T (8169S)
LinkSys EG1032 (32-bit PCI)
PLANEX COMMUNICATIONS Inc. GN-1200TC (8169S)
TP-Link TG-3468 v2 Gigabit Ethernet (8168)
USRobotics USR997902 Gigabit Ethernet (8169S)
Xterasys XN-152 10/100/1000 NIC (8169)
-
sadly the FreeBSD realtek driver is basic only. Most stuff that can be tuned on intel nic's is not tunable such as flow control, interrupt moderation and msix. In addition the driver is buggy that when offloading is enabled it doesnt behave normal. Otherwise the nic's perform providing the pps load is moderate only :) I dont expect my home setup to exceed that load and I think will be ok, I have already disabled checksum offloading on pfsense so will see how things go after I have configured the unit and it takes over the router duties, that is a while off yet tho as I have a lot of configuration to import, my router does more then just forwarding packets, it filters malware, mobile ads and performs QoS duties :) In addition to been my vpn endpoint as well.
I have ran some openssl benchmarks, with AES acceleration disabled its about 12-13 times faster than my ac68 (which itself is overclocked by 50%). With it enabled its about 740 times faster :)
-
My router was running VyOS for quite a while(which I really prefer over pfSense - but it doesn't have the community or GUI) and then I switched to pfSense.
I went for a Mini-ITX build with no moving parts, initially I chose a ASUS N3150 board for AES-NI, then I swapped it for an ASROCK AMD board which had a longer PCI-E expansion slot - I use a dual port Intel NIC and avoid the onboard Realtek.
Interesting upside to the AMD board is the AMD A4-5000 CPU - bit worse power consumption, pretty much the same performance (http://cpuboss.com/cpus/Intel-Celeron-N3150-vs-AMD-A4-5000) as the N3150, except in crypto where it hammers the N3150 -
(https://s14.postimg.org/g388oo7o1/crptamd.png)
-
by the way the aes score on cpuboss is a typo, they put a . instead of , making the value seem really low :)
-
Does anyone know if you can use BT TV with a pfSense setup easily? I assume you could use one dedicated LAN port just for the set top box multicast, but like they say - never assume...
-
The mini pcie adaptor just arrived, I decided to buy it as only one source had it in the UK and with limited stock (intel chip variant), better to get it now, then decide later I want it and not been able to source it.
-
This is hard to add whilst keeping things tidy, the issues are.
pci bracket obviously has nowhere to go, I was aware of this of course before hand.
the cables connecting the bracket to the mpcie adaptor are much thicker and longer than I expected, they also are not very flexible.
Right now I have got one end of the unit resting on top of the bracket which is underneath the unit, and raised it the other side so its sort of level, the bottom cover is not in place as impossible to do so.
My plan is to buy longer screws so the base can be secured again but with a gap so the unit would be raised and this gap will be enough for cabling to get in and out the unit as well as enough room in the unit as inside the unit where the cables attach was also too tight to fit in its normal space.
In regards to the pci bracket the two ethernet ports can be detached but I am thinking of keeping them on the bracket and having the bracket sandwiched in the space thats created with using the longer screws, then it should be ok. I will post some pics later.
So now the unit has 4 ethernet ports, 2x realtek and 2x intel.
-
ok it uses 2mmx5mm screws, I ordered some 2mmx10mm and 2x20mm screws.
Will post some pics, it has a nice blue led light inside which looks really good when the unit is as designed fully enclosed, although it leaks around my room at night with the gap I created to add the intel ports.
Some photos of what is going on. Sorry the inside view of components is blurry didnt realise until now (took 2 days ago), the right blue thing is the msata ssd, to the left of it is the intel nic addon card, in the mpcie slot which was originally populated by the wireless nic card.
The idea is I will have the base screwed on again without any slant, but with longer screws so there is a gap to allow for the new cabling and bracket to fit in place.
-
Looks like pfsense is lacking a lot of packages, 2 packages I will be making are nano and dnscrypt.
There seems a lot of opposition to dnscrypt for some reason in the pfsense forums which I find odd, but regardless I will convert freebsd packages to pfsense.
-
FreeBSD has a simple text editor called 'ee' which is quite similar to nano. You might find that meets your needs.
-
I know about ee but nano is my preference.
-
If you can SSH or FTP into the box, use WinSCP. It is then easy to double click any conf/txt files and enter text and save as easily as notepad on a PC. No skill required, unlike vi...
-
I know of alternatives but regardless the way I work is to use nano. :)
-
So I realised the 2mm screws were wrong, actually should be 3mm, and was fiddling with the bracket positioning then suddenly it was possible to get the base in place with the gap raised slightly for the pcb of the pcb bracket to fit through, I did this and its now snug (will post pics later), of course aware that a possible short could occur and damage to the nic ports, it seems fine although one of the activity leds is broken but the port is working.
-
As it turns out there is a repo for cli packages, but there is no list of what they got in it.
nano
bash
rsync are all there tho.
dnscrypt isnt but I installed the freebsd dnscrypt package which works out of the box.
Its unlikely I will be using my pfsense unit before christmas but I will likely do a online test tho.
-
pics
note it wont be sitting in this location when in production use, is where it is now to be close to monitor.
-
4th pic (3 pics per post limit)
-
pics
note it wont be sitting in this location when in production use, is where it is now to be close to monitor.
Talking of monitors, don't know if you're interested but you can buy a cheap 7" HDMI 1024x600 display for about £24 delivered - for those times when only direct access will do, or simply for monitoring things.
Link (https://www.aliexpress.com/item/7-inch-LCD-Panel-Digital-LCD-Screen-and-Drive-Board-HDMI-VGA-2AV-for-Raspberry-PI/32314647847.html?spm=2114.01010208.3.11.2J9EFz&ws_ab_test=searchweb0_0,searchweb201602_2_10065_10068_10084_10083_10080_10082_10081_10060_10061_10062_10056_10055_10054_10059_10099_10078_10079_426_10073_10103_10102_10096_10052_10050_425_10051,searchweb201603_6&btsid=09358aec-0a27-4158-b05f-86e0dfb8744a)
Can do a DIY case if necessary -
https://www.youtube.com/watch?v=UIu7eB18mps
-
cool
-
I am online now using pfsense to test it out.
Is a quick setup.
ipv4 only
using sky dns
no vlan stuff so for now my mdws will be down
-
I have done some more work now
configured dhcp (before asus was still managing my dhcp)
configured nat
configured and enabled pfblockerNG (dnsBL) - this on the asus is what I was using to filter malware etc. but on pfsense it is officially supported in a plugin.
configured unbound partially
configured QoS partially
todo
dnscrypt
finish off unbound
ipv6
vpn
finalise QoS
To compile the dns block list and load it into unbound is now 2-3 seconds, on the asus it took over 2 minutes.
with dnsbl active my ram usage is still only 14% of 4 gig.
QoS is an interesting one, pfsense has a different approach it takes to QoS and I am trying its supported implementation first to see if its good enough for my needs.
-
I'm also using PFBlockerNG, but I'm not using it to it's best. I currently have just the top 20 countries blocked (excluding UK) under GeoIP. I tried searching for an up to date guide that explained it clearly but couldn't find one. I get people from around the world trying to hack into my remote access so I'd like to block most the world. It states "It's also not recommended to block the 'world', instead consider rules to 'Permit' traffic from selected Countries only." So if for example I selected just the UK, and changed the List Action to "Permit inbound" would that effectively block the rest of the world?
I'd be interested in knowing how you have yours set up in more detail.
-
ronski I am using a private list and public, there is a couple of other public lists I may also add but this is what I did.
First of all an easy way to test is to use the easylist list as shown in first screenshot, you can add custom lists as well using the 'dnsbl feeds' option.
Third screenshot is my outbound rule I just setup to allow modem stat collecting on shared wan cable, however I could not find a way to add the right ip to the wan interface in the gui, so I had to do that in command line.
Note I am removing easylist later on my router, I added just to help test. You have to be careful filtering ads/tracking router side as if a site breaks its not so easy to whitelist, also if want to whitelist sites like kitz for ad's that is also harder to do router side.
-
example of how to add a hosts file list to dnsbl
using http://adaway.org/hosts.txt as example
-
with the box more loaded up now temp is only 33C still :)
asus ac68 on passive cooling over 70C even in winter.
latency is lower than the ac68 also
C:\Windows\system32>ping -t bbc.co.uk
Pinging bbc.co.uk [212.58.244.23] with 32 bytes of data:
Reply from 212.58.244.23: bytes=32 time=6ms TTL=56
Reply from 212.58.244.23: bytes=32 time=7ms TTL=56
Reply from 212.58.244.23: bytes=32 time=6ms TTL=56
Reply from 212.58.244.23: bytes=32 time=7ms TTL=56
Reply from 212.58.244.23: bytes=32 time=7ms TTL=56
Ping statistics for 212.58.244.23:
Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 7ms, Average = 6ms
-
Thanks for the info, not too fussed about blocking adverts to be honest. What's the Malware one do?
My box is running at about 36 degree's.
-
It blocks domains that have been found to host malware.
I am on pfsense 2.4 now as I have been trying to get ipv6 working but is a no go for me. Another buddy is trying to help me as he is also on sky but I am stuck for now.
-
What do you need to check? My 2.4 works fine on sky ipv6 and was fine for 2.3 too.
Test with IPv4 DNS record
ok (0.096s) using ipv4
Test with IPv6 DNS record
ok (0.047s) using ipv6
Test with Dual Stack DNS record
ok (0.661s) using ipv6
Test for Dual Stack DNS and large packet
ok (1.532s) using ipv6
Test IPv4 without DNS
ok (0.050s) using ipv4
Test IPv6 without DNS
ok (0.056s) using ipv6
Test IPv6 large packet
ok (1.580s) using ipv6
Test if your ISP's DNS server uses IPv6
ok (2.519s) using ipv6
Find IPv4 Service Provider
ok (0.473s) using ipv4 ASN 5607
Find IPv6 Service Provider
ok (0.428s) using ipv6 ASN 560
-
the wan DHCP6 settings, but my settings do match what the pfsense sky guy has posted on skyuser.
On the pfsense dashboard WAN_DHCP6 just stays stuck in pending status.
http://www.skyuser.co.uk/forum/ipv6/58986-sky-ipv6-settings-non-sky-routers-12.html#post463605
-
Ok give me a min will send you some screen dumps..
-
do you mind also checking the contents of your /var/etc/dhcp6c_wan.conf file?
here is mine, which doesnt look right. As it is only requesting dns servers and domain name but no prefix from what I can tell.
root@PFSENSE new # cat /var/etc/dhcp6c_wan.conf
interface igb0 {
request domain-name-servers;
request domain-name;
script "/var/etc/dhcp6c_wan_script.sh"; # we'd like some nameservers please
};
-
Done! You have mail... ;)
Did you add tracking on the lan interface on for dhcp6 etc.. have a look at my screen shots. Will have a look at the other bits when I get back.
-
yep I was missing that tracking, thanks is now working. :)
Do you have the unreleased DUID patch on your pfsense yet?
also this looks more like it :)
interface igb0 {
send ia-pd 0; # request prefix delegation
request domain-name-servers;
request domain-name;
script "/var/etc/dhcp6c_wan_script.sh"; # we'd like some nameservers please
};
id-assoc pd 0 {
prefix-interface igb1 {
sla-id 0;
sla-len 8;
};
};
-
Nice!
No not yet. Part of the fix is already on the latest snapshot- the do not release request.. I understand the duid bit is now under testing so hopefully out soon. I am happy to wait for it to be released on a patch. I only use it for the tbb ipv6 monitor so it's not really that much of an issue for my setup :D
This bit is on the latest patch.
Do not allow PD/Address release
dhcp6c will send a release to the ISP on exit, some ISPs then release the allocated address or prefix. This option prevents that signal ever being sent
-
An update :)
The GUI method to add the ip for modem stats subnet is under firewall, virtual ip's, simply add the ip as a alias to the wan interface, which in my case is 192.168.2.252 with a /24 subnet mask.
This means so far I have managed to do everything without hacks in the shell. The only shell modifying I have done is to edit the /etc/rc.initial script to add a 17) option for bash shell.
On my asus my setup involved a lot of manual cli work.
Also I sent you an email to let you know how to preserve DUID before the patch is commited. :)
Also for both you and ronski, if you want to keep traffic total stats across reboots, do as this post says, I have already confirmed it works, even preserved over 2 pfsense upgrades (I went from 2.2 to 2.3 then to 2.4).
https://forum.pfsense.org/index.php?topic=114753.msg664804#msg664804
-
Ok cheers.
I did a straight install recently to 2.4. I didn't like the idea of old files being left from previous versions. I also had a crash on my first build which was upgraded from a working 2.3.. so far on 2.4 I haven't had a crash since doing the clean 2.4 build so hopefully that was perhaps related to the in place upgrades...probably never know but a good 2 weeks since that crash...
-
Yeah I understand where you coming from, it is possible I will redo 2.4 clean at some point especially as I can use ZFS on 2.4 installer, but I have noticed no stability issues from the upgrade process so far.
-
Did you have to upgrade to FreeBSD v11 first?
I think I have to
-
with the box more loaded up now temp is only 33C still :)
asus ac68 on passive cooling over 70C even in winter.
latency is lower than the ac68 also
C:\Windows\system32>ping -t bbc.co.uk
Pinging bbc.co.uk [212.58.244.23] with 32 bytes of data:
Reply from 212.58.244.23: bytes=32 time=6ms TTL=56
Reply from 212.58.244.23: bytes=32 time=7ms TTL=56
Reply from 212.58.244.23: bytes=32 time=6ms TTL=56
Reply from 212.58.244.23: bytes=32 time=7ms TTL=56
Reply from 212.58.244.23: bytes=32 time=7ms TTL=56
Ping statistics for 212.58.244.23:
Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 7ms, Average = 6ms
Those pings are very impressive! Well done mate.
-
Did you have to upgrade to FreeBSD v11 first?
I think I have to
The OS is upgraded as part of the upgrade process, its a all in one solution.
Now days FreeBSD supports binary updates so its still a very quick process assuming you can download the update packages quickly and the box itself is powerful enough to process all the package updates quickly, on my unit it took about 10 minutes, from 2.3 to 2.4 (which includes the OS update), and about 5 from 2.2 to 2.3.
The bad news with 2.4 I think they have removed hardware crypto support on their openvpn binaries, seems an odd decision, but I may have misunderstood the post, so I have asked for clarification.
The good news is 2.4 has added newer and better QoS queuing systems so basically QoS is enhanced.
-
Oh that's good news, I was concerned about the OS update. I still plan to wait for 2.4 to be the official update so I'm on 2.3.2-RELEASE-p1 (amd64) currently.
Thanks :cool:
-
It blocks domains that have been found to host malware.
What options are there for lists?
I see there is Malware Patrol (http://www.malwarepatrol.net), and they seem to offer various lists for a fee, both for PFBlockerNG & Squid, surely you'd only need to use PFBlockerNG (https://www.malwarepatrol.net/howto_pfBlockerNG.shtml) or Squid (https://www.malwarepatrol.net/howto_pfsense.shtml) not both for the purpose of blocking Malware domains?
-
pfblockerNG also adds a lighttpd daemon so it works like this.
Dns lookup is changed to a rfc1918 ip.
C:\Windows\system32>nslookup 152media.com.
Server: PFSENSE.home
Address: 2a02:c7f:<censored>
Name: 152media.com
Address: 10.10.10.1
Then the traffic will be sent to the lighttpd daemon running on pfsense and a blank img is served.
So no squid or other package needed.
You can also whitelist sites listed on alexa to try and avoid accidental breakage of popular sites from FP's.
For this to work, you need to be using the "dns resolver"(unbound) not "dns forwarder".
-
I tested a reboot with the DUID workaround and kept the same ipv6 prefix.
I will be testing the FAIRQ+codel combo later.
-
Thanks Chrysalis, but what options are there for lists to use? It's not exactly the easiest thing to Google for.
Edit.
Have found this post which seems quite useful
https://forum.pfsense.org/index.php?topic=102470.msg573159#msg573159
-
you already found a good lot of choices there :)
for purely malware lists that are freely available there isnt many.
-
ronski I would say
malcode
mdl
dshield_sd
mpatrol - with alexa
ms2
bbc_dga
bbc_c2
Those all look malware focused including crypto malware, having a quick look at all those lists they dont seem to include any ads/tracking stuff.
just be aware its easier to add bad url's then remove cleaned out ones so it wouldnt surprise me if there is dead domains or domains that are now clean in the lists, but pfsense does allow whitelisting and alexa will whitelist the top ranked sites.
hphosts
swc
those two for sure do include ad/tracking domains.
-
I think sticky DUID feature is now commited.
https://github.com/pfsense/pfsense/commit/c337280901d3eedf98e195bd99d30d2ed9d4df1e
https://github.com/pfsense/pfsense/commit/a5d56253d87abeeb76ef480edced9a7a512ad908
-
I added these which the ac68 would explode in a fit with such a size.
===[ DNSBL Domain/IP Counts ] ===================================
1209183 total
809831 /var/db/pfblockerng/dnsbl/bambenek_dga.txt
162237 /var/db/pfblockerng/dnsbl/hphost_fsa.txt
142657 /var/db/pfblockerng/dnsbl/hphosts_emd.txt
43810 /var/db/pfblockerng/dnsbl/malwarepatrol.txt
26329 /var/db/pfblockerng/dnsbl/hphost_psh.txt
17267 /var/db/pfblockerng/dnsbl/hphosts_exp.txt
5280 /var/db/pfblockerng/dnsbl/disconnectmalvertising.txt
1207 /var/db/pfblockerng/dnsbl/malwaredomainlist.txt
345 /var/db/pfblockerng/dnsbl/zeustracker.txt
100 /var/db/pfblockerng/dnsbl/malc0de.txt
73 /var/db/pfblockerng/dnsbl/hphost_hjk.txt
43 /var/db/pfblockerng/dnsbl/dshield_sdh.txt
4 /var/db/pfblockerng/dnsbl/bambenek_c2.txt
0 /var/db/pfblockerng/dnsbl/malwaredomains.txt
0 /var/db/pfblockerng/dnsbl/disconnect_malware.txt
0 /var/db/pfblockerng/dnsbl/disconnect_basic.txt
the biggest ones are the cryptolocker domains, device is now using 25% of ram.
-
Thanks Chrysalis for the further info which is very useful. I did manage to get the Danguardian feed working last night, got tripped up with a few things, didn't realise it wouldn't work until the CRON job ran, also took ages to find/work out what a force update was. I'll detail more in my thread and add other feeds when I've got some time.
Incidentally how often should the CRON job run for pfBlockerNG?
One other thing, I understand this blocks the domain name, but would it block the actual IP address if used directly say by malware?
-
A safe setting is once a day, you dont want to upset list maintainers by swamping thewir servers for updates.
By the way a quick update.
1 - I removed the DGA list which has 800k entries, I did some digging and found these are not verified live domains, they are generated domains from seed's that ransomware admin's were found to be used for domain generation, so its a sort of catch all list that is designed to preempt new domains coming on line and been unfiltered, but my unbound started having some issues, however I may readd it later due to what I found in issue #3 see below.
2 - I added a IP BL for some ransomware servers, this has to be added in the ipv4 section as its not DNS filtered, but firewall filtered. url for list is here, site says is updated every 5 mins but I at least for now set it to once a day. https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt
3 - I discovered that unbound is restarted very frequently if it is set to host DHCP name records, basically if the static DHCP or leased DHCP boxes are ticked in dns resolver settings, pfsense seems to be coded badly in that whenever a DHCP record is updated it will restart unbound which flushes its cache and reloads dns lists, and the 800k list I had added caused unbound to be unresponsive for 30+ seconds when this happened.
Now regarding #3, on my particular network, I can see in the resolver logs, unbound was been restarted every 10-20 minutes which is way too frequent for my liking, so I copied someone else's suggestion which is to manually maintain a DHCP dns list which I load into unbound using the custom config box with an include line and keep those 2 boxes unticked, I am only maintaining for my static DHCP leases, I dont care about dns resolution on dynamic leases.
Just reread your post.
My cron is set to every 12 hours, but I have all this lists set to only update once a day. Setting the cron to run more often shouldnt be a big deal however it may (if you add lists at different start times) stagger updates which would mean more dns reloading.
Also on the DNSBL config page near the bottom is this section
"DNSBL IP Firewall Rule Settings"
I think but I am only guessing as I have not tried it that if you enable "List Action" setting and select deny, then it may add the resolved ip's to the firewall.
Just be aware that many domains can be hosted on a single ip, so lets say a ransomware dude is hosting his domain on a shared web hosting server mixed in with legal customers sharing the same ip, you could also block all those sites.
-
Was using my phone in bed earlier and there was a internet outage, when I checked pfsense later I found it had a kernel panic and self rebooted, so I seem to have a hardware issue somewhere or a configuration issue.
I think the 2 most likely culprits are the ram and addon intel card (is reported issues in pfsense with intel addon cards), I have disabled msix on the intel ports and also reverted a tunable I played with yesterday and will see if stays stable, if I get another panic I will test the ram.
-
using dnscrypt now, if anyone else using pfsense wants a guide to get it working I will print one.
-
That's good news. I would appreciate a guide ;)
-
you on pfsense 2.2/2.3 or 2.4?
-
Currently on 2.3.2-RELEASE-p1 (amd64)
-
ok its cli setup 100% as there is no official pfsense package. This relies on the FreeBSD package.
change to a working directory e.g. /root
cd /rootdownload the FreeBSD 10 dnscrypt-proxy package
fetch http://pkg.freebsd.org/FreeBSD:10:amd64/latest/All/dnscrypt-proxy-1.8.1.txzinstall the package
pkg install dnscrypt-proxy-1.8.1.txztest if there is no runtime errors, by running the binary with no arguments, should just see generic output telling you that syntax is needed
dnscrypt-proxynow the next bit is dependent on your own config, there is various dnscrypt guides around the web, We assume you will be using opendns dnscrypt servers. I cannot paste mine as its using my private dns server.
So run this command which will use the built in database to connect to opendns (cisco)
dnscrypt-proxy --ephemeral-keys --local-address=127.0.0.1:65053 --daemonize -R ciscoyou should see a warning that they do logging and also that opendns has no dnssec, but no other output aside from those 2 lines, you can verify if its running with this.
ps ax | grep dnsand look for this
dnscrypt-proxy --ephemeral-keys --local-address=127.0.0.1:65053 --daemonize -R ciscoif its running then you want it to startup auto on boot so the following 2 commands.
sysrc dnscrypt_proxy_enable=YES
sysrc dnscrypt_proxy_flags='--ephemeral-keys --local-address=127.0.0.1:65053 --daemonize -R cisco'
Now it is done but isnt actually been used.
You have created a encrypted tunnel for dns between your router and opendns, but you still need to tell the router to use that tunnel, and in this case to use the tunnel you need to forward dns queries to 127.0.0.1 port 65053
I dont think pfsense supports custom ports in its GUI so in the dns resolver settings scroll down to where you see a box for custom options, and add this
forward-zone:
name: "."
forward-addr: 127.0.0.1@65053
now unbound will forward all internet queries to the tunnel after you save and apply the settings.
That is finally done.
Notes
If you ever update pfsense to 2.4, the binary will stop working, you will need to uninstall the package, and then install the FreeBSD 11 package.
Pfsense wont manage the package meaning if you want to keep up with new versions of dnscrypt-proxy you need to keep an eye on the FreeBSD repo for updates. An easy way is checking on freshports.org.
-
Thanks Chrys I'll give it a go.
Incidentally I saw some of your posts on the pfSense forums (there are so many forums there), that fellow who was droning on about dnscrypt being unnecessary clearly lives in the USA and has no clue what life is like in other parts of the world. I doubt he's travelled outside his home town.
-
One small comment: it's not necessary to fetch the package before installing it. Just type (as root)
pkg install dnscrypt-proxy
which will fetch and install the latest version of dnscrypt-proxy in the repositories.
-
no it wont eric as it doesnt exist on the pfsense repos.
pkg install without a local package in the syntax will only install packages on the pfsense repositories.
root@PFSENSE pfblockerng # pkg install dnscrypt-proxy
Updating pfSense-core repository catalogue...
Fetching meta.txz: 100% 940 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 2 KiB 1.7kB/s 00:01
Processing entries: 100%
pfSense-core repository update completed. 7 packages processed.
Updating pfSense repository catalogue...
Fetching meta.txz: 100% 940 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 121 KiB 123.7kB/s 00:01
Processing entries: 100%
pfSense repository update completed. 444 packages processed.
pkg: No packages available to install matching 'dnscrypt-proxy' have been found in the repositories
-
Fair enough, I failed to see your comment that it isn't in the pfSense repos.
-
No worries. Obviously if it was in the repos that would be the way to install it, but when I requested it to be added, as dray mentioned I had hostile responses telling me there is no point in the package.
I also think adding the FreeBSD repo in the pfsense config (meaning pkg install would search the FreeBSD repo) is a bad idea as that could cause issues for other packages, this method I feel is the safest way.
If one wanted to remove one step of the dns processing, then they could disable the dns resolver and bind the proxy to port 53, however you then lose dns caching as well as other benefits of the unbound resolver such as been able to filter with pfblockerNG.
-
Incidentally how often should the CRON job run for pfBlockerNG?
I have made my cron more frequent now at every 2 hours, the reason is at midnight for some reason 3 of the feeds couldnt download, and so it auto retries to the next cron, but with my cron been every 12 hours it meant 12 hours to wait for a new attempt.
So the update frequency in the feed settings controls how often updates will happen for the feed, but the cron setting is how often it will 'check' if updates are needed, so a cron every 2 hours will not do updates every 2 hours if the feed is set to once a day.
The issue I have with frequent cron runs is if the dnsbl feeds get staggered at different intervals since each dnsbl feed update reload's unbound. So I will keep an eye on it and if I find they get staggered then will revert to infrequent crons again.
-
ok its cli setup 100% as there is no official pfsense package. This relies on the FreeBSD package.
change to a working directory e.g. /root
cd /rootdownload the FreeBSD 10 dnscrypt-proxy package
fetch http://pkg.freebsd.org/FreeBSD:10:amd64/latest/All/dnscrypt-proxy-1.8.1.txzinstall the package
pkg install dnscrypt-proxy-1.8.1.txztest if there is no runtime errors, by running the binary with no arguments, should just see generic output telling you that syntax is needed
dnscrypt-proxy
I have a problem at this point, this command just returns "Command not found" so I tried /usr/local/sbin/dnscrypt-proxy
which gives the error "Undefined symbol crypto_core_hchacha20"
I think crypto_core_hchacha20 is part of libsodium, but I'm stuck now.
-
yeah you need to check if these files exist
root@PFSENSE inc # ldd /usr/local/sbin/dnscrypt-proxy
/usr/local/sbin/dnscrypt-proxy:
libltdl.so.7 => /usr/local/lib/libltdl.so.7 (0x80123a000)
libsodium.so.18 => /usr/local/lib/libsodium.so.18 (0x801443000)
libkvm.so.7 => /lib/libkvm.so.7 (0x8016b3000)
libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x8018c1000)
libm.so.5 => /lib/libm.so.5 (0x801ac4000)
libc.so.7 => /lib/libc.so.7 (0x800823000)
libthr.so.3 => /lib/libthr.so.3 (0x801cef000)
libelf.so.2 => /lib/libelf.so.2 (0x801f16000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x80212a000)
run ldd on your own system tho as since I have FreeBSD 11 my /lib ones will be different.
root@PFSENSE inc # pkg which /usr/local/lib/libsodium.so.18
/usr/local/lib/libsodium.so.18 was installed by package libsodium-1.0.11_1
I think you right, I must have had it already installed as a dependency of another pfsense package.
So pkg install libsodium should fix it.
-
yeah you need to check if these files exist
root@PFSENSE inc # ldd /usr/local/sbin/dnscrypt-proxy
/usr/local/sbin/dnscrypt-proxy:
libltdl.so.7 => /usr/local/lib/libltdl.so.7 (0x80123a000)
libsodium.so.18 => /usr/local/lib/libsodium.so.18 (0x801443000)
libkvm.so.7 => /lib/libkvm.so.7 (0x8016b3000)
libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x8018c1000)
libm.so.5 => /lib/libm.so.5 (0x801ac4000)
libc.so.7 => /lib/libc.so.7 (0x800823000)
libthr.so.3 => /lib/libthr.so.3 (0x801cef000)
libelf.so.2 => /lib/libelf.so.2 (0x801f16000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x80212a000)
run ldd on your own system tho as since I have FreeBSD 11 my /lib ones will be different.
I ran ldd but not all of the files you listed were there
So pkg install libsodium should fix it.
I tried that and it installed from the pfsense repository but it didn't work so I had a look on pkg.freebsd.org/FreeBSD:10:amd64/latest/All/ and I saw libsodium-1.0.11_1.txz there which i fetched and installed and now /usr/local/sbin/dnscrypt-proxy runs :)
-
ok but I find that odd as I never had to do that and mine did work when I was on pfsense 2.2. But it works for you now which is good :) but just hope that dependency been installed from FreeBSD doesnt break any pfsense packages that have the same dependency.
-
I dont think pfsense supports custom ports in its GUI so in the dns resolver settings scroll down to where you see a box for custom options, and add this
forward-zone:
name: "."
forward-addr: 127.0.0.1@65053
now unbound will forward all internet queries to the tunnel after you save and apply the settings.
Unfortunately this causes my DNS to stop working :(
Is there any debugging I can do?
-
The commands I gave to set on bootup sadly wont be valid for pfsense.
You need to install the shellcmd addon package in the GUI.
system -> package manager -> available packages - select shellcmd
When its done goto services and select shellcmd
set shellcmd type to shellcmd and the command in left box and description in right box.
Regarding your broken dns lookups, you can check your live unbound.conf with the command 'cat /var/unbound/unbound.conf' Stick the output on pastebin and if you want to keep it confidental pm me the link.
-
As I thought would happen the mini pcie Jetway card I got for intel ports is no longer sold.
this is what I put in the unit.
https://webcache.googleusercontent.com/search?q=cache:gy2pXys8PpMJ:https://linitx.com/product/13534+&cd=6&hl=en&ct=clnk&gl=uk
-
Unfortunately this causes my DNS to stop working :(
It seems I have to add another line to this to make it work, like this do-not-query-localhost: no
forward-zone:
name: "."
forward-addr: 127.0.0.1@65053
I now have another problem in that OpenDNS servers don't support DNSSEC, so I have to turn that off
-
I've added Shellcmd with the following:
sysrc dnscrypt_proxy_enable=YES shellcmd dnscrypt start on bootup
sysrc dnscrypt_proxy_flags='--ephemeral-keys --local-address=127.0.0.1:65053 --daemonize -R cisco' shellcmd set dnscrypt parameters on bootup
-
It seems it's possible to tell if dnscrypt is working by doing
nslookup -querytype=txt debug.opendns.com
The answer includes the line debug.opendns.com text = "dnscrypt enabled (01234567890123456789)"
-
the command to add in shellcmd is this
/usr/local/sbin/dnscrypt-proxy --ephemeral-keys --local-address=127.0.0.1:65053 --daemonize -R cisco
-
Thanks :)
-
As good as pfSense is their (core) developers seem to have attitude, I been wasting my time writing up detailed bug reports as they just seem intent on hitting the reject button to keep the bug count down. So I will probably stop posting on the pfSense forum soon as well and just keep my discussion on here.
-
An update regarding my hardware.
When I was debugging some ALTQ stuff I was testing with the 2 realtek ports, at which time I discovered one of the ports is not working, someone else with the same unit has reported the same exact issue as well, so be wary if ordering the unit I purchased and are not adding extra ports.
-
this is with FAIRQ+codel
and a custom rule I added to prioritise all ack's. I think this one was with a 93.5% cap of max upload bandwidth.
http://www.dslreports.com/speedtest/8577920
This one was set to 97%.
http://www.dslreports.com/speedtest/8574510
-
pfblockerng has a very nice feature which can be used for traffic classifying.
see this data
Alias Count Packets Updated
pfB_Dhshield24 40 1 Jan 10 18:15 (2)
pfB_Dshield 101 0 Jan 10 18:15 (2)
pfB_Emerging 1731 0 Jan 10 18:15 (2)
pfB_MalwareExploits 135 0 Jan 10 18:15 (2)
pfB_Netflix 36 0 Jan 10 18:15 (1)
pfB_RansomCryptoware 11548 0 Jan 10 18:15 (2)
pfB_Steam 64 0 Jan 10 18:15 (1)
pfB_bbc 41 0 Jan 10 18:15 (1)
pfB_blizzard 44 0 Jan 10 18:15 (1)
pfB_google 6759 5 Jan 10 18:15 (1)
DNSBL_MalwareExploits 213791 5 Jan 09 19:45:33
DNSBL_PrivacyFraud 191072 2 Jan 09 19:46:15
DNSBL_Cryptolocker 830095 0 Jan 08 23:49:34
Aside from malware lists you may notice I have entries like netflix, google,steam and bbc.
What I have done is entered the ASN information into pfblockerng and it can grab the ip ranges those companies manage into a firewall table. I then used that information to add rules to send traffic based on those tables to the QoS queue I want.
For steam this is working perfect. It also works on google services if I disable ipv6. Netflix, iplayer are not that easy tho as the bbc is outsourcing to limelight networks and not using their own ip space for iplayer, netflix is coming over from local sky nodes. Also when ipv6 is enabled google and netflix come over it and the ASN feature is ipv4 only. But still I thought this was worth mentioning as it is a powerful tool.
I do have the traffic shaping working quite nicely now albeit after I had to configure it differently than how pfSense documention suggests and I caused a ruckus on the pfSense forum when trying to explain the issues I had with the default behaviour.
--edit--
ipv6 ASN's are supported, google now working good over this system as well :)
-
Since steam downloads from port 80 I could not lower its priority in conventional ways, this is classified via ASN match.
-
I am planning to install 2.4 fresh using ZFS so I can ditch UFS. Will post here when I do the work.
-
Ok I have now successfully migrated to zfs since I am not confident of using ufs for reliability, there is some gotchas that occured which I will mention here.
Please read everything before you consider doing this as is some gotchas.
So the process I did was as follows.
1 - Run the backup wizard under diagnostics menu to make a backup, make sure everything is included in the backup, a box is ticked by default which doesnt backup traffic usage data, I unticked it.
2 - If you have any custom files anywhere on the filesystem then back them up as they will be lost. Also if you want to preserve any logs back them up also.
3 - Download the pfsense 2.4 installer, which is on the development download section, pfsense 2.4 is required if you want native support for 4k alignment, trim and zfs.
4 - Put the installer files on your install media, in my case is a usb stick and I used rufus usb tool.
5 - Reboot the pfsense unit and boot of the install media.
6 - Choose zfs (guided) install
7 - select striped and pick your storage device
8 - enable forced 4k alignment option.
9 - GPT enable if you have UEFI bios or disable if you dont.
10 - proceed with install and let it finish and reboot
11 - after reboot access the web UI with default admin/pfsense login details.
12 - when the wizard appears ignore it and instead access the diagnostics menu and choose backup and restore.
13 - restore your backup and making sure the option to also restore packages is enabled (was enabled by default on my unit).
14 - Watch the console as it restores your packages to see if any issues.
15 - When completed the core pfsense system and all official plugins should be restored.
Gotchas
The first issue I had is after restoring my backup and it rebooted, it could not get internet access to redownload and install the packages, the reason for this is unbound was not working for 2 reasons (I initially thought was just one reason). This was because in my case dnscrypt was missing and as such I had no working dns tunnel, and also that unbound didnt start due to missing pfblockerng files causing a syntax error. I noticed on the console it was retrying every minute or so so I simply edited /etc/resolv.conf as follows to make the router use google dns temporarily.
nameserver 8.8.8.8
The restore packages process then successfully finished.
I then had to pkg install dnscrypt again, but that was all I had to do for that as the earlyshell cmd configuration was intact so on a reboot it started properly.
Unbound however was down still because it was trying to load pfblockerng dnsbl files that were missing, so again I had to temporarily enable google dns on the router manually, and then in the pfblockerng gui I manually ran the cron process, which downloaded all the lists and created its configuration files, after that unbound runs normally.
People who dont use pfblockerng dnsbl lists and dont use dnscrypt wouldnt have this problem.
I restored my custom files like custom loader.conf and services.inc (to fix unbound restarts) and all seemed well.
One final gotcha.
The traffic totals plugin is missing at this point, it was not restored when the restore wizard ran, I also forgot to backup its data when I backed up my custom files. So this plugin has to be reinstalled manually post restore.
Also I enabled a zfs feature that makes extra copies of every stored file, the command to do so is this.
zfs set copies=3 zroot
Bear in mind tho this will not make copies of existing files unless they get rewritten, only new writes have the automated copies, it is also nowhere near as good as a zfs mirror setup but is better than a plain single copy on a single drive setup. the reason to make 3 copies instead of 2 is so when zfs detects corruption in a copy it can determine the correct copy by a majority rules system where by where 2 copies match, they will be determined to be the correct copy.
-
I decided to check the raw smart data of the ssd, and considering I have used this pfsense unit for about a month I am surprised at how much has been written.
175 gig of data written on the ssd.
In comparison my desktop ssd, which hosts games and the OS, and I have used for over 2 years has 8.34tb of writes.
Bear in mind tho the ssd in my pfsense unit is only 60gig vs 512gig of my desktop ssd and its planar nand vs 3d nand in my pc. So the endurance of the ssd in the pfsense unit is way less than whats in my pc. both are MLC nand.
The largest dnsbl feed with 800k domains the file when downloaded is circa 100mbyte in size, this file is then converted to another format to be compatible with unbound, the converted file is 44mbytes in size. So on every update is about 150mbyte of writes from that feed, the updates are once a day but I have probably done a few dozen manual updates on top of that when testing stuff. Now I have zfs compression in play tho will continue monitoring and see if the rate of writes slows down (with 3 copies set it may well actually jump up), there is also the possibility this drive was not shipped unused to me (a return) as I didnt check before using it if the stats were all in a new state.
-
Been doing some more testing with the traffic shaper, whilst I find FAIRQ+codel superior for upstream traffic, it doesnt seem to be perfect for downstream and I am now currently using hfsc+codel which has showed better results in some quick tests using steam, dslreports and tbb speedtesting but will need time to see how it goes with days of internet use.
pfTop: Up Queue 1-14/14, View: queue, Cache: 10000 14:55:55
QUEUE BW SCH PRIO PKTS BYTES DROP_P DROP_B QLEN BORROW SUSPEN P/S B/S
qInternet 19M fair 0 0 0 0 0 0 0
qACK 0 fair 6 307812 19973104 0 0 0 0 0
qDefault 0 fair 4 248278 159647K 0 0 0 3 207
qOthersHigh 0 fair 5 13414 1717003 0 0 0 0 0
qOthersLow 0 fair 3 143551 82363014 0 0 0 0 0
qICMP 0 fair 7 1728 113444 0 0 0 2 221
qBulk 0 fair 2 649 256446 0 0 0 0 0
root_igb1 67M hfsc 0 0 0 0 0 0 0 0
qDefault 3357K hfsc 220764 249884K 0 0 0 2 357
qICMP 1342K hfsc 428 31672 0 0 0 0 0
qACK 3357K hfsc 67140 3883408 0 0 0 0 0
qOthersHigh 33M hfsc 13584 1638300 0 0 0 0 0
qOthersLow 13M hfsc 193416 284731K 35 52990 0 0 0
qBulk 6714K hfsc 179147 263734K 3894 5824621 0 0 0
Steam for me remains the ultimate test, as that floods the line with dozens of tcp sessions (seems they designed it to get round poor isp's) and under normal circumstances it will cause packet loss on small packets like ssh packets and pings. Steam testing has been more positive using hsfc than fairq and priq.
-
found this gem, add /status.php to end of the ip url so e.g. https://192.168.1.252/status.php and you get a very nice informational page :)
-
found this gem, add /status.php to end of the ip url so e.g. https://192.168.1.252/status.php and you get a very nice informational page :)
Thanks!
I can see that being very useful at times.
-
this looks pretty sweet for anyone considering pfsense.
http://www.asrock.com/ipc/overview.asp?Model=NAS-9601
Newer gen version of my CPU and the unit has 6 intel ports, and it has VGA out so no serial console stuff needed.
-
Very nice, but seems it's not readily available.
-
this looks pretty sweet for anyone considering pfsense.
http://www.asrock.com/ipc/overview.asp?Model=NAS-9601
Newer gen version of my CPU and the unit has 6 intel ports, and it has VGA out so no serial console stuff needed.
That IS nice but Very nice, but seems it's not readily available.
AKA does anyone sell it?
OTOH, bet its going to be expensive, plus I have an Asrock BeeBox N3000 (http://www.asrock.com/microsite/beebox/) and its a pile of unreliable crap.
Chunks
-
this looks pretty sweet for anyone considering pfsense.
http://www.asrock.com/ipc/overview.asp?Model=NAS-9601
Newer gen version of my CPU and the unit has 6 intel ports, and it has VGA out so no serial console stuff needed.
I looked at the on-site images and could not see a VGA port . . . So I downloaded the data sheet. That mentions VGA under the "Graphics" heading but the section under the "Rear I/O" heading states "VGA 0"!
All a bit of a mystery. :-\
-
I have reduced zfs copies back to the default 1 on /tmp and /var and left at 3 for the rest of the system.
With this it is averaging 5 gig writes a day to the ssd, which I am pretty sure is mostly due to dnsbl feed processing. If we assume my ssd was at 0 bytes writes when installed, then I believe this to be a similar rate of writes as ufs. This is because the ssd is a sandforce ssd which has native compression, on a normal ssd which doesnt compress I think zfs will reduce writes.
On pfSense /var is mostly storage for logs and dynamically generated files (including dnsbl files), so any corruption to these files should not have any persistent effects hence me changing it back, likewise /tmp is only housing short lived files.
I also changed globally logbias on zfs from latency to throughput which is more friendly to ssd's.
I am going to examine the pfblockerng code to see where it writes its files during the update process and see if some of the work can be moved to a ramdisk, I dont want to enable the built in pfsense ramdisk as it is by default as that makes alot of stuff non persistent such as logs and processed dnsbl files, meaning on a reboot logs are lost and pfblockerng has to download a new set of files due to the old ones been on volatile storage. What I am aiming to do is have the original downloads and any in between processing files on ram storage and then only the completed dnsbl files on persistent storage.
/tmp also houses the pf rule set and a config.cache file which are frequently rewritten but both are only small files.
I also have been experimenting with different power states for the cpu.
Since I had another panic last week, I disabled some power saving functions in the bios which can cause instability, with the most important one I believe disabling DVFS on the ram (so ram stays at stock clocks and voltage throughout). My current config now also has powerd set to not let the cpu drop below its stock speeds, but still allow it to ramp upto turbo speeds (so things like dnsbl updates are faster), this also removes low voltage states from the cpu which can cause instability. This config adds probably about 3-5C to my average cpu temps.
So the experiment has been to let 2 cpu cores enter C2 state when idle (very low performance cost) and the other 2 cores to C3 (moderate performance cost when going from idle to load). This actually did not achieve anything significant so I will be reverting all to C2, or even leaving at the default C1.
Here is some data, note core 0 seems to be always in a wake state doing something so is unaffected mostly by any changes.
cores 0,1 set to C2 cores 2,3 set to C3
dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.3.cx_usage_counters: 1608486 118501 844080
dev.cpu.3.cx_usage: 62.56% 4.60% 32.82% last 33390us
dev.cpu.3.cx_lowest: C3
dev.cpu.3.cx_supported: C1/1/1 C2/2/500 C3/3/1000
dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.2.cx_usage_counters: 287976 14077 383181
dev.cpu.2.cx_usage: 42.02% 2.05% 55.91% last 16161us
dev.cpu.2.cx_lowest: C3
dev.cpu.2.cx_supported: C1/1/1 C2/2/500 C3/3/1000
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.1.cx_usage_counters: 1925350 1301238 0
dev.cpu.1.cx_usage: 59.67% 40.32% 0.00% last 43us
dev.cpu.1.cx_lowest: C2
dev.cpu.1.cx_supported: C1/1/1 C2/2/500 C3/3/1000
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 39609431 170 0
dev.cpu.0.cx_usage: 99.99% 0.00% 0.00% last 102us
dev.cpu.0.cx_lowest: C2
dev.cpu.0.cx_supported: C1/1/1 C2/2/500 C3/3/1000
This shows core 0 99.99% of time is in C1 state.
Core 1 about 40% of time it manages to get in C2 state.
Cores 2,3 similar to core 1 except they drop to C3 instead of C2.
The temperatures for this config? well here it is.
dev.cpu.3.temperature: 38.0C
dev.cpu.2.temperature: 38.0C
dev.cpu.1.temperature: 37.0C
dev.cpu.0.temperature: 37.0C
Pretty much no benefit from either C2 or C3, C3 actually seems to make things worse as there is some work involved for the cpu to move between states and C3 takes much longer than C2 to move in and out of.
Here is data for this morning after it been idle and also no heating on.
root@PFSENSE tmp # sysctl dev.cpu |grep cx
dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.3.cx_usage_counters: 2572229 196691 2064691
dev.cpu.3.cx_usage: 53.21% 4.06% 42.71% last 16775us
dev.cpu.3.cx_lowest: C3
dev.cpu.3.cx_supported: C1/1/1 C2/2/500 C3/3/1000
dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.2.cx_usage_counters: 800620 38919 1055540
dev.cpu.2.cx_usage: 42.24% 2.05% 55.69% last 58977us
dev.cpu.2.cx_lowest: C3
dev.cpu.2.cx_supported: C1/1/1 C2/2/500 C3/3/1000
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.1.cx_usage_counters: 3184420 3416255 0
dev.cpu.1.cx_usage: 48.24% 51.75% 0.00% last 702us
dev.cpu.1.cx_lowest: C2
dev.cpu.1.cx_supported: C1/1/1 C2/2/500 C3/3/1000
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 112912517 423 0
dev.cpu.0.cx_usage: 99.99% 0.00% 0.00% last 252us
dev.cpu.0.cx_lowest: C2
dev.cpu.0.cx_supported: C1/1/1 C2/2/500 C3/3/1000
root@PFSENSE tmp # sysctl dev.cpu |grep temperature
dev.cpu.3.temperature: 36.0C
dev.cpu.2.temperature: 36.0C
dev.cpu.1.temperature: 35.0C
dev.cpu.0.temperature: 35.0C
Everytime i check the pattern is fairly reliable that cores 2 and 3 have higher temps than core 1, core 0 can get higher sometimes. The power savings from these mode in terms of raw watts are also very low, as the cpu itself is only rated at 6 watts, so doesnt use much power to begin with.
Quick update, C2 is actually providing a meaningful benefit, I first dropped cores 2,3 to C2, and this was the result. A 1C drop to match core 1.
dev.cpu.3.temperature: 36.0C
dev.cpu.2.temperature: 36.0C
dev.cpu.1.temperature: 36.0C
dev.cpu.0.temperature: 36.0C
Then watch what happens when I lock core 1 to C1 only.
dev.cpu.3.temperature: 36.0C
dev.cpu.2.temperature: 36.0C
dev.cpu.1.temperature: 39.0C
dev.cpu.0.temperature: 36.0C
and one minute later
dev.cpu.3.temperature: 36.0C
dev.cpu.2.temperature: 36.0C
dev.cpu.1.temperature: 40.0C
dev.cpu.0.temperature: 36.0C
However core 0 , even tho spends 99.99% of time in C1 doesnt have temps that high which is interesting, likewise if lock core 0 to C1 it has no affect as its 99.99% of time in that state anyway.
-
Found the cause of the writes, its not pfblockerng, its the rrd graphs generated by pfsense.
in /var/db/rrd is the graphing databases and they are updated every minute, on my system they are 7.2meg in size in total. Multiply that by 60 minutes and thats 480meg of writes every hour, or 11.5gig every 24 hours. (when uncompressed).
I took the idea from the scripts deployed to backup the traffic stats, so did the following.
1 - temporary disabled graphing.
2 - created a backup script which will run at a set interval, and ran it to create an initial backup.
3 - wiped /var/db/rrd but left the directory in place so can mount on it.
4 - created a small ramdisk and mounted to the location which in my case I chose 200meg which should easily be enough.
5 - created a restore script and ran it to restore the files.
6 - reenabled graphing again.
I will next set a cron entry to run the backup script at intervals, probably once an hour or maybe every 15 minutes.
Also will be the script added to shellcmd to generate ram disk and restore the backup at boot.
-
Ok the SSD I put in my pfsense unit as it turns out has a warranty write limit higher than my much more expensive samsung 512 pro.
The kingston msata ssd 60gig has a 218TB warranty coverage.
-
since i had to take unit apart for it been stuck in hibernation mode after a power cut i took a new pic to show how tight a fit to add the intel i350 nic ports
https://drive.google.com/file/d/0B7P3Ne0hzKcGd3FrWGxDdUhDbkk/view?usp=drivesdk
-
I looked at the on-site images and could not see a VGA port . . . So I downloaded the data sheet. That mentions VGA under the "Graphics" heading but the section under the "Rear I/O" heading states "VGA 0"!
All a bit of a mystery. :-\
yeah it has a vga header on board but no port on case so needs a attachment
-
You're running 2.4, you could try fq_codel instead of fairq+codel. Should be better.
https://forum.pfsense.org/index.php?topic=126637.0
-
on todo list :)
-
look like kingston have withdrawn the models of their msata from the market below the 120 gig size, the 120 gig hasnt fell down to the price range they were at so basically you can no longer get a new kingston msata in the £30 price range. All I can find now in that price range for a msata ssd brand new is chinese brands like kingfast.
-
Unrelated to the OP / thread:
Many of the posters have their kit setup in their sigs. This is getting parsed for Google searches, and as a result, when I searched for "Billion" "8800" and other search-related data, this thread popped up, but is not relevant at all.
Might I suggest that sigs not include kit, or that the site prevents parsing of sigs?
-
Funnily enough the same thing happened to me when I googled squiggle
-
camieabz this thread may have come up, but then you would find the 8800nl in my sig is clickable and points to one of my 8800nl threads.
Also i have pfsense in my sig which would then have this thread come up for pfsense searches as well.
Swings and roundabouts really.
-
dnscrypt-proxy v1 is now considered obsolete, and is been removed from FreeBSD repo's in due course, so I have v2 a replacement client developed by another developer running, so far so good and will post a guide later, there is also an official guide for it as well here.
https://github.com/jedisct1/dnscrypt-proxy/wiki/Installation-pfsense
I will post my own guide also, largely similar but added notes in relation to my tuning and using the amd64 binaries (official guide uses ARM binaries), and getting it to work with your own personal dnscrypt server.
Also a comparison with the older proxy software here.
https://github.com/jedisct1/dnscrypt-proxy/wiki/Differences-to-v1
The original dnscrypt-wrapper is still been developed and works with this v2 proxy. The v2 proxy also works with unbound DNS servers using crypto v2 or DNS over http/2.
-
pfblockerng-devel is now the recommended version of pfblockerng.
To update (keeps all existing settings so is a simple swap in), first make sure keep settings is enabled in pfblockerng settings.
Goto packages and uninstall pfblockerng.
Then in packages install pfblockerng-devel.
Then goto pfblockerng-devel and force a reload of lists.
Thats it, should be good.
There is considerable enhancements and highly recommended people switch, the dev said its very soon the old stable will be retired and devel becomes main version.
The most obvious enhancements are the feeds section and the alerts system.
https://old.reddit.com/r/pfBlockerNG/comments/9suo5q/pfblockerng_devel_version_released/
-
Thanks for posting this, when the devel becomes the main version will the old version auto update?
-
I have no idea, I will ask and put the response here.
-
heads up for those relying on the GEO databases.
https://twitter.com/BBcan177/status/1208805143535325184
Will need to register in MaxMind for updates.
-
heads up for those relying on the GEO databases.
https://twitter.com/BBcan177/status/1208805143535325184
Will need to register in MaxMind for updates.
Thanks for posting this.
Probably useful to include this: https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/
-
An update, my suggestion keep "serve expired" enabled in the DNS resolver advanced settings. Can also see prefetch is enabled as well.
Example stats, look at that cache hit rate.
root@PFSENSE ~ # unbound-control -c /var/unbound/unbound.conf stats_noreset | egrep 'total.num|cache.count'
total.num.queries=218959
total.num.queries_ip_ratelimited=0
total.num.cachehits=216339
total.num.cachemiss=2620
total.num.prefetch=28326
total.num.expired=21661
total.num.recursivereplies=2620
msg.cache.count=2333
rrset.cache.count=2034
infra.cache.count=3
key.cache.count=0