Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: New network setup with group isolation issues  (Read 4094 times)

re0

  • Reg Member
  • ***
  • Posts: 840
New network setup with group isolation issues
« on: February 03, 2018, 07:27:38 PM »

Hi Kitz Forum!

I recently setup a new network on a new broadband connection with multiple isolated groups, which was pretty much identical to the previous setup in the sense that it is using the same router (Billion 8800AXL R2) and the same devices. The only real change to the internal network is the addition of a new group which is to be used for devices exposed through the one-to-one NAT (such as VMs which have been moved to it).

I thought it was all perfect until I came across a “problem” yesterday where I could not access one of my Virtual Machines via one of my external IP addresses from my PC, despite it being correctly configured in the one-to-one NAT, and despite it working from accessing the internal IP on my second network adapter (so it is certainly reachable). External websites could ping and port scan. I tried the external IP address on my smartphone using mobile data and it could reach the webpage and SSH and, quite surprisingly, I discovered that my wireless devices (which also includes the smartphone) could also access the services running on the VM with the external IP address.

Of course, at this point, I was a bit perplexed. So I did try for the remainder of yesterday and most of today so far to try and diagnose the issue to “why” externally it was possible and “why” from wireless devices it was possible, but before I continue babbling I should provide enough network information to give a better-than-vague overview of the network (although please bear in mind that some of the information may be for the sake of example).

So here are the groups in [Group] ([Name]) – [Address/Netmask] – [LAN FW] format (please note that every group is isolated):

Group 1 (Default) – 10.0.0.0/24 – LAN side firewall DISABLED
Group 2 (VM) – 10.0.1.0/24 – LAN side firewall ENABLED
Group 3 (WLAN) – 10.0.2.0/24 – LAN side firewall ENABLED

So with every group isolated, they will not be able to contact each other (or at least not directly).

Since I have a couple static IP addresses, I wanted to one-to-one NAT one with one of my VMs so I could expose it to the internet. For example, a VM is running in Group 2 (VM) on 10.0.1.1:

Ext.: 123.123.123.123 – Int.: 10.0.1.1 on ppp1.1 interface with no Exceptional Rule Group

To build up one of the previous paragraphs mentioning which devices were working and not:

Group 1:
PC – NO
Server – NO

Group 2:
Same PC as above – YES

Group 3:
Smartphone – YES
Laptop – YES

Checking the logs on the VM, any time a device from either Group 2 or 3 made a connection to any of the services, it could log the gateway device as the source (10.0.1.254 for the former and 10.0.2.254 for the latter). But there was no evidence of devices from Group 1 even contacting any of the services.

I did some further experimentation and setup the same external IP address through the one-to-one NAT for the server in Group 1, then I could reach it via the external IP address no issue from my PC (though I can admit NOT seeing whether the source was the gateway or the device IP, but I imagine the former), but with this change the devices on the other groups could not contact it.

So I thought a lot about it, and thought it cannot be the groups isolation as the other isolated groups work fine (and it is mandatory that I instate this functionality for my network). I thought to try adding another isolated group (Group 4), but add this one without a LAN side firewall (like the default group). I was not surprised but baffled when that also did not work when trying to access the VM on Group 2 via the external IP address.

I tried a few other changes, such as disabling the LAN side firewall and to my surprise this caused the existing devices in other groups to not be able to contact the VM using the external IP address. I also noticed that re-enabling it did not rectify the issue without a reboot of the router. So I went ahead and configured the networking something like below and rebooted:

Group 1 (Default) – 10.0.0.0/24 – LAN side firewall DISABLED
Group 2 (VM) – 10.0.1.0/24 – LAN side firewall ENABLED
Group 3 (WLAN) – 10.0.2.0/24 – LAN side firewall ENABLED
Group 4 (Test) – 10.0.3.0/24 – LAN side firewall ENABLED

I put the PC on Group 4, and success – it was able to reach the services via the external IP address. Of course, the source was showing as 10.0.3.254.

Of course, the quick and dirty fix for this issue would be to simply just make an entry in the hosts file on every device in Group 1 with the internal IP address (which is reachable through the 2nd adapter locally) and the domain name in used for the external IP address. But I do not necessarily plan to use a domain for every external IP address if at all in the future.

So the results from this experimentation is that the LAN side firewall simply does not allow connection to the VM via the external IP address. But why is this happening and how to resolve it? I know a bit about networking, but I cannot see how the LAN side firewall being off actually creates the issue.

The TL;DR version:
LAN side firewall being disabled prevents accessing the external IP address (configured through the one-to-one NAT) for another device on the network which is isolated through group isolation that otherwise works when the firewall is enabled.

If anyone could provide some insight to what I can do to resolve this issue, it would be much appreciated.

re0
Logged
ISP: Gigaclear - Hyperfast 900 (up to 940 Mbps symmetrical)

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: New network setup with group isolation issues
« Reply #1 on: February 03, 2018, 11:01:41 PM »

Welcome to the Kitz forum.  :)

As for your current problem, I have carefully read through the details and, to be honest, I feel somewhat lost. We do have members well versed in networking, so hopefully one of them will be able to make some suggestions.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

re0

  • Reg Member
  • ***
  • Posts: 840
Re: New network setup with group isolation issues
« Reply #2 on: February 04, 2018, 01:35:51 AM »

Hi burakkucat,

Thanks for the warm welcome.

I do ramble on, so being lost would certainly be reasonable in any case regardless of whether it is typing or in conversation (I like to think of myself of being precise, but I am the complete opposite at the moment!). Perhaps if I had drawn something crummy in paint it would be have better illustrated the point.

I did check through all the details, but I rushed it just before posting because I was in a hurry. There may be a few discrepancies, but the ultimate goal is to try and see why disabling the LAN side firewall while the group isolation is in place prevents access to the external IP address which is in a one-to-one NAT with a device in a different group and how to get around this (since I still need access to the router settings).

I continued looking into this late yesterday evening and night with someone else and we finally looked at IP filtering. So far, it looks like enabling the LAN side firewall on the group and creating rules in both the inbound and outbound (for forwarding) filtering for the IP addresses and interfaces that need to access it allows access to both the external IP address and the router interface.

Now I am just bemused that when modifying the LAN side firewall to the on state (in a case where it is turned off) either requires a reboot of the router or for group isolation to be turned off and back on in order for access to the external IP address. I honestly do not know what the expected behaviour of this should be.

Either way, it would certainly be appreciated if someone knows a better way to do it since I am not a half-job type of person. But if this is the best way, then I can pat myself on the back and just get back to sorting out the VMs!

re0
Logged
ISP: Gigaclear - Hyperfast 900 (up to 940 Mbps symmetrical)

d2d4j

  • Kitizen
  • ****
  • Posts: 1103
Re: New network setup with group isolation issues
« Reply #3 on: February 04, 2018, 11:52:32 AM »

Hi re0

I started to read your post last night but gave up sorry

Is this home or business and could I ask why you need such a setup

I take it groups are vlans but one worry is I do not think your router has multiple firewalls unless you mean you setup global all rules to allow traffic sorry

If your connecting from internal to your external ip, it should not go live over internet as router is aware of both internal and external ip, so should deal with it at router level. Setting rules would work but kinda breaks isolation if you see what I mean. Hence why I asked why do you need it setup that way.

Many thanks

John
Logged

re0

  • Reg Member
  • ***
  • Posts: 840
Re: New network setup with group isolation issues
« Reply #4 on: February 04, 2018, 01:39:29 PM »

Hi John,

I can understand why you gave up – it was not a very precise piece of information now I look back on it. I did rush it a bit towards the end, which I can only apologise for.

It's a home network, but with potentially unnecessary segmentation (although this is subjective – I feel it is necessary). For me, it’s just so I can setup a network with multiple external IPs and isolate groups to reduce the risk and impact of being compromised.

The groups are configured under “Interface Grouping” on the router, which I imagine are not too dissimilar from VLANs. Each group is isolated and running on different local IP ranges.

In regards to firewalls, I was simply talking about LAN side firewalls. As quoted from the router manual (http://www.billion.uk.com/esupport/index.php?/Knowledgebase/Article/View/413/123/bipac-8800axl-r2-full-user-manual, page 65):

Quote
LAN side firewall: Enable to drop all traffic from the specified LAN group interface. After activating it, all incoming packets by default will be dropped, and the user on the specified LAN group interface can't access CPE anymore. But, you can still access the internet service. If user wants to manage the CPE, please turn to IP Filtering Incoming to add the allowing rules. Note that all incoming packets by default will be dropped if the LAN side firewall is enabled and user cannot manage this CPE from the specified LAN group.

So, I have enabled the LAN side firewall for each group:
- To restrict access to the router interface (IP filtering is in place to allow one machine access)
- Because if the LAN side firewall is disabled then pinging or attempting to access resources on the external IP address (which is on a one-to-one NAT in a different group) will essentially fail

Any devices configured through the one-to-one NAT will require their own firewall to be enabled (as with the DMZ), which I understand.

I understand that attempting to connect to an additional external IP address that is on the same router should not go “live over the internet”, but the point I was trying to make previously was that with the LAN side firewall disabled there are issues connecting to the additional external IP address. The only way I have managed to get this all working with Interface Grouping also enabled is by enabling the LAN side firewall on the main LAN which was disabled before – would this be normal?

In reference to IP filtering rules, I have had to use these on the local network as I needed to allow access from one local device to the internal router IP address for the router interface (no, I am not allowing it to be accessed externally).

I can only hope this clarifies the situation.

re0
Logged
ISP: Gigaclear - Hyperfast 900 (up to 940 Mbps symmetrical)

d2d4j

  • Kitizen
  • ****
  • Posts: 1103
Re: New network setup with group isolation issues
« Reply #5 on: February 04, 2018, 01:52:40 PM »

Hi re0

Many thanks

Slightly more confusing now but getting there

I think you may have tried using routing internal IP address and not correct internal ip on group (group is a vlan) so let’s say your group 1 is  192.168.1.1, you connect to router by 192.168.1.1

Now let’s say group 2 is 10.0.0.1, so you access router using 10.0.0.1 and not 192.168.1.1

Is the above what you were trying to do

Many thanks

John
Logged

d2d4j

  • Kitizen
  • ****
  • Posts: 1103
Re: New network setup with group isolation issues
« Reply #6 on: February 04, 2018, 01:58:49 PM »

Sorry I should have said I have no experience with/using billion but mostly all routers should work similar to each other

I do believe you only have 1 firewall

Dmz does not have a firewall as such - that’s why it is a dmz

Many thanks

John
Logged

re0

  • Reg Member
  • ***
  • Posts: 840
Re: New network setup with group isolation issues
« Reply #7 on: February 04, 2018, 05:24:18 PM »

Hi John,

I do not know the best way I can clarify this. I think I have got it working as intended now, but perhaps you will be able to understand the following.

Effectively there is one firewall on the device itself, yes. Or at least in reference to NAT. I know this excludes devices outside the NAT (in the previously mentioned DMZ or one-to-one NAT) which require their own firewall configured (in my case it would be on a per-device basis, on the devices themselves).

The routing, to my knowledge, was and is configured correctly. For the sake of an example:

A device (or in this case the PC) on Group 1 would have been given an IP between 10.0.0.1-253, with the gateway being 10.0.0.254. The device would have obtained an IP from the DHCP (unless reserved) which is on the router, and the gateway would have been set to 10.0.0.254 on the device automatically. Just for clarity, let’s state that it was given 10.0.0.1.

A device (or in this case the VM) on Group 2 would have been given an IP between 10.0.1.1-253, with the gateway being 10.0.1.254. The device would have obtained an IP from the DHCP (unless reserved) which is on the router, and the gateway would have been set to 10.0.1.254 on the device automatically. Just for clarify, let’s state it was given 10.0.1.1.

Group 1 and Group 2 are isolated, so they cannot contact each other in any case internally. So the PC (10.0.0.1) cannot ping the VM (10.0.1.1) or vice-versa, which is expected behaviour and therefore working as intended.

The VM in Group 2 has been put into a one-to-one NAT with the public IP address 123.123.123.123 (so it will translate to 10.0.1.1). The main IP used by other devices is 123.123.123.122.

Now here is the point that baffled me, and that is if either or if both Group 1 and 2 have the LAN side firewall disabled on the LAN groups (but of course, group isolation is still in place), the PC in Group 1 on 10.0.0.1 is unable to contact the VM through its public IP address of 123.123.123.123 (which, as mentioned above, translates internally in Group 2 to 10.0.1.1) despite devices that are not on the network (smartphone on mobile data, port scanner websites, etc.) being able to contact it.

With the LAN side firewall enabled on both Group 1 and Group 2 (with group isolation still enabled) then the PC in Group 1 on 10.0.0.1 can contact the VM via its public IP address of 123.123.123.123.

I have checked the logs for the services on the VM and, with the LAN side firewall enabled, if I contact the VM from a device on another group (not an external device) using the public IP address for the VM, the logs will show the IP of the gateway for Group 2 (the group where the VM is hosted), which is 10.0.1.254, as the source. I imagine this is the desired behaviour as since it cannot and is not contacting 10.0.1.1 directly but instead querying the router and routes it through the router to be translated and sent the group which has the one-to-one NAT IP (123.123.123.123, translating to 10.0.1.1).

So, what changes between the LAN side firewall being enabled and disabled other than the former dropping all incoming packets to the router? Because I do not understand this, especially when the LAN side firewall being enabled actually seems to route correctly. Just to note, a traceroute to this public IP will not give any clues to how it is routed internally through the router/local network since it uncovers no internal IP addresses (just external, and I cannot test right now because it seems to be working).

re0
Logged
ISP: Gigaclear - Hyperfast 900 (up to 940 Mbps symmetrical)

andyfitter

  • Reg Member
  • ***
  • Posts: 172
Re: New network setup with group isolation issues
« Reply #8 on: February 04, 2018, 06:01:28 PM »

My only comment on this would be that NAT Loopback/NAT Reflection is a bloody nightmare. Every router behaves differently, and your use case seems much more complex than most. Sorry I can't offer more help than that.
Logged

re0

  • Reg Member
  • ***
  • Posts: 840
Re: New network setup with group isolation issues
« Reply #9 on: February 04, 2018, 08:45:02 PM »

I imagine it could be one of these "in theory, but in practice" things. I appreciate the thought though!
Logged
ISP: Gigaclear - Hyperfast 900 (up to 940 Mbps symmetrical)

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: New network setup with group isolation issues
« Reply #10 on: February 05, 2018, 12:02:25 AM »

I have read both the suggestions from the more experienced members and your latest, perfectly clear, description. I understand the situation but am unable to explain why.  ???

A perplexing enigma.  :-\
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

re0

  • Reg Member
  • ***
  • Posts: 840
Re: New network setup with group isolation issues
« Reply #11 on: February 05, 2018, 12:53:06 AM »

Thanks for understanding.

As long as it works, is reliable and secure then I have no real problems at the moment. I will probably give it a few days to see whether there are any further issues that I haven't yet discovered.

It has taken longer than expected to get to this point with the router being an AIO device; I have tried to avoid rebooting it frequently to prevent the DLM making any rash changes to my line profile.
Logged
ISP: Gigaclear - Hyperfast 900 (up to 940 Mbps symmetrical)

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: New network setup with group isolation issues
« Reply #12 on: February 05, 2018, 01:01:44 AM »

. . . I have tried to avoid rebooting it frequently to prevent the DLM making any rash changes to my line profile.

A very wise decision.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

d2d4j

  • Kitizen
  • ****
  • Posts: 1103
Re: New network setup with group isolation issues
« Reply #13 on: February 05, 2018, 08:19:57 AM »

Hi re0

bless you should try reading this post using an iphone5

I think but could be wrong as time is too short to read the full manual, you need to be looking at nat group exceptions

Your description of to me looks as though it is doing what it is meant to, with firewaLL off but isolation on, and without group nat exception set/configured, the it is designed to not allow... of course, it is early here and I need to prepare for todays work load, so could have read to quickly and missed something and I have no experience of billion...

I hope that helps a little

Many thanks

John

Valid: Check whether to valid the one-to-one NAT mapping rule.
WAN Interface: Select one based WAN interface to configure the one-to-one NAT. 
Global IP address: The Global IP mapped to an internal device. It can be left empty, and under this
circumstance, it can be reached through the WAN IP of interface set in the field above.
Internal Address: The IP address of an internal device in the LAN.
Exceptional Rule Group: Select the exceptional group listed. It is to give or block access to a group
of IPs to the server after One-to-One NAT. For example, a server with 192.168.1.3 is mapped to
123.1.1.2 by One-to-One NAT, then the exceptional group can be designated to have or have not
access to 123.1.1.2.

For example, you have an ADSL connection of pppoe_0_8_35/ppp0.1 interface with three fixed
global IP, and you then can assign the other two global IPs to two internal devices respectively.
If you have a WEB server (IP address: 192.168.1.3) and a FTP server (IP address: 192.168.1.4) in
local network, owning a public IP address range of 123.1.1.2 to 123.1.1.4 assigned by ISP. 123.1.1.2
is used as WAN IP address of the router, 123.1.1.3 is used for WEB server and 123.1.1.4 is used for
FTP server. With One-to-One NAT, the servers with private IP addresses can be accessed at the
corresponding valid public IP addresses
Logged

re0

  • Reg Member
  • ***
  • Posts: 840
Re: New network setup with group isolation issues
« Reply #14 on: February 05, 2018, 05:24:54 PM »

Hi John,

Thanks for the thought and checking the information. I can imagine using a phone for this would be frustrating (especially an iPhone).

So, yeah… this is a long one (sorry).

No matter what I do and regardless of any rules I enter in any group within the Exceptional Rule Group, I cannot get any of this to work with the group isolation enabled and LAN side firewall disabled (as connections will timeout). I have also tried this before a couple of days ago without success. The only way I can get this working at all is by using a combination of group isolation and LAN side firewall, and making an IP filtering rule to allow one IP access to the router interface (as the LAN side firewall drops packets to the router when enabled).

Your example is, in theory, correct. Though in practice there is something else that seems to be preventing the routing working in a way which is expected (or at least to myself, which I could be wrong about). I have it setup with both valid IP addresses both internally and globally, and I have used the correct interface; this is evident as it does work externally and internally (the latter conditionally – only if the LAN side firewall is enabled alongside group isolation).

Let’s talk about the Exceptional Rule Group.

As in the manual:
Exceptional Rule is dedicated to giving or blocking NAT/DMZ access to some specific IP or IPs(range). Users are allowed to set 8 different exceptional rule groups at most. In each group, user can add specific IP or IP range.

By default, the rule in Exceptional Rule Group is set to blocking mode (default action is allow), which implies that any specified addresses will be blocked. So, in theory, if I just simply apply Exceptional Rule Group to the one-to-one NAT rule then it should allow me (or any other device) to connect to the external IP address from my internal network. And, to my surprise, I can connect to something, but it seems that instead of connecting to my VM, it connects me to my router. Let’s try and delve into this.

Let’s assume the following information, for sake of example:
The ISP has supplied an IP range from 123.1.1.2 to 123.1.1.3
The main PPP is connected with an IP address of 123.1.1.2
My PC is behind the NAT, in Group 1 (10.0.0.0/24) with an internal IP address of 10.0.0.1
My VM is configured in one-to-one NAT within Group 2 (10.0.1.0/24) with an internal IP address of 10.0.1.1 and global of 123.1.1.3

We have already established that if group isolation is on and LAN side firewall is enabled, I can connect from the PC (10.0.0.1) in Group 1 to the VM (10.0.1.1) in Group 2 using external IP address 123.1.1.3. Without the LAN side firewall enabled this connection cannot happen while group isolation is on (which is required), so let’s keep the firewall off and try and configure an Exception Rule Group.

As mentioned above, with the default mode being in blocking I can just select the rule in the one-to-one NAT settings and all IPs should be allowed. And while it does allow me to connect to the IP address via SSH or the browser, it simply responds with the router SSH server and web interface. So, technically, it is allowing access to the public IP address from my PC, but it looks like it is only going to the gateway. I did do a traceroute, and it completed after one hop (which would be two, if the LAN side firewall and group isolation was enabled together) – it was something like:

1 <1 ms <1 ms <1 ms host.blah-blah-blah.net [123.1.1.2]

I made a couple of changes to my VM so services were running on different ports and, unsurprisingly, my fears were confirmed – attempting to connect to the services on the VM using its public IP address from my PC was unsuccessful even considering the ports were different from what the router’s services were listening on; my PC was not even reaching the VM and the connection was being refused.

I did try and setup a rule in allowing mode, which would block any other IP address not specified. I put my own PC IP address in (10.0.0.1) and it resulted in the same as above. I also tried allowing the gateways from both Group 1 (10.0.0.254) and Group 2 (10.0.1.254) in there to see if that made any difference as the gateway does all the translation (as mentioned in a previous post, the source port from an internal connection from another group always shows as the gateway from the destination group). But none of this made any difference and I could only connect to the router and not the VM through the public IP address.

All in all, what I’ve discovered so far is that if I want to connect to any of my one-to-one NAT devices which are isolated in different groups via their public IP address then I need to have the LAN side firewall enabled alongside group isolation, and have the Exceptional Rule Group set to none because if:

- The LAN side firewall is disabled, a local connection will timeout to the VM
- A rule is setup in the Exceptional Rule Group, regardless of what changes I make, any local connections to the public IP address associated with the VM will only connect to the gateway (so attempting to connect to the ports in use with the VM’s services will be refused)

I could be doing something completely wrong in my configuration. But perhaps the Exceptional Rule Group was never intended to be used across different groups with group isolation enabled? Either way, I am grateful that people have looked into this, given their opinions on the situation and shared their ideas with me. I think the configuration I currently have in play is probably the best for my usage scenario, and I am limited by my knowledge and hardware implementation to make any further and worthwhile changes.

Of course, if anyone can see something wrong with what I have done, I would certainly be appreciative if they let me know.

re0
Logged
ISP: Gigaclear - Hyperfast 900 (up to 940 Mbps symmetrical)
 

anything