Kitz Forum

Computers & Hardware => Networking => Topic started by: Weaver on June 04, 2018, 03:05:14 PM

Title: Where am I on the global Internet?
Post by: Weaver on June 04, 2018, 03:05:14 PM
Let’s say I pick some random IP address or stick a pin in a map of the world. How can a machine that is connected to the internet there work out an answer to the question “where am I in the internet”? By which it means that it want some kind of answer that will allow it to pick some server for a particular kind of service that is close in terms of latency ?

I appreciate that anycast and other systems mean that clients often don't work this out, the network just does the right thing, requests going to the right place by magic, or failing that maybe redirects play a part in the process (and they can also help with load balancing).

I also realise that there must be various techniques for working out which territory / country you are in. But that isn't really necessarily the right thing. Russia is pretty useless as a country because it's so enormous. And even in a smaller country you might want much better geographical (better: internet topological or metric distance function) resolution. Just saying you're in the USA isn't really good enough either. But in the UK it might be that everything is connected to London only so that you always want to pick something in Docklands because it is well-connected. And if something is right next to you but you are hundreds of miles away from London, then choosing the thing that is geographically close could be a disaster because you will only end up going all the way to London and back twice, which has happened to me at times, so that is twice the latency of just choosing a London-based node rather than a supposedly ‘local’ one. Most of the speed testers make this ridiculous mistake.

Not all parts of the internet have topology like that though, I’m sure. I don’t know how you would work it out. I haven't been able to find any info on what the metric and topology of the global internet looks like. A detailed geographical pic would be nice, not one of those arty abstract IP topology diagrams that tells you nothing about physical locations.

Maybe anycast and so on are set up by interrogating the global routing table and then crunching the data you obtain, a lot.

Has anyone seen any info about this?
Title: Re: Where am I on the global Internet?
Post by: Weaver on June 04, 2018, 07:10:01 PM
I have at least found this :
Title: Re: Where am I on the global Internet?
Post by: CarlT on June 05, 2018, 12:38:41 PM
On the move but would point you to whois and DNS. Pay particular attention to that DNS is relatively easy to anycast so requests can have different responses depending on where in the world you are.
Title: Re: Where am I on the global Internet?
Post by: Weaver on June 05, 2018, 04:27:43 PM
I am wondering if there is any source of machine-readable topology info, other than going to BGP itself,  trying to make sense out of whatever can be obtained from that

As for querying eg RIPE’s databases, I have had a go at looking up a quick program for the iPad which uses the latter and it works but it is a bit crude. But that doesn't qualify as it doesn't give you any idea of interconnectedness, although it does give you a different idea of ‘where you are’, so it could be a starting point to at least perhaps tell you which country you are in, but I am not sure that would be reliable because of the case of multinational organisations.

So I don't have an answer as yet.
Title: Re: Where am I on the global Internet?
Post by: CarlT on June 05, 2018, 09:33:20 PM
Route servers and looking glasses give information that's easily parsed. Traffic can be steered to local servers based on DNS and anycasting.

Locating a client machine, depending on the databases you have access to, can be done via whois information and AS path to some extent. Going beyond that requires additional information such as the telemetry Google hold on WiFi networks.
Title: Re: Where am I on the global Internet?
Post by: Weaver on June 05, 2018, 10:47:07 PM
How do they who are wise get anycasting right, I mean choosing where to park server for best effect? I suppose sometimes it is easy. For example if in UK, answer = Docklands. Always. A no-brainer. Unless there is a possible alternative at a local internet exchange? But for say the USA you need to do better than just giving out one answer brainlessly. And as I mentioned before, you had better not get it wrong for Russia's pacific coast say.
Title: Re: Where am I on the global Internet?
Post by: CarlT on June 06, 2018, 12:19:02 AM
In Comcast's case they use CRANs - Converged Regional Access Network. Each CRAN will use either another IGP or its own private iBGP AS as part of a confederation. Have servers hosted in each one and you're good to go. For smaller networks host in regional aggregation centres connected via a couple of decent upstreams.

Have a look at

In the case of the USA having servers in datacentres in New York, Los Angeles, Texas, Florida, Chicago and Washington State takes care of most things, along with taking care of Canada's traffic too.
Title: Re: Where am I on the global Internet?
Post by: Weaver on June 06, 2018, 01:50:39 AM
Many thanks for that link!

I realise now that I went off topic in my previous post because I started talking about the whole thing from the reverse perspective - the sysadmin in charge of servers wanting to make decisions based on topology. I was originally looking at things from the other end, the client looking into the network and trying to make choices based in ‘where’ it is in topology terms.

If all else fails you could start tracerouting to things. But to what? You would already have to sort-of know the answer, it wouldn't help you if you were in a completely unfamiliar part of the internet. Maybe you could just cheat by pre-mapping the world’s interconnection relationships somehow and just handing out a list of points to trace route to. This list would need to be able to be updated, republished reasonably regularly. Perhaps that could be done by using some suitable record type in the DNS, routed under a well-known domain name. Just using the DNS as a database and effectively broadcasting readonly info. You could perhaps even be a bit more helpful, with hints. Something like, “try these nodes, pick out the ones with the lowest hop count then from that set pick the ones with the lowest round trip time”. Also need to allow for the variance in the results, so that times that are within each others' variance × k (some factor) are considered ‘equal’, or something. Needs more work to spec it properly. Anyway you end up with one or more nodes that are considered approximately equal, best candidates. Each one is an L2 node and will have an associated set of nodes to traceroute, so you now recurse and consider all the nodes listed under L2 in the same way, until you get to the set Ln, which is a leaf, and there there is whatever database of info which you might need to help you make decisions that are right for that region of the internet. For example: This might have the best NTP server to use, not a very imaginative example. It could tell you which RIR you are in. Imagination fails me, but I am certain that if I were a bit less dopey then the examples could be numerous. It would certainly have an abstract, opaque long bit-width region identifier which is also for you to use with other services. For example an organisation you know might publish a database of useful info indexed by that region id and so you could look something up by that. I am also thinking about something to bootstrap a search process to get you as far as a local database, one of many, that lists ISPs and other large organisations and those databases can give you information that will help you test whether or not you are currently connected to which ISP or other organisation’s subnet and then info you need to know about that outfit. For an ISP, this might cover the range of services available and things like which protocol types are supported or blocked, info on charging or pointer to how to find such info.

DHCP and other systems broadcast various kinds of you-need-to know-this, some of it ‘to get you started’ info onto LANs, but sometimes this is only LAN-wide. Sometimes you need wider range info that comes from further away and has a much wider range of geographical or topological validity/relevance. Also DHCP is not so easily extensible, it's fields are not self-identifying and the total length of stuff has problematic limitations iirc. And then some people do not use DHCP at all. IPv6 RA is less informative and DHCPv6 is I presume far less frequently used as it simply is not needed so it does not enjoy the popularity that DHCPv4 does. So this kind of idea could greatly extend the info publishing idea that DHCP has started up. The two will never be the same as DHCP has very fine granularity, down to the level of the individual LAN, and this is either a great virtue or a showstopper, depending on what kind of info you want to publish and what you want to achieve.

I’m just musing. What a monster post this has become. I do apologise deeply. ;)
Title: Re: Where am I on the global Internet?
Post by: CarlT on June 06, 2018, 12:09:57 PM
It's fine.

Actually some of the things you are mentioning are covered by, yes, my own employer's products. You're basically musing how to build a smarter WAN. This is more and more of a thing with networks rather than relying on IP, etc, running overlay networks over the top so that what's going on at IP layer doesn't matter as much.

For your viewing pleasure:

DNS is surprisingly powerful when extended via geolocation and other parameters - - using similar technologies to those deployed by my employer for enterprises and essentially treating entire ISPs as branch offices for the purposes of steering, but without the overlay network and other extras we provide.

Essentially clients don't need to know the best path and the Internet as it stands right now is geared towards the server-side handling the work, as is most appropriate, simply because a SaaS provider network has far more visibility of the Internet as a whole than a single client machine, so can bring way more data into making the decision which it then relays back to the client via DNS.
Title: Re: Where am I on the global Internet?
Post by: Weaver on June 06, 2018, 01:23:20 PM
DNS is an amazingly powerful tool. It could easily get abused and start to become a general broadcast planet-wide database.

A bad idea: I think that that could possibly, I don't know, be dangerous to the health of the DNS performance wise but I am not sure. To be safe we should perhaps consider starting up a separately rooted system, possibly with some of the existing well-known nasties cleaned up, as that would be an ideal opportunity to do so with no essential backwards compatibility concerns. Splitting it would be really undesirable apart from the benefits gained by a ‘reboot’ of (fresh-start with) the spec because it makes the new system far less accessible. If it can be avoided we should avoid doing so. I think it would have to be justified by the possibility of a really bad effect on performance due to the huge weight of the 'general data' placed into the existing system by using it as a general database for all purposes. Hopefully there is nothing to worry about so no need for a split.
Title: Re: Where am I on the global Internet?
Post by: CarlT on June 06, 2018, 06:38:06 PM
DNS just delivers the end result of other database lookups. The geolocation technology behind those results is probably more concerning.

Google can make an educated guess on where a device is by what wireless networks their devices have visibility of if it's permitted access. It knows them from Street View vehicles. Google war drives  ::)