We have been receiving occasional notifications, primarily via twitter from concerned customers who have visited 'isbgpsafeyet.com' and been told by the site's operator, Cloudflare, that their ISP is, apparently unsafe.
Since this has now happened a few times, we felt it worth giving some more information that may be useful to customers and others who've seen these tweets (either directed at us, or at other ISPs), explaining a bit about what BGP is and how RPKI can extend it, and also our feelings about Cloudflare attempting to build support in this manner, especially now, during the Corona Virus situation.
The way the Internet works, behind the scenes, is a complicated mix of different equipment from different vendors, and various protocols and a lot of best practice which evolves over time. Any change in the way this all works is always carefully considered by Internet providers as there are always risks and possible side effects. Some changes, like IPv6, are still being rolled out decades after they were first defined.
Right now (April 2020), like most ISPs, we are avoiding making any major changes. One change in best practice that is still just starting to be deployed by ISPs is something called RPKI which helps reduce some of the ways bad guys can break the Internet. Our transit providers still don't do this (if they did, it would be less critical for us to anyway, but we will). AAISP started work on this some time ago (signing routes), but some key steps (filtering routes) need design and work and testing on our FireBrick routers which will still take some time.
I.e. It's complicated - it's being worked on - slowly and cautiously.
Update: FireBrick are working on using the RTR (RFC6810) protocol to implement RPKI now.
Here is an example of the sort of tweet we have seen a few of now :
To most of the world, this is a horror story! And we can see why people who have done the test would be naturally worried.
The site is created by Cloudflare. Their aim is to encourage Internet providers to adopt the standard of BGP route validation known as 'RPKI' (Resource Public Key Infrastructure). Whilst it may be a good aim, we feel it is needlessly and deliberately frightening Internet users, who may have no idea what BGP even is, into believing their ISPs are acting irresponsibly, and then auto-shaming them on twitter, perhaps unfairly.
BGP (Border Gateway Protocol) is how pretty much the entire Internet handles routing. In simple terms, before BGP, the only option to allow a router to know how to pass on traffic to a destination network, was with a list 'static routes' - essentially instructions that said "for traffic for this network, pass it onto this router".
Manual lists of routes are not viable once you get beyond quite a small number of destinations. BGP is the dominant method on the public Internet of addressing this. Routes to networks are 'announced' by the owners of those networks in a way that propagates through the infrastructure to all routers.
This means that if an ISP wants to activate a new block of IP addresses, they make a 'BGP announcement' for this new block, and quite quickly, the whole Internet becomes aware of the routing for this. So far so good. However, because the whole system relies on a degree of 'trust' - that all announcements are bona fide and genuine, the system - as originally conceived - can be open to abuse, or straightforward accidental breakage, for example from a mistyped announcement. And there have been a number of famous situations where chunks of the Internet have been inadvertently 'blackholed' because of a mistake in an announcement, or deliberate attack.
As has already been said, a degree of 'trust' is required for the system to work. There is a lot of monitoring of route announcements designed to allow mistakes and malice to quickly be addressed. Major ISPs can quickly filter anything that is wrong, reducing the impact.
Of course, it would help if there was a way to be sure that these route announcements were "legitimate", and this is what RPKI aims to do. With this the network of major transit providers can ensure they don't accept mistakes or malicious announcements in the first place. Once the major networks that interconnect across the world have these measures in place, even smaller ISPs using them for transit will see the advantages of correct routes.
The whole idea has not been without controversy and concern - one issue with one authority signing these route announcements at the top level is that the local courts for that organisation can order it to remove signing and shut down a chunk of the Internet. As IP addressing is not handled by each country this gives a court a lot of power, and opportunity for mistake and abuse. However, everyone agrees something needs to be done to improve matters.
At this stage we are looking in to this. We want to be sure we take the right approach, some of which will involved asking our transit providers what they are doing about it. If we simply filtered invalid routes that we get from transit it is too late and the route is blocked. This is marginally better than routing to somewhere else (some attacker) but it still means a black hole in the Internet. So we need our transit providers sending only valid routes, and if they are doing that we suddenly need to do very little.
The other thing we can look into is ensuring all of our own routes are signed. Most are, and have been for a long time, but some PI we announce for customers need sorting out.
Both of these are relatively complicated issues in our core network and not something we want to do anything with in the middle of a pandemic!
So, yes, we do plan to implement RPKI, with caution, but have no ETA yet. In the mean time, having been made aware of an invalid route deliberately announced by Cloudflare, we have blocked that route manually.
No.
We support much that Cloudflare does in the wider community, in particular its work to ensure sites remain reliable - especially at the moment, and its work providing public resources for things like DNS, including DoH. In short, we like Cloudflare, generally speaking.
But this site cultivates unnecessary fear in the minds of people who often do not understand the (extremely important) minutiae of what they are looking at. Essentially we feel at a time when people are extremely concerned with the security and reliability of their Internet connections, Cloudflare's site is spreading fears, and even encouraging visitors of their site to automatically spread and share that fear via Twitter.
See the tweet on the right, where somebody who thinks BGP is 'a very old routing method' and is then unfairly criticising his ISP, Vodafone, for using it. A clear sign that a little knowledge may be a dangerous thing. It is also revealing that, in spite of the headline besmirching (presumably) most ISPs and writing them off as 'unsafe', when you scroll down, the list of providers who do not yet implement RPKI is far far larger than the list of those who do, including Google, Comcast, Vodafone, Zayo, Hurricane Electric and many more huge players :
If you have any questions about this, please do ask! We are not looking to hide anything!
If you want to help the overall situation, we'd suggest that if you see a tweet of this nature from someone who clearly does not understand the underlying detail fully enough to properly be aware of the risks, that you try to respond to their tweet with information that makes them better informed. RPKI is (probably) a good thing, although, for reasons given above, it does have some implementation risks, and some valid concerns surrounding jurisdiction. And in time we expect most providers will implement it.
And regarding "IsBGPSafeYet.com" ... Do you ever succeed in encouraging people to do what you want them to do by first spreading false rumours about them in public?
We feel not.
