NASDAQ Private Market"This was a hard problem that needed solving. Leave it to the team at CloudFlare to come up with a great solution that will act as a gateway to a virtually unlimited number of other cloud services."
Alex Polvi, CEO, CoreOS
"CNAME Flattening allowed us to use a root domain while still maintaining DNS fault-tolerance across multiple IP addresses."
Andrew Warner, Director of Engineering, RapGenius
Intrigued? Read on to understand the technical details of exactly what it is and why we needed to build it.
Traditionally, the root record of a domain needed to point to an IP address (known as an A -- for "address" -- Record). While it may not seem like a big deal, tying a service to an IP address can be extremely limiting. Imagine that a new blogging platform, WordPlumblr, starts up. WordPlumblr allows its users to use custom domains that point to the WordPlumblr infrastructure. Foo.com signs up and WordPlumblr gives Foo.com an IP address. The supply of IP addresses is limited so, over time, as more sites sign up, IP addresses end up getting shared between multiple customers. No problem since with virtual hostnames WordPlumblr can return different content for different domains even hosted on the same IP address.
Everything is fine until Foo.com starts using too many of WordPlumblr's resources -- maybe because they're being attacked or they're featured on Oprah or who knows why. The other customers of WordPlumblr get poor performance because they're sharing the same resources as the overwhelmed Foo.com. WordPlumblr is put in a difficult position: having to reach out to Foo.com to get them to change their DNS settings or, even harder, having to reach out to all the other customers using the IP address to get them to change. Good luck.
The solution is a CNAME. A CNAME is an alias. It allows one domain to point to another domain which, eventually if you follow the CNAME chain, will resolve to an A record and IP address. If WordPlumblr had handed out a unique CNAME for every customer then they wouldn't have had a problem. For example, WordPlumblr might have assigned the CNAME 6equj5.wordplumblr.com for Foo.com. Foo.com and the other customers may have all initially resolved, at the end of the CNAME chain, to the same IP address. However, when Foo.com started using too many resources WordPlumblr could have updated the CNAME and isolated Foo.com from the rest of the customers.
If you're familiar with programming, this is just like the idea of a pointer. When you're programming in C, in order to allow flexibility in memory management, you usually don't want to address memory directly but, instead, you set up a pointer to a block of memory where you're going to store something. If the operating system needs to move the memory around then it just updates the pointer to point to wherever the chunk of memory has been moved to. The program references the pointer and is none the wiser.
CNAMEs work great for subdomains like www.foo.com or blog.foo.com. Unfortunately, they don't work for a naked domain like foo.com itself. And, for reasons that somewhat perplex me, a lot of people are obsessed with using their naked domain for their website. So why don't CNAMEs work at the root?
DNS specification dates from 1987. At the time, no one conceived of website -- it was two years before Tim Berners-Lee’s seminal paper first laying out the idea for the world wide web -- let alone modern outsourced cloud services like Amazon Web Services (AWS) or Heroku. As a result, the DNS spec enshrined that the root record -- the naked domain without any subdomain -- could not be a CNAME. Technically, the root could be a CNAME but the RFCs state that once a record has a CNAME it can't have any other entries associated with it: that's a problem for a root record like example.com because it will often have an MX record (so email gets delivered), an NS record (to find out which nameserver handles the zone) and an SOA record.Because they follow this specification, most authoritative DNS servers won't allow you to include CNAME records at the root. At CloudFlare, we decided to let our users include a CNAME at the root even though we knew it violated the DNS specification. And that worked, most of the time. Unfortunately, there were a handful of edge cases that caused all sorts of problems.
Break the RFC at Your Own Peril
You'd never guess, but the biggest edge case had to do with email sent from Microsoft Exchange mail servers. Domains generally designate the servers that handle their email through what's known as a MX Record. The problem was that Exchange servers, under a very specific set of circumstances, could pick up the CNAME at the root record and then not properly respect the CNAME set at the MX record. You can't really blame Exchange. They were operating under the assumptions laid out by the DNS specification. However, this and a handful of other corner cases caused us to support but recommend against using a CNAME at the root record. Until now.
Introducing CNAME Flattening
Flexible and Faster