June 2010

"If you want to accomplish something in the world, idealism is not enough - you need to choose a method that works to achieve the goal. In other words, you need to be pragmatic."

Richard Stallman

It's no secret that open source software can offer substantial cost savings to startup companies. Anybody with an idea and a limited amount of capital can launch a web-based business and scale it relatively quickly using a simple and robust LAMP infrastructure. But what about startup companies contracted to provide crucial parts of the Internet's plumbing, such as a domain name registry?

This article describes how registry manager Afilias deployed open source, particularly the PostgreSQL database management software (DBMS), at the core of its business from the very outset, allowing it to scale as business grew. It also underlines the importance of actively engaging with the open source community in order to get the most out of a company's investment.

The Domain Name System

The domain name system (DNS) is the distributed, hierarchical addressing system used to translate human-readable strings such as www.example.com into the IP addresses used for Internet routing. Used hundreds of billions of times a day, the DNS underpins virtually every transaction on the Internet and is considered perhaps its most critical infrastructure component.

The role of a DNS registry service provider, such as Afilias, is to manage the authoritative list of second-level domains and the addresses of their corresponding servers under a given top-level domain (TLD). Every time a Web user attempts to browse a Web address, the address of the appropriate Web server is ultimately retrieved from the registry database via DNS.

From the creation of the DNS in 1985 until the turn of the century, the number of TLDs on the Internet did not substantially increase. Organizations wishing to establish an Internet presence could choose from essentially three generic TLDs (gTLDs) -- .com, .org and .net, all of which were operated by the same company -- or from one of over 200 country-code TLDs such as .ca for Canada.

That changed in 2000, when .info was approved for admission to the DNS by the Internet Corporation for Assigned Names and Numbers (ICANN) and Afilias became the first company to launch a completely new gTLD. Unlike its main competitor, who had built out its systems using proprietary software, Afilias chose to build its registry systems on open source software. Ten years on, Afilias has grown from a startup to the second-largest registry operator in the world. The company's engagement with the open source community has proven invaluable for its growth and scalability.

Plumbing the Internet with PostgreSQL

In 2000, Afilias was essentially a startup company with limited capital and no guarantee of success, entering a market that was already dominated by .com domain names. The very idea of the new gTLD registries was said to be a "proof of concept". A future industry would stand or fall on the experiences of its first entrants. Above all, security and stability were key.

While the .com registry had enjoyed the benefit of time when it came to gradually scaling its infrastructure in response to the growing public interest in the Internet during the mid-1990s, the .info TLD needed to be prepared to immediately handle hundreds of thousands of domain names and to quickly scale into the millions with no disruption to services. The need to scale would be less predictable than other Internet-based businesses, depending in a large part on how successful the .info registrants' own businesses became.

The database software underlying a domain name registry is one of its most critical components, storing the key records pertaining to a domain name's location on the Internet. In the case of Afilias, which operates a thick registry, it also stores the Whois record for each domain which houses the authoritative contacts of the domain owners.

Because of the way the domain name industry is structured, the registry acts as a kind of wholesale warehouse, interfacing with hundreds of registrars who act as the retailers which offer domain name registrations to end users. Registrars need to be able to tell their customers on demand whether a domain name is available for registration. After a purchase, it then has to tell the registry to assign the domain to that customer. It is critical to an entire ecosystem of companies and individuals that the registry database operates at 100% availability and with confidence that the information it stores is served accurately and consistently while multiple parties read and write data in real-time. There is no room for ambiguity.

It would have been a safe call to simply sign up for a commercial database solution such as Oracle. Nobody ever got fired for buying a tried and true solution. But in PostgreSQL, Afilias found an open source DBMS that more than suited its needs. The software was attractive due to its large capacity and its ability to handle many transactions simultaneously. PostgreSQL contained all the key functionality of Oracle or DB2 that Afilias would have utilized had it gone the commercial route, but at a fraction of the initial cost. In testing, it was quickly apparent that this open source database would deliver excellent service and better value than proprietary alternatives, thereby saving this startup at least 7 figures in initial capital.

Afilias' decision to select PostgreSQL was validated in 2003, when the Public Interest Registry (PIR), a subsidiary of the Internet Society (ISOC), won a competitive bid to take over the management of the Internet's second-largest gTLD, .org, using Afilias as its DNS and registry services provider. ICANN awarded the contract to the Afilias-supported PIR initiative despite objections from proprietary database vendors involved in competing bids, who erroneously claimed PostgreSQL lacked the security and availability features required to manage such a critical resource. Close examination by ICANN showed otherwise, a finding that has been vindicated by .org's stellar performance record (100% up-time) even as it has nearly tripled in size since the PIR/Afilias duo assumed stewardship.

You Get What You Give

While the decision to deploy an open source DBMS saved Afilias vital capital expenditure at startup and allowed it to cost effectively scale up as .info and .org grew and Afilias added yet more TLDs, everybody knows that open source does not mean no-cost. As a user, especially as a user who has chosen to use open source software in the most mission-critical part of its business, Afilias made an implicit commitment to support and engage with the community. You have to make a contribution in order to reap the biggest benefits.

Afilias honoured this commitment from the beginning, and started making direct contributions to the development of PostgreSQL. Some of its best database engineers were given the remit to work exclusively on the DBMS, in order to help strengthen and improve the software at the core of its systems. In the early days, this challenge occupied close to 100% of these engineers' work time, and they relished the challenge. A happy side effect of this commitment to open source was that, as Afilias grew as a company, it was able to attract some top-quality coding talent to the team. As time progressed, Afilias established steady support of official PostgreSQL events, conferences and seminars.

A key Afilias contribution to the PostgreSQL code base has been the development of a replication engine for high-transaction-volume installations. Known as the Slony-I replication engine, this software was released back into the open source community and continues to evolve today. It was designed for architectures where it is important to replicate data across multiple database nodes to multiple masters where all of the nodes are expected to be online simultaneously. It supported cascading replication, so slave nodes could replicate data to each other. In short, Slony-I delivers greater speed, efficiency and reliability. Afilias' engineering staff are leading efforts towards the next version of replication for PostgreSQL, Slony-II.

Not coincidentally, the Slony replication engine powered an important part of Afilias' growth strategy. A high-performance replication engine is a vital component of a registry's DBMS, if it wants to provide reliable addressing to Internet users. A domain name registry has to be both authoritative and available. To achieve redundancy and seamless switchover capability, the registry's core database is best mirrored in multiple locations on multiple networks, thereby ensuring 100% availability with high reliability. This, in turn, increases the importance that data remain authoritative: all instances of the same database need to serve the same data, requiring that changes to the master data set be reflected instantaneously at each replicated location. As the number of transactions and the number of replicated instances increase, the database system must effortlessly keep track of each committed change to the data and copy it accurately and completely across each replica.

Closing Thoughts

Afilias found that engaging seriously and directly with the open source community gave it the ability to shape the direction of PostgreSQL's development and help improve it, for everybody's benefit, based on its own experience as a power user. The beauty of open source is that it grows according to the input of participants. As a company grows and scales up, it can use that experience to help grow and scale the open source software upon which it depends.


Share this article:

Cite this article:

Rate This Content: 
10 votes have been cast, with an average score of 5 stars