|
|
|
@inproceedings{DBLP:conf/sigmod/Brewer98,
author = {Eric A. Brewer},
editor = {Laura M. Haas and
Ashutosh Tiwary},
title = {Delivering High Availability for Inktomi Search Engines},
booktitle = {SIGMOD 1998, Proceedings ACM SIGMOD International Conference
on Management of Data, June 2-4, 1998, Seattle, Washington, USA},
publisher = {ACM Press},
year = {1998},
isbn = {0-89791-995-5},
pages = {538},
ee = {http://doi.acm.org/10.1145/276304.276372, db/conf/sigmod/Brewer98.html},
crossref = {DBLP:conf/sigmod/98},
bibsource = {DBLP, http://dblp.uni-trier.de}
}
BibTeX
Inktomi provides the back-end for several well-known search engines, including Wired's HotBot and Microsoft's MS Start page. The services are supported by a highly available cluster with more than 300 CPUs and several hundred disks.
After a long evolution starting with a traditional RAID-based approach to availability, we now use an extremely low-cost software-only approach, with very little replicated hardware.
The first insight is to be Machiavellian about precisely what "highly available" means; i.e., what are tolerable failure modes? Although we have in the past provided the traditional expectation, "all the data, all the time", we now provide a weaker (but far cheaper) promise:
"some of the data all of time, and (probabilistically) all of the data most of the time". The latter provides linear degradation in document availability based on the number of concurrent failures, with single failures affecting less than 1% of the database.
The second trick is to exploit the fact that search engines (and most other web-based databases) are totally dominated by reads rather than writes. We take this notion to an extreme by building a custom database with essentially no locks and one simple atomic operation, which is an atomic swap of a contiguous set of records. In a clustered environment, this one operation allows us to replace atomically all of the following with out taking down the service: the database (part or whole), the OS, disks, CPUs, power, networking cards, etc. We have even used this technique to physically move the entire cluster 60 miles without even a minute of downtime.
Finally, we will talk about many practical issues that we have had to solve to reach quality uptimes statistics in practice.
Copyright © 1998 by the ACM, Inc., used by permission. Permission to make digital or hard copies is granted provided that copies are not made or distributed for profit or direct commercial advantage, and that copies show this notice on the first page or initial screen of a display along with the full citation.