Communications - July 2011

using a low-power CPU that consumes 10W– 20 W and costs ∼$150 in volume. We in turn give the benefit of the doubt to the server systems we compare against—we assume a 2 TB disk exists that serves 300 queries/s at 10 W.

Our results indicate that both FAWN and traditional systems have their place—but for the small random-access workloads we study, traditional systems are surprisingly absent from much of the solution space, in favor of FAWN nodes using either disks, flash, or DRAM.

Key to the analysis is a question: Why does a cluster need nodes? The answer is, of course, for both storage space and query rate. Storing a DS gigabyte dataset with query rate QR requires N nodes:

10,000

Figure 9. Solution space for lowest 3 year TCo as a function of dataset size and query rate.

1000 FAWN + Disk

Dataset size in TB

1 10 100

FAWN + Flash
Traditional + DRAM
FAWN + DRAM

0.1

1 10 100

Query rate (Millions/s)

1000

With large datasets with low query rates, the number of nodes required is dominated by the storage capacity per node: Thus, the important metric is the total cost per GB for an individual node. Conversely, for small datasets with high query rates, the per node query capacity dictates the number of nodes: the dominant metric is queries per second per dollar. Between these extremes, systems must provide the best trade-off between per node storage capacity, query rate, and power cost.

Table 3 shows these cost and speculative performance statistics for several candidate systems circa 2009; while the numbers are outdated, the trends likely still apply. The “traditional” nodes use 200 W servers that cost $1,000 each. Traditional+ Disk pairs a single server with five 2 TB high-speed ( 10,000 RPM) disks capable of 300 queries/s, each disk consuming 10 W. Traditional + SSD uses two PCI-E Fusion-IO 80GB flash SSDs, each also consuming about 10 W (Cost: $3K). Traditional+ DRAM uses 8GB server-quality DRAM modules, each consuming 10W. FAWN+ Disk nodes use one 2 TB 7200RPM disk: FAWN nodes have fewer connectors available on the board. FAWN + SSD uses one 32GB Intel SATA flash SSD capable of 35,000 random reads/s, 17 consuming 2 W ($400). FAWN + DRAM uses a single 2GB, slower DRAM module, also consuming 2 W.

Figure 9 shows which base system has the lowest cost for a particular dataset size and query rate, with dataset sizes between 100GB and 10PB and query rates between 100K

Table 3. Traditional and FAWn node statistics.

and 1 billion/s.

Large datasets, low query rates: FAWN+ Disk has the lowest total cost per GB. While not shown on our graph, a traditional system wins for exabyte-sized workloads if it can be configured with sufficient disks per node (over 50), though packing 50 disks per machine poses reliability challenges.

small datasets, high query rates: FAWN+ DRAM costs the fewest dollars per queries per second, keeping in mind that we do not examine workloads that fit entirely in L2 cache on a traditional node. This somewhat counterintuitive result is similar to that made by the intelligent RAM project, which coupled processors and DRAM to achieve similar benefits4 by avoiding the memory wall. We assume the FAWN nodes can only accept 2GB of DRAM per node, so for larger datasets, a traditional DRAM system provides a high query rate and requires fewer nodes to store the same amount of data (64GB vs. 2GB/node).

middle range: FAWN+ SSDs provide the best balance of storage capacity, query rate, and total cost. If SSD cost per GB improves relative to magnetic disks, this combination is likely to continue expanding into the range served by FAWN+ Disk; if the SSD cost per performance ratio improves relative to DRAM, so will it reach into DRAM territory. It is therefore conceivable that FAWN + SSD could become the dominant architecture for many random-access workloads.