using a low-power CPU that consumes 10W– 20 W and costs
∼$150 in volume. We in turn give the benefit of the doubt to
the server systems we compare against—we assume a 2 TB
disk exists that serves 300 queries/s at 10 W.
Our results indicate that both FAWN and traditional systems have their place—but for the small random-access
workloads we study, traditional systems are surprisingly
absent from much of the solution space, in favor of FAWN
nodes using either disks, flash, or DRAM.
Key to the analysis is a question: Why does a cluster need
nodes? The answer is, of course, for both storage space and
query rate. Storing a DS gigabyte dataset with query rate QR
requires N nodes:
10,000
Figure 9. Solution space for lowest 3 year TCo as a function of
dataset size and query rate.
1000 FAWN + Disk
Dataset size in TB
1
10
100
FAWN + Flash
Traditional + DRAM
FAWN + DRAM
0.1
0.1
1 10 100
Query rate (Millions/s)
1000
With large datasets with low query rates, the number of
nodes required is dominated by the storage capacity per
node: Thus, the important metric is the total cost per GB for
an individual node. Conversely, for small datasets with high
query rates, the per node query capacity dictates the number
of nodes: the dominant metric is queries per second per dollar. Between these extremes, systems must provide the best
trade-off between per node storage capacity, query rate, and
power cost.
Table 3 shows these cost and speculative performance
statistics for several candidate systems circa 2009; while
the numbers are outdated, the trends likely still apply. The
“traditional” nodes use 200 W servers that cost $1,000 each.
Traditional+ Disk pairs a single server with five 2 TB high-speed ( 10,000 RPM) disks capable of 300 queries/s, each disk
consuming 10 W. Traditional + SSD uses two PCI-E Fusion-IO
80GB flash SSDs, each also consuming about 10 W (Cost:
$3K). Traditional+ DRAM uses 8GB server-quality DRAM
modules, each consuming 10W. FAWN+ Disk nodes use
one 2 TB 7200RPM disk: FAWN nodes have fewer connectors available on the board. FAWN + SSD uses one 32GB Intel
SATA flash SSD capable of 35,000 random reads/s, 17
consuming 2 W ($400). FAWN + DRAM uses a single 2GB, slower
DRAM module, also consuming 2 W.
Figure 9 shows which base system has the lowest cost for
a particular dataset size and query rate, with dataset sizes
between 100GB and 10PB and query rates between 100K
Table 3. Traditional and FAWn node statistics.
and 1 billion/s.
Large datasets, low query rates: FAWN+ Disk has the
lowest total cost per GB. While not shown on our graph,
a traditional system wins for exabyte-sized workloads if it
can be configured with sufficient disks per node (over 50),
though packing 50 disks per machine poses reliability
challenges.
small datasets, high query rates: FAWN+ DRAM costs the
fewest dollars per queries per second, keeping in mind that
we do not examine workloads that fit entirely in L2 cache on
a traditional node. This somewhat counterintuitive result is
similar to that made by the intelligent RAM project, which
coupled processors and DRAM to achieve similar benefits4
by avoiding the memory wall. We assume the FAWN nodes
can only accept 2GB of DRAM per node, so for larger datasets, a traditional DRAM system provides a high query rate
and requires fewer nodes to store the same amount of data
(64GB vs. 2GB/node).
middle range: FAWN+ SSDs provide the best balance
of storage capacity, query rate, and total cost. If SSD cost
per GB improves relative to magnetic disks, this combination is likely to continue expanding into the range served
by FAWN+ Disk; if the SSD cost per performance ratio
improves relative to DRAM, so will it reach into DRAM
territory. It is therefore conceivable that FAWN + SSD could
become the dominant architecture for many random-access workloads.
Are traditional systems obsolete? We emphasize that this
analysis applies only to small, random-access workloads.
System Cost
W
Traditionals
5–2tb disks $2K 250
160Gb PCie ssd $8K 220
64Gb drAM $3K 280
FAWNs
2tb disk 20
32Gb ssd 15
2Gb drAM 15
$350
$500
$250
qPS queries/Joule GB/Watt
1500
200K
1M
250
35K
100K
6
909
3.5K
12. 5
2.3K
6.6K
40
0.72
0.23
100
2. 1
0.13
TCo/GB TCo/qPS
0.26
53
59
0.20
16. 9
134
1. 77
0.04
0.004
1. 61
0.015
0.003