Communications of the ACM

dition, the different chips in the stack need to be designed with matching pin layouts, which requires a high degree of coordination in the design of the various chips, and limits the manufacturer’s flexibility to modify the layout or use alternative suppliers.

Another major issue is heat removal, which is already a major issue for traditional chips. Combining multiple layers of heat-generating devices, and burying them further from the surface, makes the problem worse. Still, Knickerbocker said, “for low-power applications like some mobile applications and some IoT applications, the power levels are so low that getting the power in and cooling it is not a problem at all,” especially since stacking reduces the total power substantially. For high-performance computing, “even though the power delivery and the cooling challenges go up substantially, there’s still tremendous benefit at the system-performance level to make it worthwhile,” Knickerbocker said, adding that advanced power delivery may be needed, as well as cooling technologies such as flowing liquids through the chip stack or using materials that absorb heat by undergoing a phase transition.

Multiple chips also complicate manufacturing yield, which is critical to the economics of electronics manufacturing. When individual components can be proven functional before assembly, the yield of a multi-chip device can be better than that of a single chip combining the same components. However, without assurance of such “known-good die,” a failure of any layer will require trashing the whole stack.

Memory First

So far, the greatest benefit of 3D chips has been on memory. One reason is that memory consists of identical repeated units, and designers have long taken advantage of the interchangeability of memory blocks to bypass occasional defective ones. (Field-programmable gate arrays have a similar redundancy.) In addition, although heating is a major challenge for stacking of logic chips, in memory chips many transistors are inactive much of the time.

Equally important is the seemingly
insatiable demand for memory in all
sorts of systems. Indeed, two distinct
flavors of vertically stacked DRAM
have become important in the last
few years. High-bandwidth memory
(HBM) has an aggressive champion
in AMD, and is already in its second
generation, HBM2. A competing
technology, Hybrid Memory Cube
(HMC), has been developed by Mi-
cron. Although there are important
differences, both feature very high
data rates with over 1,000 or more
connections between layers. “For
the HBM stacks, the density of the
interconnect is now down at like
55μm pitch between connections,”
Knickerbocker said.

In this rapidly evolving field, stacking is not the only route manufacturers are exploring for 3D memory, however. Samsung, for example, in addition to its HBM products, has developed a monolithic flash-memory technology called V-NAND, which features strings of dozens of floating gate transistors connected vertically in series along deep etched trenches refilled with silicon, grown over a wafer of control and sensing circuitry.

Micron also has teamed with Intel to develop their own monolithic multilayer flash memory. Although they announced an end to this collaboration in January, the two companies are still collaborating on different monolithic technology, a multilayer resistive memory called 3D-XPoint.

Arms Race

In a more general stacking configu-
ration, combining different types of
chip remains challenging. “You’ve
got to have the design, you’ve got to
have the assembly, you’ve got to have
either the same die size or thin chips
are hanging out. It’s not so easy,”
Knickerbocker cautions. As an inter-
mediate step, “a lot of people over
the past five or years have been using
what’s called 2.5D, like a silicon in-
terposer, and put multiples of these
chips side by side, or some combina-
tion of chip stacks and chips next to
them,” he said. “You can get lots and
lots of connections for adjacent chips
in a way that allows that product to be
rolled out very quickly without doing
the design consistency across many
different technologies that go into a
full chip stack.”
For example, Nvidia’s latest devic-
es for artificial intelligence applica-
tions combine a high-density inter-
connect on a silicon interposer wafer
with HBM memory stacks close to
their graphics processor unit (GPU).
“That’s a good start,” Knickerbock-
er said, “but I still think 3D and full
chip stacking for many applications
will give the best and highest perfor-
mance at the system level.”
On the other hand, stacking will al-
ways be competing with the approach
of growing new devices on a wafer
during fabrication, but it is hard to
develop a monolithic process that
does not disrupt the layers below.
“Packaging is a shortcut,” said Gargi-
ni, who oversaw many generations of
this arms race during his decades in
technology development at Intel, in-
cluding the first integration of mod-
est cache memories onto the same
die with a processor. “The packaging
side buys you performance a couple
of generations ahead of technology,
then the monolithic part catches up,”
Gargini said. “At each point in time,
you take the best trade-off between
cost and performance.”
In the end, customers care more
about the price, performance, and
size of the entire packaged device than
about how many components are in it
or how it is assembled inside. As long
as advanced packaging, whether by
stacking or other means, provides an
advantage, “these companies are ab-
solutely ready to do this stuff,” Gargini
said. “They have had these capabilities
for a long time.”