this is an incredibly complex situation for an organization like Goldman
Sachs with dedicated threat, vulnerability management, and infrastructure teams. Navigation for a small or
medium-sized business without dedicated triage teams is likely harder. We
rely heavily on vendor coordination
for clarity on patch dependency and
still have to move forward with less-than-perfect answers at times.
Good cyber-hygiene practices remain foundational—the nature of
the vulnerability is different, but the
framework and approach to managing it are not. In a world of zero days
and multidimensional vulnerabilities such as Spectre and Meltdown,
the speed and effectiveness of the
response to triage and prioritizing
risk-reduction efforts are vital to all
organizations. More high profile and
complex vulnerabilities are sure to
follow, so now is a good time to take
lessons learned from Spectre and
Meltdown and use them to help prepare for the next battle.
Securing the Tangled Web
One Step Ahead
Understanding Software Patching
1. Aleph One. Smashing the stack for fun and profit.
Phrack 49, 7 (1996); http://phrack.org/issues/49/14.
2. Siron, E. The actual performance impact of Spectre/
Meltdown Hyper-V updates. Hyper-V Blog, 2018;
3. Trippel, C., Lustig, D., Martonosi, M. Meltdown Prime
and Spectre Prime: automatically synthesized
attacks exploiting invalidation-based coherence
protocols, 2018. arXiv:1802.03802; https://arxiv.org/
Rich Bennett, Craig Callahan, Stacy Jones, Matt
Levine, Merrill Miller, and Andy Ozment are on the
global technology risk and information security team at
Copyright held by owners/authors.
More specific patches for browsers came out from days to weeks after
the initial vulnerability disclosure.
We pushed those patches out rapidly.
A few hundred of our developers use
nonstandard browsers to test their
applications, so we used application
whitelisting on user endpoints to ensure that only managed browsers, or
approved and patched exceptions,
were being used.
Browser plug-ins can also execute
on untrusted code. As a partial mitigant, specific plug-ins can be locked
down to a set of whitelisted sites. Very
few plug-ins have released patches,
so this remains an area of concern.
Some firms have also chosen to virtualize browsers to isolate the application from the operating system. A
browser can be virtualized either as a
stand-alone application or as the entire desktop operating system. If any
mission-critical Web applications are
running on legacy browsers or with
plug-ins, a virtualized browser can
provide a more protected mechanism
for doing so.
Email is another common vector
for untrusted code. It is not a likely
tool for exploiting these vulnerabilities as a majority of the attack vectors
have included cache-timing attacks,
which are difficult or impossible to
exploit over email. Nonetheless, it
is important to address phishing attacks as a means of general exploitation. Most firms, including Goldman
Sachs, use a variety of techniques to
block email-based attacks.
The simplest technique is to block
certain types of attachments. If your
business supports it, this is a relatively cheap control that can have a significant impact. Unfortunately, many
businesses depend upon being able to
share office documents, such as PDF or
Excel files, that can include macros or
other types of code.
Of course, phishing emails do not
necessarily contain attachments.
They can also contain links to malicious websites. We rewrite incoming
URLs so that outbound calls have to go
through a central control point where
we can quickly implement a block.
Outbound Web connections also have
to go through the same proxy-based
controls described earlier.
In addition, we use signature-
based email blocking technologies
within our layered approach. As long
as there are no known exploits, how-
ever, there are no known signatures
to deploy. This will be an area to track
going forward when the exploits move
from research proof-of-concept to be-
There will likely be more value in
“combustion chambers,” which open
attachments in a virtual machine and
look for malicious behavior. Some
combustion chamber vendors are
looking at running unpatched virtual
machines and using them to detect
the exploitation of these vulnerabilities.
Hardware fixes. While patches
and controls are the focus here, hardware fixes are not totally out of the
question. Intel indicated in its Q4
earnings call that chips with silicon
changes (directly addressing Spectre
and Meltdown) will begin to hit the
market later this year. Similar to the
operating system patches, however,
the first generation of hardware fixes
may not fully address the vulnerabilities. Moreover, it will be years before
organizations upgrade all of their
hardware with the new chips.
Just Another Day of
These vulnerabilities pushed the
vulnerability management process
at Goldman Sachs, but they did not
break it. We are used to making risk
trade-offs in this space: for example,
do you patch more quickly to decrease the risk of cyber-exploitation,
even if that increases the risk of an
The risks posed by these vulnerabilities are a more challenging version of
that scenario. The question is not just
whether the patches would break a
system, but whether they would have a
significant performance impact. That
risk has to be assessed in a distributed
way, as it is unique to each application.
At the same time, there is a lot of uncertainty about these vulnerabilities and
how readily they could be exploited. We
therefore have to balance operational
risk with cyber-attack risk when both
risks are unclear.
The scope of vulnerabilities such as
Meltdown and Spectre is so vast that
it can be difficult to address. At best,