DATA AND ALGORITHMS
AS MATERIAL
As interaction designers, we use
color, typography, icons, and sound
to bring an experience to life. Data
and algorithms are the materials that
affect how a user experiences AI—its
personality, biases, and idiosyncrasies.
In Committee of Infrastructure, data
is used to explicitly reveal bias and
the shortcomings of relying on AI to
make decisions. Bias is a serious issue
with real-world consequences, but
speculation allows for designers to
play out bias in absurd and imagined
ways. To conceptualize how this
meeting might take place, a city council
transcript was created using the
Karpathy char-rnn machine-learning
algorithm [ 6] (Figure 4). The algorithm
learned from seminal texts important
to the ethos of each organization
participating in the meeting [ 7].
The language created is awkward
and direct. The algorithm’s narrowness
undermines the difficulty of consensus
building, central to civic dialogue and
to the meeting’s progress. However,
the transcript does effectively display
how an AI might interact with humans
and other AIs. In addition to the verbal
arguments of each organization, sets
of video evidence (machine vision
with image classification) reveal each
organization’s motivations. Specifically,
computers classify moving objects
and assign value to each object,
creating a hierarchy that allows for
communication between vehicles,
people, and animals to avoid collisions.
PETA’s video, for example, prioritizes
AI. The good aspects of bureaucracy
(e.g., checks and balances) can act as a
way to negotiate with AI that works at a
human scale and sense of time.
Committee of Infrastructure
imagines four different groups of
stakeholders that advocate on behalf
of their organization in a city council
meeting (Figure 1), wherein humans
and AIs negotiate with each other.
Stakeholders include engineers, city
council members, presidents, AI experts
(machines), smart roads, and sensors
who express conflicting positions,
ideologies, and motivations (Figure 2).
The project purposefully positions
these imaginary human and machine
stakeholders arguing among themselves
to demonstrate how this absurd scenario
might become a reality. Discussed at
the meeting: a ballot measure for the
removal of traffic lights to create a
fully autonomous intersection. Smart
street lights, autonomous vehicles, and
embedded sensors will sense objects,
things, and people through machine
vision and proximity detection,
allowing for open communication and
for vehicles to efficiently move through
traffic (Figure 3).
speculation to create tangible products
and fictional scenarios that serve as
catalysts to discuss present issues by
“collectively redefining our relationship
to reality” [ 5]. For example, their project
United Micro Kingdoms (http://umk.
techamigo.net/), completed in 2013,
envisioned four fictional societies
organized on a spectrum of ethical
and political positions, proposing how
a future society would implement
sustainability, collectivism, and
surveillance. Their project highlights
what speculation offers design: the
opportunity to propose new realities
that aren’t bound to the technical
limitations of the present and to provoke
a dialogue about how we want our
society to progress. When envisioning
new realities, designers must remember
to question techno-determinism, the
assumption that all technology is good
because it is capable of solving problems.
Designers, developers, and researchers
must work to evaluate whether or not
their work actually addresses their
defined problem space while respecting
the user and the environment in which
their creation exists.
COMMITTEE OF
INFRASTRUCTURE:
AN AI DESIGN PROVOCATION
The new realities that designers are
tasked with envisioning make space for
the introduction of human values, such
as humor and play, while addressing
the moral and ethical uncertainties of
implementing AI. My recent project,
Committee of Infrastructure, is a design
provocation that interrogates the issues
of agency, representation, and bias. The
project explores a need to scrutinize
how governance will change with the
introduction of AI. Specifically, it
considers how humans and AI systems
interact with each other in a civic
setting to negotiate issues pertaining
to a local community. The project also
argues for the introduction of civic
dialogue as a model for interaction with
The project purposefully positions
these imaginary human and machine
stakeholders arguing among themselves
to demonstrate how this absurd scenario
might become a reality.
Figure 3. The diagram illustrates how the intersection will work after the removal of traffic
lights at Sunset Blvd. and N. Alvarado St.; proximity sensors, speed sensors, and computer
vision will work together to account for and govern the different types of traffic (vehicular,
pedestrian, cyclist, and animal).
Smart Streetlight
Computer Vision
Speed Sensors
Autonomous Vehicles
Computer Vision
Speed Sensor
Children
Sensor Embedded Clothing
Accelerometer + Gyroscope
Sensored Bicycle
Computer Vision
Speed Sensor
Animal Tracking
Computer Vision