help people explore and understand
data, building on prior HCI research
on visualization. 2 Machine learning
advances yield significant gains in conversational intelligence and question
answering. Improvements in near- and
far-field speech recognition coupled
with new dialog research make conversational search feasible. Even within
current interaction paradigms, deeply
understanding query and document
semantics can help provide more intelligent responses; for example, medical symptom answers on Google and
multi-perspective answers on Bing.
Mobile devices such as smartphones and tablets are powerful and
versatile. The integration of hardware
such as accelerometers, gyroscopes,
and proximity sensors provides rich
contextual signals about user activities that are useful for search and recommendation. Evidence from self-reports and log analysis suggests people
now demand search support in more
situations—to resolve a diverse set
of questions (or arguments!)—and
question complexity continues to rise.
Complex tasks spanning devices are
also more frequent. Search systems
can utilize downtime between task
activities to perform “slow searches,”
for example, finding sets of relevant
resources or using crowdworkers to
compose answers.
Wearable and augmented reality applications support the presentation of relevant information just in
time, in anticipation of its use. Hardware such as hearables (for example,
Google Pixel Buds) or head-mounted
displays (such as Google Glass, Microsoft HoloLens) provide continuous
information access in any setting. For
some tasks (for example, monitoring
activities), relevant information can
be offered proactively, capitalizing on
signals such as user preferences and
location. Proactive notifications need
to be carefully gated and privacy must
be respected, including the privacy of
any collocated individuals.
The wealth of opportunity should not
translate to dramatically increased complexity. The prevalence of the Google
interface design has meant searchers
expect simplicity, and rightly so: search
activities are already sufficiently complex. Any new capabilities must be intuitive, simple, and add clear value.
Virtual Assistants
Integration with virtual assistants such
as Amazon Alexa, Google Assistant, or
Microsoft Cortana allows search systems to extend their capabilities to
better understand needs and support
higher-order search activities such as
learning, decision making, and action. 9
Search engines can provide an entry
point to virtual assistants when search
requests demand additional engagement (for example, are non-navigation-al). Search technology already powers
some virtual assistants, and knowledge
bases created for information finding
have utility herein. End-to-end task
completion (that is, from search interactions to action in the physical world)
has traditionally been underserved by
search engines. This can be achieved
via first- and third-party skills in virtual
assistants. Skills best suited to the current context can be recommended by
assistants and even chained together
to support multistage tasks.
Virtual assistants are particularly
amenable to supporting search interaction: they are personal and contextual, they support dialog, and they
are ubiquitous (across applications
and devices). Deep understanding of
searchers and their contexts is necessary to adapt system responses to the
situation. Natural interactions, including multi-turn dialogs, enable search
systems to clarify searcher needs. Conversational search is already attracting significant interest. 3 Ubiquity has
advantages beyond availability, that is,
richer data enables sophisticated inferences such as automatically detecting task completion or estimating task
duration, as well as supporting rapid
task resumption.
Despite its promise, search-assis-tant integration is not without challenges that require rethinking several
aspects of search interaction. For example, although virtual assistants can
foster dialog, natural language conversations can be inefficient ways to obtain answers or complete tasks. Virtual
assistants often manifest in headless
devices such as smart speakers and
personal audio, making it difficult to
communicate result lists or discover
assistant capabilities. 16 Also, the traditional search-advertising model depends on visual attention and does not
scale well to audio-only settings.
Looking Ahead
We are just beginning a journey to a
more enlightened society facilitated by
interactions with search systems. Looking ahead, the data revolution in search
interaction will gather pace, searchers
will engage with search systems in new
ways, and virtual assistants will serve
as comprehensive search companions.
Building on these and other pillars,
search systems will empower people
and support the activities they value.
This important effort will only succeed
given the expertise, collaboration, and
commitment of communities within
computer science and beyond.
References
1. Agichtein, E., Brill, E., and Dumais, S. Improving
web search ranking by incorporating user behavior
information. In Proceedings of the 29th Annual
International ACM SIGIR Conference on Research and
Development in Information Retrieval, 2006, 19–26.
2. Ahlberg, C., Williamson, C., and Shneiderman, B.
Dynamic queries for information exploration: An
implementation and evaluation. In Proceedings of the
SIGCHI Conference on Human Factors in Computing
Systems (1992), 619–626.
3. Bennett, P.N. et al. Modeling the impact of short-and
long-term behavior on search personalization. In
Proceedings of the 35th International ACM SIGIR
Conference on Research and Development in
Information Retrieval, 2012, 185–194.
4. Bush, V. As we may think. The Atlantic Monthly 176, 1
(Jan. 1945), 101–108.
5. Eckles, D., Karrer, B., and Ugander, J. Design and
analysis of experiments in networks: Reducing bias
from interference. Journal of Causal Inference 5, 1
(Jan. 2017).
6. Ginsberg, J. et al. Detecting influenza epidemics using
search engine query data. Nature 457, 7232, (2009),
1012–1014.
7. Kay, M., Matuszek, C., and Munson, S. A. Unequal
representation and gender stereotypes in image
search results for occupations. In Proceedings of the
33rd Annual ACM Conference on Human Factors in
Computing Systems ACM, 2015, 3819–3828.
8. Hearst, M.A. (2011). ‘Natural’ search user interfaces.
Commun. ACM 54, 11 (Nov. 2011), 60–67.
9. Marchionini, G. Exploratory search: From finding to
understanding. Commun. ACM 49, 4 (Apr. 2006), 41–46.
10. Metrikov, P. et al. Whole page optimization: How
page elements interact with the position auction. In
Proceedings of the 15th ACM Conference on Economics
and Computation. ACM, 583–600, 2014.
11. Pariser, E. The Filter Bubble: How the New
Personalized Web is Changing What We Read and How
We Think. Penguin, New York, NY, 2011.
12. Robertson, S.E. and Hancock-Beaulieu, M.M. On the
evaluation of IR systems. Information Processing and
Management 28, 4 (Apr. 1992), 457–466.
13. White, R. W., Bilenko, M., and Cucerzan, S. Studying
the use of popular destinations to enhance web
search interaction. In Proceedings of the 30th Annual
International ACM SIGIR Conference on Research
and Development in Information Retrieval. ACM,
159–166, 2007.
14. White, R. W. Beliefs and biases in web search. In
Proceedings of the 36th International ACM SIGIR
Conference on Research and Development in
Information Retrieval, ACM, 3–12, 2013.
15. White, R. W. Interactions with Search Systems.
Cambridge University Press, New York, N Y, 2016.
16. White, R. W. Skill discovery in virtual assistants.
Commun. ACM 61, 11 (Nov. 2018), 108–115.
Ryen W. White ( ryenw@microsoft.com) is a Partner
Researcher and a Research Manager at Microsoft
Research AI, Redmond, WA, USA.
Copyright by author.