Communications of the ACM

help people explore and understand data, building on prior HCI research on visualization. 2 Machine learning advances yield significant gains in conversational intelligence and question answering. Improvements in near- and far-field speech recognition coupled with new dialog research make conversational search feasible. Even within current interaction paradigms, deeply understanding query and document semantics can help provide more intelligent responses; for example, medical symptom answers on Google and multi-perspective answers on Bing.

Mobile devices such as smartphones and tablets are powerful and versatile. The integration of hardware such as accelerometers, gyroscopes, and proximity sensors provides rich contextual signals about user activities that are useful for search and recommendation. Evidence from self-reports and log analysis suggests people now demand search support in more situations—to resolve a diverse set of questions (or arguments!)—and question complexity continues to rise. Complex tasks spanning devices are also more frequent. Search systems can utilize downtime between task activities to perform “slow searches,” for example, finding sets of relevant resources or using crowdworkers to compose answers.

Wearable and augmented reality applications support the presentation of relevant information just in time, in anticipation of its use. Hardware such as hearables (for example, Google Pixel Buds) or head-mounted displays (such as Google Glass, Microsoft HoloLens) provide continuous information access in any setting. For some tasks (for example, monitoring activities), relevant information can be offered proactively, capitalizing on signals such as user preferences and location. Proactive notifications need to be carefully gated and privacy must be respected, including the privacy of any collocated individuals.

The wealth of opportunity should not translate to dramatically increased complexity. The prevalence of the Google interface design has meant searchers expect simplicity, and rightly so: search activities are already sufficiently complex. Any new capabilities must be intuitive, simple, and add clear value.

Virtual Assistants

Integration with virtual assistants such as Amazon Alexa, Google Assistant, or Microsoft Cortana allows search systems to extend their capabilities to better understand needs and support higher-order search activities such as learning, decision making, and action. 9 Search engines can provide an entry point to virtual assistants when search requests demand additional engagement (for example, are non-navigation-al). Search technology already powers some virtual assistants, and knowledge bases created for information finding have utility herein. End-to-end task completion (that is, from search interactions to action in the physical world) has traditionally been underserved by search engines. This can be achieved via first- and third-party skills in virtual assistants. Skills best suited to the current context can be recommended by assistants and even chained together to support multistage tasks.

Virtual assistants are particularly amenable to supporting search interaction: they are personal and contextual, they support dialog, and they are ubiquitous (across applications and devices). Deep understanding of searchers and their contexts is necessary to adapt system responses to the situation. Natural interactions, including multi-turn dialogs, enable search systems to clarify searcher needs. Conversational search is already attracting significant interest. 3 Ubiquity has advantages beyond availability, that is, richer data enables sophisticated inferences such as automatically detecting task completion or estimating task duration, as well as supporting rapid task resumption.

Despite its promise, search-assis-tant integration is not without challenges that require rethinking several aspects of search interaction. For example, although virtual assistants can foster dialog, natural language conversations can be inefficient ways to obtain answers or complete tasks. Virtual assistants often manifest in headless devices such as smart speakers and personal audio, making it difficult to communicate result lists or discover assistant capabilities. 16 Also, the traditional search-advertising model depends on visual attention and does not scale well to audio-only settings.

Looking Ahead

We are just beginning a journey to a more enlightened society facilitated by interactions with search systems. Looking ahead, the data revolution in search interaction will gather pace, searchers will engage with search systems in new ways, and virtual assistants will serve as comprehensive search companions. Building on these and other pillars, search systems will empower people and support the activities they value. This important effort will only succeed given the expertise, collaboration, and commitment of communities within computer science and beyond.

References

1. Agichtein, E., Brill, E., and Dumais, S. Improving web search ranking by incorporating user behavior information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2006, 19–26.

2. Ahlberg, C., Williamson, C., and Shneiderman, B. Dynamic queries for information exploration: An implementation and evaluation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (1992), 619–626.

3. Bennett, P.N. et al. Modeling the impact of short-and long-term behavior on search personalization. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2012, 185–194.

4. Bush, V. As we may think. The Atlantic Monthly 176, 1 (Jan. 1945), 101–108.

5. Eckles, D., Karrer, B., and Ugander, J. Design and analysis of experiments in networks: Reducing bias from interference. Journal of Causal Inference 5, 1 (Jan. 2017).

6. Ginsberg, J. et al. Detecting influenza epidemics using search engine query data. Nature 457, 7232, (2009), 1012–1014.

7. Kay, M., Matuszek, C., and Munson, S. A. Unequal representation and gender stereotypes in image search results for occupations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems ACM, 2015, 3819–3828.

8. Hearst, M.A. (2011). ‘Natural’ search user interfaces. Commun. ACM 54, 11 (Nov. 2011), 60–67.

9. Marchionini, G. Exploratory search: From finding to understanding. Commun. ACM 49, 4 (Apr. 2006), 41–46.

10. Metrikov, P. et al. Whole page optimization: How page elements interact with the position auction. In Proceedings of the 15th ACM Conference on Economics and Computation. ACM, 583–600, 2014.

11. Pariser, E. The Filter Bubble: How the New Personalized Web is Changing What We Read and How We Think. Penguin, New York, NY, 2011.

12. Robertson, S.E. and Hancock-Beaulieu, M.M. On the evaluation of IR systems. Information Processing and Management 28, 4 (Apr. 1992), 457–466.

13. White, R. W., Bilenko, M., and Cucerzan, S. Studying the use of popular destinations to enhance web search interaction. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 159–166, 2007.

14. White, R. W. Beliefs and biases in web search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, 3–12, 2013.