ple of this pattern is work on You Tube recommendations, which started in various engineering groups, but then moved to a research team, where the work continued using a different, and perhaps deeper, algorithmic basis.

successes

In the same way that it is difficult to define what exactly constitutes “ research,” it can be difficult to measure its “success.” In our opinion, a research project is successful if it has academic or commercial impact, or ideally, both. Commercial impact at Google is perhaps easier to measure, and the company has benefitted from numerous advances in systems, speech recognition, language translation, machine learning, market algorithms, computer vision, and more.

By academic impact we refer to impact on the academic community, other companies or industries, and the field of computer science in general. Of course, this type of impact has most traditionally come from publications, and Google continues to publish research results at increasing rates (from 13 papers published in 2003, to 130 in 2006, to 279 in 2011). Some of our papers are highly regarded and have been extensively cited. 3, 4, 7 But we feel that publications are by no means the only mechanism for knowledge dissemination: Googlers have led the creation of over 1,000 open source projects, contributed to various standards (for example, as editor of HTML5), and produced hundreds of public APIs for accessing our services. In some cases, we have used these different channels in symbiotic ways, following up an initial publication describing the high-level ideas (MapReduce, GFS, BigTable) with open source implementations of particular aspects (Protocol Buffers). In other cases, projects have started as open source initiatives from day one: Android and Chromium are probably the two most well-known examples of open source projects and demonstrate the effectiveness of this approach.

Discussion

Technology companies invest in re-
search for a number of reasons, in-
cluding: importance to the company’s
products and services, prestige and
contributions to the public good, and
reducing the risk of getting blindsided
by new technology developments.

Conclusion

Many of the world’s computer science
research questions are of great rel-
evance to Google’s business, our tech-
nical leaders, and our user community.
We have chosen to organize computer
science research differently at Google
by maximally connecting research
and development. This yields not only
innovative research results and new
technologies, but also valuable new ca-
pabilities for the company. Our hybrid
approach to research enables us to
conduct experiments at a scale that is
generally unprecedented for research
projects, generating stronger research
results that can have a wider academic
and commercial impact. We also pro-
vide flexible opportunities across the
R&D spectrum for our team members.
While our hybrid research model ex-
ploits a number of things particular to
Google, we hypothesize that it may also
serve as an interesting model for other
technology companies.

References

1. baluja, s. and Covell, M. Waveprint: efficient wavelet-based audio fingerprinting. in Pattern Recognition, 2008.

2. buderi, r. Engines of Tomorrow: How The World’s Best Companies Are Using Their Research Labs To Win The Future. simon & schuster, 2000.

3. Chang, F. et al. bigtable: a distributed storage system for structured data. in Proceedings of OSDI 2006.

4. dean, J. and ghemawat, s. Mapreduce: simplified data processing on large clusters. in Proceedings of OSDI 2004.

5. dodgson, M., gann, d. and salter, a. The Management of Technological Innovation: Strategy and Practice. oxford university Press, 2008.

6. enkel, e., gassmann, o. and Chesbrough, h. open r&d and open innovation: exploring the phenomenon. in R&D Management, 2009.

7. ghemawat, s., gobioff, h., and leung, s.t. google file system. in Proceedings of ACM SIGOPS 2003.

8. leifer, r., o’Connor, g. and rice, M. implementing radical innovation in mature firms: the role of hubs. in The Human Side of Managing Technological Innovation. r. Katz, ed., oxford university Press, 2004.

9. reis, C., barth, a., and Pizano, C. browser security: lessons from google Chrome. ACM Queue 7, 5

(May 2009).

10. schalkwyk, J. google search by voice: a case study. in Advances in Speech Recognition: Mobile Environments, Call Centers, and Clinics. a. neustein ed., springer, 2010.

11. stokes, d.e. Pasteur’s Quadrant—Basic Science and Technological Innovation. brookings institution Press, 1997.

12. uszkoreit, J., Ponte, J., Popat, a., and dubiner, M. large scale parallel document mining for machine translation. in Proceedings of COLING 2010.

additional references can be found at http://research. google.com/pubs/papers.html.

Alfred Spector ( [e-mail]) is vice President
of research and special initiatives at google, inc.
Peter norvig ( [e-mail]) is director of
research at google, inc.
Slav Petrov ( [e-mail]) is senior research
scientist at google, inc.

We acknowledge many discussions on this topic with dan huttenlocher, who spent a summer at google in 2008, and contributions and reviews from bill Coughran, Úlfar erlingsson, Fernando Pereira, Matt Welsh, and John Wilkes. We also thank the anonymous reviewers for their valuable feedback.