Jeff Dean from Google recently held a lecture at the University of Washington which is highly interesting to anyone curious about the current and future challenges that face companies that operate computing on such a large scale as Google. And of course, it will give you some nice insights into how Google does things.
Since this is from Google’s perspective there are also several aspects specific to search companies, SaaS and distributed computing. It’s a very interesting lecture, easy to follow and well worth the time (it runs almost exactly an hour). You can download it or stream it from here.
Some of the things discussed are:
- Energy efficiency.
- Scaling and distributed systems.
- The potential of an OS designed from the start to run on huge server clusters with 1000s of nodes (Google currently uses Linux).
- There are some interesting pictures from inside one of Google’s data centers.
- Some anecdotes of challenges that Google’s operations teams have faced.
- The need for internal, automatic monitoring systems.
- Handling data consistency across multiple data centers for distributed applications.
- Privacy issues.
- Efficient retrieval and storage systems for data.
- The challenges of correlating data across different websites (which is one of the challenges present to make search data even better).
- Machine learning and scalability (processing huge amounts of data is a problem with existing algorithms).
- Machine translation systems.
- Speech recognition systems and their robustness.
- Computer vision.
- Security concerns and security reviews inside Google.
Though the lecture is of course from Google’s perspective, most of the points discussed really are relevant to any large-scale operation. It doesn’t go really deep into any aspect, but is more meant as a review of areas that need further research and improvement, i.e. the technical challenges facing Google.
