Yahoo! Uses Distributed Computing to Speed up Search Index Processing

After wading through the official blogs http://www.ysearchblog.com/archives/000521.html
on this subject, which mainly deal with the fact it is the world’s largest Hadoop installations, I’ve got a few interesting facts from the SEO point of
view.

• The Webmap (which is the database that feeds their algorithm) now generates in a third less time
  than it did before
• It keeps track of roughly 1 trillion links
• It uses 10,000 Linux cores (which doesn’t mean 10,000 computers or even 10,000 processors, as processors are multi-core, but I guess it makes a nice round number)

So hopefully that’s some information that might actually be of use when talking to someone non-technical or if you have a need to discuss it with a client.

Related Posts

1. What is the Supplemental Index?
2. Yahoo Search Marketing Update for 2010
3. Google announce that site speed may become a ranking factor
4. Yahoo Opens Search Results
5. More on the Supplemental Index
6. How to get out of the Supplemental Index
7. Yahoo Sponsored Local Search
8. Problems for Yahoo! Google drop ad sharing deal and Yahoo ask Microsoft to come and buy them!

This entry was posted on Tuesday, February 26th, 2008 at 10:18 am. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.