April 16, 2019
How Google Search Works
A brief look at how Google and other search engines deliver instant results to your searches.
Today’s internet has changed a bit from the nascent 90s, but many of the fundamental concepts are still alive and well. The web has always been a mechanism to deliver data to people around the globe. Finding what you are looking for on the web is much more complicated than it may seem when you type your daily queries into Google or your other favorite search engine. To understand Google and the mechanisms that power it, we’re going to break it down into three parts.
- The Crawler
- The Index
- Secret Sauce
Google has an automated bot which navigates around the web parsing web pages and analyze every page it visits. You may sometimes hear this referred to as the crawler, the spider, or just the Google bot. In the early days, Google took roughly 30 days to analyze the entire web. This meant that Google’s search results could be out of date for up to 30 days. To become the worlds number one search engine they needed to stay as up-to-date as possible when delivering search results to users, and thus in the Summer of 2003 Google introduced the Fritz update which allowed Google to index a substantial portion of the web every day. Fritz broke the web into a number of different segments and incrementally updated the index of the web. The crawler provides the words on the page and additional metadata to the Google index.
You can think of Google’s index like a filing cabinet. However Google’s index is a bit more complicated than pieces of paper organized alphabetically. Google’s index amounts to hundreds of Petabytes of data distributed in data centers across the globe. The index stores words and references to the web pages that store them. When you use Google to search for something on the internet Google sends your query to the data center closest to you. Once it receives your query it will determine which document is the best for your search query. How it does that is Google’s secret sauce.
What makes Google the most popular search engine on the web today is the Google algorithm(s). When you create a search Google attempts to determine the web page which will offer the user the highest value and ideally (but not always) the most correct information. To do this, a search is sent to hundreds of machines to be processed against the index. Pages relevant to the search term are then ranked by over 200 factors and delivered to the user in order of supposed usefulness. We will dive into the ranking factors in a later post to allow us to break down the individual ranking factors.
We hope this post gave you a brief introduction to the way search engines, and Google in particular, work on the web today. Stay tuned for some additional deep dives into the inner workings of search engines.
For some additional resources, we highly recommend the Google Webmasters Youtube channel.