How Do Search Engines Work? Though it may seem like magic, a search engine is a computer program designed to help us navigate the endl...

 

How Do Search Engines Work?

Though it may seem like magic, a search engine is a computer program designed to help us navigate the endless information on the World Wide Web. The most popular search engines in the world are Google, Bing, Yahoo! and Baidu, China’s leading search engine. Because so many of us use search engines in our daily lives, these few companies and the systems they created possess a remarkable amount of power. You could even say they run the world. And here’s why.

Think of a search engine as both the library and the head librarian of the web. Not only do search engines scan, index, and track every website in the world — they’re also charged with helping us find just the right website when we enter a keyword phrase or “query” into their search tool. This is what you might call “Googling” something. We rely upon their answers without thinking much about how they come up with their results. But how do they work?

You See X, Google Sees Y

When search engines view websites, they don't see them exactly as humans do. Google and friends have developed elaborate programs to try and mimic human behavior as closely as possible, but at the end of the day search engines are computers, not people. 

When you see a beautiful website filled with colors, images, fonts and buttons, search engines peer behind the scenes to the code of the site. The pages of code are known as “documents" — and to most humans they look like gibberish. The search engines are able to make sense of this data, however, and use it to understand what a website is all about.

By examining the code, they can differentiate between headlines in large, bold text or small fonts that describe details about a business. They can tell which buttons are clickable and what pages on your site are interlinked to each other. They can't, however, absorb and make sense of images and pictures like humans do. Instead, search engines rely on "alt text" to understand what's depicted in each given image. 

From Searching to SERPs

Using complex algorithms that are constantly improving, search engines actually perform three major duties: crawling, indexing and retrieving data.

1. Crawling

Search engines send out automated computer programs known as “spiders” or “bots” to scan the more than 30 billion web pages on the Internet and take note of their information. The links contained within any given web page lead the bot to their subsequent destinations. The process never truly ends — not only because new websites are constantly being added, but also because the bots continually return to previously crawled websites to look for new content, new links and other updates. 

2. Indexing

What do the search engines do with all of the information they gather as their bots crawl the web? They store it in carefully organized databases that hold the billions of webpages they are monitoring on the web. This index is stored in vast data centers where search engine robots can quickly and easily access it to update information about the websites or to retrieve an answer for a user’s search query.

3. Data Retrieving

The third function of a search engine is likely the one with which you are most familiar. When a user enters a query, the search engine checks its database of websites and retrieves a list of web pages that are relevant to the query. What’s more, the search engine also ranks those results in order of relevance. And it’s all done in under a second. Pretty remarkable, right?

When you look for something online, Google isn't actually scanning the entire World Wide Web — rather, they are searching through their own library, or index, of websites. By giving your website great SEO, you can ensure that Google’s bots will know when your website is a relevant result for someone’s search. 

Note:

“A single Google query uses 1,000 computers in 0.2 seconds to retrieve an answer.” - Internet Live Stats

No one knows just how search engines determine which websites to select from their database and present on a SERP. The algorithm is intentionally kept secret in an attempt to keep the web a fair place. If Google and other search engines were to release the algorithm behind their retrieval techniques, webmasters would likely be able to hack the system and manipulate results to make their websites show up higher in searches. Despite the fact that the details of the search algorithm are kept under wraps, we do have a good understanding of which factors can influence how a site ranks — that’s what giving your site good SEO is all about. 

Understanding Rankings

Google’s algorithm relies on more than 200 parameters to determine which websites they think will be most relevant for a given query. They scan their index, looking at how relevant each website is to a search phrase while taking into consideration the site’s popularity and reputation. Popular websites will rank higher in searches than websites that receive little or no traffic.

Again, while the exact algorithms used by search engines are kept secret, experts agree that the following elements have an impact on how well a website ranks in searches. These elements form the basis of SEO. 

• Domain - Is the user’s search phrase found in a website’s domain name?
• Titles and Descriptions - Is the user’s search phrase found in the website’s page titles and descriptions? 
• Keyword Frequency - How often is the user’s search phrase found in the website content or as image alt text?
• Freshness - How regularly and how recently was the website updated?
• Backlinks - How many others websites link to the site?
• Quality of Links - How reputable are the sites who link to the site? Are they spammy or professional and helpful?
• Engagement - How many people click on the website in search results and how much time do they spend on the website?
• Bounce Rate - How many people click on the website and immediately leave? A high bounce rate will negatively affect the website's rank.
• Brand Reputation - How often is the brand, business or site domain mentioned in news and media?
• Social Media - How often are people mentioning the website in Tweets, on Google+ and on Facebook posts?

Using a combination of these factors and many more, search engines strive to provide each user with a list of web pages that will be most likely to answer their query.

Buried Treasures: Why Links Matter

Search engines are not infallible, and they don’t find every site on the web. There are over 1 billion websites on the Internet and crawlers do their best to find and index them all. But search engines find websites by following links — if a website is not linked from other pages on the web, bots have a harder time finding and indexing it. To make sure your own website is not overlooked, submit your sitemap to Google. You also want to try and get links from other websites. These are called “backlinks” and they can help make sure you avoid creating a beautiful website that search engines never discover. Find out more about how to get backlinks here.