What is Search Engine?

What is Search Engine?

Search Engine :

Search Engines are nothing but a software which is designed to search and display information in the world wide web. Search results are displayed in line wise on multiple pages (SERPs). Generally, each search engine results (SERP) page displays 10 results. But, we can change it accordingly on search engine settings. There are number of websites which contains related information about our query. But, it follows some metrics to rate all those websites; depending on that it decides which webpage to display first and which webpage to next. We have trillions of websites in world wide as well daily billion websites are registering. Search Engines are mainly focus to bring and display genuine and perfect results to users. Its ultimate goal is to satisfy the user by providing proper information. For this purpose Search Engine uses Bots & Spiders which are nothing but special type of algorithms. These algorithms strictly check each and every pages information and also checks for spam. The resulted information in SERP may be a combination of webpages, videos, images and other type of files. Few search engines also mine data from data bases or open directories. Search Engine maintains real-time information by run an algorithm on a web crawler.


How Search Engine works:

In recent time, search engines maintains 3 processes. They are:

  1. Web crawling
  2. Indexing
  3. Searching

1.  Search Engines analyse the information by web crawling from site to site and page to page. The web spider checks for site’s robots.txt file which addresses to it. robots.txt is also known as ‘robots exclusion protocol’. It is nothing but a standard used for communication between web crawlers and websites. It specifies the robot that which pages it shouldn’t be crawled or processed. After checking for robots.txt file, the web spider will send certain info back to be indexed depends on various factors like Content, Title, CSS, Script, headings, etc. Mostly, it gets information from page’s HTML metadata where it has various meta tags.

2.  In general indexing is useful for faster searches. In the same manner web indexing also works. Web indexing means associate words and other definitions of the word found on web-pages, in their domain names and in html fields. All the associations are store in public database which available for search queries of web. Users can give their query as a single word or multiple words. With the help of index, the information which is related to the query finds very quickly. Apart from Index, there is a cache which is used to store recent computations again in index manner. When user searches for a query, then it first searches in the cache. If the info is there in the cache then it will be serve faster. If the info not there in the cache then it will search in index. The techniques use for caching and indexing are trade secrets; But web crawling is a straight process where it crawls all the websites on systematic manner.

3.  While web spider visiting the websites, it stores the pages (some content or all the content) in the search engine working memory. Between the visits of the spider if anyone inquires about those pages then cached version of those pages which are stored in search engine working memory will quickly sent to the inquirer. If visit becomes overdue then the search engine acts as a web-proxy. In this scenario, the page may differ from indexed search terms. The cached page (whose words were indexed) acts as an actual one in the absence of actual page. So, if the actual page has lost then cached page can be useful to the website. But, this issue can be considered as a mild form of linkrot. (Linkrot is nothing but hyperlinks which points to webpages and other resources have become unavailable.)


When the user enters his keywords (Keywords are nothing but search terms of user) in the Search Engine search box then SE searches for that in its index, which is stored in SE’s database. The index already contains all the websites names with their corresponding keywords. Then depending on those keywords, the results will instantly obtain from index. Now, the real process begins by generating webpages list which has to show as search results. By following few metrics, every webpage in the list must be weighted depends on info in the indexes. Now the top result needs to prepare for its lookup and markup of the snippets, which shows the keywords matched. And this process is for one result. The post results will also require more of this process to display. And, all this process will complete in very less time.

Apart from preparing to display webpages depends on their weights, search engine also offers their own GUI or command driven operators & search parameters to filter the search results. These give users an extra advantage over their searches. They can further filter results by giving their requirements. This will give great control for the user to select his desired ones while refining the results. For example, Google proving so many options to filter the results in its SERP (search engine results page). If user enters his keyword as “seo online videos” then search engine displays certain results which are related to the search term. All those webpages doesn’t contain videos. The results are combination of all. Now if you want to access only videos, then click on Videos tab from serp’s menu. Then it shows only videos by further filtering the results. Like this, user can select anything he want from the serp menu like Images, News, Maps. Now, if user wants to access results of desired dates. Then he has to click on Tools which is rightside on Google. Then it shows lots of options below to that. Click on Any time and click on Custom range. Give your custom range. Now, it shows the results which has posted in that date range. Yes, it’s possible to refine the results by its date range because every page has modification date. Most of the search engines support boolean operators And, Or, Not which helps users to filter search queries. These operators are useful for literal searches where user can filter the results and extend his terms of search. Like in above example, to select his custom date range he has to give one date and another date. So, boolean operators are useful in refining the results where conditions takes place.

The greatness of search engines depend on how accurately they are providing results to the end users. Even-though there are millions of webpages which include search term, they have to filter the most relevant, authoritative and popular ones. In the other hand, it has to avoid fake content, copy righted content, malware, hacking , phishing and spam websites. Almost every search engine has its methods to rank the filtered results to displays the accurate results first. These methods are vary from one search engine to another. Anyway, a search engine decides which webpages are most relevant to the search term and in what order the filtered results have to display. The order of results are vary in individual search engines depending on their own methods. These methods are also change time to time when new techniques evolved.

Almost all the search engines are commercial ventures. They are getting revenue from advertisements. Some search engines are charging fee from the advertisers to move their listings to top in SERP’s advertisement space. And these listings will display with symbol Ad along with regular results. For this, advertiser has to participate in the bid for a particular keyword. At first search engine shows suggested bid amount for a particular keyword in its business page (Ex: For Google it is AdWords). Then advertiser decides, what amount will workout for him through that keyword. If that keyword is very essential for his business then he bids for more amount. In the bidding time, there it asks for advertisement type. There are different types of ad types. But the famous one’s are, 1. Search Ads and 2. Display Ads. The process includes,

i) While displaying search results for a keyword; the search engine checks, is there any bids for that keyword. Then depends on bid value it filters the top ones and display on SERP along with regular results. The number of this ad listings may vary from keyword to keyword and search engine to search engine. It depends on individual search engine methods it follow.

ii) When comes to display ads, the ad will display on websites which are related to that ad. i.e., All those websites contains that ad’s keywords. For example, if an advertiser creates an ad with the keyword Samsung Mobile, then that ad displays in websites which contains that keyword. Then, if any user clicks on that ad then the search engine charges money from that advertiser. For every click, advertiser has to pay money and in otherside search engines earning money.

Apart from Search ads and Display ads, there are some other ad types like Search and Display ads, Shopping ads, Videos ads, App ads, etc.


Why only Google:

Now a days, Google is the most advanced and popular search engine in the world. It has the market share of 80.52 percent as of mid 2017. It is more popular in the countries like USA, India, etc. Apart from search engine, it has other popular products like Gmail, Google Drive, Google Maps, Google Plus, Google Photos, Blog, etc. Even-though it’s more advanced, it is updating itself in regular intervals to serve better than previous. It places the google in king’s position, while other search engines are following the foot steps of Google.

The top 10 search engines in the world from rank#1 to rank#10 are as follows,

  1. Google
  2. Bing
  3. Yahoo
  4. Ask.com
  5. AOL.com
  6. Baidu
  7. Wolframalpha
  8. DuckDuckGo
  9. Internet Archive
  10. ChaCha.com