Swati Lathia

Learning ways

Author : Swati Lathia

Session Ids in URLs : Crawler Confusion

Session IDs are most common in e-commerce sites and are embedded in a URL so the website can track their users or consumers from page to page and they are used to keep track of items in a consumer’s shopping cart.

But these IDs cause problems for search engine crawlers because they create a large number of links for the spider to crawl. This can create a situation where the search engine indexes essentially the same page over and over. Search engines like Google refer to it as a ‘spider trap’, which we will discuss later on.

Below are a few examples of how session IDs can give the appearance of an endless number of pages within a single site. A crawler coming to your website may find a page with the following URL:

http://www.yoursite/shop.cgi?id=dkom2354kle03i

This page gets indexed but when the spider returns later to look for new content, it finds the following:

http://www.yoursite/shop.cgi?id=hj545jkf93jf4k

This is actually the same page as before, just with a different special session ID but the spider sees it as a new URL. Because of this confusion, search engine spiders are programmed to avoid pages containing these session IDs.

Dynamic URLs : Caution For SEO

During earlier times, all websites used static HTML pages and so the first search engines were oriented towards static web pages. As the web technology developed several new methods to generate websites and dynamically generated web pages came into being. Dynamic web pages differ from static pages in that there is no physical equivalent to the page you may be viewing on the server.

Dynamic URLs are generally used if the content of a site is stored in a database and pulled for display on pages on demand.

But it is said that URLs have a distinct disadvantage in that different URLs can carry the same content.

 

A simple example would be a 1000000 page site with each page generated with the format below

http://fishbase.org/Country/CountrySpeciesSummary.cfm?c_code=356&id=24516
http://fishbase.org/Country/CountrySpeciesSummary.cfm?c_code=356&id=24517
http://fishbase.org/Country/CountrySpeciesSummary.cfm?c_code=356&id=24518
http://fishbase.org/Country/CountrySpeciesSummary.cfm?c_code=356&id=24519
http://fishbase.org/Country/CountrySpeciesSummary.cfm?c_code=356&id=24520
http://fishbase.org/Country/CountrySpeciesSummary.cfm?c_code=356&id=24521……

All the above mentioned six URLs end up with six different landing pages. But when the search engine tends to remove the information subsequent to the first offending character, the question mark (?), ultimately all the six pages become identical :

http://fishbase.org/Country/CountrySpeciesSummary.cfm
http://fishbase.org/Country/CountrySpeciesSummary.cfm
http://fishbase.org/Country/CountrySpeciesSummary.cfm
http://fishbase.org/Country/CountrySpeciesSummary.cfm
http://fishbase.org/Country/CountrySpeciesSummary.cfm
http://fishbase.org/Country/CountrySpeciesSummary.cfm

At this instant, your unique pages have become identical, and consequently the URL’s are duplicated which won’t be indexed by the search engines.

Search Engines Around The World

Google

Google is founded on September 4, 1998

Google Search Engine is the best search engine in the world. It is also the most popular search engine in the world. and it is also one of most popular product from Google. According to latest report from Netmarketshare, more than 75 percent of Search Engine market has been acquired by Google. Both desktop & mobile search engine market is dominated by Google

According to Alexa, google.com is the most visited site in the whole web.

Bing

Bing is a web search engine owned and operated by Microsoft which had launched on June 1, 2009. The service has its origins in Microsoft’s previous search engines: MSN Search, Windows Live Search and later Live Search. Bing provides a variety of search services, including web, video, image and map search products. Bing is available in 40 languages.

According to Alexa in November 2016, Bing is on No. 26

Yahoo

Both Yahoo & Bing are not giving competition to Google but giving competition to each other. According to latest report on netmarketshare, Yahoo have a market share of 5.2 percent. Yahoo is still a leader in among most popular free email providers but does not able to convince users in search engine area.

Rest of the search engines – Baidu, Ask.com, AOL, Excite, DuckDuckGo, Lycos and the list goes on.

 

Foreseeable Future

SEO – Search Engine Optimization is the art and science of optimizing your web pages to be found by the search engines like Google, Yahoo, and Bing etc. But why part art, part science?

The science part of SEO deals with technology and all the engineering & technicalities of a website such as code, information architecture & user experience etc.

While Art is all about understanding the business, and the customer behavior and try to positively influence it by adding value to them (in the form of a product, service or a solution) and getting value in return (as leads, sales or branding).

SEO plays a vital role in scaling up sales and accelerating the growth of a completely online business (such as Myntra.com) or an offline business (such as Pizza Hut).

Scroll to top