DIGIT-ARC read-only archive · snapshot: July 23, 2026

.:: Has Google Stopped Deep Crawl? ::.

9 posts · originally from https://geek.digit.in/community/threads/has-google-stopped-deep-crawl.7972/

tuXian
tuXian Original post #1
Apr 6, 2005 at 8:09 AM
The other day somewhere in the General Discussion a user wrote that Google's Deep crawl wherein the bot indexes the site completely isnt happening from the long time. I found the statement to be true coz the googlebot indexes only the index page of my site and goes away while other search engines have done a deep crawl.

Is Deep crawl really not taking place? Can someone please confirm this??
rajesh
rajesh #2
Apr 6, 2005 at 8:30 AM
No. I dont think so.

I was trying to help someone out in another forum and when i was searching for some clarification on Google, it returned me a link to the very post on the forum. That post was posted on the same day and it was not on the index page also.
ferrarif50
Apr 6, 2005 at 11:40 AM
No, Infact they are bringing out new strategies and plans for it. It is a matter of wait and see!
tuXian
tuXian #4
Apr 6, 2005 at 12:16 PM
rajesh said:
No. I dont think so.

I was trying to help someone out in another forum and when i was searching for some clarification on Google, it returned me a link to the very post on the forum. That post was posted on the same day and it was not on the index page also.

May be that site is old and first deep crawled long ago thats why its been again deep crawled. Is it possible that the new sites are not be deep crawled as my site is not an old one and is just a month old.

can anyone throw light on my first question above in the first post and as well the one I have stated just above this
Deep
Deep #5
Apr 6, 2005 at 1:51 PM
TuxiaN said:
The other day somewhere in the General Discussion a user wrote that Google's Deep crawl wherein the bot indexes the site completely isnt happening from the long time. I found the statement to be true coz the googlebot indexes only the index page of my site and goes away while other search engines have done a deep crawl.

Is Deep crawl really not taking place? Can someone please confirm this??

it is not true...
google indexes all the pages pretty well..you just need to give direction to google..

have site map and proper link structure so that google can follow links...

check ur raw access logs or awstats also....

i have many inside pages scanned and listed very well..

Deep
S
sunnydiv #6
Apr 6, 2005 at 2:08 PM
nope, cant be, as major of google business depends on number of pages spidered

so , i dont think, no go
tuXian
tuXian #7
Apr 7, 2005 at 11:19 AM
Well then I think the fault lies with my web pages. I think this might be happening coz of this:

I have a cookie on the site that asks the first time visitor by means of a Javascript propmt to enter the user name which is then displayed as [ Hi! username ]. So when the URL is entered and as soon as the page loads the first thing a user gets is a java script propmt to enter the name. Those who have already entered the name before ofcourse wont get this prompy.

May be the googlebot has got problem with that.

Is my doubt correct? Can anyone elaborate on it??
Deep
Deep #8
Apr 7, 2005 at 11:21 AM
ah yes it seems to be correct coz bots cant read javascript....

and thats why it stops...

Deep
enoonmai
Apr 7, 2005 at 11:01 PM
While Google is testing a new GoogleBot that tries to read JS to check for valid redirects and to cut down unauthorized doorway redirects, the current one does only text. If it encounters an error in parsing your site on the home page, it WILL not proceed further with indexing your site. For more information on optimizing your site for the GoogleBot, please refer here:
*scribbling.net/help_the_googlebot_understand_your_web_site