jumpenjuhosaphat

One thing that I've noticed about the major search engines is that on some sites they will never index every page. I had one site in particular that had several thousand pages, but only 150 indexed, supplemental or not. I think the reason in my case was because each page, while different, was very similar to almost every other page in my site. I don't know if that is the case with your site, but it is a possible cause. Another site, which is a forum, not every page is indexed, supplemental or not, even though every page of the forum is unique in it's content. I don't know if it has anything to do with page design, but on forums, especially SMF, there are a lot of links pointing to different pages on the site, and each page looks nearly identicle with the exception of the content, and that also might be a cause of not being indexed. Other than those reasons, both of which pertain to duplicate web pages, I've never experienced a problem with getting pages indexed. But it is something to look into.

__________________
Storage Sheds
Lost Forum

jfranktoo

Jumpen thanks. It is comforting to know that experts can also have problems.
There may be something in what you say. The format of each page is similar -the words are different. I expect that others took an in terest in what you said.

Each page on the site is a child of one main section page and that is the only one to have a decent serp. (I don't know the right technical terms for this) The others are nowhere in serp which is disappointing because I had hopes for each of them.

John

__________________
Scotland last minute short breaks and special deals, self catering cottages

Self catering cottage in Appin by Glencoe Scotland

jfranktoo

Thanks everyone. It has all worked and the site traffic is up by 50%. The idea of putting in links to the new pages from another site was excellent.

The two other sites I used were both blogs, and relevant to the topic, and each of them had a decent pr.

I've also added a robots.txt page.

You are a great bunch. Hope someone else also gets something out of this.

John
ps Under the new rules am I allowed to quote the blog sites I used as well as using the signature sites?

__________________
Scotland last minute short breaks and special deals, self catering cottages

Self catering cottage in Appin by Glencoe Scotland

Chart Music

Somehow it's the first time I read this thread. Which is quite a pity as I've been trying different methods to crack this and have spent many hours looking through bot logs to see what's going on.

Things that help.
1) Add a sitemap, yes it don't really matter but it helps indexing. Make sure it's in the Google sitemap info or in robots.txt, or better still both! Google at one point will read everything in a sitemap.xml and visit all listed pages, but it won't do it twice(but drop pages), so this is no permenant solution.
2) For things not listed, create a backlink from another site. I found that low PR pages deep within a high PR domain are idea. It seems these pages on high PR domains are crawled more often. So go for this. I once added a forum post on a thread on a PR7(?) domain and saw the Google bot come and visit my link within 20 minutes! Now that could be luck, but I tend to think not. Of course it's not possible to keep posting each of your URL's but if you have two sites then consider linking them.
3) Make your site fast or have a fast host, if you have lots of pages Google adjusts to visiting less on slow sites. So take the time and optimize the server(if you can).
4) Use short URL's.
Google will more likely visit yourdomain.com/x.html and leave a URL like yourdomain.com/index.php?topic=48871&pid=1458367&mode=normal&star t=0

What doesn't matter.
nofollow - So far as I can tell with lots of example, pages that only available from nofollow are just as good as normal to get indexed. So far as I'm concerned this makes no difference to indexing at all.

Maybe useful, but I doubt it now.
Adsense - Some time ago I use Adsense and this helps pages get listed, but I think after a while Google dropped the link between Adsense bot and Googlebot so I don't think this will help now.

__________________
Eco Tours | Gift Baskets | Random Keyword

See a good post?
<--Give thanks by clicking left of post.