sg

Çarşamba, Kasım 23, 2005

RE: URL's restricted by Robots.txt, but I dont have one


> Why are these pages even listed?

Because spiders "crawl" sites and visit more pages than are explicitly
listed. Its possible that the links followed are from a site which is
still linking to the old pages or from an archival copy of the pages
that are being reindexed. If you don't want them listed remove the pages
and make sure the error page returns a 404 for them. Eventually they'll
be removed.

> ...and why does it say they are restricted because
> of a robots file that isnt even there?

It's possible the files are using a "robots" meta tag. Without a link
its hard to provide any assistance.

> I dont know why they would be showing old files
> when I told it to update yesterday.

You're absolutely right - Google /should/ have reindexed the entire
internet /last night/. After all, what *are* we paying these people for?

Seriously though, search engines take time. Sometimes lots of time. A
change you make today might not be visible for months.

Have patience.

Regards,

Shawn K. Hall
http://12PointDesign.com/

0 Comments:

Yorum Gönder

<< Home


Komik Videolar   islam  şarkı sözleri  yemek tarifleri  gelibolu  huzur   sağlık