sg

Salı, Kasım 29, 2005

Question about SiteMap Filters


Hello all,

I am trying to configure the filters to index all .html files and
exclude all other files. However, there are also several directories
that I would like to exclude from .html walking and indexing. The
problem is, it seems that the filter of:

<filter action="pass" type="wildcard" pattern="*.html*" />

supersedes all other filters such as (example):

<filter action="drop" type="wildcard" pattern="*/mydirectory/*" />

The directories within my drop filters are still being walked and
indexed.

Here are example filters in my config.xml file:

<filter action="pass" type="wildcard" pattern="*.html*" />

<filter action="drop" type="wildcard" pattern="*/mydirectory/*" />

<filter action="drop" type="wildcard" pattern="*" />

I tried adding disallow directives to my robots.txt file as in the
following example:

User-agent: *
Disallow: /mydirectory/

My robots.txt has passed a validation test so the robots.txt file
should prevent the disallowed directories from being walked and indexed
but unfortunately they are still being walked and indexed. Has anyone
got any suggestions on successfully setting up certain directories from
being walked and indexed while still using the *.html* filter?

Sincerely,
Scott

0 Comments:

Yorum Gönder

<< Home


Komik Videolar   islam  şarkı sözleri  yemek tarifleri  gelibolu  huzur   sağlık