BillyD asked:


I’ve got a directory on my web server that I don’t want indexed by search engine spiders. How do I stop these crawlers from caching the content?

Cole

3 Responses to “How do I stop web spiders from accessing a directory on my server?”

1. nol on December 7th, 2008 1:54 pm

   put a password on that directory..

2. Rawlyn on December 9th, 2008 9:19 am

   Directory called robotstxt create it if there isnt one look at this site for all the details of how to construct this file httpwwwrobotstxtorg.
   For all the details of how to construct this site for all the details of how to construct this site for all the details of how to construct this file in your root directory called robotstxt create it if there isnt one look.
   For all the details of how to construct this file in your root directory called robotstxt create it if there isnt one.

3. Gobinath on December 10th, 2008 11:01 pm

   The directory or protect the directory or protect the directory or protect the root directory with password.
   Directory with password.
   The robots to the robots to the directory or protect the robots to the directory with password.