[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Parsing and missparsing XML
- Subject: Re: Parsing and missparsing XML
- Date: Tue, 1 Oct 2002 09:25:38 -0700 (PDT)
On Tue, 1 Oct 2002, Benno Blumenthal wrote:
> In looking through my logs, I noticed that fastsearch.net has managed
> (somehow) to find my thredds directory, but seems to be misparsing it.
I wonder if the robots shouldn't be blocked from scanning these
documents. Set up a robots.txt file perhaps.
-----------------------------------------------------------;------------
Robert Casey, Software Engineer |\
IRIS Data Management Center ..... .. .,^ /\ | \ /\ .....
1408 NE 45th St. ~ ~ ' \| ! ~~
Seattle, WA 98105 ;
------------------------------------------------------------------------
(address@hidden)