Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Ed

Doom Center Robots.txt

Recommended Posts

I raelly hate robots.txt as it assumes that archiving is bad. There should be some exceptions for sites that used to not have it. Such attitude is why a lot of classic pages are gone.

Share this post


Link to post

There's nothing wrong with robots.txt that a bit of fine tuning can't fix. In any case, just like the lock on your front door it only keeps the honest robots out.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×