Doomworld Forums - Doom Center Robots.txt

Doomworld Forums : Powered by vBulletin version 2.2.5

Doomworld Forums > Classic Doom > Doom General > Doom Center Robots.txt

Author

All times are GMT. The time now is 21:45.

Ed
Senior Member

Posts: 1036
Registered: 07-01

For one reason or another, robots.txt has been removed from www.doomcenter.com. Archive searches have been unavailable for years

http://web.archive.org/web/20010331...doomcenter.com/

Quite a nice piece of Doom history available for viewing again.

10-15-11 22:55 #

Profile || Blog || PM || Email || Search || Add Buddy

IP || Edit/Delete || Quote

Csonicgo
This post is probably useless

Posts: 3960
Registered: 03-04

I raelly hate robots.txt as it assumes that archiving is bad. There should be some exceptions for sites that used to not have it. Such attitude is why a lot of classic pages are gone.

10-15-11 23:40 #

Profile || Blog || PM || Email || Homepage || Search || Add Buddy

IP || Edit/Delete || Quote

GreyGhost
a ghost... only grey

Posts: 5700
Registered: 01-08

There's nothing wrong with robots.txt that a bit of fine tuning can't fix. In any case, just like the lock on your front door it only keeps the honest robots out.

10-16-11 01:34 #