In-Span, More Like In-Gone!

[offtopic]I actually prefer FilePlanet out of all of those "Wait in line" download sites. The last few times I was there I didn't have to wait in line ever. It was quite nice.[/offtopic]

I suppose I'll find a better mirror next time I see something worth downloading.

eh, well I guess this isn't so bad as the loss of cdrom.com back in the day. Though now I have to regret using in-span download links on my review site.

sargebaldy said:

eh, well I guess this isn't so bad as the loss of cdrom.com back in the day. Though now I have to regret using in-span download links on my review site.

I fixed my sites html files with on simple linux command on console :)

This requires perl:

perl -i -pne 's#3darchives.in-span.net/pub/idgames/#ftp.sunet.se/pub/pc/games/idgames/#g' *.html

It looks like the greece server is my server bitch now, no more indiana (the state) for meh!

As speaking of wget... i think i am going to do a wget -m ftp.sunet.se/pub/pc/games/idgames/
Not that someday nobody hosts doom stuff anymore...
Maybee we should get used to use edonkey/kazza/appleseed for downloading our .WAD files, or someone in the community comes around with a p2p wad-file sharing tool that all flatrate doom users should let run as long as they can give away the bandwith and CPU-Power.
I would run a copy of it for sharing doom levels!
I would even code it, but i lack the knowledge of tcp-ip and i am involved in other projects ATM.

So Long, The Werelion!

Bastet Furry said:
Maybee we should get used to use edonkey/kazza/appleseed for downloading our .WAD files, or someone in the community comes around with a p2p wad-file sharing tool that all flatrate doom users should let run as long as they can give away the bandwith and CPU-Power.

No thanks. If anything I'd host a genuine /idgames mirror, but p2p isn't what we need.

I wonder how big all of /idgames is...

Surprisingly small. I (or rather FlashFXP) measured it at 6.44 GB nearly six months ago. Some of that is non-Doom too.

Edit: I've just rechecked it. It's now 8.32 GB (27010 files), of which 5.03 GB is in /idstuff (360 files - some big Doom3 files have appeared in recent months).

FlashFXP now estimates it as being somewhat over 8 GB. /combos, /deathmatch, /graphics, /levels, /music, /skins, /sounds, /themes, and /utils (the most relevant portions) take up about 3GB total. The biggest (and probably the only) non-Doom offender is /idstuff, which is about 5GB.

EDIT reflecting Grazza's EDIT: In this case, we might see a new mirror pop up soon.

Bummer. :(

This probably makes people worry, what if all of the other mirrors eventually die out. Well, remember me if that happens, because I backup the lot bi-monthly...

Bastet Furry said:

Maybee we should get used to use edonkey/kazza/appleseed for downloading our .WAD files, or someone in the community comes around with a p2p wad-file sharing tool that all flatrate doom users should let run as long as they can give away the bandwith and CPU-Power.

I like ftp, but bittorrent would have benefits. I would think that of all p2p systems the bittorrent would be the right one for wad sharing.

Isn't BitTorrent more meant for distributing files as opposed to sharing files? Seems like it'd be quite difficult to make torrents for hundreds of files (or groups of files) and then have seeds for them. I'm no expert, but I don't think it'd work too well.

FTP's the best in this case, IMO.

mystery8 said:

Isn't BitTorrent more meant for distributing files as opposed to sharing files? Seems like it'd be quite difficult to make torrents for hundreds of files (or groups of files) and then have seeds for them. I'm no expert, but I don't think it'd work too well.

FTP's the best in this case, IMO.

You're right about that. It would be a better idea to use edonkey for this, I think - it scales better for large numbers of independent files, URLs don't change (since they're based on the checksums of the actual file), and it's distributed like bittorrent.

SirRobin said:
I fixed my sites html files with on simple linux command on console :)
This requires perl:
perl -i -pne 's#3darchives.in-span.net/pub/idgames/#ftp.sunet.se/pub/pc/games/idgames/#g' *.html [/B]

Wow, that you pop up once in a year :P
I've got it a little easier:
pesubst -s '3darchives.in-span.net' -d 'ftp.sunet.se' -mg *.html
but it's just a nice wrapper that does it without perl's in-place-editing options (-i),
btw there's an error: -p and -n do not mix.

Hirogen2 said:

pesubst -s '3darchives.in-span.net' -d 'ftp.sunet.se' -mg *.html

Replacing 3darchives.in-span.net with ftp.sunet.se isn't enough, as the paths are different. You need to replace 3darchives.in-span.net/pub with ftp.sunet.se/pub/pc/games, as in Sir Robin's post.

Mirror mirror on the net, from where should I, my files, get.

Okay this is what I know and/or have ascertained in the past, it may be helpful

archives.3dgamers.com is, as everybody knows, the upload site. ftp.gamers.org is an alias (resolves to the same IP address) of archives.3dgamers.com (and is quicker to type)

http mirrors are faster for downloads than ftp mirrors because you don't have to muck around with anonymous logins, passive mode transfers, etc. Nobody blocks port 80.

http://www.gamers.org/pub/idgames/ is, I believe, an http front-end to ftp.gamers.org. In other words it is the "freshest" mirror, so that's where you look for 0-day leaks of popular WADs. It used to be a very slow mirror (I'd get 32kb/s off it) but since last August or so it began maxing out my download speed (85-90kb/s)

http://ftp.sunet.se/pub/pc/games/idgames/ is, apart from being a few hours behind gamers.org, probably the best mirror, especially for those in Northern Europe (Sweden is topologically nearer than Greece to England, at least) It is accessible by both http and ftp. Furthermore it is also capable of rsync for those of you who maintain your own private idgames mirrors. However, it can sometimes take a few seconds to respond to http requests.

Summary list

• http://www.gamers.org/pub/idgames/ for newstuff,
• http://ftp.sunet.se/pub/pc/games/idgames/ for everything else,
• http://www.doomworld.com/idgames/ (not forgetting the front-end, obviously)

But it makes sense for all concerned if people spread the load between the various mirrors. Isn't that one of the main reasons for having several mirrors?

Edit: I just noticed that the German mirror is still up-to-date, but the url is a bit odd - to get the up-to-date version, you need to repeat the "idgames/" bit:
ftp://ftp.fu-berlin.de/pc/msdos/games/idgames/idgames/
According to this info, it's a mirror of the Swedish mirror.

There's also:
http://ftp.au.xemacs.org/pub/idgames/
ftp://ftp.au.xemacs.org/pub/idgames/
http://mrtg.planetmirror.com/pub/idgames/ (presumably that's the same as the standard planetmirror.com though)
http://www.au.horde.org/pub/idgames/ (also planetmirror, I think)

Awesome, more Australian mirrors. Props to Grazza of course.

Isn't in-span supposed to be down now? Because it isn't...

Maybe they're just taunting us, like a bully waving your shoes in front of you right before throwing them up onto the telephone/power lines...

sargebaldy said:

Isn't in-span supposed to be down now? Because it isn't...

Shhh, don't complain. :)

Jacknife said:

Son of a bitch. Well it could be worse, we could bedownloading from gamespy.

AAAAA! DONT EVER MENTION THAT
ABOMINATION!

>:-[=]
]

Just a note to clarify (for some people) what's going on...

In-span.net is/was a mirror. Every mirror going away the same day wouldn't make any difference to the integrity of the archives--they will continue to exist on our server at 3darchives. This is far different from when cdrom.com went away (or infant2 before it) in that then we had to find a new home for the archives those times.

I'm glad to hear that some folks have a mirror of the archives, though having a mirror of idstuff/ within idgames/ is probably useless--that in itself is just a mirror of id Software's files. Anyway, I have one too--we got kind of paranoid when cdrom.com pulled the plug rudely and we couldn't even get to the files, but now we're the primary so no worries.

And as Grazza said, spread out your usage of mirrors to something close (net topology-wise anyway) and fast enough for you. No point everyone moving to one other mirror.

When I'm downloading a specific file, in fact, I typically use the Doomworld database because it gives me an option right then to pick a mirror. Yay Doomworld. Maybe those of us with web links could reference the http://www.doomworld.com/idgames/?id=whatever kind of link from that instead of linking to some mirror specifically, if that's allowed?

Ty Halderman said:

Maybe those of us with web links could reference the http://www.doomworld.com/idgames/?id=whatever kind of link from that instead of linking to some mirror specifically, if that's allowed?

Yeah, that's what I use on the Doom and ZDoom wikis and would on my site if I cared enough :P

Ty Halderman said:

Maybe those of us with web links could reference the http://www.doomworld.com/idgames/?id=whatever kind of link

There might be some pitfalls there. If the database needs to be rebuilt for whatever reason, then the numbers might change, destroying these links (and in a way that a search/replace for part of the url wouldn't fix). Or maybe Ling can confirm whether the numbers in the database are completely set in stone. Can't see why they would be though, if it's generated from the fullsort document, and given that files do sometimes get deleted from the archive.

Grazza said:

There might be some pitfalls there. If the database needs to be rebuilt for whatever reason, then the numbers might change, destroying these links (and in a way that a search/replace for part of the url wouldn't fix). Or maybe Ling can confirm whether the numbers in the database are completely set in stone. Can't see why they would be though, if it's generated from the fullsort document, and given that files do sometimes get deleted from the archive.

I doubt they will change even if files get deleted etc. - it would be rather stupid to regenerate the entire database everytime something changes somewhere, so the data (including the id numbers) are most likely stored in the database permanently.

Not being Linguica, I don't know, of course, but everything else would be too stupid (and inefficient) for words. :)

Uhhh I certainly didn't code anything to change any of the number designations. I could maybe code in a "permalink" thing that generates a URL with the path/filename instead of an arbitrary number. I used numbers because it's faster to query a database for that than for a string of text.

I wasn't thinking that the numbers would be changed for the sheer hell of it. :)

What I had in mind was if there was a server failure or the database got corrupted, or something like that, causing a need for it to be regenerated from scratch. Catastrophic data loss does happen sometimes. Or if Linguica decided to implement a major upgrade of the information in the database (as suggested here), then having to keep the urls for each page the same might be an unwelcome restriction.

...plays taps for in-span.net...