Download Everything?

Im at:

ftp://ftp.fu-berlin.de/pc/msdos/games/idgames/levels/doom2/

Isnt there a smart way to download ALL the Doom2 wad files at once, so i dont have to download them one by one?

There are plenty of FTP tools that can help you to do this, such as FileZilla (you just need to install the Client software).

Try wget

Started Dowloading them now with FIleZilla.

But it disconnects and reconnects all the time :S

Is there anything that can be made to remove that?

All the Doom 2 levels?

I don't know how much space that takes up, but I doubt it'll fit on an average hard drive.

Disk space is not a problem ;)

Processingcontrol said:

All the Doom 2 levels?

I don't know how much space that takes up, but I doubt it'll fit on an average hard drive.

Idgames (excluding the idstuff & incoming folders) is only ~8.25GB @ 32,891 files. I think you over estimate how big doom PWADs are.

Only 8.25? WTF

Even including stuff from the /ports section? I mean there's a lotta big stuff in there, especially ZDoom or Skulltag wads that use new monsters and MP3 music and stuff. But then again with all the 1994 pwads in there that are less than 100kb each those probably doesn't even add up to a full gig by themselves.

40oz said:

Even including stuff from the /ports section? I mean there's a lotta big stuff in there, especially ZDoom or Skulltag wads that use new monsters and MP3 music and stuff.

That number is half a year old, and while I'm updating it right now anyway, I doubt that number will change much.
Edit: Finished updating. It's now 8.72GB @ 33,250 files.

40oz said:

But then again with all the 1994 pwads in there that are less than 100kb each those probably doesn't even add up to a full gig by themselves.

It gets even more strange when I have ~1GB of wasted space because of all the less than 1KB text/demo files (I'm not so sure what the cluster size is on my external).

What is the point? You'll never really have the entire collection of Doom wads unless the repository gets locked down. They'll just keep coming.

Also, your collection will be filled with redundant textfiles, which are in 99% of cases already in their accompanying zipfiles.

And you'll never play them all in your lifetime, or your next lifetime or your next next lifetime.

Just use this.

Only justifiable reason is if you wanted to save the current state of /idgames on a DVD in case an Internet cataclysm happens.

Processingcontrol said:

Only 8.25? WTF

If you're only after levels/doom2/ it's 5.21 Gig and 15,069 files.

printz said:

Also, your collection will be filled with redundant textfiles, which are in 99% of cases already in their accompanying zipfiles.

The question in the thread is "a smart way to download ALL the Doom2 wad files at once" so this criticism seems rather irrelevant; if you had a smart script to do the downloads for you, it could also inspect the zip archives for duplicated text files and automatically correct that problem.

I think it'd be interesting if someone played all Doom II WADs in the A category, then someone else could do B, someone else for C, etc. People can post a brief description of the level to give insight to what it was like and if they'd recommend it. Sort of like the Maximum Doom guide.

valkiriforce - There are at least two recent threads that I know of with /idgames recommendations:

DoomHero's thread: http://www.doomworld.com/vb/wads-mods/55897-doomheros-idgames-findings/

My thread: http://www.doomworld.com/vb/wads-mods/48476-idgames-i-liked-it-cool-maps/

So far there are only recommendations for wads from '94 and '95.

valkiriforce said:

I think it'd be interesting if someone played all Doom II WADs in the A category, then someone else could do B, someone else for C, etc. People can post a brief description of the level to give insight to what it was like and if they'd recommend it. Sort of like the Maximum Doom guide.

Grimm was doing something like that on his own until he ran out of steam - here's the review site

Campaign to record demos on every level in idgames/levels/doom.
Demos by Michael Novikov: http://doomedsda.us/player5lmps.html

Most of those demos have textfiles packaged with them. You can see them as mini-reviews.

edit: oops, didn't notice that we are talking about Doom II wads.

GreyGhost said:

Grimm was doing something like that on his own until he ran out of steam - here's the review site

This is actually something i can see myself doing! Seems alot of fun!

I just need to find one of those nice free templates to use ;D

Any ideas of a good one?

Two sites worth checking out are Open Source Web Design and Open Designs. See what takes your fancy.

This is from a local mirror of /idgames/levels, from Dec. 2010:

474M doom/
4.3G doom2/

That's not very much, and it's actually the Ports subdirs which eat up most of the space:

151M doom/Ports/
2.9G doom2/Ports/

The next time I sync up, I'm just going to omit the Ports subdirs entirely. After all, I pretty much just play vanilla maps anymore, and I expect that the other stuff will continue to grow in size at a much faster rate.

I would like to mirror other portions of /igames also, like for example /igames/combos, but unfortunately the directory structure makes no distinction between the vanilla mods and those requiring an advanced port.

Btw, the web server doesn't like wget, so if you use that you have to change the user-agent string, for example:

wget -mULynx -t0 -np -X /pub/idgames/idstuff http://www.gamers.org/pub/idgames/

Yeah, I exclude the idstuff subdir, since all the non-classic Doom stuff in there takes up space...

I'd get all of it. In case the Internet explodes and the entire world is thrown intro destruction, as long as you have Doom WADs you should be fine.

Personally whenever I want to mirror idgames I just do:

wget -c -r --no-parent -t 10 --progress=dot:mega -N --limit-rate=16k -w 7 --random-wait --restrict-file-names=lowercase,nocontrol,ascii,windows --retry-connrefused -nH --max-redirect=5 --no-remove-listing -l 50 -m -S ftp://mancubus.net/pub/idgames/