WadArchive

I'll just try to forget this world cup even happened

Not worth getting up for with most of the top players out for IPL/rest/injury

I try to forget that the 2019 world cup happened.....

Try reading the readme https://archive.org/download/wadarchive/README.MD

Have a read of README.MD, it will tell you the structure files. You have the hash already (E27E8B77D94E3A4C6FAE0F0310D36F881484C951) so won't take forever. It will be in DATA/e2.zip

You could just use the torrent provided by archive.org:

They are all there, have a look at the archive listing (the 'Show All' link). I have also upload a CSV that lists all the WAD SHA1 hashes and the filenames: https://archive.org/download/wadarchive/wads-list.csv

Yes it is dump from mongodb, have a look at README.MD. Yea lumps.json is a biggie which is why I uploaded it compressed.

I have attached a list of all the sites that are scraped. Some may not longer exist or provide WADs anymore If you are wanting a list from ID to map name then just parse wads.json. Being JSON it is easy to handle with javascript. sites.zip

I can generate a list of WADs and upload it when I get home from work. The SEO part was probably due to what my job was at the time, glad it was working haha. As for the list of sites that are scraped, I could list them here but not sure if I will include it in the internet archive. Most likely

I have decided to shut down Wad Archive as I don't have time to dedicate to it anymore and just not motivated. I don't have an exact date/time yet but it is currently no longer updated. Since I would rather not have the archive go to waste I have uploaded all the WADs, images and database data to the internet archive https://archive.org/details/wadarchive . The entire unpacked archive is around 1TB and is stored in a way that made it easier/faster for the server to use rather than for someone manually browsing the files. I won't be releasing the code for the website itself or the indexing code and I am not interested in allowing someone else to host it, feel free to create your own with the data. All the database files fit in the free tier at https://www.mongodb.com/atlas/database except for lumps.json.

I have roughy 84,000 wads on my site and that is almost at 1TB so good luck hosting 80,000,000 (if you can even find that many)

If only there was a contact link on the website https://www.wad-archive.com/contact . I used to have a 'Megawad' section (>=15 maps) but when redesigning everything I didn't add it in. I might do it some day in the future

The IWAD and source ports listed a generally guesses from different sources but mainly from @MTrop's library https://mtrop.github.io/DoomStruct/index.html (however the one I am using is probably out of date). It should be seen more as 'it should work on this but might work on source ports'. There is no way I am going over 90,000 WADs manually