Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
larsboy

Doom Map Database

Recommended Posts

I have been applying my tool to a complete dump of idgames (from March 1st 2020). I'd like to analyse every file in levels. It is a good test, and it would be cool to get the statistics. Has anyone generated extensive gameplay statistics for all of idgames before? Like total monsters, total hp, total linedefs, total secrets, etc.?

 

So far I have been able to process all of levels/doom, so all wads for doom1. I had to improve handling of errors and corner cases, so I've made a new release. Here are some key figures:

 

2656 wads* analysed, with 7029 maps (*wad here is technically the zip with everything in it)

 

Geometry statistics
                Total           Average        Min        Max
Linedefs    8698327     1237.5        4           41954
Sectors     1602870     228.0          1          11757
Things       2127468     302.7         0           8381
Secrets      22211        3.2             0           51
Area (MU)                    12.0           0.012    1057.069
Maps with computed area: 6933

 

Gameplay statistics (UV)
                    Total           Average        Min        Max
Enemies        808177       123.9           1        8188
Enemy HP     11869091    1820.1         2        835056
Ammo ratio                     5.58            0.00    5870.63
Health ratio                     0.60            0.00    148.00
Armor ratio                     0.77            0.00    953.85
Maps with gameplay stats: 6521

 

Also attached is a screenshot showing the breakdown by year. Most of the maps are from '94 (and obviously a few of the file dates are wrong!)

 

If I can complete the analysis of all level wads, I will upload the database somewhere.

years.png

Share this post


Link to post

This is an extremely cool resource. I think this will be very useful. Not just for informational uses like finding trends in mapping, but also for intelligent filters in launchers.

 

Interesting to see 2020 is so far setting to be a record breaking year across many metrics. We're in an era of excess, apparently.

 

Awesome work, looking forward to the Doom 2 stats.

Share this post


Link to post

I have some statistics on nearly 80,000 wads (not all have levels). I don't have area or HP but do have things, sidedefs etc..

Share this post


Link to post

Wow, quite interesting actually. Can you generate graphs like "linedef increase as years passed on", "average amount of enemies per WAD in a given year" and stuff like that? That might be an interesting addition to "The Evolution of Doom Mapping" by NotJabba, as well.. :p

Share this post


Link to post
14 hours ago, holaareola said:

This is an extremely cool resource. I think this will be very useful. Not just for informational uses like finding trends in mapping, but also for intelligent filters in launchers.

 

Interesting to see 2020 is so far setting to be a record breaking year across many metrics. We're in an era of excess, apparently.

 

Awesome work, looking forward to the Doom 2 stats.

Yes, I thought it was interesting that 2019 had the highest number of Doom1 maps since 1996. Note that some of these might be total conversions, so not really Doom1.

 

I also like to think of how long it would take to kill all those 808 thousand enemies, or find all 22211 secrets! And the Doom2 folder is almost 20 times as big, in MBs.

 

12 hours ago, elend said:

Wow, quite interesting actually. Can you generate graphs like "linedef increase as years passed on", "average amount of enemies per WAD in a given year" and stuff like that? That might be an interesting addition to "The Evolution of Doom Mapping" by NotJabba, as well.. :p

 

I can get a lot of statistics from the data. I haven't made views for everything, but more than is shown here. I don't have plans for creating my own graphs in the application, as there are other applications which can do that better, but I could add some export features.

 

The database I am building here is only what is extracted with the automatic analysis. I will do some cleanup later, such as for the obviously wrong dates. But the application is also very much for entering data manually, which is what I do in my own database for wads I have played (screenshot, rating, comments, authors, style tags, etc).

Share this post


Link to post
On 5/3/2020 at 4:36 AM, WadArchive said:

I have some statistics on nearly 80,000 wads (not all have levels). I don't have area or HP but do have things, sidedefs etc..

 

I checked out wad-archive.com, and it is very impressive.That is many times the content of idgames!

 

How to you generate the screenshots?

Share this post


Link to post

I created a script that runs in a virtual machine to fetch the wads and takes a screenshot. It doesn't work for all wads as some wads will depend on other wads or the engine/iwad guessing isn't 100%

Share this post


Link to post

This looks great! I can see the appeal for the more dedicated fans, and this sure is one great achievement.

Share this post


Link to post

Some joker made a wad with 100,000 maps, as well as a 30k one. I'll leave those out of the database, or else the majority of maps are little squares.

 

I've entered a lot of the Doom2 wads. The database so far has about 10,000 wads and 29,000 maps. I only do a little each night, and sometimes I have to improve the code to handle special cases, but it should be done in a few more days.

Share this post


Link to post

Yea I had to put in some exceptions for those WADs so that it didn't take screenshots and the HTML is partially pre-built to speed up the page load

Share this post


Link to post

This sounds amazing - I’ve wanted to have the ability to preview how much grief a WAD will give me :)

Share this post


Link to post

I have finished entering all wads from idgames/levels into a database. The database has a total of 12,768 wads and 42,355 maps (this includes Heretic, Hexen, Strife and Hacx). There has been a few zips DMDB couldn't process even with some manual help - where the content wasn't packaged as a wad or pk3, or where map data was in an exceptional form it couldn't handle. There's even two maps for Doom64 EX, with map data in a format I haven't seen before (Doom64, I presume). But I think this is a representative data set.

 

The screenshot shows some map statistics by year (a few wads have file dates outside this range). We see that 2019 was a very good year, with 1927 maps, the highest number after 1996! The low point was 2002, with 771 maps, after a gradual decline from 1996. The amount of content per map has of course increased a lot over the years as maps have gotten more detailed as well as bigger in terms of square area. From an average number of linedefs of 780 in 1994, its 3333 in 2019. Sectors likewise from 159 to 601, and THINGs from 291 to 683. Average square area (the unit here is 1024x1024 map units) has gone up from 10.9 to 28.6. Note also that the averages from 1995 and 96 are down from 1994, probably because a lot of the maps were for deathmatch.

 

I will add some more statistics and filters to DMDB, and make a new release of the application. Then I will also make the idgames database available for download.

years.png

Share this post


Link to post

Good job! Although I have to wonder if that much data affects the program's size. Must be quite huge, I guess.

Share this post


Link to post
1 minute ago, InDOOMnesia said:

Good job! Although I have to wonder if that much data affects the program's size. Must be quite huge, I guess.

 

Thanks! The application (DMDB.exe) and the database I have created with it are two separate things. But the core database - the wad and map tables - is actually very small, less than 5 MB in total. I have hand-coded the data model and storage to use as few bytes per entry as possible, and am not using any SQL engine. But it has generated a map drawing for each map, and these pngs are a few hundred MBs.

 

It was a clear goal when making DMDB to be able to always have the whole core database in memory, so that all scrolling, filtering and statistics generation is instantaneous. I like that I can scroll through the list of 42k maps freely, with no lag.

Share this post


Link to post

A new release of DMDB is now ready on github, and the idgames database is available.

 

DMDB version 1.03 now includes a version of the idgames database. Folder idgamesDB, just extract everything and select this folder when starting the application. Note that this version does not include the map images, to keep the file size reasonable for the application itself.

 

A version with map images for all 42355 map entries is available from my Google Drive, here:

https://drive.google.com/file/d/1Umo2YD62WRIGxOOFnvWg_3MIafeUe2V5/view?usp=sharing

 

This database allows you to browse all wads and maps in idgames/levels, with the gameplay stats, statistics etc. Keep in mind that this is the raw output of the wad analysis, without human input, so not all data is perfect. You can also use it as a starting point if you want to keep track of what you have played, with own comments, rating, etc.

 

The latest version of DMDB generates wad statistics, as in the attached screenshot. Here you can see the average maps per wad, which have climbed from 1.59 in 1994 to 8 in 2019. So these days there are fewer wads released, but with more maps in each. The average wad size has gone from 0.18 MB in 1994 to 11 MB in 2019.

years_wads.png

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×