Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
7hm

Wad / Map Database

Recommended Posts

I'm not entirely sure if this should go here, or if it should go into General, but the tagging thread I refer to below was here, so here it goes. I'm kind of looking just to get other people's thoughts on how I should structure a database I'm building and further suggestions about what I should add / remove.

Here's the concept:

We launch wads, but we play maps; and yet when you want to search for a particular map (using IDGames, or DSDA, or just google), you generally need to know what wad it's in. It's surprisingly hard to search for an individual map. This is particularly an issue when it comes to identifying map authors. I ran into this problem when I wanted to play through the collected works of certain map authors, to see how their style developed over time.

Even if I could use IDGames to find the wads that contained an author's maps but for which they were not credited as an author (and finding them through IDGames is not always a given), it's a real hassle going through text files to ascertain which particular map was theirs. Stuff will often appear where you don't expect it - for example Erik Alm made a bunch of speedmaps in the Nightmare speedmapping sessions.

On top of that, going through and determining what IWad the wad is designed for, what ports the wad is compatible with, what additional resource files are required, is a also an annoyance. It's even an annoyance of something like ZDL - even when I have a fancy launcher that does make my life easier than writing a batch file for every wad I want to play, I still have to go into the text file and check all of that stuff before running the wad. It's not the end of the world, but if I didn't have to do that, it would be a nice quality of life thing for me, and I assume for other people as well. I've currently got a batch program set up to do just that, being able to quickly launch an individual map with all that work automated is nice.

So that's part of the impetus for this, and it was why I did the whole batch program thing (https://www.doomworld.com/vb/doom-general/68764-map-specific-batch-loader). The other part though is the idea that Memfis had a while back about creating a tagging system (https://www.doomworld.com/vb/wads-mods/70087-wad-tagging-system). I listened to the critisim on Doom Radio, and in that thread, and I think it skewed a bit too negative.

Tags have a couple purposes that I think could really help people looking for certain types of maps or wads. Firstly, at a wad level, you could use them to identify PCs / TCs, Cacoward winners, stuff reviewed by kmx, stuff the DMWC thread had played, community projects, project with constraints, etc. Secondly, at a map level, there are "map types" that are pretty obvious: hell level, techbase, puzzle, adventure, short, gimmick, etc. And thirdly at an author level. I might be interested in what guys from TNT did outside of their TNT work, or want to see what some Russian map makers came up with that flies under the radar, or what the Italian doom community is doing. A tagging system that allows for multiple tags at each of these levels could help you narrow down something to play.

So what I'm starting to work on is a database to kind of tie this stuff together. As it stands now I've got a couple excel spreadsheets that I'm using to handle this stuff, but it's not exactly a solid foundation for a database that other people can eventually use.

The database design (real bare bones) is below, but I'm looking for feedback on what I might have missed. How can I make this better, is there something I'm missing, is there something I have that's superfluous?

Spoiler

I have an obsession with normalizing databases. I figure the amount of data here isn't enough for it to really cause a problem, but if need be I'll revisit.

Wad_Table
:: The Wad table contains information about the overall wad
- Wad_ID*
- Title
- Year (Date?)
- IWad [FK]
- IWad Indicator (is this map an IWad?)
- Note (long text)

IWad_Table
:: The IWad table contains a list of allowable IWads, as well as their file name
- IWad_Name*
- File

Wad-Port_Table
:: Wads can be designed for multiple ports when it comes to stuff like Legacy / Vavoom / ZDoom / Boom etc. This table contains the ports that the wad is designed for.
- WP_ID*
- Wad_ID [FK]
- Port_ID [FK]

Wad-File_Table
:: The Wad-File table contains the relationship between filenames (that are loaded by the game, not text files) and the wad itself.
- WF_ID*
- Wad_ID [FK]
- File
- File_Order (integer denoting order in which files should be loaded if there are multiple files)

Map_Table
:: The Map table contains information about the individual maps
- Map_ID*
- Title (or "untitled")
- Note

Wad-Map_Table
:: The same map can exist in multiple wads, including maps we would want to index (example - Heroes.wad contains Crossing Acheron). This table contains the relationship between wads and maps.
- WM_ID*
- Map_ID [FK]
- Wad_ID [FK]
- Slot (format E#M#, MAP##, MAPXXXX)

Author_Table
:: The Author table contains the author ID and the "main" name used by the author. I figure nobody has more than 3 nicknames, so I can leave it at that. I think normalizing further is a bit much
- Author_ID*
- Author Name
- Nickname 1
- Nickname 2
- Nickname 3

Map-Author_table
:: The map-author table contains the relationship between maps and their authors.
- MA_ID*
- Map_ID [FK]
- Author_ID [FK]

Series_Table
:: Some wads are released individually, but are part of larger series. This table contains possible series entries.
- Series_ID*
- Name
- Note

Map-Series_Table
:: Series are determined at the map level. This table contains the relationship between maps and series as well as the ordering.
- MS_ID*
- Map_ID [FK]
- Series_ID [FK]
- Series_Order

Tag_table
:: Tags can be assigned to Wads, Maps, Authors, or Series. This table contains the allowed tags.
- Tag_ID*
- Short Description
- Tag Type (Wad / Map / Author / Series)

Wad-Tag_Table
:: Wad-Tag contains the relationship between wads and tags, only allowing tags of the correct type.
- WT_ID*
- Wad_ID [FK]
- Tag_ID [FK]

Map-Tag_Table
:: Map-Tag contains the relationship between maps and tags, only allowing tags of the correct type.
- MT_ID*
- Map_ID [FK]
- Tag_ID [FK]

Author-Tag_Table
:: Author-Tag contains the relationship between authors and tags, only allowing tags of the correct type.
- AT_ID*
- Author_ID [FK]
- Tag_ID [FK]

.......

When I'm done the database should be searchable by:

Wad:
Wad Title (Wad_ID from Wad_table)
Port (Wad_ID from Wad-Port_Table)
IWad (Wad_ID from Wad_Table, filtered on IWad)
Year (Wad_ID from Wad_table)
Notes

Map:
Map Title (Map_ID from Map_table)
Map Slot (Map_ID from Wad-Map_table)
Series (Map_ID from Map-Series_Table)
Notes

Author:
Author (Author_table; searches nickname fields as well)

Tag:
Tag (Wad_ID from Wad-Tag_Table, Author_ID from Author-Tag_Table, Map_ID from Map-Tag_Table)

I know this kind of thing won't interest anyone, but I have this weird historian / collector itch and this sets it off for some reason. Again, I'd guess I'm not the only one, so the next question is: does anyone want to help with tagging the individual maps?

Right now my tracking has just shy of 1200 wads comprising 9000 maps made by 1000 authors, but I've only played a fraction of the maps, obviously. I know others have played more. I speak positively about tags, but only in two contexts: when you have enough anonymous users that the overwhelming opinion drowns out minor dissent / trolls, or when you restrict the ability to tag items to a smaller subset of users. I think for Doom we're looking at the latter option, and that's where I'd really like some help.

Initially I'm probably going to toss this into an access DB for download, but I'll try to figure out hosting and go back to using php and MySQL or whatever for the first time in about 15 years.

Down the road, I'd like to integrate the capabilities into a wad launcher, but that's pretty far down the road.

Share this post


Link to post

As someone who has wanted to play certain author's maps but not the rest of the WAD they're encompassed in from time to time, I fully support your efforts here. I haven't got much to add, but keep it up.

Share this post


Link to post

I like this project. I can help with tagging I think. Just tagging a few wads a day should be easy and not time-consuming at all. I would probably start with obscure maps that have just a few votes on /idgames, I've played a lot of these and it would be nice to be able to share my knowledge.

Share this post


Link to post

Cool, glad to see someone respond. These are my initial thoughts on tagging. I'm open to adding anything or removing anything from these lists.

Map Tags

Location / Feature Based:
I figure I'd add the location ones when a significant portion of the map takes place in one of the locations or uses one of the features. Maps could have multiples of these tags.

Spoiler

Hell
Techbase
Starbase
Castle
Spaceship
Outdoors
Graveyard
Sewers
Caverns
Crates
City
Church
Mountains
Oceans (what would be an example of this?)
Ruins
Egypt
Aztec
Jungle
Snow
Void
3D Bridge
My House
Real World (attempts to mimic the real world - trinity college for example)
Factory
Gothic (these next ones are "in the style of" and normally require resource files...)
Plutonia
TNT
Eternal
Icarus
Requiem
CC4Tex

Type of map:
These are pretty straightforward. Not all map 7's are dead simple remakes, but maybe I'd like to play through the ones that are. Or maybe I'd like to play some map 7s that aren't dead simple remakes.
Spoiler

Icon of Sin
Secret (maybe super secret too? I think that's not neccesary, and designating secret is enough)
Secret exit
Dead Simple
Boss
Size constraints
Line constraints
Sector constraints
Monster constraints


Type of gameplay:
These are much less straightforward. I think once again you would have maps that fit under multiple categories
Spoiler

Slaughtermap
Tyson
Starvation (maybe I should say ammo starvation? I mean those conservation-y maps)
Puzzle
Aventure / Exploration (should these be one tag, should they be two tags? they mean the same to me but others might disagree)
Maze
Non-linear (Sandbox?)
Linear
Dark (horror?)
Combat Scenario (I'm thinking cyberdreams / AV jump type maps... puzzle but revolving around monsters)
Hub-Based
Arena
Empty (or some other word, for maps with very few monsters)
Speedrun (maps designed for speedrunning - this is one of 40oz's tags, I'm not sure what it would apply to)

Other:
These are mainly warnings / flavour tags.
Spoiler

Mandatory secrets
Obscure progression
UV Only (for when no difficulty levels are implemented)
Cutscenes
Escort
Low detail
High detail
Height variation (for maps that use extreme height variation)
Speedmap
Death traps
Inescapable pits
Ambushes (for maps with a lot of ambushes)
Open
Death exit
Forced pistol start
Cramped
No pistol start (should this go at a wad level?)
Heretic (used for Doom maps that use heretic monsters)
Contest (with name of contest, probably, as in contest: xxx)
Project (with name of project, as in Secret Santa Project)
Remake (for maps that are a direct remake of another map but with some changes - for example the Doom to Doom 2 conversions that were common in the mid 90s)


Wad Tags

Developed by...:
For team efforts.
Spoiler

TIC
TNT
Wraith Corp
ACE Team
Rogue
Raven
iD
CP: NewDoom (CP stands for community project)
CP: ZDoom
CP: Doomworld
[B0S] Clan

Awards / Reviews:
For denoting awards, prominent reviewers, etc
Spoiler

DWMC (Doomworld megawad club)
Cacoward
Top 100
Cacoward Runner Up
Mordeth

Wad Status:
The status of the wad, whether or not it was a commercial release. I generally only gather info on demo / beta wads that are very unlikely to ever be finished.
Spoiler

Beta
Demo
Unfinished
Commercial

Authorial intent:
What was the intent of the author
Spoiler

Jokewad
Compilation
Doom E4 Remake
Doom E3 Remake
Doom E2 Remake
Doom E1 Remake (are there Doom 2 episode remakes? I can't really think of anything offhand)

Doominess
How close does this stick to vanilla doom
Spoiler

Partial Conversion (does this need to be here if we tag the weapons / sounds / monsters below? textures are done on a per-map basis)
Total Conversion
New Weapons
New Sounds
Unique Monsters
Realm667 Monsters

...

On the db front, I'm still getting it set up. I'll post the actual db for anyone who has access to look at once I at least convert all of the stuff I have sitting in excel. My "to sort" folder of maps worth playing is pretty damn huge too, but that'll go on the back burner for now.

Share this post


Link to post

When recording WADs you should also include their MD5 and SHA-1 checksum so that they can be located that way. This helps with WADs named the same but different versions, and completely different WADs. Otherwise you may have conflicting tags for the same named WAD.

Share this post


Link to post

That's a pretty good list already. Some comments...

"Oceans (what would be an example of this?)" - I guess Map01 of this, Vacation Doom, Map01 of Pirate Doom is on a ship I think, etc.

"Non-linear (Sandbox?)" - As I understand, these are different things. For example, Doom 2 Map19 is very non-linear, but I think it's not open enough to be a sandbox. A sandbox should be closer to Doom 2 Map16 where it's like a bunch of buildings placed in the same "box". The difference is a little blurry but it is there.

"Starvation (maybe I should say ammo starvation? I mean those conservation-y maps)" - sounds okay I think, another term could be Survival.

"Aventure / Exploration (should these be one tag, should they be two tags? they mean the same to me but others might disagree)" - I'm not sure what they even mean. I guess like a pretty large non-linear map where the emphasis is more on finding the way and less on the action?

"Speedrun (maps designed for speedrunning - this is one of 40oz's tags, I'm not sure what it would apply to)" - helltrak for example, it was made for a speedrunning competition and isn't of much interest outside of speed-oriented playing. Also Pagan's Run I suppose.

"No pistol start (should this go at a wad level?)" - should go at the wad itself, it means that the author only tested the wad by carrying weapons from the previous levels, like in Odyssey.

"Remake (for maps that are a direct remake of another map but with some changes - for example the Doom to Doom 2 conversions that were common in the mid 90s)" - hmm, to me a remake means that a map was remade from scratch, without copying anything from the original. What you're talking about probably should be called Edit or Remix or something like that.

Share this post


Link to post
7hm said:

These are my initial thoughts on tagging.

I find too many of these categories and particular tags unintuitive or questionably-logical and hard to search by, while their purpose is to be the exact opposite of that. Some of them seem superfluous too. I don't support the idea of tagging wads in the first place (mainly because of subjectivity of tag choices and impossibility to cover everything), but in response to your list, I propose my own one, trying to make it more logical and intuitive to use. Just the map-specific ones for now:

Disclaimer: * (this should be the first category to act like a quick warning for players)

Spoiler

Speedmap
Jokemap
Sadistic
Death traps
Inescapable pits
Cutscenes
Ammo starvation
Obscure progression
Mandatory secrets
Mandatory jumping
Mandatory crouching
Mandatory mouselook
Unplayable

Location:
Spoiler

Techbase
City
Hell
Outdoor
Indoor *
House **
Underground
Space
Ocean
Surreal
Real World ***

* "Indoor" means that majority of the map takes place inside a single building
** "House" means that the map takes place in a house/mansion/castle and possibly around it, for example in its garden
*** "Real World" means that the map is based on a real location

Thematic style:
Spoiler

Prehistoric
Ancient
Egypt
Aztec
Gothic
Eastern
Western
Contemporary
Futuristic
Idiosyncratic

Mapping style: * (only set this tag if the style is a DIRECT RIPOFF of another wad's or game's style)
Spoiler

Doom E1
Doom E2
Doom E3
Doom E4
Doom2 Base
Doom2 City
Doom2 Hell
Wolfenstein
TNT
Plutonia
Heretic
Hexen
Duke3D
Quake
Quake 2
Eternal Doom
<insert another inspirational wad or game>

Thematic elements:
Spoiler

Barrels
Crates
Nukage
Blood
Lava
Water
Summer
Winter
Christmas
Easter
Halloween
Wood
Metal
Brick
Sewer
Cave
Vegetation
Castle
Temple
Ruins
Church
Graveyard

Mapping gimmicks:
Spoiler

Fake 3D
Vanilla tricks
Size limit
Line limit
Sector limit
Monster limit
Time limit *

* "Time limit" as in limit for the player to do something within the map

Special map type:
Spoiler

Boss
Secret
Secret exit
Dead Simple
Icon of Sin

Gameplay:
Spoiler

Relaxing
Slaughter
Puzzle
Experimental *

* "Experimental" would also refer to wads with newbie-ish/broken/lazy balance or wads like WOW or NUTS

Progression style:
Spoiler

Linear
Semi-linear
Non-linear
Open world

Share this post


Link to post

I quite like most of your suggestions scifista, especially as it relates to sorting. I'd say that sorting of the tags is really something designed more just for ease of use - as far as the DB is concerned you can have eighteen different theme tags, just on the offchance that a map actually does that.

When recording WADs you should also include their MD5 and SHA-1 checksum so that they can be located that way. This helps with WADs named the same but different versions, and completely different WADs. Otherwise you may have conflicting tags for the same named WAD.


This is a good idea, but is a ton of extra work and doesn't really fit the goal close enough for that work to be needed. I don't plan on capturing multiple versions of the same wad, and different wads can be captured by their title / other details. There will be instances where two wads reference a .wad file of the same name, but so be it.

Anyway I've been working on getting the DB up to where my excel file is. If you have access, you can poke around in it. It's super duper basic, for now my goal was just to make it functional enough for me to use. I've uploaded it here. At the moment the db has the first few years of the Top 100 list and the main iD releases (including Heretic).

In terms of adding tags, for anyone who is interested in helping, I guess the question is - how? I see a couple possibilities:

1) Wait until everything is done, I port this back out of access and build a real frontend (it'll be a while)
2) Post wad / map details with tags in a thread, at which point I update the db with that info.
3) Do the same as step 2, but offsite somewhere (google docs?).

Share this post


Link to post

yes!!!!! this is what I've been waiting for a decade! would love to see this implemented.

Other ideas may include monster counts (so maybe an average per map):
This could be an integer or an enumerator

1-100 (small)
101-249 (medium)
250-499 (large)
500-999 (larger)
1000-2499 (huge)
2500-9999 (extreme)
10000+ (holy hell)

Share this post


Link to post

^I'd set the limits a little differently:

1-50: Small
51-250: Medium
251-600: Large
601-1500: Huge
1501+: Extreme

Share this post


Link to post
7hm said:

Type of map:
These are pretty straightforward. Not all map 7's are dead simple remakes, but maybe I'd like to play through the ones that are. Or maybe I'd like to play some map 7s that aren't dead simple remakes.

Spoiler

Icon of Sin
Secret (maybe super secret too? I think that's not neccesary, and designating secret is enough)
Secret exit
Dead Simple
Boss
Size constraints
Line constraints
Sector constraints
Monster constraints


Type of gameplay:
These are much less straightforward. I think once again you would have maps that fit under multiple categories
Spoiler

Slaughtermap
Tyson
Starvation (maybe I should say ammo starvation? I mean those conservation-y maps)
Puzzle
Aventure / Exploration (should these be one tag, should they be two tags? they mean the same to me but others might disagree)
Maze
Non-linear (Sandbox?)
Linear
Dark (horror?)
Combat Scenario (I'm thinking cyberdreams / AV jump type maps... puzzle but revolving around monsters)
Hub-Based
Arena
Empty (or some other word, for maps with very few monsters)
Speedrun (maps designed for speedrunning - this is one of 40oz's tags, I'm not sure what it would apply to)

"Type of maps" is redundant. Icon of Sin, Dead Simple, Boss map are gameplay types. Secret, secret exit, and the various rule constraints would better be placed in "other" (or in "mapping gimmicks" if you use Scifista's approach).

Share this post


Link to post
scifista42 said:

in response to your list, I propose my own one, trying to make it more logical and intuitive to use. Just the map-specific ones for now:

Disclaimer: * (this should be the first category to act like a quick warning for players)
Jokemap
Sadistic; Death traps; Inescapable pits
Cutscenes
Ammo starvation
Obscure progression; Mandatory secrets
Unplayable

Mapping gimmicks:
Time limit *

* "Time limit" as in limit for the player to do something within the map

Special map type:
Boss
Secret
Secret exit
Dead Simple
Icon of Sin

Gameplay:
Experimental *

* "Experimental" would also refer to wads with newbie-ish/broken/lazy balance or wads like WOW or NUTS


this all alienates anyone who makes non-standard maps from teh Plutonia/Scythe mainstream, and even if you don't care about that sort of social dynamic, it would still heavily spoil the content of every map in the DB. your definition of experimental is the worst of the bunch because no-one would want to use the tag for fear of having their work being construed as a broken wad. I'm sure there would be more than a few mappers who would simply stop submitting work under a system like this

Share this post


Link to post
7hm said:

1) Wait until everything is done, I port this back out of access and build a real frontend (it'll be a while)

Please not this option. I'm enthusiastic right now but who knows what will happen in a few months. Both this thread and Google Docs would be fine, just give me something to work with. :)

Share this post


Link to post
yakfak said:

this all alienates anyone who makes non-standard maps from teh Plutonia/Scythe mainstream, and even if you don't care about that sort of social dynamic, it would still heavily spoil the content of every map in the DB. your definition of experimental is the worst of the bunch because no-one would want to use the tag for fear of having their work being construed as a broken wad. I'm sure there would be more than a few mappers who would simply stop submitting work under a system like this

I don't think this really excludes anyone. I want to find maps with obscure progression, for example, because I think they are fun. Hell, I was playing Laura Beyer's Doom yesterday just to double check what the monsters looked like and holy shit that is a disaster, but I liked the weird progression part of it at least. It may be too subjective though.

I don't like using "jokemap" (are there real wads that include jokemaps? seems like a wad thing, and intentional) or "unplayable" (too subjective), but mandatory secrets, death traps, inescapable pits, survival gameplay are all things some people really like. Ghost monsters is another one like that, although a lot more rare.

Gez said:

"Type of maps" is redundant. Icon of Sin, Dead Simple, Boss map are gameplay types. Secret, secret exit, and the various rule constraints would better be placed in "other" (or in "mapping gimmicks" if you use Scifista's approach).


Yeah like I said I wasn't really organizing them for anything but to make that post a bit cleaner. I'll re-work it though keeping scifista's approach in mind.

As for monster numbers, I don't think I'll go through and do that manually. I'm sure that's something that can be automated given there are tools to strip that data from wads anyway. That's something for down the road. In general I do think it would be really useful to have though, and I definitely find myself looking at the wiki for that sometimes.

Memfis said:

Please not this option. I'm enthusiastic right now but who knows what will happen in a few months. Both this thread and Google Docs would be fine, just give me something to work with. :)


In that case, I guess we should just use this thread, I know some people had issues with google docs when Linguica used it for the voting thing in April.

If anyone wants to post tags for maps / wads here, I'll add them to the db and continue to release it as I update it. Anything is fair game to post, but I'll still working at adding content to it.

I'd suggest the following format:

Icarus - http://www.doomworld.com/idgames/themes/TeamTNT/icarus/icarus

Wad tags:TNT, story, DWMC, kmx review

Map tags:
Map 1: spaceship, obscure progression (just based on complaints in the dwmc thread)
Map 2:spaceship
...
Map 24:Ghost monsters, spaceship, puzzle

etc. If you want to skip maps, or just hit certain maps, or whatever, that's cool. I'll add what people provide.

I'm also going to be going through some of the threads / reviews to pull some of this stuff without necessarily playing every map myself.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×