Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Quasar

UDMF v1.0 RFC

Recommended Posts

http://doomworld.com/eternity/engine/stuff/udmf10.txt

Hopefully I caught everything. Changes from 0.99 include:

* Unquoted string support removed by CodeImp's request. Replaced with predefined keywords.
* Language added to clarify that UDMF defaults do not dictate editing defaults.
* Minimum width/precision of data types required specified.
* Suggested behavior of implementations on hard limitation overflow added.
* namespace required first, and required unique in file.
* Standard namespaces defined for vanilla executables.
* Support for MBF friend flag added due to inclusion of MBF in Doom namespace.
* Support for Strife, including currently unknown bits.
* Fields renamed for consistence (ie. floorpic -> texturefloor).
* Difficulty skill flags separated.
* comment field added to most entities.
* Vertex and thing coordinates changed to floats.
* Type of flag fields changed from string to bool.

If this spec is accepted, it will automatically become v1.0 final.

Share this post


Link to post

Very good. Read it all and I don't think I have any comments right now.

I like the "comment" fields, they can be very usefull to mappers. I can see Doom Builder 2 supporting them with a special color and a tooltip popup on mouse-over or something like that.

Share this post


Link to post

Additional info for Strife:

Line flag 1024 (0x400): Block floating monsters only
Thing flag 512 (0x200): Total invisibility

The other 2 bits being mentioned seem to be unused by the engine. I could never find a trace of them in the disassembled code.

Other:

The thing height value should also be a float. For ZDoom's slope things this would be very useful.

EDIT: Something I almost forgot:

In Hexen, activation type 'monstercross' means precisely that. But 'monsters can activate' in Boom means that such a linedef can be activated by anything except projectiles. And there are some maps which use the Boom specials with non-monsters, so I think there has to be an additonal activation type 'anycross'.

EDIT2:

For consistency the class bits should be

Class1, Class2 and Class3 instead of Class0, Class1 and Class2. Skills also start with 1.

EDIT3:

How are thing specials with Doom type specials to be handled? There is nothing that defines what should be the tag for them. Arg0?

Share this post


Link to post

These are the thoughts I had while playing with converting Doom maps to text format.

Basically I think in a few places you're forcing unnecessary specification of variables because their defaults aren't the same as the most common values encountered in wads. I think if you change the defaults you'll have less information to parse, which is faster and uses less space.

For example: currently you have to have most of the 8 skill and game mode flags set for most things, which seems like a huge waste of space. Most things appear in nearly all skills and game modes so I think it makes more sense to default all of said flags to true.

With linedefs, the twosided flag seems very redundant. As this is a new map format why bother with the flag? Just use the presence of a left sidedef to set or unset the flag when you load the map into the engine. Is it just there so you can convert old maps to the new format and still have demo compatibility?

Again with linedefs, you're going to get id = 0 a lot, as - if I am reading it right - id is apparently the tag number, but it defaults to -1. (If I remember rightly this is because of Hexen?)

Finally, despite this being a text format it's still unsuitable for putting in version control because you still have the renumbering problem. You're directly referring to offsets into an array so if I, say, delete sector 0 it will change every sidedef. What happened to giving each object a unique static id and referring to it by that? Maybe that's for version two...

Feel free to ignore, accuse me of bikeshedding, etc.

Share this post


Link to post
RjY said:

For example: currently you have to have most of the 8 skill and game mode flags set for most things, which seems like a huge waste of space. Most things appear in nearly all skills and game modes so I think it makes more sense to default all of said flags to true.


No. File size is absolutely secondary here so let's keep the defaults as the most easy to handle. All flags should default to false.

With linedefs, the twosided flag seems very redundant. As this is a new map format why bother with the flag? Just use the presence of a left sidedef to set or unset the flag when you load the map into the engine. Is it just there so you can convert old maps to the new format and still have demo compatibility?


Absolutely NOT! Yes it is needed for compatibility. Two-sided lines without this flag handle very differently in the engine so it can't be omitted.

Again with linedefs, you're going to get id = 0 a lot, as - if I am reading it right - id is apparently the tag number, but it defaults to -1. (If I remember rightly this is because of Hexen?)


Default in Hexen is -1, yes. And this will definitely be a problem for Doom format specials which expect 0 to be the default. As I see it it's a bad idea to use the id for this double purpose, just because Doom never had a line ID. IMO a special's tag should be stored in the first arg as a separate parameter, then the problem would go away.

(And yes, I know precisely why they were combined so it's unlikely to change.)

Finally, despite this being a text format it's still unsuitable for putting in version control because you still have the renumbering problem. You're directly referring to offsets into an array so if I, say, delete sector 0 it will change every sidedef. What happened to giving each object a unique static id and referring to it by that? Maybe that's for version two...


Nice to have but it will make the format MUCH more complicated to handle. Right now it is just straightforward parsing and storing the data in an array. Once we start adding indirections it can no longer be addressed just by the index alone. You have to implement some search algorithm.

If we want to get this going we really need to avoid things that complicate matters and might keep port and editor (and node builder) developers from supporting it.

Share this post


Link to post

My only comment is that the lack of a freeform comment syntax (// or #) really sucks. I understand the reasoning for not having one (editors not preserving it), but it would still be nice to have one, even if it would mean that the comments would be lost if loaded with an editor.

I'm thinking, for example, from the perspective of someone who might want to generate scripted levels, with comments inserted for debugging purposes. There really might be any number of other reasons that you would want to put a comment into the file.

Also, might it be a good idea to include a header line at the start of files to denote the UDMF version, like what dehacked patches do, or XML files? Just in case you want to change the syntax something in the future and maintain backwards compatibility ...

Other than that, this looks really cool!

Share this post


Link to post

Strife flags info: Thanks Graf, will amend the standard.

Thing height as float: Missed this, should definitely be float, for consistency if for no other reason.

SPAC flags with Boom specials: I thought we agreed that the activation info for DOOM-style line specials (including BOOM generalized specials) should be implicitly defined - IE, line type 1 is "Dr Door Open Wait Close (monsters can activate)" regardless of what the SPAC flags say. Having it any other way makes the namespace standardization of considerably less value. Consider that an automated map converter would then need to understand the semantics behind every special in the Doom, Heretic, and Strife namespaces in order to properly set the SPAC flags, rather than being able to simply leave them zero and have the special work as normal.

In your ZDoom namespace you only deal with parameterized line specials, so it shouldn't be a problem for you. When converting Doom maps you already never had that information. Not having to deal with it now means you can use the same code you use now to load Doom maps when dealing with UDMF namespace Doom. Introducing SPAC into that formula will complicate things for you. If people want to use the SPAC data, shouldn't they be using either Hexen or ZDoom namespace? This is the way that I look at it. It may not be ideal but it does have some advantages.

Class bits: Not a big deal to me what number they start at, I was just thinking in programmer-ese when starting them at 0. I can change this.

Doom/Boom specials on things: See the above point about using SPAC with Doom specials. I think the same thing applies here. It makes no sense to try to use them on things because Doom has no mechanism for dispatching its line specials from mobj_t's. Its system is hardwired to work only from linedefs (and employing hacks to accomplish it, like Lee Killough's LineSpecial codepointer, can lead to some nasty side effects on the map).

Defaults: I think keeping flags off by default is still the best policy. I understand that it can be a pain to type these bits when creating a map by hand, but remember that we consider hand generation of maps a low priority. This format is primarily text due to the advantage this presents to universal support and infinite extensibility vs. the port lockdown and limitations inherent in the binary map format.

Tag in args[0] for Doom-style specials: I am not opposed to that at all. It makes a lot of sense actually because this is where Hexen specials store their tags. It would keep the whole tag vs. id issue from being so irritating. This barely requires any change to the specification.

Comments: I am in favor of them. CodeImp's cfg parser already handles them. I left them out only because it is impractical for most editor utilities to maintain them when rewriting the file. If CodeImp is not opposed, I will add support for at least single-line comments. Multiline comments, however, can be considered harmful due to their inability to nest. This would not override the inclusion of the comment field in blocks, however, since it CAN be maintained by editors, and as CodeImp notes, could actually be displayed in-editor as helpful info to the end user.

Share this post


Link to post

My thoughts after reading it all:

1) It is a good spec and I couldn't find anything major wrong with it.

2) The linedef "id" field is confusing. I'm assuming it's the tag for DOOM maps and ignored for HEXEN maps. At the very least, rename it to "tag", but I agree with Graf Zahl that using the arg0 field would be easier/better.

3) I suggest renaming the thing "tid" field to "tag" as well, for consistency.

4) As far as I could see, all booleans have a default of false and most integers have a default of zero. The spec would be simpler / less cluttered with a extra paragraph in section III like this:

Unless otherwise stated all booleans default to false and all integers default to zero.

5) thing "angle" field should specify degrees, and perhaps the range (0-360)

Share this post


Link to post
Ajapted said:

2) The linedef "id" field is confusing. I'm assuming it's the tag for DOOM maps and ignored for HEXEN maps. At the very least, rename it to "tag", but I agree with Graf Zahl that using the arg0 field would be easier/better.

3) I suggest renaming the thing "tid" field to "tag" as well, for consistency.

Line IDs/Thing IDs aren't the same as tags, though. The lack of a way to natively set a line ID in Hexen format is one of its major shortcomings.

Share this post


Link to post
Quasar said:

Comments: I am in favor of them. CodeImp's cfg parser already handles them. I left them out only because it is impractical for most editor utilities to maintain them when rewriting the file. If CodeImp is not opposed, I will add support for at least single-line comments. Multiline comments, however, can be considered harmful due to their inability to nest. This would not override the inclusion of the comment field in blocks, however, since it CAN be maintained by editors, and as CodeImp notes, could actually be displayed in-editor as helpful info to the end user.

My parser does know what real comments are, even multiline comments, although not nested. And it ignores them. So like fraggle said, as soon as you load it in the editor, they are gone.

Ajapted said:

4) As far as I could see, all booleans have a default of false and most integers have a default of zero. The spec would be simpler / less cluttered with a extra paragraph in section III like this:

Unless otherwise stated all booleans default to false and all integers default to zero.

And ofcourse:
all floats default to zero and all strings default to a zero-length string: ""

Share this post


Link to post
Quasar said:

Strife flags info: Thanks Graf, will amend the standard.

Thing height as float: Missed this, should definitely be float, for consistency if for no other reason.

SPAC flags with Boom specials: I thought we agreed that the activation info for DOOM-style line specials (including BOOM generalized specials) should be implicitly defined - IE, line type 1 is "Dr Door Open Wait Close (monsters can activate)" regardless of what the SPAC flags say. Having it any other way makes the namespace standardization of considerably less value. Consider that an automated map converter would then need to understand the semantics behind every special in the Doom, Heretic, and Strife namespaces in order to properly set the SPAC flags, rather than being able to simply leave them zero and have the special work as normal.


I know but I thought since the activation type is already needed implicitly it might as well be added to the spec. It's not a big deal because I already defined it for the ZDoom namespace.

Class bits: Not a big deal to me what number they start at, I was just thinking in programmer-ese when starting them at 0. I can change this.


ok.

Doom/Boom specials on things: See the above point about using SPAC with Doom specials. I think the same thing applies here. It makes no sense to try to use them on things because Doom has no mechanism for dispatching its line specials from mobj_t's. Its system is hardwired to work only from linedefs (and employing hacks to accomplish it, like Lee Killough's LineSpecial codepointer, can lead to some nasty side effects on the map).


Fair enough. But things like this need to be clearly stated in the spec (at least for the default namespaces) to avoid confusion when implementing it.

Tag in args[0] for Doom-style specials: I am not opposed to that at all. It makes a lot of sense actually because this is where Hexen specials store their tags. It would keep the whole tag vs. id issue from being so irritating. This barely requires any change to the specification.


Ok. The only potential problem would be on the implementation side of 'traditional' ports that don't have any Hexen features. I'll only address this in ZDoom when the spec is changed. Right now I wrote my test map converter to always write the line ID so that it doesn't rely on a default.

Comments: I am in favor of them. CodeImp's cfg parser already handles them. I left them out only because it is impractical for most editor utilities to maintain them when rewriting the file.


There lies the big problem. All this information needs to be maintained by every tool modifying such levels. And to be honest, making it a requirement to preserve them will certainly not help in generating tools for the map format. CodeImp is not the only person to consider here - even though he is the most important one.

And considering that this is primarily a machine generated format, what's the point anyway?

Share this post


Link to post

You misunderstand. The spec will specify that editors do *not* have to maintain such syntactic comments, because it is highly impractical to do so. Syntactic comments would only be useful for the unlikely hand-created maps and for debugging of generators. As well as the more likely possibility of by-hand tweaks to otherwise finished UDMF maps.

For comments that ARE saved by editors, there are the new comment fields in the various blocks. This feature was suggested on IRC by esselfortium and I thought it a particularly brilliant solution to the issue.

Also, on the issue of tag in args[0] for ports that don't support Hexen: it shouldn't be an issue. arg0 as a field name can simply be mapped to the native tag field of the mapline_t structure in such ports :) arg0 is just a name, it doesn't have to necessarily mean that the data ends up in the Hexen arg0 field :) The use of named fields allows a layer of abstraction that can be exploited in cases like these.

Share this post


Link to post
Quasar said:

You misunderstand. The spec will specify that editors do *not* have to maintain such syntactic comments, because it is highly impractical to do so. Syntactic comments would only be useful for the unlikely hand-created maps and for debugging of generators. As well as the more likely possibility of by-hand tweaks to otherwise finished UDMF maps.


Ok, then all is fine. :) My parser can handle such comments but I wouldn't ever want to bother with actually parsing and maintaining them.

Also, on the issue of tag in args[0] for ports that don't support Hexen: it shouldn't be an issue. arg0 as a field name can simply be mapped to the native tag field of the mapline_t structure in such ports :) arg0 is just a name, it doesn't have to necessarily mean that the data ends up in the Hexen arg0 field :) The use of named fields allows a layer of abstraction that can be exploited in cases like these.


The issue is a bit tricky so it should be clarified.
Boom uses the tag as both ID and tag parameter for the line's special. As a result this either has to be stated explicitly in the spec that this is the expected behavior or some semantics have to be defined how both tag and ID can be used separately in Doom format maps, if this is desired. I don't really think it's needed so if we say that the ID field is completely ignored with Doom format specials I'd be ok with that.

Share this post


Link to post

I'm not really happy with the whole tag/id issue. I don't know what the best solution is. Tag and id are really the same thing, excepting the -1 or 0 default issue. The only reason which I initially implemented them as separate numbers in EE is because lines really *do* need some kind of identifier that can be detached from what sector they're trying to refer to. For ExtraData, we needed to accomplish this in a manner which didn't change the way Doom specials work.

But I see plenty of problems with this, and we've already started to face them in EE. Script functions for Small end up needing to be able to address lines via either tag or id. Users get confused because there are now effectively two tags, and wonder why setting one vs setting the other doesn't result in what they expected. I don't like it, but I don't know the best solution either.

I think we need to step back and give this particular problem a whole lot of thought.

I hate to dictate any port's implementation details in UDMF, but I wonder if "correcting" the mistake of lines referring to other lines via the tag field in BOOM should not be undertaken, by having such lines use the id instead like you seem to be suggesting. This wouldn't be as arduous as trying to use SPAC for DOOM/BOOM specials, since the number of line specials that refer to other lines with their tag is fairly limited (only one I can even think of right now is the Translucency control special).

This would establish a precedence that a line's tag refers to sectors it wants to affect. Whereas the id is strictly just that: a not necessarily unique line identification that has nothing to do with sectors. I could then patch this stuff up in EE when loading maps, and hopefully eliminate the entire concept of referring to lines by tag.

Share this post


Link to post

I don't care that much as long as it is properly defined in the spec. For ZDoom there's two possibilities:

1. Hexen format specials: The ID must default to -1.
2. Doom format specials: The ID must default to 0 and is also used as tag parameter for any line special.

Combinations of both in the same map are not possible so all I need is a decision which value to use where.

Share this post


Link to post

OK. There are only three concrete possibilities which I can devise. Which of the following seems the best in your opinion?

namespace = "Doom";

// option A
linedef
{
   special = 1; // DR Door Open-Wait-Close (monsters can activate)
   id = 0;      // id is tag for Doom namespace maps
}

// option B
linedef
{
   special = 1; // same
   arg0 = 0;    // arg0 is tag for Doom namespace maps
                // (line id is free, not used in namespace Doom)
}

// option C
linedef
{
   special = 1; // Ditto
   tag = 0;     // tag is a separate field like in the original
                // idea for the specification.
                // id is not used in namespace Doom.
}
In Hexen (or a Hexen-derived) namespace, nothing is different from usual. Tags for specials are in args[0] and line ids are set by the id field.

In any of the above situations it doesn't matter what the default value is. If id default is -1, however, Doom namespace maps will have to explicitly specify the tag on every line in the map. This means using the id field is automatically less than ideal.

When ZDoom loads a Doom-format map, what precisely does it do with the line tag? Does it turn it into a line id? Does it write it into args[0] unconditionally for all specials? Or does it have special cases for individual specials? I need to understand this in order to see where you are coming from.

Share this post


Link to post

Excuse my ignorance, but there is no "id" or equivalent field in the normal Hexen linedef structure, so why has it been added to this UDMF spec?

Seems like adding it has caused more problems than benefits.

Share this post


Link to post
Quasar said:

Which of the following seems the best in your opinion?

Certainly NOT option B, because args are used to refer to OTHER elements. At least thats how I feel about it.

Ajapted said:

Excuse my ignorance, but there is no "id" or equivalent field in the normal Hexen linedef structure, so why has it been added to this UDMF spec?

Not having to use the special 121 (SetLineIdentification). Now it is done properly like Doom's sectors tags (but for linedefs).

Share this post


Link to post

As far as I see it, line id should always be used to refer to lines (i.e. for a trigger or script to tell line id 5 to change its texture and become nonblocking), and tag should always be used to refer to sectors (i.e. for a trigger to tell sector tag 5 to open as a door). Treating this differently in different gamemodes is just going to make something that should be simple into a confusing mess.

Share this post


Link to post

Option B was suggested because in Hexen args[0] is more or less always a tag. I could probably count on one hand the specials for which args[0] is not a tag ;) Still, I'm not really defending it. It's a wonky option IMO.

Share this post


Link to post
esselfortium said:

As far as I see it, line id should always be used to refer to lines (i.e. for a trigger or script to tell line id 5 to change its texture and become nonblocking), and tag should always be used to refer to sectors (i.e. for a trigger to tell sector tag 5 to open as a door). Treating this differently in different gamemodes is just going to make something that should be simple into a confusing mess.



In general you are correct and ZDoom would have no problem whatsoever with a strict separation of both values as all Boom specials affecting lines are implemented to handle this properly. It's just in the translation step that the tag value gets copied to both so a separation of the 2 fields in the map wouldn't involve more than creating 2 entries in the linedef structure of the map from the tag value when saving as UDMF. Of course this would mean that no ID default could be used and the size of the map would be larger but does anyone really care?

If Quasar really implemented this differently in Eternity, well, that was probably not the smartest idea. ;)


@Ajapted: The lack of id field for linedefs has always been the biggest shortcoming of Hexen's map format and since it is relatively easy to fix it should be done. It's just the same as the separation of the skill filter flags for things: Removing a limitation in the original design that can be trivially implemented.

Share this post


Link to post
esselfortium said:

As far as I see it, line id should always be used to refer to lines (i.e. for a trigger or script to tell line id 5 to change its texture and become nonblocking), and tag should always be used to refer to sectors (i.e. for a trigger to tell sector tag 5 to open as a door). Treating this differently in different gamemodes is just going to make something that should be simple into a confusing mess.

What about things then? No, I think they all should be called "tag". Because you can tell from the function/special name which uses the tag if it is for things, sectors or lines. Floor_LowerToNearest, DamageThing, etc.

Share this post


Link to post
CodeImp said:

What about things then? No, I think they all should be called "tag". Because you can tell from the function/special name which uses the tag if it is for things, sectors or lines. Floor_LowerToNearest, DamageThing, etc.

What if you have several lines that you want to use as switches to lower lift tag #7, but you want to also have one of those lines change some of their properties from a script or another trigger? With only a tag field and no line id, you wouldn't be able to refer to those lines individually.

Share this post


Link to post

Yes we could make it infinitely complex and give everything 3 different tags to refer to lines and things and sectors seperately, but no, thank you.

Share this post


Link to post
esselfortium said:

What if you have several lines that you want to use as switches to lower lift tag #7, but you want to also have one of those lines change some of their properties from a script or another trigger? With only a tag field and no line id, you wouldn't be able to refer to those lines individually.


With Hexen map format this isn't a problem because the tag parameter is one of the args but not the ID.

All this confusion only comes from the fact that some people don't want to draw a clear boundary between both. And I repeat myself: If Quasar really did as he said and implemented the line ID separately from tags that are being referenced by Boom specials that affect lines not sectors, then this was very, very poor planning.

Share this post


Link to post

Wait... so what you guys (CodeImp/Graf) are saying is you don't want the ability to simultaneously reference groups of lines/sectors via a tag and reference individual lines/sectors via IDs?

Share this post


Link to post

I never saw the need for it. It's quite obvious that we have a serious compatibility issue here because ZDoom and Eternity are handling these values so differently that a reasonable compromise is not possible.

As I see it the ID is what the line is referenced with and the tag is what it uses to reference a group of sectors or lines (i.e. it's a parameter to the line's special, nothing more.)

That's how ZDoom implements it.

I know that Boom used the tag for both but that was due to map format limitations, not because it was a good idea.

ZDoom has to work around this by transferring the tag's value to both the line's ID field and the respective argument of the translated line special which, of course, for old maps doesn't cause a problem because the map format is fixed and not open to interpretation anymore.

If you need such constructs for Eternity, I'm sorry to say that they have no place in the general spec.

There won't be a perfect solution because what Eternity seems to require is not compatible with what ZDoom needs. From what I understand you treat both tag and ID as valid means to address groups of lines but ZDoom does not.

So, the only working solution seems to be to restrict the spec to the lowest common denominator which is one tag value and forbidding the use of the ID field in all namespaces that are affected.

Share this post


Link to post

Also keep in mind that the tag a special is referring TO is set in the arguments, which are already there. The line tag/id is not used for that. Line tag/id and sector tag are always only used for identification, they do NOT serve to refer to another element. So you can mark a bunch of sectors of an elevator with tag 7 and some lines on that elevator with id 2 and when you have a special line on the elevator is can specify both individually because the tag/id it is referring TO is written in the arguments and is NOT taken from the special line's own tag/id. I don't see the problem at all. We are not adding more limitations here, we are just making the tag/id an actual property of the line and not a special of the line so that the special of the line can be used for what it is meant for.

Share this post


Link to post

That's precisely how I'd define the spec.

However, from the discussion I gathered that Eternity is using the tag for both although it also implements the line ID. Well, obviously this is a big problem that will inevitably clash with a proper definition - if it needs to be addressed in the base spec and not in an Eternity extension of it.

Share this post


Link to post

ONLY the original Doom map format uses the tag on a linedef to refer TO a sector (1). Because Doom had no specials referring to linedefs, that wasn't a problem. But for Hexen this became a problem and Hexen fixed this problem with it's arguments for the special, but introduced another problem with the SetLineIdentification special (2) and we're solving that problem now.

This is not the Doom map format and doesn't suffer from problem 1, though, we need to fix problem 2 as decently as possible. I say we use option A quasar suggested, but I also suggest calling it "tag" because I really so no reason to call it "id" just because it's default value differs depending on the namespace.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×