Textual Map Format: Non-Democratic Discussion

All I have to say is I really hope everyone can come to an agreement on this ;) I'd certainly implement support for the map format in SLADE 3.

Quasar said:

1) In Eternity, the activation parameters, which are stored in the "extflags" field accessible via ExtraData, include the following flags:

* CROSS
* USE
* IMPACT
* PUSH
* PLAYER
* MONSTER
* MISSILE
* REPEAT
* 1SONLY

This allows a single line to be activated by any combination of thingtypes, methods, etc. I understand that ZDoom does not currently support this. How can it be handled?

Uhm, now I am confused. Aren't these the same as the ones I took from hexen in my proposal? just different names?

Quasar said:

2) Z height should remain relative, as it was in Hexen. Otherwise, changing a sector floor height will require updating all z-positioned things inside the sector. Either that, or two different z fields could be created, one for relative height and one for absolute coordinate.

No, if your engine internally works with relative Z heights, then you only have to take the absolute Z height from the map data and calculate the relative Z height for your entity at load time, once only. I think the value in the map format should be an absolute value, because X and Y are absolute as well.

Quasar said:

3) Hexen id's and Doom line tags have slightly different semantics. Tags default to 0, whereas id's default to -1 (lines without an id in Hexen cannot be addressed by a tag-style search at all, since they're never added to the line id lookup table). Should there be a separate "id" field for Hexen? This is how EE handles it internally, and even allows the id to be set explicitly via ExtraData, separately from the line tag.

Lets just define 0 as nothing and if your engine encounters zero, don't add it to the list or simply do if(tag == 0) tag = -1; ? Tags are everywhere and are everything, lets not make an exception for the linedefs here because Hexen wanted to be a racists :P And we really shouldn't be using two fields for this IMO.

Quasar said:

4) While I'm aware this is just the basic specification, here are the new fields that EE currently supports for sectors:

* damageamount (int, damage every leveltime & damagemask tics)
* damagemask (int, determines frequency of damage)
* damagemod (string or int, method of damage (ice, fire, etc))
* floorterrain (string, name of TerrainType)
* floorangle (float, floor flat angle in degrees)
* floorxoffset (float)
* flooryoffset (float)
* ceilingterrain (string, name of TerrainType)
* ceilingangle (float)
* ceilingxoffset (float)
* ceilingyoffset (float)
* colormap_upper (string, colormap lump name)
* colormap_middle (string, colormap lump name)
* colormap_lower (string, colormap lump name)

Sure why not, I don't see a problem here. Just one little detail: Would you agree on putting the x/y at the end of the field name and do this throught the map format as a naming convention rule (highest significant identifier on the left)? So "floorxoffset" would be "flooroffsetx". Oh and while I'm on to naming conventions: why the underscore after colormap?

Quasar said:

When we get into things like these, determining how and what DB will handle may get tricky. What sort of process will we use to work it out? While the standard that I specified allows for unknown keywords and fields, what recourse will we have if DB does not implement support for a given feature?

Is there perhaps a practical way for DB to support adding arbitrary information to entities? Such as a dialogue box that would pop up and have a text field into which the user could type a keyword/value pair? This would allow for adding fields that DB does not yet or will never support. I understand it adds more work to implementing this format, since things, lines, and sectors will be required to hold a list of string objects that describe their user-added fields. Seems fairly simple on the surface though.

Yes I think I will probably add a tab panel on each editing window for Things/Linedefs/Sectors that will just have a list of custom values to edit. Better yet, I'll probably make the list a sort of editor that recognizes names (from the Game Configuration) and gives the proper editing method (when the int field is supposed the be an action, it will allow you to choose from a list of actions) so that the Doom Builder interface won't need any changes for additional fields you might add later: just add the to the Game Configation as well and they are easier to edit!

How about a iwad identifier of sorts in the format? to distinguish which main game wad file it was intended to run with?

This may help clarify extreme similarities between one or two main wad files (ie plutonia or tnt or doom2)

Quasar said:
Suggestions:

1) In Eternity, the activation parameters, which are stored in the "extflags" field accessible via ExtraData, include the following flags:

* CROSS
* USE
* IMPACT
* PUSH
* PLAYER
* MONSTER
* MISSILE
* REPEAT
* 1SONLY

This allows a single line to be activated by any combination of thingtypes, methods, etc. I understand that ZDoom does not currently support this. How can it be handled?

That's precisely how I'd implement it, too. I don't like the limitations of ZDoom's current method and change it as soon as possible. Although ZDoom has one more activation type but that one is just a combination of two which are mentioned here already.

2) Z height should remain relative, as it was in Hexen. Otherwise, changing a sector floor height will require updating all z-positioned things inside the sector. Either that, or two different z fields could be created, one for relative height and one for absolute coordinate.

No need to change it. I see no need to have a second z-field. The only port which has implemented a few absolutely positioned things is Vavoom with its static lights but I see these more as an aberration than something that needs to be explicitly supported by another feature.

3) Hexen id's and Doom line tags have slightly different semantics. Tags default to 0, whereas id's default to -1 (lines without an id in Hexen cannot be addressed by a tag-style search at all, since they're never added to the line id lookup table). Should there be a separate "id" field for Hexen? This is how EE handles it internally, and even allows the id to be set explicitly via ExtraData, separately from the line tag.

That fully depends whether the format allows specifying doom format specials. If so it needs the tag parameter. I don't know Eternity's precise needs here but for ZDoom doesn't need to make the distinction. All Doom format specials are translated to Hexen format when the map is loaded so after that the tag as such is never needed anymore. This means it could take the place of the first of Hexen's line special args. Of course, in order to support Doom format maps it needs to set the id the same as the tag. But for the new format I'd do a strict separation between tags (which essentially are merely parameters for the line special) and id's.

4) While I'm aware this is just the basic specification, here are the new fields that EE currently supports for sectors:

* damageamount (int, damage every leveltime & damagemask tics)
* damagemask (int, determines frequency of damage)
* damagemod (string or int, method of damage (ice, fire, etc))
* floorterrain (string, name of TerrainType)
* floorangle (float, floor flat angle in degrees)
* floorxoffset (float)
* flooryoffset (float)
* ceilingterrain (string, name of TerrainType)
* ceilingangle (float)
* ceilingxoffset (float)
* ceilingyoffset (float)
* colormap_upper (string, colormap lump name)
* colormap_middle (string, colormap lump name)
* colormap_lower (string, colormap lump name)

ZDoom could support all of them - except 'ceilingterrain'. What's that being used for? For the 'damagemod' parameter, how would that work. I'm asking because ZDoom allows custom damage types so some minimal standard is required. I'm assuming that any damage type that is not known gets treated as 'no special damage type', right?

Another question: How to specify RGB colors? For ZDoom's colored lighting this is important. Just a string in "RR GG BB" format or better a hexadecimal integer 0xRRGGBB? I think this is something that needs to be cleared because color selection in an editor is logically best done with a special color selection dialog.

I know ZDoom also has sector specials that can be set via scripting and activate on various actions, such as an actor entering the sector or hitting the sector floor. Seems like another natural field to add.

These are done with special things that are placed in the sector so it's not needed to add them to the sector's properties. The current method has the advantage that an unlimited amount of actions can be placed in the same sector with each one having its own trigger type and I don't see it making editing any harder if it stays that way.

When we get into things like these, determining how and what DB will handle may get tricky. What sort of process will we use to work it out? While the standard that I specified allows for unknown keywords and fields, what recourse will we have if DB does not implement support for a given feature?

Is there perhaps a practical way for DB to support adding arbitrary information to entities? Such as a dialogue box that would pop up and have a text field into which the user could type a keyword/value pair? This would allow for adding fields that DB does not yet or will never support. I understand it adds more work to implementing this format, since things, lines, and sectors will be required to hold a list of string objects that describe their user-added fields. Seems fairly simple on the surface though.

[/quote]Precisely what I think, too. The user should be able to add new fields at will to a simple property list where the fields are listed by name and the values can be edited. That way any port has the flexibility to add some extravagant stuff and it is immediately accessible to mappers.

Moriarti said:

In regards to flags, why represent them as a bitfield at all? Why not make each flag its own boolean field? This would simplify the map format so a reader wouldn't need to parse a flags string to extract each property out of it. Instead, it just sets a bit in its own internal flag(s) field(s) as it encounters a flag. Presumably any flag that doesn't appear as a field will default to off.

So the example from the opening post could be better written as:

thing 245 { easy = 1; normal = 1; hard = 1; ambush = 1; }

Yeah. Why not? ;) It's certainly easier to parse than a bitmask with 'or' operators.

Yagisan said:

In summary:
Eternity: For
GZDoom: For
Doomsday: Against
Deng-NG: Against

[/B]

You can add ZDoom as well to that list because I'd never implement it in GZDoom but not ZDoom. And if ZDoom leads, Vavoom will eventually follow. I think Legacy will, too. Which only leaves the Doomsday faction out. I intentionally don't count the more 'classic' ports because due to their very nature I think it's unlikely that they will participate. But still, with the question mark behind Legacy (and EDGE, of course) that would mean broad support by ports that specialize in advanced mapping features.

Ajapted said:

Comments can be useful for debugging (especially writers). Quake 'map' format is text based and allows comments.

Comments in the text format: No. Makes no sense at all.
A 'comment' property for each sector/line/thing: Yes, why not. But that doesn't require any change in specification. If the property list is implemented any mapper could use some unused property names for comments. Let's just say that no active property must start with the word 'comment' and you are already done. ;)

CodeImp said:
Uhm, now I am confused. Aren't these the same as the ones I took from hexen in my proposal? just different names?

No. Hexen uses a bit mask that defines 8 types. This is a bit field that uses one bit for each so they can be freely combined (like a line that can both be activated and shot or a line that can be crossed by a player or a projectile.

No, if your engine internally works with relative Z heights, then you only have to take the absolute Z height from the map data and calculate the relative Z height for your entity at load time, once only. I think the value in the map format should be an absolute value, because X and Y are absolute as well.

I disagree. Relative is easier to use for mapping because you don't have to think actively about z-positioning. 0 means 'on floor' or 'on ceiling' depending on the SPAWNCEILING flag. If you make z absolute the mapper has to actively think about the z field each and every time he adjusts something in the map.

Lets just define 0 as nothing and if your engine encounters zero, don't add it to the list or simply do if(tag == 0) tag = -1; ? Tags are everywhere and are everything, lets not make an exception for the linedefs here because Hexen wanted to be a racists :P And we really shouldn't be using two fields for this IMO.

As I said before, the tag is better doubled by the first arg rather than the ID. For any new map format it's better to think of the tag as a parameter to the line's special type, not an integral linedef property. That's the ID now. Hexen's developers were correct when they made this distinction. They only made the fatal mistake to forget an 'id' field in their linedef structure to save an irrelevant amount of disk space (approx. 50 kb to be precise.)

Sure why not, I don't see a problem here. Just one little detail: Would you agree on putting the x/y at the end of the field name and do this throught the map format as a naming convention rule (highest significant identifier on the left)? So "floorxoffset" would be "flooroffsetx". Oh and while I'm on to naming conventions: why the underscore after colormap?

Yeah, x and y should be put to the end. As for underscores, why not? In fact I'd have used some for ceiling_/floor_* properties as well. If the property names are accessible in a list to the mapper their names should be made a bit more readable. 'colormap_upper' is certainly more readable than 'colormapupper'

Yes I think I will probably add a tab panel on each editing window for Things/Linedefs/Sectors that will just have a list of custom values to edit. Better yet, I'll probably make the list a sort of editor that recognizes names (from the Game Configuration) and gives the proper editing method (when the int field is supposed the be an action, it will allow you to choose from a list of actions) so that the Doom Builder interface won't need any changes for additional fields you might add later: just add the to the Game Configation as well and they are easier to edit!

Sounds good. ;)

RTC_Marine said:

How about a iwad identifier of sorts in the format? to distinguish which main game wad file it was intended to run with?

This may help clarify extreme similarities between one or two main wad files (ie plutonia or tnt or doom2)

Might be useful. Then again, might not be. For KDiZD it would have been a big help but for other maps it may become a nuisance. I often single maps and build episodes out of them and in such a case coding the required IWAD into the map is not that smart. I think for things like this external control lumps like (E)MAPINFO are better suited. Let's keep non-map related things out of here.

Graf Zahl said:

Might be useful. Then again, might not be. For KDiZD it would have been a big help but for other maps it may become a nuisance. I often single maps and build episodes out of them and in such a case coding the required IWAD into the map is not that smart. I think for things like this external control lumps like (E)MAPINFO are better suited. Let's keep non-map related things out of here.

It wouldn't have to be a mandatory requirement, it'll be
useful if a situation ever arises when its needed.

And to state the bleeding obvious, not all modern ports
support MAPINFO either.

No, but all modern ports have some kind of equivalent. I wouldn't want to put information into the map data that's related to the game, not the map itself. This is precisely the things that would make a new format more complicated than necessary.

And for the record:

ZDoom (and derivates), Legacy (2.0) and Vavoom all support MAPINFO, Eternity has EMAPINFO and EDGE has DDF. And that's all ports that might support this new map format.

And Doomsday has its DED files even though, as has been made clear several times, Doomsday won't support the new map format - at least for now.

Ok, after a little discussion with Quasar on IRC we have come to a a few conclusions.

1) The Activation flags issue;
The problem with Quasar's flags was that it is not completely flexible. here is an example:

PLAYER | MONSTER | MOVE | BUMP

Is the player allowed to bump in here, or move over or both? And what about the monster? So I extended the flags I suggested earlier with which we can do:

PLAYERBUMP | PLAYERMOVE | MONSTERMOVE

See? And if your engine doesn't support a flag, you can ofcourse ignore it. And better yet, that flag won't be in my Game Configuration for your sourceport then :)

2) The Thing Z field
Agreed, lets keep this relative. Because it is relative and not absolute like X and Y are, it needs a better name, so we call it 'height'.

3) The tags issue
After a lengthy discussion, we think its best to adapt Hexen's way of working with tags. Thing and Sector tags default to 0, Linedef tags default to -1 (which essentially means 'no tag at all'). Linedefs DO have a seperate tag field (like Doom) which makes the Line Identification action obsolete. All linedef actions (and therefor also the ACS functions) still use the 5 arguments as input, NOT the tag field. The tag field is for referring TO the line only.

===================================================================
With the above conclusions, here is the revised format.
===================================================================

Moriarti suggested flags to be splitten up in seperate fields:
example...

thing 245 { easy = 1; normal = 1; hard = 1; ambush = 1; }

I agree as well.

Graf Zahl said:

Yeah, x and y should be put to the end. As for underscores, why not? In fact I'd have used some for ceiling_/floor_* properties as well. If the property names are accessible in a list to the mapper their names should be made a bit more readable. 'colormap_upper' is certainly more readable than 'colormapupper'

I agree.

EDIT:

===================================================================
And with these suggestions, the map format would become like this revised format.
===================================================================

CodeImp said:
1) The Activation flags issue;
The problem with Quasar's flags was that it is not completely flexible. here is an example:

PLAYER | MONSTER | MOVE | BUMP

Is the player allowed to bump in here, or move over or both? And what about the monster? So I extended the flags I suggested earlier with which we can do:

PLAYERBUMP | PLAYERMOVE | MONSTERMOVE

See? And if your engine doesn't support a flag, you can ofcourse ignore it. And better yet, that flag won't be in my Game Configuration for your sourceport then :)

I haven't seen that part but you are definitely correct about making it unambiguous.

One important issue: For compatibility with old maps PROJECTILEHIT must be activatable by hitscans crossing which can be unwanted. Maybe a compatibility activation mode that allows projectiles impacting and hitscans impacting and crossing is needed here? I don't like the idea but I fear it is needed if this is supposed completely backwards compatible with the original maps.

3) The tags issue
After a lengthy discussion, we think its best to adapt Hexen's way of working with tags. Thing and Sector tags default to 0, Linedef tags default to -1 (which essentially means 'no tag at all'). Linedefs DO have a seperate tag field (like Doom) which makes the Line Identification action obsolete. All linedef actions (and therefor also the ACS functions) still use the 5 arguments as input, NOT the tag field. The tag field is for referring TO the line only.

Great. That will cause the least problems.
So just to make the definition as solid as possible: The line's id/tag (whatever you'd like to call it) should *ALWAYS* be set by the 'tag' field and *NEVER* by any line special. ZDoom had to add the SetLineID functionality to several other special types like translucency etc. Since this isn't needed with a map format that has a separate field for it I think the definition should explicitly disallow setting the ID this way.

Graf Zahl said:

The line's id/tag (whatever you'd like to call it) should *ALWAYS* be set by the 'tag' field and *NEVER* by any line special.

Exactly.

I guess new wads built with this new map format will be port specific.

It depends. If they only use a feature subset used by multiple ports all of them would be able to use it. Of course the main motivation behind a new format is to make the port specific features easier to access and to remove some existing limits in the current format. Imagine maps twice as large as KDiZD. ;)

rpeter said:

I guess new wads built with this new map format will be port specific.

Maybe it should be clarified that the main motivation behind this is not to make all ports support all maps made with it, but to agree on a format so that editors can support it aswell. It wouldn't be hard for a port author to come up with his own format, but no editor author would support it if it was just for one sourceport (unless the editor in it self was port specific)

Graf Zahl said:

Imagine maps twice as large as KDiZD. ;)

.... maybe this isn't a good idea after all. :P

I'm more interested in making non-standard features easier to access. However, for KDiZD it would have been nice not to have the 65535 sidedef limit. Z1M10 had to suffer because it needed to stay below it.

Graf Zahl said:

But still, with the question mark behind Legacy (and EDGE, of course) that would mean broad support by ports that specialize in advanced mapping features.

As far as EDGE goes, I've pointed them to it on their forums, so there should be a response one way or another from them sometime in the next few days, probably in the form of "if someone else is willing to write the parser for it, we'll do it".

Graf Zahl said:

Imagine maps twice as large as KDiZD. ;)

Wait...do you mean maps twice as large as a single KDiZD map....or maps twice as large as the entirety of KDiZD? D:

This new format isn't going to limit itself to 5 arguments is it? Also, judging from the format provided by CodeImp, is there a reason to keep these arguments limited to just integers (though I suppose this won't be as big of a problem since it won't be limited to a range of 0-255)?

I have no plans to support this format in any of my projects (EDGE, glBSP or Oblige).

I am finished with glBSP and won't add any major features to it, only bug fixes.

Both EDGE and Oblige depend on glBSP, so that is a show-stopper in itself, but with EDGE I'm also rather tired of working on it, and probably wouldn't have the energy to add a big feature like this anyway.

So you guys blaze the trail. I'm just spectating this time :)

Blzut3 said:

This new format isn't going to limit itself to 5 arguments is it? Also, judging from the format provided by CodeImp, is there a reason to keep these arguments limited to just integers (though I suppose this won't be as big of a problem since it won't be limited to a range of 0-255)?

A big part of the reason for this text-based format is to have the ability to arbitrarily extend it as much as is needed. A limit of 5 arguments would not defeat the purpose entirely. :p

Blzut3 said:

This new format isn't going to limit itself to 5 arguments is it? Also, judging from the format provided by CodeImp, is there a reason to keep these arguments limited to just integers (though I suppose this won't be as big of a problem since it won't be limited to a range of 0-255)?

If you can add arbitrary values to any linedef/sector/thing/whatever the only limits are what the respective engine allows you to do.

In other words, if the engine allows you to script events yourself you should be able to add any value you need.
But the 5 arguments are just what is being used by all current implementations. None of the action specials used in ZDoom and other ports has any use for more of them. As for integers, that's the only format these values can be in at the moment. The 0-255 limit has to be lifted though. ZDoom can use any integer value here.

Ajapted said:

stuff

Which means that as far as EDGE support, it's down to one of two things - either I write a compiler for it into "normal" format (in which case all of the ports will end up indirectly supporting it, whether or not they like it, HA HA) or I write EDGE's parser and support it.

I'm open to persuasion as far as which is preferred from other people, including those who are annoyed by the idea or process in the first place.

Yeah there's no reason that only five arg values have to be accepted. Any above five are viewed as an extension, however, since the minimum needed for Hexen compatibility is five.

I'll be doing some more work on this stuff soon, btw.