32-Bit Map Format Proposal

[Quasar]

Fair enough -- the parser can always accept a language more general than what you're allowed to input. So even if Eternity's parser will allow multiline comments or single-quoted strings, you're still not allowed to use them if you want your map to be compatible with any port (which is the point of all this).

I see one other problem, however. libConfuse allows # to indicate single-line comments, so color values will mess up. Is it possible to use something else, like perhaps $ ? $ is a traditional indicator of Hex values so it makes sense. Other punctuation marks would be available as well.

[Graf Zahl]

The '#' was just a suggestion by Ajapted. There's no problem changing that. I'll use '$' instead.

[Manc]

With all of the discussion here, will this make obsolte the phsyical sidefs/sectors limits in maps?

[Graf Zahl]

No. For that you'd either need a new binary format or replace the standard format completely with this. I think the second option is something worth thinking about when we have support in the editors.

But honestly, the largest possible maps with the current format are already huge and before establishing a completely new map format it'd be necessary to have an unlimited node format (unless any port is willing to implement ZDoom's compressed node format of course.) The segs are the first resource that overflows long before any of the map limits are reached.

[andrewj]

Graf Zahl said:

What do you think?

Looking very good. A few thoughts:

- typo: only 'three' section types now (spec says 'four'). BTW it should allow unknown sections, but editors are not required to remember any section it doesn't recognise.
- the spec has '-' (minus) allowed in names, I think you meant '_' (underscore).
- for colors: allow $RRGGBB (where alpha defaults to FF). Also the examples have alpha of 00 which is totally transparent (if we follow GL convention). The spec needs mention that too.
- it says editors don't need to show fields they don't recognise. I don't agree with that, we need editors to allow unrecognised fields to be added and edited so the feature development in ports is not dependent on the editors.

Each port supporting this format must unpack compressed sidedefs so that each sidedef is guaranteed to be used only once in the internal data.

Makes sense (after thinking it through).

EDIT: may as well make '#' the line comment instead of '//'. Because this format is not meant to be edited directly by users, there is little need for comments though.

[smite-meister]

Quasar said:

libConfuse allows # to indicate single-line comments, so color values will mess up. Is it possible to use something else, like perhaps $ ? $ is a traditional indicator of Hex values so it makes sense. Other punctuation marks would be available as well.

You can always make the color values strings and parse them with a special validating callback in libConfuse, right? Like this:

color = "#65313422"

[smite-meister]

Ajapted said:

Truncating the name probably won't work, someone could have MYLEVEL1, MYLEVEL2, MYLEVEL3.

Maybe this: prefix the name with the ']' character, but if 8 letters then replace the first letter. For example: "]E1M1" "]MAP01" "]ITLEMAP".

Consider this: If someone wants to use the new extradata lumps, he has to modify his WAD anyway. Why can't we demand that while he's at it, the map lumpnames be modified also, for example to fit the ExMy/MAPxx formats. The editor could even ask the user for permission to do this automatically.

After the renaming / name validation the extradata could be stored in a more descriptively named lump, for example MAPxx#ED or something similar.
(The hash char is there just to make random name clashes more improbable.)
Slight problem: existing MAPINFO lump should be modified too.

[Graf Zahl]

[QUOTE]Ajapted said:
[B]
Looking very good. A few thoughts:

- the spec has '-' (minus) allowed in names, I think you meant '_' (underscore).

Of course I meant '_'. I must have missed the shift-key. ;)

- for colors: allow $RRGGBB (where alpha defaults to FF). Also the examples have alpha of 00 which is totally transparent (if we follow GL convention). The spec needs mention that too.

Done.

- it says editors don't need to show fields they don't recognise. I don't agree with that, we need editors to allow unrecognised fields to be added and edited so the feature development in ports is not dependent on the editors.

I really like some input from the editor developers. So far they haven't said a word and a lot depends on them. Anything that improves the format it welcome so this is certainly something to think about.

EDIT: may as well make '#' the line comment instead of '//'.

AFAIK '//' is more widely supported by existing parsing tools than '#' so it will stay.

Because this format is not meant to be edited directly by users, there is little need for comments though.

Agreed. But since someone wanted comments I put them in the spec.

[Graf Zahl]

smite-meister said:

Consider this: If someone wants to use the new extradata lumps, he has to modify his WAD anyway. Why can't we demand that while he's at it, the map lumpnames be modified also, for example to fit the ExMy/MAPxx formats. The editor could even ask the user for permission to do this automatically.

I'd agree - if ZDoom wouldn't use TITLEMAP as a special case. Aside from this I don't have any problems with limiting the level name to 5 chars. It not only would make things easier here - it also would make GL node handling easier.

[DarkknightXC]

Graf Zahl said:

What do you think?

Work for me, although I would prefer for colour number format to be start with a '0x' to represent the hex format used. Gets around the '#' comment problem.

[Graf Zahl]

Since apparently everybody has different thoughts about the color, how about making it an 6/8 character string,like

sector 1
{
color = "ffaa44ff"
fade = "227766"
}

That should protect it from the oddities of the various parsers in existence.

[smite-meister]

Graf Zahl said:

Since apparently everybody has different thoughts about the color, how about making it an 6/8 character string

Works for me. Of course you could also use a 3 or 4 element list of floats clamped between [0, 1] to denote RGBA values OpenGL style, but that's probably overkill.

[smite-meister]

Graf Zahl said:

I'd agree - if ZDoom wouldn't use TITLEMAP as a special case.

You can remedy that by introducing another special case. You could use for example the lump MAP00#ED or MAP-1#ED or something similar for TITLEMAP extradata.

[DaniJ]

The spec looks good to me.

I can't pledge I'd implement support for it in Doomsday atm for many reasons though...

Personally I would prefer colours specified as clamped floats ala OpenGL as smite mentions.

I wonder why did Randy choose TITLEMAP anyway? Seems a bit of a rash choice to me (as well as allowing none MAPXX EXMX map identifiers)...

IMO MAP00/E0M0 would have been a better choice.

[TheDarkArchon]

DaniJ said:

I wonder why did Randy choose TITLEMAP anyway?

Because it's similar to TITLEPIC, I suppose.

[Graf Zahl]

[QUOTE]DaniJ said:


Personally I would prefer colours specified as clamped floats ala OpenGL as smite mentions.


I would not for parsing reasons. After all it's only an internal representation of something that should be set in an editor.
But having this a simple hex string makes the parsing much simpler than having to deal with multiple parameters.

I wonder why did Randy choose TITLEMAP anyway? Seems a bit of a rash choice to me (as well as allowing none MAPXX EXMX map identifiers)...

TITLEMAP is a replacement for TITLEPIC so it should be obvious where the name came from.

Non MAPxx/ExMy names aren't such a big problem - as long as they aren't longer than 5 or 6 characters. For example I like to be able to do E1M10 on occasion.

ZDoom doesn't auto-generate map names ever (as it really should be) from level numbers so the ability to specify arbitrary names comes naturally from removing that.

[DaniJ]

TITLEMAP is a replacement for TITLEPIC so it should be obvious where the name came from.

Obviously. IMO it was a bad choice.

Non MAPxx/ExMy names aren't such a big problem - as long as they aren't longer than 5 or 6 characters. For example I like to be able to do E1M10 on occasion.

That is different to allowing ANY name to be used. AFAIC there is no need to use map identifiers that don't conform to either MAPXXX or EXXMXX/EXMXXX/EXXXMX.

What benefit is there in allowing ANY 6 character string as a map identifier?

[andrewj]

DaniJ said:

What benefit is there in allowing ANY 6 character string as a map identifier?

Imho: none. Users wanted the feature, and port authors added it (including myself), and it's too late to revert it.

[Graf Zahl]

Since the longer map names cannot be undone how about:


If the map name is 5 chars and shorter: extradata uses ?????#ED
For 6 chars: ??????#E
For 7 chars: ???????#
for 8 chars: #??????? (to deconflict with 7 char names and to account for a higher probability of differences in the last chars.)

As it is too late to undo longer than 5 char map names some solution has to be found and the perfect one for 8 char names doesn't exist (unfortunately.)
But there's nothing that should keep the spec from strongly discouraging the use of names longer than 6 chars.

[andrewj]

Make it simpler, e.g.:

5 chars: X#MAP12
6 chars: X#MAP123
7 chars: X#AP1234 (first char is lost)
8 chars: X#P12345 (first two chars are lost)

Admittedly that's worse for the 7/8 letter map names, but it's better for recognition by users. If user recognition didn't matter (though I think it does matter), then a single '#' without the X would suffice.

I agree that we should discourage non-standard map names from now on, and the spec can reflect that.

[DarkknightXC]

The major reason I suggested a hex number for the colour format is simple. You can parse it with the standard format specifier.

[Graf Zahl]

DarkknightXC said:

The major reason I suggested a hex number for the colour format is simple. You can parse it with the standard format specifier.

And that's why it will stay that way. And it is the way the color values are stored internally in most ports.

[DarkknightXC]

Ajapted said:

Make it simpler, e.g.:

5 chars: X#MAP12
6 chars: X#MAP123
7 chars: X#AP1234 (first char is lost)
8 chars: X#P12345 (first two chars are lost)

Admittedly that's worse for the 7/8 letter map names, but it's better for recognition by users. If user recognition didn't matter (though I think it does matter), then a single '#' without the X would suffice.

I agree that we should discourage non-standard map names from now on, and the spec can reflect that.

I think we've trying to solve the wrong problem here. Rehashing an extended name has the problem of handling where two or more names clash.

Perhaps we should have an alias mapname lump, so that all these custom names are remapping to a standard format. We shouldn't care so long as the name is unique in relation to other mapnames.

[Graf Zahl]

What's keeping us from using a naming method that is designed for 5 char names but make spacial cases for 6 and 7 char names that don't remove any chars from the name? Only for 8 char names some loss is unavoidable and making it perfectly clear that 8 char names are discouraged from being used (except for special map names like TITLEMAP.)

I really don't want to introduce some remapping lump here because such a thing would break the easy association between the extradata and the level itself.
We have to live with the fact that 8 char map names will cause problems but I see no point in developing more complex means to handle them unambiguously. So far I know of one WAD that uses an 8 char name aside from TITLEMAP. That's Simplicity which names its last map CREDITS1. These cases are so rare that a mapper had to willfully break the system for this issue to show.

[DarkknightXC]

Graf Zahl said:

I really don't want to introduce some remapping lump here because such a thing would break the easy association between the extradata and the level itself.

The extradata <-> map data association works if the mapname is a standard MAPXX format. The aliasing of non-standard names to the standard format means that the method of assocation is easy and unbroken. Its up to application to refer to the aliasing lump to get the proper standard mapname from the non-standard mapname.

I think its best to solve the non-standard mapname problem as part of the spec as a whole.

[Graf Zahl]

Why do we need aliasing by an external lump? Anything shorter than 7 chars can be handled without any problems as it is.

Introducing any intermediate lump is a concept I really don't like at all. I'd rather prohibit 7 and 8 char names completely with an explicit exception for TITLEMAP before doing that.

[Anders]

What is the problem with prefixing the map name with ], or replacing the first charactar in the map name with ] if the map name is 8 characters to get the name of the extradata lump as proposed earlier in this thread?

[Graf Zahl]

Anders said:

What is the problem with prefixing the map name with ], or replacing the first charactar in the map name with ] if the map name is 8 characters to get the name of the extradata lump as proposed earlier in this thread?

IMO nothing. But apparently some people prefer more complicated solutions. ;)

[Fredrik]

Anders said:

What is the problem with prefixing the map name with ], or replacing the first charactar in the map name with ] if the map name is 8 characters to get the name of the extradata lump as proposed earlier in this thread?

The only reason I can think of is that it's hideously ugly.

[Graf Zahl]

Fredrik said:

The only reason I can think of is that it's hideously ugly.

So what is prettier and works as well? Suggestions are welcome.