32-Bit Map Format Proposal

[Anders]

Fredrik said:

The only reason I can think of is that it's hideously ugly.

Well the only prettier solution I've seen mentioned is adding a map lump. But people seems to worry alot about outdated wad tools not liking this.

Restricting map names with special cases for zdoom seems like a very ugly solution in comparison with the ] solution.

[cycloid]

just read the thread in entirety, sheesh you guys got stuck up on some bizzaro issues, my thoughts.

from the new-draft-spec-in-progress
"block_floaters = true;"

make it so that it can be 0 or 1 (or even -1 for true but that's not as intuitive to the human but would be more so to the parser), filesize could be a lot smaller this way if they stack up. optional.





the proposed text format is a winner IMO, (i edit CSS by hand every day and it's pretty much the same):
.button
{
background-color: #FFFFFF;
color: #5F8EC1;
text-align: center;
font-weight: bold;
text-align: left;
padding: 6px;
}

one thing css does have is the shorthand properties so you can do

font-weight: bold;
font-size: 10pt;
font-family: arial;

or this:

font: 10pt bold arial;

though i dont know how that would apply in this situation.




i remember many hours spent editing quake and half life .map files by hand too but they got very bloaty with default values, until someone released a tool to strip out the default values that the editor had saved in (as the editor wasnt clued in itself as to what the default values were it just saved all of them!). similar but somewhat clunky:

{
"classname" "func_door"
"angle" "0"
"speed" "16"
"targetname" "t1"
"sounds" "1"
"wait" "16"
{
( 128 0 0 ) ( 128 1 0 ) ( 128 0 1 ) GROUND1_6 0 0 0 1.0 1.0
( 256 0 0 ) ( 256 0 1 ) ( 256 1 0 ) GROUND1_6 0 0 0 1.0 1.0
( 0 128 0 ) ( 0 128 1 ) ( 1 128 0 ) GROUND1_6 0 0 0 1.0 1.0
( 0 384 0 ) ( 1 384 0 ) ( 0 384 1 ) GROUND1_6 0 0 0 1.0 1.0
( 0 0 64 ) ( 1 0 64 ) ( 0 1 64 ) GROUND1_6 0 0 0 1.0 1.0
( 0 0 128 ) ( 0 1 128 ) ( 1 0 128 ) GROUND1_6 0 0 0 1.0 1.0
}
}

the last chunk being the coords of each face with texture and offsets

[smite-meister]

Fredrik said:

The only reason I can think of is that it's hideously ugly.

That :), and maybe collation order too (more natural to have the extra character at the end of the name).
But since the lumps will be handled by editors, all this is of little importance.

What is important is to have as few special cases as possible. Therefore I think it would be best to either use only one char (] or # or whatever) to denote the extradata lump so only 8-char names need special handling, or do the renaming I suggested and use MAP00 for TITLEMAP. After considering all the trouble the renaming would cause to the MAPINFO lump, I'm inclining to support the ] scheme after all.

[Anders]

cycloid said:

make it so that it can be 0 or 1

Imho it's good if the value type can be deduced from the given property=value pair. (in this case the editor can't know if it's a boolean or an integer) Since otherwise the editor would have to know about what data type the value for each property should be. And the one of the points of this is that all ports can add whatever properties they like.

[cycloid]

i get it so:

true/false = boolean
$ = color
"" = string literal
1234 = integer
12.3 = float

[Anders]

I've now read the latest draft and I have some thoughts.

Since it should be kept as simple as possible to parse it should be case sensitive. Case insensitivity can't just be solved by a call to tolower() in the ReadLine function in Graf Zahls example code since that would break strings.

It should also be kept as simple as possible for editing software to handle, thus comments should be out, or atleast editing software should be allowed to remove them. Forcing editing software to store comments in their internal structures and then outputting them at the right place in the record when writing the internal structures to the lump is just too complex.

I really agreed on the "Machine generated" approach suggested early in this thread.

[Fredrik]

Anders said:

Since it should be kept as simple as possible to parse it should be case sensitive. Case insensitivity can't just be solved by a call to tolower() in the ReadLine function in Graf Zahls example code since that would break strings.

Um, what's difficult about using tolower() on a token *after* you've extracted it?

[Anders]

Because you can't just blindly use tolower(), you'll have to check what type of data it is first. It'll be more complex, but maybe not enough to warrant case sensitivity now that I think of it. But I don't see why something that should be machine generated should be case insensitive. One of Graf Zahl's opening arguments for using a text based format was that one could be made that's super easy to parse, and I agree on this.

[cycloid]

i'm tending to agree, fix all tokens at lowercase unless they need to be two words for whatever reason, then use the c variableNamingSyntax.

i also agree about comments somewhat, i suggest you allow them to exist but dont require that wad editors preserve them.

however because different source ports may use different token/value pairs any port or editor should be required to preserve any that it does not understand (so opening it in an editor that doesnt, say, support a new zdoom feature wont kill that feature). so you can hack in any comments you like

thing 100
{
notetoself="this is a comment";
}

this brings up an issue with the use of the semi colon, what if we want to embed script in a token/value pair?

thing 100
{
cycdoom.onkill = "if( something global ){\ndo this;\ndo that;\n}";
}

i assume that the quotes will null any control characters within it? so

thing 666
{
obituary = "you died {squishily}";
}

should not break either.

[Anders]

cycloid said:

...so you can hack in any comments you like

thing 100
{
notetoself="this is a comment";
}

Yeah ofcourse, just like Fredrik suggested earlier. The only caveat here is that you cannot make comments on specific lines in a record in this manner since any editor may reorder the lines when saving. But this is really a minor issue.

If we really want comments we could put them after the optional ; after the value, that way an editor would just have to store anything between ; and '\n' with the property and put it back when saving. But this could cause problems for port authors hoping to reuse their textparsers (libConfuse, sc_man).

[cycloid]

agreed, with my hack you could only reliably add comments per item and not per line.

but allowing comments to be used (and preserved) after the ; and before the \n, IMO breaks the rule of simplicity that the token/value setup is trying to achieve as you are now parsing the line 3 ways.

so it may be worth predefining a "comment" token in the spec, just for completeness.

edit: oh yeah, i just went back and read fredriks post.

[DarkknightXC]

Graf Zahl said:

Why do we need aliasing by an external lump? Anything shorter than 7 chars can be handled without any problems as it is.

It is possible to use 8 characters at the present time and it seems something of a backward step to not support that in the new spec. Rehashing a non-standard mapname is open to error. I think the spec should not define something that is quite so open to error as this. Therefore I am providing a way of working around that.

Should we choose to restrict name length I think it preferable to restrict the format of the name also. Say MAPXX in all cases. Fudges - especially those in specs - rarely result in a good working solution.

I suppose one could argue that the limitation of the wad format is at fault here and we could write a new one..... ;)

[Graf Zahl]

That doesn't sound good at all. The WAD name restriction should be as minimal as possible and introducing yet another lump is ultimately counterproductive because there is no longer a direct association between extradata and level. But that association is needed if we don't want to put the entire burden on the mapper. If we did that we'd have already lost. I'd rather live with the 8-char problem and the limitations resulting from it. IMO it's the least evil of all the suggestions made here. After all, there are hardly any maps using 8 character map names.

[andrewj]

Go with the ']' method.

Having an extra lump for associations is making things too complicated.

[cycloid]

forgive my dumbness but

why is restricting the map name to 8 such a big deal when we can include the facility for the descriptive name as you would in dehacked? i.e. MAP01 = "entryway"



another question, if you have a multilevel wad and the whole thing is text format would you nest the declerations?

map "map01"
{
name = "entryway"

sector 1
{
cieling.height = 128
floor.height = 0
}

etc.
}

map "map02"
{
name = "underhalls"

etc.
}

this way you dont need 3 lumps to describe a map (the map lump itself, the etradata and the mapinfo).

but this is not backwards compatible unless it's compiled.

[Graf Zahl]

No. It is one extradata lump per map. Ideally the mapper should never bother with this lump himself. The editor is supposed to take care of that. The current spec doesn't deal with per-map global data. Since MAPINFO (and similar solutions) already exists and it is more practical to have one central place where to define the relationship between maps I don't think that should change.

[DarkknightXC]

Graf Zahl said:

That doesn't sound good at all. The WAD name restriction should be as minimal as possible and introducing yet another lump is ultimately counterproductive because there is no longer a direct association between extradata and level. But that association is needed if we don't want to put the entire burden on the mapper. If we did that we'd have already lost.

I'm now confused. Why does a mapper care about the naming of their map within a WAD? So long as the application - be it editor or engine - knows where to source the data then the mapper should not have reason to care.

I'm not fussed in how you its done - be it by mapname restriction or by remapping lump etc.. - but I would suggest that allowing for a known problem to exist in a spec, makes the spec bad.

Having thought about it, only new maps will support this use this format so a limitation on the length of mapname is a simple answer.

[DaniJ]

Or better still - if you want ExtraData then the map marker MUST conform to the standard identifier rules.

[Manc]

For a quick interjection from a non-programmer, I just want to say that keeping names restricted to a MAPXX is a really dumb move. The fact that most ports of interest can load map names other than this is a great thing. I can load multiple wads with different maps and be able to play them for example, instead of loading 2 maps both with MAPXX and only being able to access one of them. After reading all of this, I still don't quite understand why it's necessary to do anything to the mapname. It would almost seem like using map names OTHER than MAPXX would be better for any sort of newer map format.

[Graf Zahl]

Mancubus II said:

After reading all of this, I still don't quite understand why it's necessary to do anything to the mapname.

The only problem is that map names with 8 characters don't have any room to add another character to name the extradata lump. So one character has to be sacrificed. Judging from this discussion this appears to be unacceptable to some of the participants.

In reality it is no big deal. Such map names are exceedingly rare and clashes are even more unlikely.

[DarkknightXC]

Graf Zahl said:

The only problem is that map names with 8 characters don't have any room to add another character to name the extradata lump. So one character has to be sacrificed. Judging from this discussion this appears to be unacceptable to some of the participants.

In reality it is no big deal. Such map names are exceedingly rare and clashes are even more unlikely.

True. My issue is that the spec should not be vague on such matters. Either exclude the possibility of 8 character names or handle it without possiblity of error (IMHO).

[Graf Zahl]

That was my intention from the beginning. I was just waiting for an agreement.

The spec will define a behavior for 8 character names but at the same time state that these names can not be used unless they are some internal defaults (like TITLEMAP)

[DarkknightXC]

Graf Zahl said:

That was my intention from the beginning. I was just waiting for an agreement.

I Agree. Just not sure on what :).

[Graf Zahl]

Let's end this. I added this paragraph for naming the lump. After due consideration I think it is best to completely prohibit 8 character names and force source ports to adjust to this instead of developing crappy workarounds.

The lump containing this data will be called #mapname for any level called 'mapname' with a name that is shorter than 8 characters. 8 character names can not be used with this data.

Any source port using 8 character names for some special maps must rename these maps if it wants to allow this data for such a level.

[DarkknightXC]

Graf Zahl said:

Let's end this.

Anyone else image of some grey bearded bloke thumping the ground with a staff and the earth shaking. OK, just me then....

(Ok yeah, Mapname restriction works for me).

[cycloid]

handy how "mapname" is 7 chars...

[smite-meister]

Graf, where can the current ED draft be found?

[Graf Zahl]

Here you find it.

[andrewj]

It's apparently lacking a name, so I propose:

The Joint ExtraData Specification

[Julian]

I have an alternative to this extradata naming lump: reverse the map lump name, fill with # or _ or whatnot for lumps names < 8.

"E1M1"     becomes "####1M1E"
"TITLEMAP" becomes "PAMELTIT"
"FACTORY"  becomes "#YROTCAF"

Name collisions should end up being minimal. What do you think?