Demon
Register | User Profile | Member List | F.A.Q | Privacy Policy | New Blog | Search Forums | Forums Home
Doomworld Forums : Powered by vBulletin version 2.2.5 Doomworld Forums > Classic Doom > Source Ports > UDMF v1.0 RFC
Pages (6): « 1 2 3 [4] 5 6 »  
Author
All times are GMT. The time now is 13:52. Post New Thread    Post A Reply
Retox
Mini-Member


Posts: 73
Registered: 11-00


The spec looks good, but it's missing one thing. It doesn't specify whether key names are case sensitive or case insensitive. I assume it's case insensitive, but you should explicitly say this to avoid confusion later.

Old Post 05-18-08 14:07 #
Retox is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
CodeImp
Moderator


Posts: 1084
Registered: 12-03


I vote key names should all be lowercase. no exceptions.
That makes the identifier definition as follows:
code:
identifier := [a-z0-9_]+

__________________
Doom Builder 2 is coming!

Old Post 05-18-08 16:06 #
CodeImp is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


The spec is strictly case-insensitive. I forgot to mention this in the spec! strcasecmp should be used on everything. Literal string values should be read as they are, though, of course.

Old Post 05-18-08 18:15 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03



Graf Zahl said:
Ok, so far, so good. Is there any reason why you didn't add my other clarifications? I think they are necessary to avoid problems.

The Strife flags are unlikely to be supported by most Doom ports, I'd think, so they should be excluded from non-Strife games - at least in the default namespaces.

And the thing special remark is necessary because there are absolutely no semantics defined how thing specials are to be used with Doom format specials.
The spec contains nothing about this which I consider a serious problem that may easily cause incompatibilities.



Quasar, could you please say something about this? I think it is important.

Old Post 05-18-08 18:33 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


I thought it was pretty well implied that the default, or compatibility namespaces, are provided to allow for lossless port-independent conversion of vanilla game maps to and from UDMF format. Note the "from" part would imply that you could move back to binary, if you were compelled to do so. This would imply that the map already doesn't use UDMF features that do not apply to maps of that particular gamemode. So Strife flags should be false in a Doom namespace map. Thing specials should be zero in a Doom namespace map. This means they can default and there is no need to have them specified in the files for every object. What about this is not already obvious from the spec? If you can boil it down to a single point, I'll add a blurb. Otherwise it just seems to be restating the obvious.

Old Post 05-19-08 03:19 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03



Quasar said:
If you can boil it down to a single point, I'll add a blurb. Otherwise it just seems to be restating the obvious.



This is a specification. Therefore it needs to be precise - and should state the obvious to avoid gray areas. Besides, what may be obvious to you is not necessarily to others. For example the spec says [u]nothing[/b] about how thing specials are to be handled with Doom format specials, IOW it's wide open to interpretation.
Even I had to ask how you see it and my first implementation saw it differently and did implement them.
The Strife flags are minor by comparison, but does it really hurt that much to add 2x3 lines of text to clarify things? If you have to answer that question for yourself with 'yes', I'm sorry we will eventually run into compatibility issues with some implementations.

Old Post 05-19-08 07:40 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


Except we seem to be confusing issues relating to the compatibility namespaces with issues relating to individual ports' private namespaces.

Doom namespace should never have specials on things, because there's no mapthing_t::special field to convert to it in the first place.

In Eternity, Doom-style specials cannot be used on things. Use on things was always a feature exclusive to Hexen's parameterized thing specials. This is an implementation detail specific to Eternity. I see no reason to dictate this requirement onto other ports, however. If they want to go to the trouble of adapting Doom-style specials to run on things, that is their business alone, isn't it?

The purpose of the spec is not to unify the semantics of all fields across *private* namespaces. The inclusion of namespaces in the first place was meant to avoid such debates and arbitrary rules that would discourage implementation of the standard. It's not a Ten Commandments for Doom ports.

As far as the base UDMF standard is concerned, the only unified semantics are with respect to the standardized namespaces. I think maybe you're seeking, with this suggestion, to do something with the standard that it is not meant to do. Now I would certainly be interested in participating in a later process to define a higher level standard where we seek to get some compatibility between our individual private namespaces' custom fields. Such as if you implement a 1sonly SPAC flag, well, EE already has such a thing. So if we go to the trouble of making it work the same and use the same field name as part of a higher-level spec, call it "UDMF Advanced Standard" or something, then it would be good for both our ports, for CodeImp, and for editors in general.

Old Post 05-19-08 16:09 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03



Quasar said:
Except we seem to be confusing issues relating to the compatibility namespaces with issues relating to individual ports' private namespaces.



No, we don't. All I am missing is some statements of these things.



Doom namespace should never have specials on things, because there's no mapthing_t::special field to convert to it in the first place.



Agreed - but it must be mentioned in the spec! As it stands the spec would (or would not) allow it, depending on how you interpret it.




As far as the base UDMF standard is concerned, the only unified semantics are with respect to the standardized namespaces. I think maybe you're seeking, with this suggestion, to do something with the standard that it is not meant to do.



Absolutely not! All I am seeking is explicit clarification on two issues which, I think, may create problems because it is nowhere stated explicitly how they are to be treated.
The first rule of any spec should be: Never rely on implicit meaning! Someone will miss it if it is not clearly written out.

If you just copy the 3 sections from my version of the spec that are missing in your draft I'll be happy because they handle everything I am concerned about.


Regarding custom namespaces, I will support everything, including thing specials in ZDoom's Doom format namespace because the engine can do it so it should be obvious that nothing of this is of any concern to me regarding ZDoom extensions. I just don't want to see someone messing up the base namespace by accident.

Old Post 05-19-08 16:45 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
esselfortium
Senior Member


Posts: 1823
Registered: 01-02



Graf Zahl said:
Regarding custom namespaces, I will support everything, including thing specials in ZDoom's Doom format namespace because the engine can do it so it should be obvious that nothing of this is of any concern to me regarding ZDoom extensions. I just don't want to see someone messing up the base namespace by accident.

Wait, what? Let me get this straight: You want to completely ignore the fact that the base namespaces are intended to be the same on all ports (the basic stuff only, with no sourceport-specific additions), for no benefit other than confusing other ports that try to read those maps (which, for all intents and purposes, have no reason not to just use the ZDoom namespace).....and then you say you don't want someone else "messing up the base namespace"? Unless I'm misunderstanding something here, it sounds like you want to ignore part of the spec and then tell everyone else to follow it?

__________________
sl4.poned.com: Free Esselmusic
Esselmusic: The Album
Knee-Deep in KDiZD: Relive KDiZD, in doom2.exe: (WiP Page) (Doomworld Thread)

Old Post 05-19-08 17:10 #
esselfortium is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


Alright, I gotcha. ;)

I'll try to get the revised spec up later tonight.

Old Post 05-19-08 17:11 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00



esselfortium said:

Wait, what? Let me get this straight: You want to completely ignore the fact that the base namespaces are intended to be the same on all ports (the basic stuff only, with no sourceport-specific additions), for no benefit other than confusing other ports that try to read those maps (which, for all intents and purposes, have no reason not to just use the ZDoom namespace).....and then you say you don't want someone else "messing up the base namespace"? Unless I'm misunderstanding something here, it sounds like you want to ignore part of the spec and then tell everyone else to follow it?



What I gathered was that he means a namespace for Doom but ZDoom-specific. Kinda like "Doom in ZDoom" or something. IF a standard Doom namespace map contains a thing special and ZDoom interprets it relative to its private implementation, it's not the end of the world. It would just mean ZDoom's behavior for that standardized namespace is not 100% compliant (which would be bad only in that it would encourage people and tools to create 'namespace = "Doom"' maps that should actually be under another port-specific namespace).

But again, I don't think this is what he meant.

Old Post 05-19-08 17:15 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03


@esselfortium: You mean this?


Graf Zahl said:

Regarding custom namespaces, I will support everything, including thing specials in ZDoom's Doom format namespace because the engine can do it so it should be obvious that nothing of this is of any concern to me regarding ZDoom extensions. I just don't want to see someone messing up the base namespace by accident.




No, this is not about the base Doom namespace but the one I'll call 'ZDoomTranslated'. Since ZDoom has no problems whatsoever with thing specials I see no reason to exclude them here.

I want to keep the difference for the specs of ZDoom's 2 namespaces as small as possible so the only diffrence is that line, thing and sector specials are processed by the translator. Everything else will be the same.

Of course for the Doom namespace I want to adhere to something definitive which means that thing specials will be disabled.

Old Post 05-19-08 18:15 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
esselfortium
Senior Member


Posts: 1823
Registered: 01-02


Ah, okay. Sorry for the misunderstanding :)

__________________
sl4.poned.com: Free Esselmusic
Esselmusic: The Album
Knee-Deep in KDiZD: Relive KDiZD, in doom2.exe: (WiP Page) (Doomworld Thread)

Old Post 05-19-08 18:47 #
esselfortium is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


Spec updated.

I took your point to its logical conclusion, Graf, and also added points explaining that passuse is Doom-only, MBF friend is Doom-only, and the dormant/pclass flags are Hexen-only. Let me know if this meets with your approval.

Old Post 05-21-08 06:31 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03


Yes, that's ok. I didn't see these ones as critical though but having this as clearly defined as possible will certainly help avoid misunderstandings - and nothing prevents me from implementing them properly in ZDoom's namespace. :)

I wouldn't mind calling this spec final now.

Old Post 05-21-08 08:01 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
CodeImp
Moderator


Posts: 1084
Registered: 12-03


Looks good to me. I agree on making this final.

__________________
Doom Builder 2 is coming!

Old Post 05-21-08 09:33 #
CodeImp is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03


There's one thing I just remembered while writing a Hexen map format converter.

What about Line_SetIdentification? Shall this be supported or should we require that line IDs are to be set via its own property?

For the ZDoom namespace I will disallow such use in specials in general and any use of Line_SetIdentification or the ID defining arg of other specials will be disabled in UDMF maps but for Hexen this still needs to be clarified or we might end up with improperly converted maps that don't work everywhere.

Old Post 05-21-08 10:13 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
CodeImp
Moderator


Posts: 1084
Registered: 12-03


Can we just say that a map converter must remove any instance of Line_SetIdentification (special 121) and put whatever was in arg0 in the id field?

So a map converter would do this:
code:
special = 121 ----> special = 0; arg0 = 51 ----> arg0 = 0; (or not even include arg0) ----> id = 51;

__________________
Doom Builder 2 is coming!

Old Post 05-21-08 10:23 #
CodeImp is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03


That's precisely my idea.

Although a ZDoom aware converter will have to do a few more things but I will describe those in the spec for ZDoom extensions.

Old Post 05-21-08 11:08 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Retox
Mini-Member


Posts: 73
Registered: 11-00


Looks good to me too.

Old Post 05-21-08 17:54 #
Retox is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


I suppose as long as it is always clear to a Hexen binary->UDMF/UDMF->binary map converter, there's no problem with simply setting the line id instead of writing the Line_SetIdentification special.

However again I won't put in language forbidding such a special from existing in any namespace, because that's feature dictation, and it isn't really necessary. It's out of the way to set line id's this way in UDMF, but it's not harmful in any way that I can imagine.

So, the standard behavior for Hexen namespace will be to write the arg0 for the Line_SetIdentification as the line's id, and set the line's special to zero.

I'll add it this evening :)

Old Post 05-21-08 18:31 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03


Oh, just in case you are interested. I did some size tests.
I used KDiZD Z1M10 because it's a rather large map.

In binary format without nodes it is 1.7 MB uncompressed and 450kb zipped. In UDMF it is 8 MB uncompressed and 650kb zipped.

This average factor of approx. 1.5 is what I got for most maps I tested so for distribution it shouldn't be a size issue and HDs should be large enough by now so that even ports without Zip support don't need to worry.

Old Post 05-21-08 18:50 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
CodeImp
Moderator


Posts: 1084
Registered: 12-03



Quasar said:
UDMF->binary map converter
We should not think about that, this won't work without breaking the map and I don't think it has to work.

__________________
Doom Builder 2 is coming!

Old Post 05-21-08 21:53 #
CodeImp is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


So far there's no reason why UDMF maps in the standard namespaces could not be converted back to the binary map format. There is a one-to-one correspondence between all the data. I'm not saying that there will be high demand for a UDMF -> binary conversion tool, just that it is technically possible.

Graf: *gulp* 8 MB? Before nodes? EE would require +16 MB of RAM to process this map, since libConfuse caches the file, then copies data out of the file, and has overhead for each structure holding the copied-out data. 16 MB is not a big deal in modern memory terms, but it's a bit alarming since currently the entire game can run on a 16 MB heap O_O

I knew there would be overhead with text maps, but I didn't realize it could reach that kind of size. It compresses very well for storage on disk, but that won't help the memory footprint once it's loaded into RAM.

Looks like EE may have to increase the default heap size a bit (by around 300%) if we are to support maps of that size.

Last edited by Quasar on 05-21-08 at 22:28

Old Post 05-21-08 22:17 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
CodeImp
Moderator


Posts: 1084
Registered: 12-03


No, because with UDMF you can put a special on a line with an id which originally was impossible because the special was abused by Line_SetIdentification. A map which uses this UDMF feature cannot be converted to binary format without breaking its special. But it is far from important anyway, no reason to do such a thing.


Quasar said:
Looks like EE may have to increase the default heap size a bit (by around 300%) if we are to support maps of that size.

Or you write a decent parser that doesn't require so much memory. You CAN read UDMF sequential and store the data in memory as binary before you process it, so no need to allocate the whole file in memory.

__________________
Doom Builder 2 is coming!

Last edited by CodeImp on 05-21-08 at 22:36

Old Post 05-21-08 22:31 #
CodeImp is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Quasar
Moderator


Posts: 2203
Registered: 08-00


No, you cannot. Not in Hexen namespace. It is not allowed. A binary Hexen map cannot represent two specials for a line. If you edit the map, you must put it under some other namespace if you set another special on the line. This was the whole point of the compatibility namespaces.

If the purpose of eliminating Line_SetIdentification in the Hexen namespace was to try to allow effectively two line specials on one line, then I am going to decline to implement that change to the spec.

Yeah, I COULD write an entire second parser, even though the one I already have works perfectly for the format, which I designed to work perfectly with it, and even though having two parsers that do the same thing is gigantic code bloat.....

It would probably be better for me to write an IO buffer layer, so that libConfuse can continue to parse files from memory buffers, but not require the entire thing to be loaded at once.

Last edited by Quasar on 05-21-08 at 22:45

Old Post 05-21-08 22:38 #
Quasar is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03



Quasar said:
[B]So far there's no reason why UDMF maps in the standard namespaces could not be converted back to the binary map format. There is a one-to-one correspondence between all the data. I'm not saying that there will be high demand for a UDMF -> binary conversion tool, just that it is technically possible.

[quote]
Graf: *gulp* 8 MB? Before nodes? EE would require +16 MB of RAM to process this map, since libConfuse caches the file, then copies data out of the file, and has overhead for each structure holding the copied-out data. 16 MB is not a big deal in modern memory terms, but it's a bit alarming since currently the entire game can run on a 16 MB heap O_O



That's the largest map available. And if you want to play such a monster the normal zone heap probably won't do anymore unless it's considerably increased. Completely loaded into memory with textures precached and everything I doubt this will run well with anything less than 50MB. At least ZDoom reports an increase in memory usage of 30MB just by loading this level.

Oh, and before I forget: That's 8 MB with fully compressed sidedefs. With uncompressed sidedefs it's 9.2 MB. ;)

Old Post 05-21-08 22:49 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
CodeImp
Moderator


Posts: 1084
Registered: 12-03



Quasar said:
No, you cannot. Not in Hexen namespace. It is not allowed. A binary Hexen map cannot represent two specials for a line. If you edit the map, you must put it under some other namespace if you set another special on the line. This was the whole point of the compatibility namespaces.

If the purpose of eliminating Line_SetIdentification in the Hexen namespace was to try to allow effectively two line specials on one line, then I am going to decline to implement that change to the spec.


No that change stays, regardless if we can have an id AND a special in hexen (note: its not 2 specials). I chose my words carefully: Line_SetIdentification was an abuse of the special field and it is good that we fixed that properly, even if the special MUST be 0 when an id is given to the line.

But I just don't see why we have to limit the hexen namespace in UDMF with the same limits the binary map format has.

__________________
Doom Builder 2 is coming!

Old Post 05-21-08 22:54 #
CodeImp is offline Profile || Blog || PM || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Graf Zahl
Forum Legend


Posts: 4194
Registered: 01-03



Quasar said:
No, you cannot. Not in Hexen namespace. It is not allowed. A binary Hexen map cannot represent two specials for a line. If you edit the map, you must put it under some other namespace if you set another special on the line. This was the whole point of the compatibility namespaces.



What? When was exact 1:1 convertibility ever mentioned?

That's not how I saw it and it renders 90% of the line ID discussion we had obsolete. If that's all the compatibility namespaces are there for why did we even bother? I thought they were supposed to be minimal supersets of the original map features so that maps can be created that still restrict themselves to the game's features but without the map format related restrictions.

Now with your reasoning here we suddenly have *NO* common ground between Eternity and ZDoom anymore except for what has been there before with all its limitations. Now mappers which want to use UDMF's features are forced to choose a source port instead of mapping for some less advanced namespace that covers the common features of a larger group but makes full use of UDMF's features. Yuck!

Old Post 05-21-08 23:03 #
Graf Zahl is offline Profile || Blog || PM || Email || Homepage || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
Creaphis
I will deliberately take a contrary position just for the sake of writing incredibly long arguments


Posts: 1296
Registered: 10-05


Well, I'm not the developer of a Doom port or utility so that limits my sway here, but I agree with Graf. Backwards convertibility is not necessary even in theory, and will be impossible for maps that have exceeded the old map format's limits anyway. Also, the end users who will be making UDMF Hexen maps some day will see that restriction as pointless and aggravating ("Wait, WHY can't I put a special on this line?") and this could force map makers into considering only a single port-specific namespace or avoiding UDMF for its apparent lack of advantages.

Old Post 05-22-08 00:22 #
Creaphis is offline Profile || Blog || PM || Email || Search || Add Buddy Report to mod || IP || Edit/Delete || Quote
All times are GMT. The time now is 13:52. Post New Thread    Post A Reply
Pages (6): « 1 2 3 [4] 5 6 »  
Doomworld Forums : Powered by vBulletin version 2.2.5 Doomworld Forums > Classic Doom > Source Ports > UDMF v1.0 RFC

Show Printable Version | Email this Page | Subscribe to this Thread

 

Forum Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is OFF
vB code is ON
Smilies are OFF
[IMG] code is ON
 

< Contact Us - Doomworld >

Powered by: vBulletin Version 2.2.5
Copyright ©2000, 2001, Jelsoft Enterprises Limited.

Forums Directory