printz Posted December 14, 2015 EDIT: the branch is now part of the main Eternity codebase, https://github.com/team-eternity/eternity, though it's not merged in any way. First post may be outdated; scroll down to my last post __________________ I really wanted UDMF in Eternity, so I decided to develop it. Now it's pretty much working but it needs fixing in some places. The GitHub fork is here: https://github.com/ioan-chera/eternity/tree/udmf I wonder if I should push this branch into the main Eternity repository and make it official. It's an old planned Eternity feature, even though this was started by me without notifying Quasar first. What is important to do: define the "eternity" namespace. Here's where: Decide what lumps need to be placed between TEXTMAP and ENDMAP. The current ad-hoc implementation basically requires NODES, SEGS, SSECTORS, REJECT, BLOCKMAP and optionally BEHAVIOR to be, in any order, between TEXTMAP and ENDMAP. Implement the base UDMF fields for the vanilla games (Heretic, Hexen, Strife) as they get integrated into Eternity (currently they're not available) Implement fields that exist in ExtraData but not in the base UDMF specs. Currently "alpha" works for linedefs, but that's it and totally unofficial. Fix some hacks that I had to do to make mapthings work (mainly, coordinates are rounded to integer because mapthing_t is too coarse) Be careful with fractional floating-point vertex coordinates! Won't they be parsed in a platform-dependent way, causing demo desyncs? Why not: optimize the loading time. It's nothing outrageous, but can be noticed in complex maps. In addition, the most popular map editors and nodebuilders need to be updated to support this new flavour of UDMF. Currently GZDoom Builder fails to read the test map I converted. You can get the Windows build here. Paste the exe where you have Eternity.exe already placed. You can get the Macintosh build here. Here's more:OMGIFOL Python script that reads a vanilla Doom map and outputs a TEXTMAP to stdout. Unmodified version of Back to Saturn X E1* from my computer Same MAP01 from that megawad, but converted to UDMF namespace="doom". What you can do: record a -vanilla demo in the converted BTSX map, and then use doom2.exe to play the demo in the base BTSX map in vanilla. It should be identical. _____ *Back to Saturn X 0 Share this post Link to post
esselfortium Posted December 14, 2015 ❤ Awesome! I'm really glad you're working on this, since I know Quasar is too busy to deal with everything on his own. It's great to see some substantial outside contributions to EE again. 0 Share this post Link to post
Gez Posted December 14, 2015 printz said:What is important to do: define the "eternity" namespace. That's something that I think should be left to Quasar. For the moment you should only implement the "doom" namespace. printz said:Decide what lumps need to be placed between TEXTMAP and ENDMAP. The current ad-hoc implementation basically requires NODES, SEGS, SSECTORS, REJECT, BLOCKMAP and optionally BEHAVIOR to be, in any order, between TEXTMAP and ENDMAP. Eternity supports extended nodes. printz said:Be careful with fractional floating-point vertex coordinates! Won't they be parsed in a platform-dependent way, causing demo desyncs? Set the numeric locale to "C" before parsing a TEXTMAP, and you'll have no problem. 0 Share this post Link to post
Mechadon Posted December 15, 2015 This is very cool news! Thanks for you hard work :D 0 Share this post Link to post
printz Posted December 15, 2015 It seems that ZNODES is the only currently supported BSP data in UDMF, together with optional REJECT, BLOCKMAP and BEHAVIOR. So the easiest path is to add ZNODES support in this branch. More exactly, it should support XGLN, XGL2 and XGL3 (see the ZDoom wiki on nodes for details). Eternity currently supports ZDoom nodes, but only XNOD (non-GL nodes). As I see it, adding the GL ones looks like a simple edit in the non-GL reader, and choosing some other lumps. By the way, and this applies for traditional ZDoom maps: can non-GL (in NODES) and GL (in SSECTORS) ZDoom nodes coexist for one map? If so, which has priority? 0 Share this post Link to post
Gez Posted December 15, 2015 printz said:By the way, and this applies for traditional ZDoom maps: can non-GL (in NODES) and GL (in SSECTORS) ZDoom nodes coexist for one map? If so, which has priority? Yes, GL nodes and non-GL nodes can coexist, that's why the GL nodes are in a different lump. Both node types may be used simultaneously! GZDoom will use GL nodes for rendering, and normal nodes for gameplay. ZDoom will use normal nodes for everything, unless textured automap is enabled, then it'll use GL nodes for rendering as well. 0 Share this post Link to post
printz Posted December 15, 2015 Gez said:Both node types may be used simultaneously! GZDoom will use GL nodes for rendering, and normal nodes for gameplay. ZDoom will use normal nodes for everything, unless textured automap is enabled, then it'll use GL nodes for rendering as well. And can GL ZDoom nodes be the only nodes in a traditional level? 0-length NODES lump, existing SSECTORS. 0 Share this post Link to post
Gez Posted December 16, 2015 Yes. If GL nodes are present but normal nodes are missing, ZDoom will use the GL nodes. 0 Share this post Link to post
Graf Zahl Posted December 17, 2015 Gez said:That's something that I think should be left to Quasar. For the moment you should only implement the "doom" namespace. Didn't Quasar say he'd want to base Eternity's native namespace on Hexen specials? That said, be aware that the "doom" namespace intentionally limits the allowed line specials to MBF! If you want Eternity extensions you'll need another one. ZDoom strictly enforces this limitation and clears out any special not covered by Boom and MBF. The UDMF spec said:The use of one of these namespaces indicates that all line specials, sector specials, thing types, and if applicable, thing specials contained in the UDMF map conform to the "vanilla" types for the respective gamemode (with an exception for Doom) and are therefore 100% compatible with all UDMF implementors which support the specified gamemode. An exception is made for the "Doom" translation type. It is considered to represent the final fork of the v1.9 Doom engine, and therefore it includes all line types added in DOOM II. It is also additionally considered to include all specials defined in the BOOM and MBF source ports. Implementors who support BOOM linedef types can handle these in the normal manner; ports which do not support them should zero or otherwise ignore the offending specials. 0 Share this post Link to post
printz Posted December 17, 2015 ZNODES support has been implemented. I'll bump this thread when I release an Eternity/UDMF GZDoom Builder configuration and eternity-udmf binaries for beta-testing. Please don't use it to release UDMF maps until they become official. Here's a screenshot of a removed HPACK map written in UDMF. Don't mind the missing textures; the resources weren't loaded, and they were ZDoom textures anyway. Currently loading time seems quite slow so I'll have to optimize it. 0 Share this post Link to post
printz Posted December 20, 2015 The UDMF functionality in the branch is pretty much complete, by now. I even optimized the loading speed. What's left are some details. For instance, required fields aren't enforced by now (after I optimized the speed). Someone serious would want those fields to throw an error if they're missing (or wrong type)… For nodes: it uses the same format as ZDoom. Things that don't depend on me: - setting up namespaces and new fields corresponding to them. Currently you only get MBF capabilities (albeit joinable with EE's mighty EDF system), which is a bit underwhelming. EXTRADATA stuff needs to be ported to UDMF; for now it's not. Enjoy and godspeed! 0 Share this post Link to post
esselfortium Posted December 20, 2015 Fantastic, thank you so much! This will hopefully be a great foundation to build from. 0 Share this post Link to post
Altazimuth Posted December 20, 2015 It's so good to see things start to come together for Eternity. I can't understate how amazing this is for the port. Any word on an Eternity-UDMF config for DB2/GZDB? 0 Share this post Link to post
esselfortium Posted December 20, 2015 Altazimuth said:Any word on an Eternity-UDMF config for DB2/GZDB? For a meaningful one to be developed that won't be immediately obsoleted by EE development, an extended EE namespace (based on Hexen's line types, like ZDoom's) still needs to be created first. Without that, accessing ExtraData functionality, parameterized lines, or anything else not already doable in Doom format (aside from sub-unit vertex precision, at least) wouldn't be doable. So, what printz has done here is an awesome foundation of the base functionality for UDMF, that still needs to be built on to be truly useful. 0 Share this post Link to post
Graf Zahl Posted December 20, 2015 esselfortium said:or anything else not already doable in Doom format (aside from sub-unit vertex precision, at least) wouldn't be doable Let's keep in mind what I already stated: The "Doom" namespace should not allow any extended features to avoid problems with other ports that may implement UDMF later. So, strictly speaking, anything beyond MBF would be on shaky ground, depending on how strictly Quasar is going to enforce this. It may work right now due to lax checks, but my suggestion would be to obey the spec to the letter and do the same as ZDoom here, i.e. clear all line specials above 272. EDIT: I just saw this: // Special handling for tag if(gUDMFNamespace == unsDoom || gUDMFNamespace == unsHeretic || gUDMFNamespace == unsStrife) { int tag[2]; tag[0] = uld.id; tag[1] = uld.arg0; ld->tag = tag[1] ? tag[1] : tag[0]; // support at least one of them } To make it clear: This is a violation of the UDMF spec! Tags (as in 'parameter for line special') and IDs MUST be treated separately. This was the major point I discussed endlessly with Quasar but the end result was to strictly separate them and put explicit wording into the spec to make this separation clear. Just make sure this isn't forgotten later as it is incompatible with both the spec and ZDoom! You also shouldn't make map format decisions this way. Better have one place somewhere where you say either 'Doom format' or 'Hexen format' Just imagine someone decides later to support one of ZDoom's namespaces (or something else, like a combined 'enhanced' namespace which contains the features supported by both ZDoom and Eternity.), they can sift through the entire code to add support for this - one can only pray that nothing is overlooked... 0 Share this post Link to post
printz Posted December 20, 2015 Graf Zahl said:To make it clear: This is a violation of the UDMF spec! Tags (as in 'parameter for line special') and IDs MUST be treated separately. This was the major point I discussed endlessly with Quasar but the end result was to strictly separate them and put explicit wording into the spec to make this separation clear. Just make sure this isn't forgotten later as it is incompatible with both the spec and ZDoom! This is a bit difficult to me. Internally Eternity has tag and args[0..4] for each line_t object. But it ignores args[0] and uses only tag for all classic, non-parameterized specials. What should I do, modify their internal functions to use args[0] for the target sectors instead of tag? I understand that one of the big problems in what I did was to put either of id or arg0 into tag, instead of putting them in tag and args[0] respectively. Having separate, decoupled arg0 and id for line teleportation or translucency means I can do things in namespace="Doom" not possible in binary MBF. I suppose it's fair in the scope of implementing ports? Graf Zahl said:You also shouldn't make map format decisions this way. Better have one place somewhere where you say either 'Doom format' or 'Hexen format' Just imagine someone decides later to support one of ZDoom's namespaces (or something else, like a combined 'enhanced' namespace which contains the features supported by both ZDoom and Eternity.), they can sift through the entire code to add support for this - one can only pray that nothing is overlooked...I don't understand 100% what you're saying here. Do you mean that I should minimize the places that I check for namespace? Or that should I use a higher-level variable that aggregates the namespace and other factors (such as current IWAD or broad game category)? 0 Share this post Link to post
Graf Zahl Posted December 20, 2015 printz said:This is a bit difficult to me. Internally Eternity has tag and args[0..4] for each line_t object. But it ignores args[0] and uses only tag for all classic, non-parameterized specials. What should I do, modify their internal functions to use args[0] for the target sectors instead of tag? I know. It's a bit of a problem to change on the fly. I brought this up because the danger here is simply that it's almost a given that some mappers won't use it correctly. You can bet that they just use the tag for both, see that it works and release it in a broken state, as far as the spec is concerned I think this will require some input from Quasar to ensure a smooth transition. I can only give some hints what I'd do in this situation: - add two access functions to linedefs - getTag() and getID(), for a first conversion let both return the tag value. By doing this you get an abstraction layer that'd make later changes a lot easier to implement. - once all places have been adjusted to use the proper value, rename the 'tag' variable to 'id'. - change the binary map loader to set up both 'id' and 'args[0]' to the tag value from the map. - change the UDMF-loader to initialize the 'id' field from the 'id' property and 'args[0]' from the 'arg0' property. - and finally change the 'getTag()' function to return 'args[0]' instead of 'id'. Obviously the biggest issue here is to decide which to use where. The rule should be, if something is about to be done to the tagged line, use the id, if the line is supposed to be doing something to another line, use the arg0. But let's not jump the gun and wait for Quasar. I think the best approach for now is to run some check on the loaded map to see if it contains linedef actions that affect other lines which are only marked by tag and error out. The most important thing right now is that such maps cannot accidentally be used. printz said:Having separate, decoupled arg0 and id for line teleportation or translucency means I can do things in namespace="Doom" not possible in binary MBF. I suppose it's fair in the scope of implementing ports? Actually, this was the hottest issue when the specs were written. In the end it was considered that for an implementing port, the necessary changes should be manageable, since the required changes are small, compared to adding an UDMF-parser. Let's not forget that this isn't the only thing where the feature set is expanded. There's also the fully separated skill flags and the full precision vertices so it made sense to address the lack of separate IDs for lines as well. printz said:I don't understand 100% what you're saying here. Do you mean that I should minimize the places that I check for namespace? Or that should I use a higher-level variable that aggregates the namespace and other factors (such as current IWAD or broad game category)? I'm a bit worried that the code is littered with stuff like 'if (namepsace == a || namespace == b)... For now that is sufficient, but you never know if some other namespace will be supported later on. Then every single of these lines needs to be verified and changed if applicable. I just think it's better to filter out the more important properties here and store them separately. The Doom vs. Hexen specials are the most obvious thing because they are clearly the ones which get checked the most. So if you assign that information to one global 'useHexenSpecials' variable, you get rid of all those namespace checks and better, since you inevitably set this for new namespaces as well, don't have to check so many places in the code where distinctions need to be made. 0 Share this post Link to post
printz Posted December 20, 2015 On "id" vs "arg0": I tested in ZDoom a "doom" namespace map obtained by using UDMF-Convert over a vanilla map. There is a line in the map with the WR-Teleport (97) special. It has the target sector tag assigned both in "id" and "arg0". Commenting out just "id" causes the teleporter to fail. "Arg0" has no effect. It's like ZDoom only checks "id" to determine the destination of the teleport special in the "Doom" namespace. 0 Share this post Link to post
Graf Zahl Posted December 20, 2015 Can you give me the test map(s)? Due to lack of material the Doom namespace was never fully tested. These conversions all need to be run through XLAT and it may be possible that something in there is wrong (AFAIK it was changed after UDMF was added to allow some mucking around with tag expressions, which may cause problems.) 0 Share this post Link to post
printz Posted December 20, 2015 Here's what I'm testing: * Back to Saturn X (some version) as a resource wad * add over it: UDMF version of MAP01 as generated by UDMF-convert. This is different from the ad-hoc one I posted upwards. How to test it: jump to linedef #49 and comment the id and arg0 fields respectively; see what happens. It's located at (-1952, -1920). 0 Share this post Link to post
Graf Zahl Posted December 21, 2015 Thanks. This was indeed broken. When I fired up the debugger it immediately broke with an uninitialized variable report. The two variables had indeed been mixed up, but due to the lack of test maps when this was implemented nobody ever noticed. 0 Share this post Link to post