MREJECT (alternately, REJECTXT) lump idea

[Blastfrog]

I had an idea for a lump that would make hand-made REJECT table tweaks far easier to deal with. Possible names could be "manual reject" or "reject text" (MREJECT or REJECTXT).

 

It would be a very simple human readable and editable text format. All it does is allow you to define deliberate alterations to the reject table. This would have two advantages: first is that it'd simply be easier and more convenient than messing about in a hex editor, second is that you could "fire and forget" the changes, continuing to work on the level, yet your intended tweaks remain without having to reimplement them every time the REJECT lump needs to be rebuilt for any reason.

 

Editors could make use of this, having a frontend UI to make these changes instead of typing them, as well as applying the changes to the REJECT lump automatically after building a new one. Source ports could also make use of this, by applying the changes upon level load.

 

Is this something that is worth pursuing?

[scifista42]

I like the idea, especially the concept that the lump could be processed both by map editors while building a REJECT lump (for vanilla compatible maps), and by advanced ports while loading a (port-specific) map without a REJECT lump.

[Memfis]

How reliably can that stuff be stored though? I mean, during the editing sectors might be cut in parts, maybe renumbered, etc, which could interfere with your alterations. This would require good planning I guess.

[scifista42]

I imagine that if the mapper splitted a MREJECT-affected sector into 2 parts, the (compliant) map editor would automatically update the MREJECT lump to affect both parts of the original sector in the same way the original sector was affected, and similarly for other actions.

[Graf Zahl]

Storing this is just as reliable as any other way to store it. Unless this is added to UDMF as a sector property it will always be a separate lump that'd be invalidated once the sectors change.

 

[scifista42]

What I mean is that the map editor would update it during every elementary change of map geometry, not just after every REJECT lump rebuild.

[Blastfrog]

3 hours ago, scifista42 said:

What I mean is that the map editor would update it during every elementary change of map geometry, not just after every REJECT lump rebuild.

Exactly what I had in mind. Although...

 

3 hours ago, Graf Zahl said:

Unless this is added to UDMF as a sector property

That's a good idea too! Why not have both the lump for the old map spec, and have a proper implementation in UDMF?

[boris]

Or you could take a different approach and let the editor handle everything. In the editor you could draw polygons and define which polygons are in a "reject" group. After the map is saved (and reject data is built) the editor can check which sectors are inside the polygons and then modify the REJECT lump. While that's a more hackish approach it would work with every port, and also with vanilla.

[Graf Zahl]

It would also lose the data it edited with a non-supporting editor. To be persistent it needs to be stored elsewhere.

 

[scifista42]

As long as the non-supporting editor only erased/rebuilt REJECT without modifying VERTEXES/LINEDEFS/SIDEDEFS/SECTORS, storing the custom REJECT info in a separate MREJECT lump should be safe enough.

[Gez]

• Sector numbers. Advantage: pretty much how it already works. Drawback: has to be handled by the editor.
• Sector tag. Advantage: simplified creation of rejected zones (mass select sectors, change all tags at once), already handled by the editor. Drawback: can interfere with other uses of sector tags.
• New property. Advantage: simple and straightforward, can mass-edit from editor. Drawback: basically requires mapping in UDMF (map could potentially be converted to other formats after, depending on which namespace and feature sets are used).

 

[printz]

I think UDMF and sector tags is the easiest approach. If UDMF is not available, you can have something like Eternity's ExtraData.

[Blastfrog]

18 hours ago, Gez said:

• Sector numbers. Advantage: pretty much how it already works. Drawback: has to be handled by the editor.

This is the best option for broadest compatibility. Not a big deal that it has to be handled by the editor. Besides, as I said in the OP, it should be easy to make these by hand (despite the limitations of doing it by hand, such as when sectors get reordered and such).

 

18 hours ago, Gez said:

• Sector tag. Advantage: simplified creation of rejected zones (mass select sectors, change all tags at once), already handled by the editor. Drawback: can interfere with other uses of sector tags.

Advantage: not actually an advantage at all, MREJECT is better in every way so long as it's handled well by the editor. The editor would ideally have GUI-based ways of simplifying it and making it easy, just like so many other things you don't have to do by hand in editors.

Drawback: For the reason you listed, it becomes useless and limited by that unavoidable conflict of this method.

 

18 hours ago, Gez said:

• New property. Advantage: simple and straightforward, can mass-edit from editor. Drawback: basically requires mapping in UDMF (map could potentially be converted to other formats after, depending on which namespace and feature sets are used).

Advantage: Just what you listed.

Drawback: Not a drawback at all, as long as MREJECT is also an option.

 

I say, why not both? Both MREJECT and a cleaner UDMF implementation are worth pursuing. Do it right in the modern spec, do it the cleanest way possible within the existing parameters of the old spec. Bam, maximum usefulness. No sense in excluding either option.

 

Other ideas:

 

* MREJECT lump can be also be used in a UDMF map

*The UDMF new property could be referenced to generate an MREJECT lump

Edited May 22, 2017 by Blastfrog

[kb1]

On 5/21/2017 at 2:13 PM, Graf Zahl said:

It would also lose the data it edited with a non-supporting editor. To be persistent it needs to be stored elsewhere.

 

Exactly. It is a great idea to have a way to alter the reject. But, here's the hurdles the programmer must cross:

• Direct REJECT edits become invalid when the map is edited.
• If your MREJECT lump lists sector numbers, it will become invalid when the map is edited.
• You cannot expect the map editor to update the REJECT table at runtime. A proper REJECT map build can take a LONG TIME to build, as it is basically a line intercept formula for num_lines squared * number_of_traces (precision), depending on implementation. So, for a 20,000 line map, for, say, 20 traces per line, that's an intercept formula for 8,000,000,000 lines! Because of this, you cannot expect the editor to start out with a good REJECT map, and you cannot expect the editor to keep it up to date through changes, such as cut and paste.

Manually typing in a sector number sucks, even in UDMF. Changes to the map will change sector numbers. Sure, you can attempt to keep them valid as you manipulate the map. But, consider doing a cut and paste, including one of the 2 sectors - how would the editor reconnect and fixup the sector numbers? The asnwer is that it cannot do the right thing, in every case - it's not possible.

 

These are only approaches that I think could be considered reliable:

• 1. Adding a new sector "reject tag" value, along with a new reject_effect property to control what happens. Unlike sector numbers, tags are reliable, because, unlike sector numbers, they can be duplicated, and "carried around" through cut, copy, paste operations.
• 2(a) Hexen/UDMF: Creating a special Thing type that uses its Tag property (Hexen/UDMF only) to match a Thing of the same type in another sector. You'd have a "Sectors Visually Connected" type, and a "Sectors NOT Visually Connected" type.
• 2(b) Doom: Create a set of, say, 64 Things that work, as in 2(a), but, instead of matching on Thing.Tag (which is not available in Doom format), it would match on Thing.DoomedNum, and you'd have a maximum of 64 of these per map.
• 3. Doom (super-hack): Use a "magic" identifier + a Tag value in either the upper or lower sidedef of one of the sector's 1-sided walls, in both sectors, to set up a special visibility condition. Some of Wesley's linedef proposals for Boom+ involve overloading the sidedefs. Also, Legacy (and others) put colormap/color blend/3D-floor info into sidedefs, so there is precedent for using some of these methods. The editors must be careful to not clear upper/lower sidedefs on 1-sided walls for this to work.

Out of these possibilities, 1. is the cleanest, and 3., though hackish, does not have the limitations of 2(b), and avoids the "use things to mark sectors" ordeal of 2(a). Anything else leaves you entering sector numbers, which is a recipe for disaster.

 

Note to other port devs: Please verify this info - I don't feel like having a debate today. Thanks!

 

[Blastfrog]

3 hours ago, kb1 said:

.Direct REJECT edits become invalid when the map is edited.

That's kinda the whole point of MREJECT.

 

3 hours ago, kb1 said:

If your MREJECT lump lists sector numbers, it will become invalid when the map is edited.

So then have the editor translate the sector numbers to their new values in the MREJECT every time the sectors are reordered for whatever reason.

 

3 hours ago, kb1 said:

You cannot expect the map editor to update the REJECT table at runtime.

Hence the need for a stable "patch" of sorts outside of the REJECT itself, such as MREJECT.

 

3 hours ago, kb1 said:

Manually typing in a sector number sucks, even in UDMF.

Okay, but it's better than using a hex editor! Besides, editors could provide a more user-friendly GUI to manage MREJECT entries.

 

3 hours ago, kb1 said:

Changes to the map will change sector numbers. Sure, you can attempt to keep them valid as you manipulate the map.

So then, do that.

 

3 hours ago, kb1 said:

But, consider doing a cut and paste, including one of the 2 sectors - how would the editor reconnect and fixup the sector numbers?

It wouldn't, and probably shouldn't.

 

3 hours ago, kb1 said:

The asnwer is that it cannot do the right thing, in every case - it's not possible.

Not an issue. It's the mapper's responsibility to ensure their MREJECT is set up as intended. MREJECT itself is to make their lives easier, and the editor can carry as much weight as it can, but it can't be expected to magically do everything for the user. That is no reason to reject the whole idea, it still is a massive improvement over the status quo.

 

3 hours ago, kb1 said:

Adding a new sector "reject tag" value, along with a new reject_effect property to control what happens. Unlike sector numbers, tags are reliable, because, unlike sector numbers, they can be duplicated, and "carried around" through cut, copy, paste operations.

For UDMF, this would be the best solution IMO.

 

2(a) is good too, but I think 1 seems like a better idea. 2(b) is ugly, and 3 is unacceptable for obvious reasons. Not to mention, the point of MREJECT is to have broad compatibility, including vanilla, so having it as metadata meant to be "baked" into the REJECT rather than supported by new engine-side feature is required.

 

3 hours ago, kb1 said:

I don't feel like having a debate today.

I hope this isn't some kind of slight aimed at me. If you're referring to about what went down in the GEC thread, I was merely responding to unwarranted crap being tossed my way. I'm not that argumentative of a person, and anyone who says otherwise doesn't know me well enough.

[kb1]

42 minutes ago, Blastfrog said:

That's kinda the whole point of MREJECT.

 

So then have the editor translate the sector numbers to their new values in the MREJECT every time the sectors are reordered for whatever reason.

 

Hence the need for a stable "patch" of sorts outside of the REJECT itself, such as MREJECT.

 

Okay, but it's better than using a hex editor! Besides, editors could provide a more user-friendly GUI to manage MREJECT entries.

 

So then, do that.

 

It wouldn't, and probably shouldn't.

 

Not an issue. It's the mapper's responsibility to ensure their MREJECT is set up as intended. MREJECT itself is to make their lives easier, and the editor can carry as much weight as it can, but it can't be expected to magically do everything for the user. That is no reason to reject the whole idea, it still is a massive improvement over the status quo.

 

For UDMF, this would be the best solution IMO.

 

2(a) is good too, but I think 1 seems like a better idea. 2(b) is ugly, and 3 is unacceptable for obvious reasons. Not to mention, the point of MREJECT is to have broad compatibility, including vanilla, so having it as metadata meant to be "baked" into the REJECT rather than supported by new engine-side feature is required.

 

I hope this isn't some kind of slight aimed at me. If you're referring to about what went down in the GEC thread, I was merely responding to unwarranted crap being tossed my way. I'm not that argumentative of a person, and anyone who says otherwise doesn't know me well enough.

It's not a slight aimed at you - I'm just tired. But, I will briefly reply - You're not thinking it through. When sectors numbers change due to editing a map, you have created a discrepancy which is not visible to the mapper. It's hidden, implementation-specific information...basically housekeeping data. It's a direct sign of a piss-poor unprofessional interface. If I use the cut tool, and paste a map section to a different map, I expect the properties of the objects I highlighted to come along for the ride...and they do, except for this one little lump that doesn't behave like any other piece of map data. It's the ugly red-headed stepchild that requires "special" care.

 

WTF, I was careful to not be condescending. I gave a very detailed, considered insight into why you and everyone else would come to dislike your proposed implementation, and even went so far as to design 4 different possible alternatives that do not suffer from the original proposal's shortcomings, for each type of map in existence, all the while taking care to not diss you or anyone in this thread, and you went and split up my entire post, and replied with snarky one-liners, which is, in fact, disrespectful. Why?

 

I gave you sensible reasons, and took the time to try to get you a REAL implementation that works *properly*, and you get shitty about it, why? My time is very limited these days. I wasn't the only person that noticed the inherit issues with your proposal, but I took the time to try to explain them. Please take the time to understand them, so you will know why too, before spitting out "So then, do that", as if it's some fucking obvious thing that anyone should know. My answer: "No. I don't want to have to track sector numbers: That's what I bought a computer for." How's that for obvious?

[Blastfrog]

I can see how my tone could be interpreted as snarky, but I assure you that was not the intent. I split the post up so I could individually respond to every point and provide my own views on them. It's easy to mistake terseness and directness as snark, especially in text. I was not being remotely shitty with you.

 

Anyhow, sector numbers; they're not hidden, GZDoom Builder can already tell me a sector's number when I hover over one with the mouse. Even if they were hidden in the editor, that's no reason to bring it up as if it were an obstacle to anything; the natural conclusion is "why doesn't the editor let me see the sector's number? obviously, this should be a thing it can do, because it's highly valuable info and is trivial to retrieve and display"

 

My proposal is the cleanest, simplest and most convenient way to legitimize manual REJECT table tweaks as an actual editing feature that is compatible with all engines. My proposal isn't perfect (nothing really is), but I don't find myself agreeing much with what you have proposed for binary formats, as they all require engine-side changes (and in the case of 3, questionable and inflexible ones at that). MREJECT is purely just some metadata used to more easily tweak the REJECT table, engines are free to ignore it entirely.

 

I very much like your UDMF "reject tag" idea. This is also something that should be looked into, for UDMF. For binary, I simply disagree with your proposals. That doesn't make them bad at all, just not appropriate for my specific goals.

 

2a is a good solution, but I think that Hexen format is not worth developing anymore and that 1 is preferable in the case of UDMF. 2b is not ideal to me because it occupies more new actors than 2a would and sets a hard limit on how many changes you can do. 3 is unacceptable to me because it is, in your own words, a "super hack". The situation with Doom format is bad enough, I don't like the idea of delving into parts of the format so deeply (and in ways beyond unintended) just for one effect. What if some other effect is desired and then there's no more room?

 

I don't think it's worth stressing over how or whether to automate MREJECT adjustments in ambiguous cases. A mapper even considering using MREJECT will not be a novice, they will know of its limitations in this regard, and will take appropriate steps in response. In such cases, only a human being can really decide these things.

 

It wouldn't be hard to add MREJECT entries when the level starts with none, why would it a problem to just put it in again when an entry got cut due to a non-elementary geometry change? I think it should go without saying that the editor should notify you when this happens. Besides, like anything else "tricky" in Doom mapping, your geometry should be close to final before you start playing around with details of any kind. Your responsibility if you corner yourself.

 

It's a waste to come up with 'solutions' for a minor inconvenience (rather than a true problem) that will more likely hinder than help. This applies to things I've said too, such as "lol dummy sectors are teh worst get rid of them". I'm a dummy sector for thinking that. MREJECT, on the other hand, addresses an actual problem, though, which is that REJECT editing is next to useless in the current status-quo because right now, you can only do it in a hex editor, and you have to redo it every time the REJECT is rebuilt. MREJECT doesn't do anything new, it just tries to automate this process as much as possible in order to make it truly useful, convenient and accessible.

Edited May 24, 2017 by Blastfrog

[kb1]

19 hours ago, Blastfrog said:

I can see how my tone could be interpreted as snarky, but I assure you that was not the intent. I split the post up so I could individually respond to every point and provide my own views on them. It's easy to mistake terseness and directness as snark, especially in text. I was not being remotely shitty with you.

 

Anyhow, sector numbers; they're not hidden, GZDoom Builder can already tell me a sector's number when I hover over one with the mouse. Even if they were hidden in the editor, that's no reason to bring it up as if it were an obstacle to anything; the natural conclusion is "why doesn't the editor let me see the sector's number? obviously, this should be a thing it can do, because it's highly valuable info and is trivial to retrieve and display"

 

My proposal is the cleanest, simplest and most convenient way to legitimize manual REJECT table tweaks as an actual editing feature that is compatible with all engines. My proposal isn't perfect (nothing really is), but I don't find myself agreeing much with what you have proposed for binary formats, as they all require engine-side changes (and in the case of 3, questionable and inflexible ones at that). MREJECT is purely just some metadata used to more easily tweak the REJECT table, engines are free to ignore it entirely.

 

I very much like your UDMF "reject tag" idea. This is also something that should be looked into, for UDMF. For binary, I simply disagree with your proposals. That doesn't make them bad at all, just not appropriate for my specific goals.

 

2a is a good solution, but I think that Hexen format is not worth developing anymore and that 1 is preferable in the case of UDMF. 2b is not ideal to me because it occupies more new actors than 2a would and sets a hard limit on how many changes you can do. 3 is unacceptable to me because it is, in your own words, a "super hack". The situation with Doom format is bad enough, I don't like the idea of delving into parts of the format so deeply (and in ways beyond unintended) just for one effect. What if some other effect is desired and then there's no more room?

 

I don't think it's worth stressing over how or whether to automate MREJECT adjustments in ambiguous cases. A mapper even considering using MREJECT will not be a novice, they will know of its limitations in this regard, and will take appropriate steps in response. In such cases, only a human being can really decide these things.

 

It wouldn't be hard to add MREJECT entries when the level starts with none, why would it a problem to just put it in again when an entry got cut due to a non-elementary geometry change? I think it should go without saying that the editor should notify you when this happens. Besides, like anything else "tricky" in Doom mapping, your geometry should be close to final before you start playing around with details of any kind. Your responsibility if you corner yourself.

 

It's a waste to come up with 'solutions' for a minor inconvenience (rather than a true problem) that will more likely hinder than help. This applies to things I've said too, such as "lol dummy sectors are teh worst get rid of them". I'm a dummy sector for thinking that. MREJECT, on the other hand, addresses an actual problem, though, which is that REJECT editing is next to useless in the current status-quo because right now, you can only do it in a hex editor, and you have to redo it every time the REJECT is rebuilt. MREJECT doesn't do anything new, it just tries to automate this process as much as possible in order to make it truly useful, convenient and accessible.

Ok, I misinterpretted - sorry about that. I agree with you that reject editing is a useful thing. But your implementation forces the user, expert, or not, to do housekeeping that, with a better format, the editor can and should do. Using sector numbers is a hack. It forces the map author to do the reject stuff at the end, instead of during editing, where it should happen. There are simple mechanisms to make it work properly - why do you insist on making it work in a mickey-mouse fashion? And, yes, Doom (and Hexen) support is important to me, to the spec I'm building, and the only way to add tagging in Doom format is to hack it in. That's the price you pay when using Doom format. Having it impossible to do is worse than having to hack it in. Believe me, once support is in for UDMF (which can handle anything), the machanics for getting it in Doom format are just an exercise in mundaneness. There is precedent for having a need for a similar technology for Doom mapping.

[kb1]

Here's my attempt at a better explanation:

 

Sector visibility is conceptually a property of the sector, just like top, bottom, and lightlevel. Imagine if we needed a lump MLIGHT to set sector brightness: Everyone would hate it. Sure, it's a simple enough structure to understand, but it sucks to use.

 

In the editor, you want to be able to set sector lightlevel in the same place you set sector top and bottom, right? You want sector lightlevel to come along for the ride when you cut a bunch of sectors, and paste them elsewhere, right? Sector visibility is no different. Inside the engine, maybe it does require edits to a different data structure. But, within a map editor, you want to abstract those details away from the map author, so he/she can concentrate on map making.

 

Internally, using a tagging system, instead of raw sector numbers, allows the map editor program to generate sector numbers any way it sees fit. Internally, the map editor need only copy the data, it does not need to do lookups and fixups on the sector numbers. Furthermore, the map author just sets the appropriate tag numbers once, and those 2 sectors are automatically locked together, through just about any edit possible. And, if one of the sectors disappears, it is easy to check for missing tags, and it's easy for the map author to fix the problem in an intuitive way.

 

You still get all the functionality you wanted, without the limitations - Does this makes sense?