Idea: METADATA lump for Slade and Doom Builder

Oh, just for in-dev projects? Right, nvm.
I read something about storing more specific author data, presumably for the long term.

It does store author data. The main use case for that, though, is for resources. You compile a resource wad from multiple sources and the metadata for each texture will tell you whether you got it from GOTHICTX.WAD or wherever else, and if metadata is present in any of the wads you compiled from, that'll follow them as well, so you'll get specific author credits.

The really nice thing about this is that if someone then uses your compiled wad as a source to get their resources from (think about how many people have probably gotten texture edits from Scythe 2, Deus Vult II, or CC4-tex rather than from their original sources, for instance), the credits and origins of those resources will still be kept accurate for each one.

Look, I'm not on a crusade my point is can we keep the integration hash data out of the package being built until finalized by the author and released.

DaniJ said:

Look, I'm not on a crusade my point is can we keep the integration hash data out of the package being built until finalized by the author and released.

Let me get a quick check here.
Your suggesting a folder design and thus eliminating duplicate lump names and the need for hashing?
It might work, but what happens if there is a duplication in a folder? (Not sure why anyone (but Id) would do that though)

example structure

maps
    map01
    map02
textures
    mytex1
    mytex2
sounds
    dspistol
    dsshotgn
music
    d_runnin
    d_stalks
graphics
    titlepic
sprites
    ceyea0
    ceyeb0
    ceyec0
patches
    mypat1
    mypat2
flats
    myflat1
    myflat2

Moving these into a new archive would auto create missing folders and copy the metadata?

Gez said:

"Do we need to validate that the metadata corresponds to the same file?"
Yes -> Then we need a hash even for zip files
No -> Then we don't need a hash even for wad files

There are no situations where "we need a hash for wad files but not for zip files" makes sense...

...checking whether the entry has been modified is necessary for checking that it's indeed the same entry...

...The approach we're going with, on the other hand, is by-software, for-software. "User tagged this file as being by Rand Omname, and this other file as being by Sam Wanelse" and it doesn't matter in what folder they are.

I think it, and Gez types it :) Stated very eloquently, Gez. Thanks for "getting it" :)

Graf Zahl said:

This entire discussion of DaniJ vs. everybody else mirrors nearly all other discussions about design philosophies, i.e. there seems to be a profound phobia for Doom-centric handling among the Doomsday devs.

I remember similar issues about map formats, map format storage and a few other things that all ended with DaniJ trying to discuss the topic to death without even trying to get a consensus with the other source ports and tool developers who are nearly uniformly opposed to the kind of abstraction that seems so prevalent in everything coming out of the Doomsday corner.

Graf, you noticed that too? :) I cannot understand energy being devoted to purposely disrupt movements towards universal compatibility. Actually, it pisses me off, because I do not believe that the motives are innocent. And, that amazes me as well. I'm left asking "Why?".

DaniJ said:

Look, I'm not on a crusade my point is can we keep the integration hash data out of the package being built until finalized by the author and released.

That doesn't match the OP's request. The OP wants to know what versions of lumps he's included. See post #1 of this topic.

DaniJ, didn't you say you were done with this discussion, a couple of pages back? You know, DoomsDay wads and zips would benefit from this new feature, because what's being proposed is a universal solution to the OP's request.

By all means, please stay. I'm not asking you to agree. I'm asking you to stop derailing the proposal. 1/3 of the posts are attempts to respectfully explain the concept to you. Another 1/3 are getting people back on track after you've disrupted the idea, by discussing completely irrelevant topics, like Zip vs. WAD.

I, and others would like to use this feature, and I, and others, are being forced to invest significant time and energy to keep typing the same answers over and over. I am investing my time, because I believe it is a worth-while endeavor. What goal are you investing for?

Randy87 said:

Oh, just for in-dev projects? Right, nvm.
I read something about storing more specific author data, presumably for the long term.

The proposal is for in-dev projects, but because of that, it also naturally covers the storage of long-term author data, as well, on a lmup-by-lump basis (as long as the final edit, lump-per-lump, preserves the author data.

In other words, if I import your wall/flat resource wad, and your wad contains metadata, listing you as the author, my wad will also list you as the author of those lumps.

But, if I edit one of your wall images, the metadata no longer applies to that image. The Wad Editor program then has the choice to do one of the following:

1. Prompt me to adjust the metadata.
2. Change the Date Modified stamp to today's date.
3. Modify the Author field. Maybe originally it was "Randy87". Maybe, the Wad Editor leaves you as the author, but adds a metadata field "Modified by: kb1". That would be nice.

We haven't really defined exactly what happens in that case. As it stands, if I modify your image lump with an editor that does not support metadata, then later I open the wad in an editor that *does* support metadata, your author info for that lump would be gone, because, technically, it is no longer the same lump.

So, by the nature of how the proposed solution works, yes, the metadata *is* long-term, and it is virtually *guaranteed* to maintain your author info, until you are no longer the author of the current lump, if that makes sense.

esselfortium said:

The primary intent of lump metadata is for display in lump editing tools. Source ports already have their own methods for displaying author info for maps, and compatibility options vary based on different ports' needs, so that's also something probably best kept to port-specific mapinfo lumps or similar. This is more for keeping track of timestamps for in-development projects, and for keeping track of the origins of shared resources as they travel from one project to the next.

There's probably not much reason for source ports to implement metadata reading -- in the immediate future, I'm planning on making use of it for vanilla wads and texture resource packs.

Yes, this is a very typical usage. In this manner, esselfortium's metadata will be carried through to the Wads that make use of his resources in maps.

What's confusing some people is that hashes are to be stored in the metadata. This is the technical mechanism that allows editing programs to confidently carry metadata through the edit process, preserving date/time stamps, author, and other info. The preserving of this data is done automatically by the program. Edit programs can remain confident about the integrity of the metadata, even if the wad happens to get edited by an old editor that does not know how to maintain metadata. Because of this, metadata can be though of as being very permanent, and very suited for storing author info.

Randy87 said:

(questions about the proposed implementation)
I don't want to slog through this mess of a thread again, so I don't remember all the listed uses.

I don't blame you :) Please see below:

--------------------------------------------------------------------
I feel that there are enough of us that agree on the base implementation idea that we can go ahead and start hashing out the exact specs. I've started working on just that document, but it'll take me a few days to complete.

So, I think we should proceed as follows:

Please give me a couple of days. I'll start a fresh thread (if that's ok with the admins). The thread will be exclusively for those ready to proceed with specs. I'll post my specs, and we'll all critique and fine-tune them to perfection.

Alternately, if you're ready with some specs of your own, go for it.

Basically, my specs have already been described, and I think we agreed on most points. Here's a summary:

. single metadata lump, maybe called "~META~" (something very unique)
. multi-line text-only lump, resembling .ini/UDMF/JSON
. header line is lump name, maybe in [brackets]
. details are one name/value pair per line, like "name = value;"
. ? lines terminated with <CR> 0x0D. <LF> 0x0A ignored? Or the reverse?
. must contain hash field. CRC32 preferred over MD5.
. Multiple ~META~ lumps are concatenated in lump index order.
. Entries are in lump index order.
. Unconditional 1 entry per lump.
. Match is on both name and hash.
. 1st matching meta entry corresponds to 1st matching lump entry.
. meta entries apply to one and only one lump, in entry order + in lump order.

. Lumps with no match normally get new default metadata, but the editor could try to attach unmatched meta data with mismatched hash, after user prompt: "'CEMENT9b' was originally created by 'kb1' on 5/28/2015, but it has been changed. Preserve original metadata?".

. Edit tools might store the user name in a config file, to be used automatically be the editor, to fill in the Author field.

Other things in the spec should describe what editors should do in certain situations.

Please see my earlier post on specs, and my pseudocode that shows my take on how the editor can handle the loading of the metadata lump.

This is a nice idea - let's make it happen!

kb1 said:

@DaniJ - By all means, please stay. I'm not asking you to agree. I'm asking you to stop derailing the proposal. 1/3 of the posts are attempts to respectfully explain the concept to you. Another 1/3 are getting people back on track after you've disrupted the idea, by discussing completely irrelevant topics, like Zip vs. WAD.

Believe me, I do understand the concept. I am not intentionally trying to 'derail' the topic.

The problem as stated in the OP starts off fairly well-defined. Essentially essel wants (a) tool(s) to assist him in the task of tracking assets and their metadata, through the development process of putting together a new project.

From here, things start to veer into less well-defined territory. Rather than simply being an editor/tool based mechanism to support that goal we are now in the realms of defining a common standard for metadata that will exist in final projects released to the wild. (Clearly a solution had already been partially specified given the title of this thread).

Please understand that once you start introducing hashes for asserting this relationship in released projects - strictly speaking, you are no longer solving the same problem. This is now veering into the realms of data signing (a topic which I freely admit to having very little knowledge of).

All I am really saying is - shouldn't the problem as stated originally be solvable entirely within the editor? If so then we don't need hashes in released projects in order to track the state of assets and their relationship to their metadata because one should be able to assume that they do relate - the project has been finalized and released. If we can't assume this then I agree we need a way to check whether the data has been altered. However, are we saying we need to do this for every lump individually, or can we sign the package itself and on that basis determine that the reset of the metadata/assets have been integrated by the editing tool correctly? If we can't even assume that - it seems to me we need both a common metadata standard and a mechanism to track changes and in which case, is inserting hashes into the common metadata logical (or is there a better way that separates these concerns)?

If we are going to define a common standard for metadata - that's one thing. Tracking the state of partially integrated projects during the development process is another, different thing. Checking the integrity of the subcomponents of released packages using embedded hashes is yet another.

So let us just examine if what we are purporting to do with a common standard for metadata makes sense for solving the original problem, before diving into specifying a solution for it. If we can logically separate the fundamentally different goals (as I believe we absolutely should) - it simplifies the problem.

I am not a fundamentalist radical trying to derail the whole process. I am simply trying to highlight an issue which I believe has been skirted over.

The reason I brought ZIP into the equation is because of the revised goals (tracking asset/metadata changes in released projects), leading to defining a common standard for metadata that includes hashes (which from my perspective, concern a relationship that is already known to be good because the final project has been released in this form). A unified standard would be great for this but understand that I want to use the same representation for metadata in ZIP, as well as WAD.

Edit+2: Reordered paragraphs for clarity. (2)Expanded on arguments to avoid potentially 'derailing' again.

To be honest I think you're reading too much into the hash thing. It's purely to help the editor link metadata to lump, in the case that the lump is moved or renamed, nothing more.

If the hash is not there, the editor can't determine with certainty that a particular piece of metadata is up-to-date for the lump it represents (eg. the lump may have been modified outside the editor).

DaniJ said:

Please understand that once you start introducing hashes for asserting this relationship in released projects - strictly speaking, you are no longer solving the same problem. This is now veering into the realms of data signing (a topic which I freely admit to having very little knowledge of).

That's a slippery slope argument.

The hash isn't there to validate the lump itself, it's there to validate that the metadata describes it.

This is not the first tentacle of DRM-WAD slithering its way into our free software collectivist commune.

Here's a quick Q&A:
Q: TOTOR.WAD has a TITLEPIC lump, but its metadata hash is invalid! What should my port do about it?
A: Just load the TITLEPIC normally; it should make no difference in a port's core functions whether there is no metadata, invalid metadata, or valid metadata.

Gez said:

DaniJ said:

Please understand that once you start introducing hashes for asserting this relationship in released projects - strictly speaking, you are no longer solving the same problem. This is now veering into the realms of data signing (a topic which I freely admit to having very little knowledge of).

That's a slippery slope argument.

In terms of specifying a solution to the original problem? Ostensibly it is a slippery slope argument, without context. I'm not saying "don't do this because we don't know where it might lead", I am saying "do we even need to do it like this in the first place (and in doing so, inviting otherwise unrelated concerns)?".

There seems to be a split focus in this discussion; a) signing data in a released project, and b) managing an edit time integration problem (as originally stated). I think we need to be clear and decide - which is it?

I personally think we need to look into separate solutions to these problems. If you guys don't agree at this point, now that all the arguments have been laid bare - I am happy to walk away and leave you guys to it (just say the word).

Lumps are identified by both name and md5 for several reasons that have already been outlined in the thread:

Multiple lumps can (and, in the case of map lumps, often do) have identical names, so name alone is not enough to match a metadata entry to a lump.

Having the md5 hash will help to preserve the accuracy of metadata if a user edits the wad using a non-metadata-aware tool.

It can't be separated into two separate problems as you suggest, because the only reason for the md5 hash to be there at all is to improve accuracy and resilience in the matching of metadata to lumps. It has nothing to do with signing lumps.

I don't agree that whether a project is finished or not makes a difference. The metadata being discussed here describes each single asset separately (the dreaded 1:1 relationship), the status of the project as a whole does not enter the equation.

Furthermore, it has also been established that resources can have identical names (the WAD format allows to have several lumps with the same name, and it is in fact a requirement of the format in order to have several maps; and even though most generic zip archive managers do not allow it, it is also a capability of the ZIP format). So to the question "do we need hashes?" the answer is that we need something that will be absolutely non-ambiguous, knowing that names are ambiguous. That leaves either directory index position or content hashes.

If we go with the former, then an edit by a non-conforming tool will probably destroy the validity of all metadata. Maybe just inserting a readme.txt file in a pk3 (something that could be done to a finished, released project by someone who repackages it) will cause the zip software to sort all the entries by alphabetical orders. Now none of the name+index pairs match, so all the metadata is invalidated!

If we go with hashes, we don't have these problems. Order can change. Entries can be deleted or inserted, this will not invalidate metadata for the others. And if an entry is replaced by another, this will be detected. The only entries without metadata will be those that have been added or modified, instead of all of them.

TL;DR:
1. We need a unique identifier for each entry
2. It cannot be the name (ambiguous because of duplication)
3. It cannot be the directory index (unreliable in case of repackaging, potentially invalidating all the content despite no changes besides reordering)
4. The content hash is non-ambiguous (except for 100% identical entries) and reliable (it will survive repackaging in a non-conforming tool).

As it would seem we simply don't agree, by all means carry on with defining your spec.

DaniJ said:

Believe me, I do understand the concept. I am not intentionally trying to 'derail' the topic...

...From here, things start to veer into less well-defined territory. Rather than simply being an editor/tool based mechanism to support that goal we are now in the realms of defining a common standard for metadata that will exist in final projects released to the wild. (Clearly a solution had already been partially specified given the title of this thread).

I am not sorry that we get to kill 2 birds with one stone - I'm quite proud of noticing it...a fact which I do not try to hide.

DaniJ said:

Please understand that once you start introducing hashes for asserting this relationship in released projects - strictly speaking, you are no longer solving the same problem. This is now veering into the realms of data signing (a topic which I freely admit to having very little knowledge of).

All I am really saying is - shouldn't the problem as stated originally be solvable entirely within the editor? If so then we don't need hashes in released projects in order to track the state of assets and their relationship to their metadata because one should be able to assume that they do relate - the project has been finalized and released. If we can't assume this then I agree we need a way to check whether the data has been altered. However, are we saying we need to do this for every lump individually, or can we sign the package itself and on that basis determine that the reset of the metadata/assets have been integrated by the editing tool correctly? If we can't even assume that - it seems to me we need both a common metadata standard and a mechanism to track changes and in which case, is inserting hashes into the common metadata logical (or is there a better way that separates these concerns)?

I assert that "No, there is not a better way". I don't want to assume that the "finalized product" has not been altered - this proposal couldn't care less. When is a project ever "finalized" anyway? Why invalidate the ownership/last modified dates of every lump just because someone changes a script? Adds a song? Modifies the TITLEPIC?

There's no need to build a "sign the package" function. CRC32, MD5, and a million other file-based tools exist, and are in use. Yes, that's an important tool. But, it has nothing to do with this project. This proposal is lump-based.

The OP wants per-lump tracking. The proposal gives it to him. And, by adding one more field, we get all this other shit for free. That's as far as I need to think about it, to know that it's a good thing - nice, in fact. How that becomes a "skirted-over issue" is beyond me.

DaniJ said:

I am not a fundamentalist radical trying to derail the whole process. I am simply trying to highlight an issue which I believe has been skirted over.

Ok, then I ask you to approach this from a view of the future where it is already in place, and being used by many projects. How are you, or your source codes, or your methods affected? What problem is this implementation causing you and your friends? I'm not being sarcastic - I am asking for you to access the damage that has been caused by including hashes to lumps, within a lump.

DaniJ said:

The reason I brought ZIP into the equation is because of the revised goals (tracking asset/metadata changes in released projects), leading to defining a common standard for metadata that includes hashes (which from my perspective, concern a relationship that is already known to be good because the final project has been released in this form). A unified standard would be great for this but understand that I want to use the same representation for metadata in ZIP, as well as WAD.

Go ahead. It does not conflict with that goal whatsoever. It's just a lump. Add whatever you wish to your zip. And, if someone uses an editor with metadata support, your zip will contain a lump called ~META~.

DaniJ said:

I personally think we need to look into separate solutions to these problems. If you guys don't agree at this point, now that all the arguments have been laid bare - I am happy to walk away and leave you guys to it (just say the word).

Ok, I'll be the asshole. "word". Is that what you were waiting for? Just realize: You were the one arguing.

I'm kinda blue-in-the-face (hands) from discussing this simple feature to death. I think what I and others provided and tweaked, is more than sufficient for going out and just adding the feature into the editor of your choice.

I'm not currently in a position to hack up someone else's source at the moment. I'm in the middle of a total reorganization of my source folders (source code, both Doom and other, resources, media, etc), spread across 6 hard drives. I'm organizing, finding and removing duplicates, renaming files, adding/removing media tags, etc. I've decided to halt all development until I am done. This could be a while.

However, I will assist anyone attempting to add metadata support to their editors. I think it's a nice, useful idea, and I know that it could have helped me personally, quite a bit in the past.

I am hoping that what's in this thread is good enough for pseudo-specs, until a prototype is built, and empirically tested by a few good testers. That tweaked prototype can serve well to provide a formal specification. Besides, the first person to build support for metadata will be in a position to dictate the exact specs!

Eventually I will add support to a fork of an editor or two, but I've got such a huge backlog, I don't want to promise anything. Hopefully, the ugliness in this thread does not distract from the coolness of the idea, and someone closer to the source can just give it a shot.

Again, I am happy to provide any assistance on the implementation. But, I'm done explaining why it's good, and why it works.

Yeah I'm going to start looking at implementing it soonish, though it will probably be for SLADE 3.1.1, which is still a ways off.

sirjuddington said:

Yeah I'm going to start looking at implementing it soonish, though it will probably be for SLADE 3.1.1, which is still a ways off.

That's great news! Again, anything I can do - let me know. What language is SLADE written in?

kb1 said:

What language is SLADE written in?

https://github.com/sirjuddington/SLADE should give you an idea.
Looks like C++.

Thanks, duh!

esselfortium said:

I think my biggest question here is: Why would someone who doesn't want to use Slade, want to spend the time to manually create metadata that has no relevance to source ports and literally only exists to be displayed alongside the lump list in Slade or similar specialist tools?

People might want to generate slade compatible lumps via other tools. For example the freedoo, build system might want to compile such a lump.

What ever became of this?

Having spent the last 20 minutes skim-reading through the thread, I'd be interested to know too! Sounds like a useful idea.

A lot of people are still interested in this idea. To encourage discussion and some solidity to the idea, here is an example specification to start with:

https://gist.github.com/jmickle66666666/df81f82adc91df406dafc7ff24dd7a83

As far as I can see this would be a good basis to work with, but I'd love to know people's thoughts and changes that could potentially be made regarding it.

Edit: Clarified issue with duplicate names

That looks like a good start, I think :) Two years after the original post, I'd still really love to see this idea implemented in Slade and GZDB, so I'm happy to see it revived.

Name aliases would be really useful, and it'd be exciting to eventually see their scope extended to the contents of TEXTURE1/2 lumps somehow as well, so that virtual folder structures and long/descriptive names can be utilized for big texture packs. I know that's a bit of a gravy thing on top of the basics here, though, and probably shouldn't be included in the initial spec for the sake of simplicity.

To be clear: my specification of the METADATA lump has the following goal: Transparently extend the Lump format to support a number of user-defined fields, without modifying a Lump's data structure at all.

I don't think it's reasonable to have to put duplicate METADATA information for all lumps that make up a map. Besides, the format can be changed (UDMF->Hexen, etc).
So perhaps for maps, it should point to the map marker only.

Also, there can easily be duplicate lump names in different namespaces (one example being, again, maps).
You can even have duplicate lump names in single namespace (like multiple DECORATE lumps, and not only in WAD, but also in PK3).
See Project Brutality, which is a PK3 that has two lumps both called DECORATE. (I remembered about this because it's something I was going to fix in GZDB...)
How is this handled?

Also, do we log lump deletion into METADATA or just delete the record?
If we log it, how? If we don't, then after deleting and recreating a lump (instead of replace, which SLADE actually can't do IIRC, if you want to replace a lump, you need to delete old and import new) the record will be lost.

As for GZDB, not sure if I want to even try to support this thing, the lump will grow to massive proportions, especially if each new GZDB revision gets added.
Additionally, GZDB has this fancy overcomplicated script editor which will have to also update METADATA for when you are editing DECORATE lumps, etc... (and every new revision of GZDB will be a separate editor version). All this just makes me want to go and suicide.

I'm not convinced that every editor version needs to be listed specifically in metadata.

As far as map lumps, this should be completely agnostic to what is actually contained in the lump: these are additional properties given to each and every lump, and trying to detect which ones 'ought' to have it or not is only going to make a mess of things. Map format and anything else about the actual content of the lumps is irrelevant here, outside of the hashing checks to match those lumps up with metadata records.

Deleted lumps shouldn't be in metadata.

Why having both MD5 and SHA-1? Isn't it a bit redundant?

ZZYZX said:

I don't think it's reasonable to have to put duplicate METADATA information for all lumps that make up a map. Besides, the format can be changed (UDMF->Hexen, etc).
So perhaps for maps, it should point to the map marker only.

METADATA shouldn't care what a lump is. it is extension data for *every* lump, regardless of what it is. No exceptions or it leads to unnecessary complications

Also, there can easily be duplicate lump names in different namespaces (one example being, again, maps).
You can even have duplicate lump names in single namespace (like multiple DECORATE lumps, and not only in WAD, but also in PK3).
See Project Brutality, which is a PK3 that has two lumps both called DECORATE. (I remembered about this because it's something I was going to fix in GZDB...)
How is this handled?

PK3s don't need metadata. the METADATA lump is specifically for wads. This is not an issue. If there are two lumps called DECORATE in a wad, they can still be told apart by checksum and position in the wad, both of which are stored in METADATA.

Also, do we log lump deletion into METADATA or just delete the record?
If we log it, how? If we don't, then after deleting and recreating a lump (instead of replace, which SLADE actually can't do IIRC, if you want to replace a lump, you need to delete old and import new) the record will be lost.

No need to log deletions, there's no practical application. The METADATA is just for storing extra information for existing lumps. When you create a new lump it is treated as such: a new lump.

As for GZDB, not sure if I want to even try to support this thing, the lump will grow to massive proportions, especially if each new GZDB revision gets added.
Additionally, GZDB has this fancy overcomplicated script editor which will have to also update METADATA for when you are editing DECORATE lumps, etc... (and every new revision of GZDB will be a separate editor version). All this just makes me want to go and suicide.

This is a valid point, i agree it would be unwise to log every editor version.

The editors property is also quite non-essential anyway, if it turns out there's no practical application of it then I wouldn't mind not seeing it implemented.

Why having both MD5 and SHA-1? Isn't it a bit redundant?

Quite possibly. I know it is standard to provide both for checksums but it might not make sense built into metadata. I can see there being filesize issues if both are stored for every lump. The original justification was the extra layer would add a layer of protection against checksum conflict.

I'm glad to see this getting some traction and interest again - it's a neat, rather simple idea. It's simple, but crucial for editors to handle it exactly, in a specific careful manner, to keep it from being a mess. The maintenance of METADATA should be transparent and completely automated, and, with proper handling, it can be.

ZZYZX said:

I don't think it's reasonable to have to put duplicate METADATA information for all lumps that make up a map. Besides, the format can be changed (UDMF->Hexen, etc).
So perhaps for maps, it should point to the map marker only.

Also, there can easily be duplicate lump names in different namespaces (one example being, again, maps).
You can even have duplicate lump names in single namespace (like multiple DECORATE lumps, and not only in WAD, but also in PK3).
See Project Brutality, which is a PK3 that has two lumps both called DECORATE. (I remembered about this because it's something I was going to fix in GZDB...)
How is this handled?

You must track every single lump in the WAD, or it will not work 100% of the time. Lumps should be uniquely identified by three identifiers:

Lump Name
Hash
Index

The inclusion of all 3 identifiers is mandatory to completely automating the process, and preventing false identifications. Lump name and hash are not enough, since lump names are not unique, and hashes for 0-length lumps will all be the same.

Imagine the usage case of having 2 WADs open in an editor, with the goal of copying a map from WAD 1 to WAD 2. Also, let's say that both WADs have METADATA lumps. So, the user highlights the lumps of a map, but misses one. Then, the user pastes the lumps into WAD 2, mistakenly forgetting to include one of the lumps. You want the user to be able to highlight the lump they missed, paste it into WAD 2, and everything to work ok. For this to be successful, the editor must do the following:
1. Load up the METADATA from both WADs at WAD-load time.
2. Check the integrity of each METADATA entry against the lumps at WAD-load time, and clear the METADATA for each entry that does not match. Possibly, the user could be prompted, in some situations, for what to do with the invalid METADATA
3. Track all 3 identifiers for every lump operation.
4. Be sure to copy METADATA info, in every lump copy operation.
5. Be sure to delete METADATA info, for every lump delete.
6. Keep up the METADATA on lump rename, lump move, lump edit.
7. Prompt for METADATA for each lump add, or, have a function for editing lump properties.

Some notes:
. Non-compliant editors will alter lumps without maintaining METADATA, so a compliant editor must properly "reconnect" the METADATA on WAD load.

. Some lumps could be marked with a special flag that tells the editor to maintain METADATA info even if the lump changes. An example would be the altering of a map, in a non-compliant editor. If the map lumps are marked as "changeable", the compliant editor will default to trying to reconnect the map lumps with their METADATA, even if the hash doesn't match.

. METADATA will get invalidated by non-compliant editors, so METADATA-aware editors must be resilient against this.

. Editors could default to adding the author's name to new lumps, optionally.
. It would, in fact, be nice to have a texture METADATA lump.
. METADATA could be used for support of long lump names/texture names. Ports could be modified to support long lump/texture names as well. This adds a new importance to the proper handling of METADATA.

. Ports could be modified to show some metadata about the maps as they are loaded, such as map author, description, long map name, etc.

I'm skeptical of having sourceports read METADATA. We already have MAPINFO formats for that stuff, for one thing, and a lot of standard Doom content (like a map!) isn't contained within a single lump, anyway, which makes METADATA very unsuited to displaying information about a map. Long map names already exist ingame: they're just the display name of the map :)

This stuff is just not METADATA's purpose, and complicates the premise to accomplish things that are already accomplished elsewhere.