Vanilla engine + custom sprites in a PWAD

OK. I've heard and also experienced that you cannot simply have custom sprites in vanilla Doom. I've also read this, this and this. I've made some experiments myself but wasn't able to get a PWAD working. I have two questions regarding the topic of vanilla PWADS with custom sprites.

1. From player's perspective: I'd like to be able to play vanilla PWADs with custom sprites in vanilla engine. Particularly for Batman Doom, but I think BTSX is the case too, as all Doom sprites had to be reincluded in it because of the new palette. Can anyone give me quite exact instructions what to do with DEHACKED, DEUSF or anything other I may need?

2. Now from the author's perspective. Let's say I wanted to make a PWAD for vanilla with custom sprites. On one of the linked pages I've read that to achieve so, I'd need to insert ALL existing sprites in the PWAD. Really all? Assuming that yes, I'm curious about this: How about the sprite naming? Does it have to exactly match too? For example, can I place one sprite POSSA0 instead of all 8 POSSA1-8 rotations, and would vanilla deal with it?

scifista42 said:

Let's say I wanted to make a PWAD for vanilla with custom sprites. On one of the linked pages I've read that to achieve so, I'd need to insert ALL existing sprites in the PWAD. Really all?

No. This problem only occurs if you try to replace the iwad sprites, but you can be more clever: rename the sprite name (I mean stuff like TROO, etc) in DEHACKED, make the object use that newly named group of sprites and, of course, add the sprites with new names. Congrats, you bypassed this limitation.
You can check Wraith's Alpha Accident demo for an example. (he is the author of this method)

Memfis said:

No. This problem only occurs if you try to replace the iwad sprites, but you can be more clever: rename the sprite name (I mean stuff like TROO, etc) in DEHACKED, make the object use that newly named group of sprites and, of course, add the sprites with new names. Congrats, you bypassed this limitation.
You can check Wraith's Alpha Accident demo for an example. (he is the author of this method)

That is a clever method, and I'm surprised I've never heard of it before. Extremely useful info. Is this documented anywhere besides forum threads?

plums said:

Is this documented anywhere besides forum threads?

http://www.doomworld.com/tutorials/fx2.php

Thanks, that sounds like a pretty good trick and if it's actually vanilla engine compatible, it solves almost everything.

However I worry about one thing:

What if I wanted to replace item pickup sprites? Wouldn't that break their behaviour? Because you know, the item you get by picking them up depends on the hardcoded sprite name, I believe.

No, again, check out Alpha Accident where Wraith does this without any side effects. As I understand it, by changing that line in DEHACKED you change the hardcoded name.

Another amazing find of his: so in the iwad you have sprites BON2A0-BON2D0 for armor bonus. You can add a new set of sprites called BON2E0-BON2H0, make some object use them and add "can be picked up" flag to that object. Congrats, you have a new, different looking armor bonus. :) You can do some nifty things with this, for example always put new armor bonuses stacked together in your map and then it will look like a new, better armor bonus.

Just one note: in this case you need to include the original armor bonus sprites as well after all. Doom can't just take the first four sprites from doom2.wad and the other four sprites from your wad.

The reason this method did not emerge during the vanilla heyday is that a) It depended on an external tool hacking the executable and b) The executable version changed a lot of times between 1993 and 1995 (1996?) so this is definitively not an easy or highly portable method.

I can imagine that even if some mod used it back in the day, it will most likely be broken with source ports, especially if applied pre-v1.9, resulting in yet another Obituary or OttawaU-like debacle and endless butthurt, especially if the author had the sadism of bundling it with the infamous "you MAY NOT modify" clause.

Of course, after the vanilla engine was settled at v1.9, there'd no problem, for the most part.

Thanks for all the information. Alpha Accident is unfortunately down right now and I don't have its dehacked file, but I've made some experiments myself anyway and achieved partial success.

(Sorry, now I'll be browsing more to the Doom editing category, but when I started this thread, I intended to ask primarily about playing mods with sprites - and my question regarding that still remains unanswered.)

I've come across this thread from 2004, and quite interesting things are mentioned there.

I attempted to create a custom monster with new sprites, replacing Hellknight. I added new sprites named "BOS3" (which I changed via dehacked, replacing BOS2). I used 2 more sprite "letters" than the usual BOS2 have - namely BOS3P and BOS3Q. Using DEHACKED I hacked the Doom executable, and after several attempts, I got my monster to succesfully inhabitate the maps and work in vanilla, as well as sourceports!

Then I wanted to try the harder thing: to place this monster in an Ultimate Doom map, run with Ultimate Doom IWAD. As advised in the thread linked above, I changed the editor number of Hellknight to a totally new and unused value - 301. Then I created a test map (E1M1) with a thing 301 placed in it. When I launched the map in sourceports, it worked. Then I tried vanilla. I hacked the (new and clear) executable with the new .deh patch, but vanilla crashed upon entering the map. Chocorenderlimits, which I tried later, crashed too - with an error message "R_ProjectSprite: invalid sprite frame 43 : 1". Does anyone have a clue why it didn't work?

http://www.doomworld.com/vb/doom-general/58920-r-projectsprite-crash/

Rather than try to understand it myself I'll just link to that thread.

I've just found what the problem was. You forgot to tell me that I still need to use DEUSF to merge my PWAD with sprites with an IWAD. Pretty obvious, but I somehow automatically assumed that this sprite trick should work as itself. When I'm thinking about it... It actually ruins the efficiency of the trick, because all sprites need to be in the PWAD anyway! At least I finally learnt how DEUSF work and how to use it to play vanilla. Which means that now I know what I've asked for. Thanks for all the advices!

I think the "problem" is that by hacking the executable you got past the limitation of having to merge your PWAD (or providing placeholders/duplicates for all IWAD sprites in your PWAD) by using your own sprite names -preferably ones that DON'T clash with the IWAD sprites at all.

Of course, even if you use this trick, you have still to replace either ALL or NONE of the hardcoded sprite names, you can't only change SOME.

E.g. let's say that you invent your own sprite class e.g. NAZIA0, NAZIBO, etc. and you decide to "sacrifice" the Former Human as a placeholder, so you replace the POSS string in the .EXE with NAZI.

What happens to the OTHER sprites like TROO (Imp), BOSS (Baron), SARG etc.? When loading your PWAD, once again, the limitation of having to keep all sprites in one continuous sequence between the S_START and S_END marker lumps is NOT lifted simply by having hacked a name: that's why you can't modify just SOME sprites with this method: you either modify them ALL (and ALL of your sprites reside in ONE external PWAD), otherwise you'll have to go by traditional merging, and you lose what -potential- little benefit this method has.

scifista42 said:

I've just found what the problem was. You forgot to tell me that I still need to use DEUSF to merge my PWAD with sprites with an IWAD. Pretty obvious, but I somehow automatically assumed that this sprite trick should work as itself. When I'm thinking about it... It actually ruins the efficiency of the trick, because all sprites need to be in the PWAD anyway! At least I finally learnt how DEUSF work and how to use it to play vanilla. Which means that now I know what I've asked for. Thanks for all the advices!

The link Grazza posted says that with that technique, you shouldn't have to merge sprites, but it doesn't work with "SSWV, POSS, SKEL, BSPI and SPID", whereas "I had no trouble replacing the FATT, CYBR, MANF, FATB and BOS2 entries."

Thanks for moving the thread.

plums and Maes: I was actually "replacing" BOS2 sprites that way I changed BOS2 to BOS3 via DEHACKED. Then I only had the BOS3 sprite set in between SS_START and SS_END markers in my PWAD, and nothing else. That sounds complying to Grazza's link and what are both of you claiming, doesn't it?

But the BOS3 set had more "letters" (you know what I mean) than an usual BOS2 set. Would that make the problem? After reading the linked text, I was convinced that it wouldn't make a problem, I may be mistaken though.

I'm sorry, now I have everything I wanted and I am totally satisfied. I'm just curious why the discussed method didn't work as expected.

scifista42 said:

SS_END

Try with S_END (one s) instead?

That's right, the closing marker must be S_END. For flats it also carries a single letter (F_END).

Memfis said:
You can check Wraith's Alpha Accident demo for an example. (he is the author of this method)

He may have discovered it independently but it's known since 1996 at least. Same goes for custom sprite frame values.

I've used these methods a lot, generally editing any PWAD that "requires" DEUSF merging and storing it in this method with a corresponding DeHackEd patch.

plums said:
it doesn't work with "SSWV, POSS, SKEL, BSPI and SPID",

SSWV works, at least. I think I've swapped some of the others too.

Some failures have to do with the placement of the lumps once you make many changes, and moving the sprites to the top of the PWAD may fix this.

So is it possible to add custom monsters to a vanilla wad, like from realm667?

odysseyofnoises said:

So is it possible to add custom monsters to a vanilla wad, like from realm667?

No.


You cannot add anything. At most, you can change existing stuff. Some of the stuff is unused or redundant (two different "gibbed players" object, the "dead lost soul" object, etc.) so you can sacrifice them without losing actual content, but then you run into another problem. Vanilla doesn't just have a limited number of object type slots, it has a limited number of state slots. So while there are also unused or (mostly) useless states available (like the pain elemental resurrection frames), there's a severe limit to the complexity of the "new" actors you'd bring. Even worse, you cannot add a codepointer to a state that didn't have one to begin with!

So in most cases, if you want to bring a new monster to the game in vanilla, that means you'll also have to sacrifice an old monster.

Also, most of the Realm667 monsters use coding features from ZDoom that are impossible to replicate in vanilla Doom because the relevant codepointers just don't exist and don't have anything remotely similar.

I forgot to reply in this topic, thank you for giving me a notice about S_END, it worked and I've finally succeeded in making a vanilla monster for Doom 1.

Now a question for anybody who knows something about it, maybe:

Would boss spawners be put into work in Doom1 somehow if I provided sprites and did DEHACKED changes, or is it impossible because of hardcoded forbidden thing spawn IDs, too? You know, I can change the IDs via DEHACKED, but what about the dynamically spawned cubes? And monsters?

I think you can "activate" Doom II-specific code even in Ultimate Doom -the .exe is the same, and AFAIK there's no check against spawning Doom II specific items (monsters, however, are checked).

It might be possible to do monkeying like e.g. adding Megasphere and SSG pickups in Ultimate Doom, if you put the proper thingno in a map, and it may even work if you provide frames. However, you won't be able to manually select the SSG it or switch it with the normal SG, because that is actually checked. Hmm....

Edit: from a quick check of the source code, spawning any of the Doom 2-specific items & decorations is not expressibly forbidden, and since they are defined anyway in the code (even if not used in maps), they will -mostly- work. Now, the Megasphere actually has a check for Doom II, so even if picked up, it won't work. However the SSG has no checks against picking it up (and giving you the ammo that comes with it, so if anything it could be used as a gimmick one-time source of shells), but it might be tricky (or impossible) to select it.

However, some TCs like the The Quest For The Necronomicon actually use the SSG codepointers. It's also possible to use e.g. revenant projectiles and the such with DEHACKED, or actually link to the megasphere codepointer.

Edit 2: Boss Brain & Shooters are checked against placing them in a map (actually, things 88 and 89), but not spawn cube targets (87) flaming barrels (for which there also are sprites...)